Joined: |
Apr 22, 2008 03:56 AM |
Last Post: |
Oct 22, 2008 11:31 AM |
Last Visit: |
Oct 22, 2008 11:33 AM |
Website: |
|
Location: |
|
Occupation: |
|
Interests: |
|
|
AIM: |
|
ICQ: |
|
MSN IM: |
|
Yahoo IM: |
|
|
jacobi has contributed to 6 posts out of 21251 total posts
(0.03%) in 6,024 days (0.00 posts per day).
20 Most recent posts:
Hi,
We had a second similar occurrence of the problem about a week ago.
We now subscribe to two feeds that run identical systems on two different computers (one paper trader, one actual trader) . October 14th, several time stamps were missing from both feeds.
Feed P: 9:30, 9:32-9:44, 10:46-10:15, 10:17-10:22 etc. Feed T: 9:31-9:37, 9:39-10:30, 10:32-10:37 etc.
To be able to debug, we save each line of text that is received from the IQFeed socket (along with our own local timestamp), and the above-mentioned time stamps from IQFeed are simply missing. Also, many of the given times of trade were severely out of sync with our local timestamp (time of saving the data to the log file). As you explained it previously, I cannot see how the missing time stamps can be attributed to our software since we save every line received, as mentioned, and if IQConnect's internal buffer queues infinitely as you explained nothing should be lost along the way, or maybe I'm missing something.
The size of the text file with a day worth of text lines from the socket usually takes up about 10 GB. That particular day, the text file was only 1.2 GB large. I conclude from this that several price updates were also missing from the feed. I am pretty sure that the page file was not in action (no other application are running), and furthermore, paging shouldn't leave holes in the feed as I see it, just a premature end, right?
Our systems have had several days with working feeds, nothing changed, and then the problem appeared one day, and was gone the next.
Do you have any idea of what is happening? We lost some pretty big opportunities and have now build in several alarms that will go off if it happens again. But obviously it would be best to actually get the timestamps and data which is not delayed.
The specs of the paper trader system is as explained above, the live system runs on Vista 64 with 4GB of RAM on an Intel Core2 Duo @2.66GHz.
I hope that you can shed some light on this. Let me know if you need additional information. Thanks.
Thanks for your prompt reply.
My system: Windows XP Pro/SP3. Pentium D 3.20 GHz (dual core), 3.25 GB RAM.
I was watching updates on all S&P 500 stocks using the IQFeed API via TCP/IP to localhost. The 500 symbol updates probably accounted for a good deal of net traffic.
You are correct that it is my own app (C#) that uses the IQFeed API. No other applications using the feed or other intensive calculation were running (3rd party or otherwise), no DTN.IQ.
I didn't record CPU utilization. The only intensive app running was the above-mentioned app that more or less just stored the TCP/IP stream from IQFeed directly as received in a flat text file. I did not get any socket time-outs or other exceptions from the system.
Is it possible for IQFeed's internal buffer to reach its limit, and what will happen if it does?
I experienced some problems with the feed yesterday that I haven't seen before. Can anyone shed some light on what happened?
I am tracking about 500 real time symbols and did not receive any time stamps 9:32 AM-9:37 AM (inclusive) (but I did receive symbol information updates).
Also, around 9:31 AM data was received a bit too late. The following is a dump of the TCP/IP feed I received (with my own time stamps at the beginning of each line). Note the 25 seconds delay.
2008-09-15 09:31:25.858: Q,CVH,D,35.34,500,35.16,35.30,100,300,09:30:03b,35.34,36.15,0.14,c,09/15/2008, 2008-09-15 09:31:25.858: Q,JNS,D,23.71,0,22.37,22.86,300,1000,17:27:27b,0,23.71,0.49,c,09/12/2008, 2008-09-15 09:31:25.858: T,20080915 09:31:00 2008-09-15 09:31:25.858: Q,PRU,D,76.23,3122,76.09,76.11,100,100,09:30:51b,76.80,80.59,0.02,t,09/15/2008, 2008-09-15 09:31:25.858: Q,NTRS,F,85.1100,23086,85.0100,85.0900,1000,500,09:30:58b,85.8900,87.0900,0.08,t,09/15/2008,
I'm subscribing to the following fields: S,CURRENT UPDATE FIELDNAMES,Symbol,Exchange ID,Last,Total Volume,Bid,Ask,Bid Size,Ask Size,Last Trade Time,Open,Close,Spread,Market Center,Last Trade Date
There were no mentionable delays at neither 09:30 nor 09:38. The delay could probably be attributed to the heightened market activity of yesterday, but the lost timestamps worry me. Do you stop sending them, if the delay is too large?
I would appreciate assistance. Thanks in advance.
No, the problem is not there anymore, neither using COM or sockets. I'm now getting the full amount of requested MSFT data without any errors.
Thanks.
Correction: The lookup2 interface gives good data but only for yesterday (2008-04-21) and not further back as does the socket interface. Lookup1 gives junk dates. How come?
I am requesting MSFT 1 minute data for the last 7 days via the COM interface. I'm getting a result that looks like this (time,OHLC,vol,accvol):
1999-11-30 10:34:43,30.03,30.03,30,30,1200,1200 1999-11-30 10:34:43,30.03,30.03,30.03,30.03,1175,2375 1999-11-30 10:34:43,30.03,30.03,30.03,30.03,525,2900 1999-11-30 10:34:43,30.03,30.03,30.03,30.03,200,3100 1999-11-30 10:34:43,30.03,30.03,30.03,30.03,430,3530 1999-11-30 10:34:43,30,30,30,30,570,4100 1999-11-30 10:34:43,30.09,30.09,30.09,30.09,200,4300 1999-11-30 10:34:43,30.04,30.04,30.04,30.04,250,4550 1999-11-30 10:34:43,30.1,30.1,30.1,30.1,160,4710 1999-11-30 10:34:43,30.15,30.36,30.15,30.3,11324,16034
The error has been reproduced on two different machines running respectively the latest stable release and the beta version of your API. The error appears both when using your demo C++ examples (using Lookup1) and from C# where I do
_iqHistory = new DTNHISTORYLOOKUPLib.HistoryLookupClass(); _iqHistory.MinuteCompleted += new DTNHISTORYLOOKUPLib._IHistoryLookupEvents_MinuteCompletedEventHandler(_iqHistory_MinuteCompleted); _iqHistory.MinuteProgress += new DTNHISTORYLOOKUPLib._IHistoryLookupEvents_MinuteProgressEventHandler(_iqHistory_MinuteProgress); _iqHistory.RequestMinuteHistory("MSFT", 7, 1);
Over 400 other symbols that I have requested seem to produce good data. Using the Lookup2 interface on MSFT also gives good data.
Help would be appreciated. Thanks in advance.
|
|