Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I am very pleased with the DTNIQ system for quotes and news." - Comment from Larry
"I am very happy I changed. I love the product, but more so I am thrilled with Tech Support. You are knowledgeable, polite, pleasant and professional." - Comment from Pat
"IQ feed is brilliant. The support is mind-bending. What service!" - Comment from Public Forum Post
"Thanks for the great product and support. During this week of high volume trading, my QuoteTracker + IQ Feed setup never missed a beat. Also, thanks for your swiftness in responding to data issues. I was on ******* for a few years before I made the switch over early this year, and wish I had done it a long time ago." - Comment from Ken
"Thanks for following up with me. You guys do a great job in tech support." - Comment from Phelps
"I will tell others who want to go into trading that DTN ProphetX is an invaluable tool, I don't think anyone can trade without it..." - Comment from Luther
"You are much better than lawyers or the phone company because you answer the phone when I call! I just love your customer service." - Comment from Isreal
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"Just a thank you for the very helpful and prompt assistance and services. You provided me with noticeably superior service in my setup compared to a couple of other options I had looked at." - Comment from John
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Timestamp Message.
Author Topic: Timestamp Message. (8 messages, Page 1 of 1)

Craig
-DTN Guru-
Posts: 284
Joined: Apr 16, 2010


Posted: Aug 3, 2010 02:33 PM          Msg. 1 of 8
I notice that if I leave IQFeed connected for more than one day, often it seems to silently stop working just before the second day starts (usually about 8:30 EST). The only indication that anything is wrong seems to be that I would stop receiving time-stamp messages.

Questions:
1. Is IQFeed designed to run for more than one day?
2. Is there some kind of server reset happening at your end during this time?
3. Is the lack of time-stamp messages significant of something having gone wrong?
4. Shouldn't I be getting some sort of server disconnect message instead of 3?

DTN Brian Wood
-Interested User-
Posts: 17
Joined: Jun 2, 2004


Posted: Aug 3, 2010 05:32 PM          Msg. 2 of 8
>1. Is IQFeed designed to run for more than one day?
Yes.

>2. Is there some kind of server reset happening at your end during this time?
This should be the start of the market. You shouldn't have noticed anything other than a lot more data coming in.

>Is the lack of time-stamp messages significant of something having gone wrong?
Yes. You should be expecting the next time stamp on time, unless you are backing up data because you have a high CPU load and are not processing it quickly enough which would then cause it to be delayed.

>4. Shouldn't I be getting some sort of server disconnect message instead of 3?
That is odd. You should see a S,SERVER DISCONNECTED message, if IQFeed becomes disconnected from its servers. You could then issue a S,CONNECT command and the symbols you were watching should automatically be rewatched for you, when the connection is reestablished. Did you loose your connection to IQFeed?

Can you describe a little more what you are doing, so I can better help you?

Visit our new online forums! DTN employees actively monitor the forums to provide you the highest level of support in the industry. Our forums also has an announcements area where you can learn about new features or changes within DTN. You can even use the forums to exchange trading ideas or tips and tricks on how you use your DTN data to be more profitable.

Go to http://forums.dtniq.com now to start sharing information with fellow DTN subscribers!

Craig
-DTN Guru-
Posts: 284
Joined: Apr 16, 2010


Posted: Aug 3, 2010 06:14 PM          Msg. 3 of 8
I'm running a server program which is connecting to IQFeed via TCP, basically I usually stop and start this thing each day, however yesterday I left it running overnight. I have a timer which is latched on the time-stamp message as I have seen the time-stamp messages stop before. Basically everything is chugging along just fine, IQFeed is not chewing CPU or memory, in fact there is not much load at all at 8:30 EST. Then for no reason the time-stamp messages stop (I wait for 5 minutes before timing out), as do the level 1 messages. My program breaks the TCP connection to IQFeed and tries to re-connect, after a few attempts I usually get reconnected.

I log out all data from the TCP connection, and I don't see any SERVER DISCONNECTED message.

Please let me know parts you require more info. on.

Edit....
Couple of extra notes:
1. If I stop and start IQFeed from the context menu, I see the SERVER DISCONNECTED messages in my logging, so I'm definitely handling them.
2. What is the correct response for a client that receives a SERVER DISCONNECTED? (drop connection, do nothing etc).
Edited by Craig on Aug 3, 2010 at 06:40 PM
Edited by Craig on Aug 3, 2010 at 06:40 PM

Craig
-DTN Guru-
Posts: 284
Joined: Apr 16, 2010


Posted: Aug 5, 2010 02:57 PM          Msg. 4 of 8
This happened again today, here is my log.

07:21:00.211227 0x954c400 Tokenizer::Parse, 'T' [T,20100805 07:21:00]
07:21:03.006119 0x954c400 Tokenizer::Parse, 'F' [F,MBT,D,14.7,1918000,23.5920,16.3160,23.5920,17.2000,,,,,05/06/2009,1,7,,3242378,,1.52,6.42,7.07,12,,MOBILE TELESYSTEMS ADR,MBT MBT1,21.,1.63,,4395.0,4257.9,03/31/2010,6482.2,958435,517210,0.40 05/03/2010,0.25 01/03/2005,,t,14,4,4899,30.55,1,7,03/17/2010,08/17/2009,03/17/2010,02/05/2010,19.56,,,,,517210]
07:21:04.985537 0x954c400 Tokenizer::Parse, 'F' [F,WTW,D,12.3,277000,30.4100,24.3900,30.4100,24.3900,2.50,0.18,0.70,07/16/2010,06/30/2010,1,7,,2816451,,2.26,2.44,6.77,12,,WEIGHT WATCHERS INTL,WTW,44.,1.08,,200.0,536.1,03/31/2010,1238.0,77083,812191, , ,,p,14,4,7299,18.14,1,7,02/23/2010,03/04/2010,02/23/2010,03/04/2010,29.16,,,,,812191]
07:21:05.425600 0x954c400 Tokenizer::Parse, 'F' [F,COG,D,27.3,1852000,46.4600,28.8900,46.4600,28.8900,0.40,0.03,0.12,08/19/2010,08/03/2010,1,7,,8632381,,1.20,1.14,-1.99,12,,CABOT OIL & GAS A,COG,,1.08,,281.5,308.7,06/30/2010,805.0,104135,211111,0.50 04/02/2007,0.67 04/01/2005,,p,14,4,1311,36.82,1,7,01/06/2010,05/25/2010,01/06/2010,05/25/2010,43.59,,,,,211111]
07:21:06.305470 0x954c400 Tokenizer::Parse, 'F' [F,LTD,D,15.2,4830000,28.7800,12.5600,28.7800,18.3400,2.30,1.15,0.60,06/15/2010,06/02/2010,1,7,,9620683,,1.71,1.78,-5.59,01,,LIMITED BRANDS,LTD,84.,1.78,,3250.0,1322.0,04/30/2010,2723.0,325191,448120,0.50 05/31/2000,0.50 06/20/1990,,t,14,4,5621,38.56,1,7,04/26/2010,08/05/2009,04/26/2010,01/06/2010,19.24,,,,,448120]
07:21:07.133532 0x954c400 Tokenizer::Parse, 'F' [F,MAT,F,12.5,5870000,24.6000,17.0800,24.6000,19.0700,3.40,,0.75,12/17/2009,11/25/2009,1,1,,9689657,,1.74,1.79,5.83,12,,MATTEL,MAT,97.,0.99,,2554.6,1060.9,06/30/2010,700.0,359724,339932,0.80 03/04/1996,0.80 01/23/1995,,t,14,4,3944,39.50,1,1,04/16/2010,08/06/2009,04/16/2010,01/29/2010,19.98,,,,,339932]
07:21:10.159038 0x954c400 Tokenizer::Parse, 'F' [F,RYN,D,14.8,710000,51.2900,37.8800,51.2900,39.7000,3.90,0.50,2.00,09/30/2010,09/08/2010,1,7,,2033954,,3.42,2.15,13.89,12,,RAYONIER,RYN,87.,1.19,,509.7,175.1,06/30/2010,695.0,80255,531190,0.67 10/18/2005,0.67 06/13/2003,,t,14,4,6798,27.67,1,7,08/04/2010,11/02/2009,08/04/2010,02/10/2010,42.16,,,,,531190]
07:21:11.585490 0x954c400 Tokenizer::Parse, 'F' [F,NU,D,16.4,1337000,29.0300,22.2000,29.0300,24.6800,3.60,0.26,1.03,09/30/2010,08/30/2010,1,7,,2711446,,1.75,1.94,,12,,NORTHEAST UTILITIES,NU,81.,0.55,,1267.9,979.4,06/30/2010,4492.9,175996,221122, , ,,t,14,4,4931,20.30,1,7,08/04/2010,11/02/2009,08/04/2010,02/10/2010,25.79,,,,,221122]
07:21:12.025526 0x954c400 Tokenizer::Parse, 'F' [F,WAT,D,18.8,880000,73.7800,49.3400,73.7800,55.9700,,,,,11/26/1996,1,7,,1868824,,3.56,3.96,15.3,12,,WATERS CORP,WAT,,1.00,,1172.4,394.6,06/30/2010,500.0,92945,334516,0.50 08/28/2000,0.50 06/11/1999,,t,14,4,3826,33.13,1,7,04/28/2010,09/02/2009,04/28/2010,02/05/2010,61.96,,,,,334516]
07:21:22.642119 0x954c400 Tokenizer::Parse, 'F' [F,MAN,D,973.8,781000,63.0000,39.9400,62.4800,39.9400,1.50,0.37,0.74,06/15/2010,06/01/2010,1,7,,2731536,,0.05,1.41,,12,,MANPOWER,MAN,,1.51,,4332.4,2603.0,06/30/2010,715.6,81875,561330, , ,,t,14,4,7363,32.31,1,7,10/20/2009,06/08/2010,04/22/2010,06/08/2010,54.58,,,,,561330]
07:21:28.085517 0x954c400 Tokenizer::Parse, 'F' [F,AYE,D,11.5,1953000,27.7000,18.9650,23.9900,18.9650,2.60,0.15,0.60,09/27/2010,09/09/2010,1,7,,7388516,,2.04,2.09,,12,,ALLEGHENY ENERGY,AYE,92.,0.89,,1345.3,985.6,03/31/2010,4417.0,169579,221111,0.50 11/19/1993, ,,p,14,4,4911,24.04,1,7,09/22/2009,05/25/2010,03/23/2010,05/25/2010,23.48,,,,,221111]
07:21:44.257416 0x954c400 Tokenizer::Parse, 'F' [F,NIHD,F,22.4,1744000,43.9500,22.1100,43.9500,30.0000,,,,,,1,1,,2534322,,1.77,1.88,14.48,12,,NII HOLDINGS,NIHD,,2.11,,3791.1,1529.7,06/30/2010,3016.2,167287,517210,0.50 11/22/2005,0.33 03/23/2004,,t,14,4,4812,45.44,1,1,04/05/2010,08/27/2009,04/05/2010,02/08/2010,33.58,,,,,517210]
07:26:00.211857 0x954c400 IqFeedConnection::IqFeedConnection::ConnectionStateLive::OnTimer, no timestamp for 5 minutes

Last time-stamp is rx'd at 07:21:00.21122, I timeout at 07:26:00.211857.
As you can see, I'm receiving other messages after the last time-stamp.
Edited by Craig on Aug 5, 2010 at 02:58 PM

DTN_Steve_S
-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005


Posted: Aug 6, 2010 01:47 PM          Msg. 5 of 8
Thanks for the log. I do see that you received other messages after the last timestamp but it doesn't appear as though you recieved any messages after the point when the next timestamp would/should have been received. I only point this out because it is not means that we are not necessarily dealing with a Timestamp issue itself but rather an issue causing the feed to stop, yet not register a disconnection from the servers.

The main question I have at this point would be what the stats show in the connection manager at the time that this happens (or the S,STATS message that is returned on the admin port or L1 port). I suspect that since you aren't getting a disconnection message (and the feed isn't going into a reconnection attempt loop) that the stats will show you are still connected and that the seconds since last update will be increasing. If that is the case, it sounds like the actual socket connection is still connected and your machine is able to send data to the servers but the servers are unable to send anything back. Unfortunately, the only thing I've seen cause that scenario in the past is a saturated internet connection but that doesn't seem to be the case here since it is happening during low market volume times.

Craig
-DTN Guru-
Posts: 284
Joined: Apr 16, 2010


Posted: Aug 6, 2010 02:22 PM          Msg. 6 of 8
It did not happen today, but I have added more logging and will monitor over the coming week.
I have seen similar problems to the one you describe on TCP connections in other contexts, it depends on the keep-alive settings on the intervening hardware as to how the connection behaves. If this is the case, monitoring the time-stamp messages as I'm doing now, maybe the only way of detecting the problem.
Edited by Craig on Aug 6, 2010 at 02:23 PM

Craig
-DTN Guru-
Posts: 284
Joined: Apr 16, 2010


Posted: Aug 26, 2010 02:34 PM          Msg. 7 of 8
I believe I have found the problem here, hopefully this post will stop somebody else making the same mistake. I was waiting until my rx buffer was terminated with a '\n' before I tokenized the buffer. This lead to a problem that if I ended up without a '\n' at the end due to TCP fragmentation the buffer may not get processed in a timely manner. This lead to 'bursting' behavior in the CPU and stuck messages, everything seems much smoother now this is fixed.

Craig
-DTN Guru-
Posts: 284
Joined: Apr 16, 2010


Posted: Aug 29, 2010 05:38 PM          Msg. 8 of 8
Alas, the problem is still happening, will keep trying to track.
 

 

Time: Thu May 26, 2022 1:29 PM CFBB v1.2.0 0 ms.
© AderSoftware 2002-2003