Trender
-Interested User-
Posts: 16
Joined: Sep 22, 2009
|
Posted: May 21, 2013 01:59 PM
Msg. 1 of 11
I recently upgraded from 4.9.x to 5.0.x. I have a C# program that records streaming data form level 1 and level2. Since I am still testing the move to the new version, I am running the same program twice. One is using the old protocol (I am running my old program, I don't specify protocol so it defaults to 4.9) and the other explicitly using the newest protocol 5.0.
It has been working correct for several days, today I am getting repeatedly the message "the parameter is incorrect" as a popup dialog window. I noticed that level 2 streaming data is broken and works very slow and for both applications data is coming in but with bad characters.
I am connected to 66.112.148.222
Level2 data of 5.0 protocol:
2,@NQM13,MD06,3027.,3029.75,86,96,14:48:30.524,2013-05-21,52,14:48:30.526,T,T,T,,05/21/2013 09:48:30.110 PM T<???5??/a?/[I???)????A??`???-d?,05/21/2013 09:48:31.030 PM T,05/21/2013 09:48:31.405 PM
I thought it could be a problem with my new application, but also the old program that was working very good for many months now is receiving similar garbage characters. You can also see connection problems with level 2 server.
by the way, Level 1 works fine on both applications.
Level 2 data with 4.9 protocl:
U,@ESM13,MD01,1669.25,1669.5,456,195,14:49:46,2013-05-21,20,52,43,14:49:46,T,T,,05/21/2013 09:49:45.598 PM U,@ESM13,MD06,1668.,1670.75,2050,1394,14:49:46,2013-05-21,20,52,43,14:49:32,T,T,,05/21/2013 09:49:45.598 PM S,SERVER CONNECTED,05/21/2013 09:49:45.953 PM C,05/21/2013 09:49:45.953 PM T???????,05/21/2013 09:49:48.323 PM T,05/21/2013 09:49:48.753 PM S,SERVER CONNECTED,05/21/2013 09:49:49.553 PM C,05/21/2013 09:49:49.553 PM S,SERVER CONNECTED,05/21/2013 09:49:49.608 PM C,05/21/2013 09:49:49.608 PM S,SERVER CONNECTED,05/21/2013 09:49:49.853 PM C,05/21/2013 09:49:49.853 PM S,SERVER CONNECTED,05/21/2013 09:49:50.788 PM C,05/21/2013 09:49:50.788 PM S,SERVER CONNECTED,05/21/2013 09:49:50.898 PM C,05/21/2013 09:49:50.898 PM S,SERVER CONNECTED,05/21/2013 09:49:50.923 PM C,05/21/2013 09:49:50.923 PM S,SERVER CONNECTED,05/21/2013 09:49:52.805 PM C,05/21/2013 09:49:52.805 PM T?(?^Us?c,05/21/2013 09:49:53.020 PM T?(??k?????,?9\j?,?W??`???`????&???&o??+??|?q8?,05/21/2013 09:49:53.555 PM S,SERVER CONNECTED,05/21/2013 09:49:53.555 PM O,05/21/2013 09:49:53.555 PM S,SERVER CONNECTED,05/21/2013 09:49:54.470 PM C,05/21/2013 09:49:54.470 PM T,05/21/2013 09:49:55.620 PM T??|?,h8'??B,?????2,05/21/2013 09:49:56.885 PM U,@YMM13,MD01,15378,15380,9,13,14:49:57,2013-05-21,20,52,42,14:49:46,T,T,,05/21/2013 09:49:57.355 PM U,@ESM13,MD01,1669.25,1669.5,373,195,14:49:57,2013-05-21,20,52,43,14:49:46,T,T,,05/21/2013 09:49:57.355 PM U,@YMM13,MD01,15378,15379,9,11,14:49:57,2013-05-21,20,52,42,14:49:57,T,T,,05/21/2013 09:49:57.355 PM U,@YMM13,MD01,15378,15379,8,11,14:49:57,2013-05-21,20,52,42,14:49:57,T,T,,05/21/2013 09:49:57.355 PM U,@NQM13,MD01,3028.25,3028.5,20,12,14:49:46,2013-05-21,20,52,43,14:49:57,T,T,,05/21/2013 09:49:57.355 PM
|
aQuant
-Interested User-
Posts: 49
Joined: Jul 20, 2012
|
Posted: May 21, 2013 03:25 PM
Msg. 2 of 11
I have exactly the same problem (and same scenario-running 4.9 and 5.0 versions at the same time), also happening today (coincidence?).
|
aQuant
-Interested User-
Posts: 49
Joined: Jul 20, 2012
|
Posted: May 21, 2013 03:29 PM
Msg. 3 of 11
Also, to add, I have run the applications (4.9 and 5.0 version) side by side for a few weeks now without the issue.
|
DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005
|
Posted: May 21, 2013 03:32 PM
Msg. 4 of 11
Hello Trender, the Parameter Is Incorrect error is a general error that simply means something went wrong (most likely when parsing the data from the server) so unfortunately, it doesn't tell us a whole lot as to what to look for in this case. Can you answer a few questions for me to help narrow this down?
Are the timestamps that are on the end of the messages your local time when they were received?
Did the feed recover by itself eventually? We are showing that you have been connected to the Level 2 server since 11:44:49 Eastern time. If your errors occurred at 14:49:32 (or sometime after) based on the timestamps in the Update messages you posted, that would imply that your current connection to the servers is the same connection that showed these errors.
It looks like the only messages that were improperly formatted were the Timestamp messages. Is that correct?
In your first example (with your extra timestamps removed): 2,@NQM13,MD06,3027.,3029.75,86,96,14:48:30.524,2013-05-21,52,14:48:30.526,T,T,T,, T<???5??/a?/[I???)????A??`???-d?, T,
Is this to indicate you received 2 timestamps in a row (one with gibberish and a second with just a T)?
Also, it looks like your 2 different apps had problems at different times. Your first example occurred about 75s apart. Did your app running using the 4.9 protocol out of IQFeed receive the timestamp message correctly that was corrupted 75s earlier in the 5.0 protocol?
Lastly, you mentioned connection problems on the Level 2 data but I only see S,SERVER CONNECTED messages (no S,SERVER DISCONNECTED messages). Does your app maintain a connection to both Level 1 and Level 2 for the duration that it is running? Did you receive any S,SERVER DISCONNECTED messages on the Level 2 port? Was the improperly formatted timestamp messages received after the disconnection or before?
|
DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005
|
Posted: May 21, 2013 03:34 PM
Msg. 5 of 11
aQuant, can you tell me if any of the above also applies to your scenario? Did you also receive improperly formatted timestamp messages? Were any other messages improperly formatted as well? Server disconnections?
Any extra information you can provide will help us track down the cause.
|
Trender
-Interested User-
Posts: 16
Joined: Sep 22, 2009
|
Posted: May 21, 2013 04:02 PM
Msg. 6 of 11
Hi, Few answers to the questions above:
1. I did not close my applications, they are working from morning (my local time) until now. 2. Both applications are using the TCP sockets to get data. (I used the C# examples as the base for both applications). 3. The last time-stamp is my local time, I just add it to every message I read from the feed. 4. I don't parse the message, at least not in this application, and I save the entire string that arrived from the feed (I just add my time stamp at the end). 5. I think the "T," was just my mistake, I cut it when copying, at the bottom I added a larger section of the same data. 6. seems that the problem started around 11:44:49 EST
New app: 2,@EUM13,MD01,1.2912,1.2914,48,71,11:44:49.551,2013-05-21,52,11:44:49.885,T,T,T,,05/21/2013 06:44:49.287 PM T<?n,05/21/2013 06:44:49.692 PM T?n?????|?W ?/??????Yb ?n 5w?"-F,05/21/2013 06:44:51.081 PM T??Q?4??`q?*B5"?#R?Hh??ZE????k+n.hFZ?E?M??(x?cJ?G???S? ????????q,05/21/2013 06:44:51.783 PM T?H??J??8^?@???Q:o q??B,05/21/2013 06:44:52.672 PM T<?????????,05/21/2013 06:44:53.031 PM T"????P<,05/21/2013 06:44:54.965 PM 2,@ESM13,MD01,1665.25,1665.5,1096,684,11:44:49.419,2013-05-21,52,11:44:56.128,T,T,T,,05/21/2013 06:44:55.511 PM
Old App:
U,@EUM13,MD01,1.2912,1.2914,48,71,11:44:49,2013-05-21,20,52,110,11:44:49,T,T,,05/21/2013 06:44:49.287 PM T<?n,05/21/2013 06:44:49.692 PM T?n?????|?W ?/??????Yb ?n 5w?"-F,05/21/2013 06:44:51.081 PM T??Q?4??`q?*B5"?#R?Hh??ZE????k+n.hFZ?E?M??(x?cJ?G???S? ????????q,05/21/2013 06:44:51.783 PM T?H??J??8^?@???Q:o q??B,05/21/2013 06:44:52.672 PM T<?????????,05/21/2013 06:44:53.031 PM T"????P<,05/21/2013 06:44:54.965 PM U,@ESM13,MD01,1665.25,1665.5,1096,684,11:44:49,2013-05-21,20,52,43,11:44:56,T,T,,05/21/2013 06:44:55.511 PM
7. It seems that there was a disconnection ~30 minutes before the garbage started on both apps, before that everything was working correctly for hours (from 5/20 ~11:52:44 PM EST to 5/21 ~11:16:55 AM EST)
New app: 2,@NQM13,MD10,3019.5,3024.25,79,119,11:16:55.772,2013-05-21,52,11:16:50.260,T,T,T,,05/21/2013 06:16:55.154 PM S,SERVER DISCONNECTED,05/21/2013 06:16:55.154 PM S,SERVER DISCONNECTED,05/21/2013 06:16:55.731 PM S,SERVER CONNECTED,05/21/2013 06:16:56.340 PM O,05/21/2013 06:16:56.340 PM Z,@EUM13,MD04,1.287,1.2877,181,168,11:16:55.873,2013-05-21,52,11:16:55.137,T,T,T,,05/21/2013 06:16:56.761 PM
Old App: U,@NQM13,MD10,3019.5,3024.25,79,119,11:16:55,2013-05-21,20,52,43,11:16:50,T,T,,05/21/2013 06:16:55.154 PM S,SERVER DISCONNECTED,05/21/2013 06:16:55.154 PM S,SERVER DISCONNECTED,05/21/2013 06:16:55.731 PM S,SERVER CONNECTED,05/21/2013 06:16:56.340 PM O,05/21/2013 06:16:56.340 PM U,@EUM13,MD04,1.287,1.2877,181,168,11:16:55,2013-05-21,20,52,110,11:16:55,T,T,,05/21/2013 06:16:56.761 PM
--------------------------------------------------------------------------------- Sample of more data for item 5 above.
2,@NQM13,MD01,3028.25,3028.5,16,22,14:48:30.526,2013-05-21,52,14:48:30.525,T,T,F,,05/21/2013 09:48:30.110 PM 2,@NQM13,MD06,3027.,3029.75,86,96,14:48:30.524,2013-05-21,52,14:48:30.526,T,T,T,,05/21/2013 09:48:30.110 PM T<???5??/a?/[I???)????A??`???-d?,05/21/2013 09:48:31.030 PM T,05/21/2013 09:48:31.405 PM S,SERVER CONNECTED,05/21/2013 09:48:31.945 PM C,05/21/2013 09:48:31.945 PM T???g,05/21/2013 09:48:33.720 PM T?h???{O?&???H???yE,05/21/2013 09:48:35.090 PM T??\?E?~????|,05/21/2013 09:48:38.502 PM T2?x??,05/21/2013 09:48:38.687 PM S,SERVER CONNECTED,05/21/2013 09:48:39.037 PM C,05/21/2013 09:48:39.037 PM S,SERVER CONNECTED,05/21/2013 09:48:39.252 PM C,05/21/2013 09:48:39.252 PM S,SERVER CONNECTED,05/21/2013 09:48:39.307 PM C,05/21/2013 09:48:39.307 PM S,SERVER CONNECTED,05/21/2013 09:48:42.127 PM C,05/21/2013 09:48:42.127 PM T???U`L?^_?)_??c???M?m\,05/21/2013 09:48:43.137 PM S,SERVER CONNECTED,05/21/2013 09:48:43.482 PM O,05/21/2013 09:48:43.482 PM S,SERVER CONNECTED,05/21/2013 09:48:43.692 PM C,05/21/2013 09:48:43.692 PM S,SERVER CONNECTED,05/21/2013 09:48:43.957 PM C,05/21/2013 09:48:43.957 PM S,SERVER CONNECTED,05/21/2013 09:48:44.012 PM C,05/21/2013 09:48:44.012 PM S,SERVER CONNECTED,05/21/2013 09:48:44.277 PM C,05/21/2013 09:48:44.277 PM T?,05/21/2013 09:48:44.462 PM TQy?jj???????'V????x\??=??G44,?D,05/21/2013 09:48:44.992 PM 2,@YMM13,MD01,15379,15380,17,15,14:48:45.450,2013-05-21,52,14:48:45.450,T,T,T,,05/21/2013 09:48:45.042 PM 2,@NQM13,MD01,3028.25,3028.5,16,23,14:48:30.526,2013-05-21,52,14:48:45.450,T,T,F,,05/21/2013 09:48:45.042 PM
|
aQuant
-Interested User-
Posts: 49
Joined: Jul 20, 2012
|
Posted: May 21, 2013 04:05 PM
Msg. 7 of 11
Some info on my side: applications using 4.9 protocol don't use (or subscribe) to Level2 data. Only the application running 5.0.x protocol (latest official release) uses Level2 data.
Here are a few lines of raw Level2 data, at some point between 0-1am CST I was disconnected from Level2 (not sure if it was internet outage on my side).
Here is how a correct time stamp message looks like before disconnect:
2~NÍ#Ð;T,20130521 01:00:15 (note: the part starting with "2~" and ending with ";" is my own stuff, rest is IQfeed generated)
Here are a few lines at and after disconnect:
2,@SFM13,MD01,1.0327,1.033,8,2,01:42:27.612,2013-05-21,52,01:42:27.613,T,T,T, 2,@SFU13,MD02,1.0335,1.0341,9,8,01:42:27.613,2013-05-21,52,01:42:27.613,T,T,F, 2,@SFU13,MD03,1.0334,1.0342,9,9,01:42:27.612,2013-05-21,52,01:42:27.613,T,T,T, T 2~Ѭ?4Ó#Ð;TQY~U¤¸ëÖ‚>²gqüLu¹œ†6E›êjÙæ+ìïÓ¶'È 2~8²8Ó#Ð;S,SERVER DISCONNECTED 2~`z“;Ó#Ð;S,SERVER DISCONNECTED 2~ƒ©;Ó#Ð;S,SERVER CONNECTED O 2~k±ò;Ó#Ð;2,QMGCM13,MD01,1396.1,1397.1,2,2,01:42:28.109,2013-05-21,52,01:42:21.169,T,T,F, 2,QMGCM13,MD02,1396.,1397.8,1,1,01:42:28.109,2013-05-21,52,01:42:21.169,T,T,F, 2,QMGCM13,MD03,1395.,1398.,1,1,01:42:28.109,2013-05-21,52,01:42:21.169,T,T,F, 2,QMGCM13,MD04,1394.6,1398.4,1,1,01:42:28.109,2013-05-21,52,01:42:21.169,T,T,F, 2,QMGCM13,MD05,1394.2,1398.8,1,1,01:42:23.743,20130S,BINARY_HEADER1,COMPRESSION_TYPE1,52,01:42:21.169,T,T,T, Z,QSIN13,MD05,22.785,22.83,12,10,01:42:40.707,2013-05-21,52,01:42:41.209,T,T,T, Z,QSIN13,MD06,22.78,22.835,10,9,01:42:40.707,2013-05-21,52,01:42:41.017,T,T,T, Z,QSIN13,MD10,22.76,22.855,8,2,01:42:40.989,2013-05-21,52,01:42:33.249,T,T,T, Z,QSIN13,MD07,22.775,22.84,3,17,01:42:40.707,2013-05-21,52,01:42:41.017,T,T,T, Z,QSIN13,MD01,22.805,22.81,3,3,01:42:40.710,2013-05-21,52,01:42:40.948,T,T,T, Z,QSIN13,MD02,22.8,22.815,3,2,01:42:40.710,2013-05-21,52,01:42:24.763,T,T,T, Z,QSIN13,MD03,22.795,22.82,10,8,01:42:40.791,2013-05-21,52,01:42:36.493,T,T,T, Z,QSIN13,MD08,22.77,22.845,5,5,01:42:40.707,2013-05-21,52,01:42:40.767,T,T,T, Z,QSIN13,MD09,22.765,22.85,2,7,01:42:40.707,2013-05-21,52,01:42:40.767,T,T,T, Z,@SU13,MD04,1291.75,1294.,2,1,01:42:00.283,2013-05-21,52,01:42:24.769,T,T,T,
After that disconnect, the connection was reestablished but since then the time stamp messages have been all gibberish: 2~‹H5TØ#Ð;TQùݤpô (again note: the part starting with "2~" and ending with ";" is my own stuff, rest is IQfeed generated)
Also the above line containing "BINARY_HEADER1,COMPRESSION_TYPE1" is unexpected to me.
The popups with that error have never gone away and I also noticed that I only have about 1/2 of the volume of Level2 data for today compared to most days (including yesterday), so the feed recovered somehow but not fully. Let me know what more specifics you would like, maybe I can dig them out.
Similarly as the other poster, I use sockets in C# to get raw data in.
|
Trender
-Interested User-
Posts: 16
Joined: Sep 22, 2009
|
Posted: May 22, 2013 11:07 AM
Msg. 8 of 11
Just happened again in level 2 connected to 66.112.148.111 I am thinking to revert to 4.9 client.
This is the application using the 5.0 protocol, less than 15 minutes ago.
2,@EUM13,MD01,1.2879,1.288,15,44,11:54:13.291,2013-05-22,52,11:54:13.281,T,T,T,,05/22/2013 06:54:24.091 PM 2,@EUM13,MD01,1.2879,1.288,14,44,11:54:13.291,2013-05-22,52,11:54:13.281,T,T,T,,05/22/2013 06:54:24.091 PM 2,@EUM13,MD01,1.2879,1.288,15,44,11:54:13.292,2013-05-22,52,11:54:13.281,T,T,T,,05/22/2013 06:54:24.091 PM 2,@EUM13,MD01,1.2879,1.288,14,44,11:54:13.310,2013-05-22,52,11:54:13.281,T,T,T,,05/22/2013 06:54:24.091 PM S,SERVER DISCONNECTED,05/22/2013 06:54:27.206 PM S,SERVER DISCONNECTED,05/22/2013 06:54:27.611 PM S,SERVER CONNECTED,05/22/2013 06:54:28.221 PM O,05/22/2013 06:54:28.221 PM TlJ~???1??q<??)U??q"qj?(??>?=z????????xk??2n<s?>???!1H??B??v? ????\????P{?wH??ve+???w?,05/22/2013 06:54:28.636 PM S,SERVER CONNECTED,05/22/2013 06:54:29.241 PM C,05/22/2013 06:54:29.241 PM 2,@ESM13,MD01,1679.,1679.25,543,182,11:54:28.783,2013-05-22,52,11:54:13.224,T,T,T,,05/22/2013 06:54:30.066 PM 2,@NQM13,MD02,3042.5,3043.25,32,28,11:54:28.783,2013-05-22,52,11:54:11.542,T,T,T,,05/22/2013 06:54:30.066 PM 2,@NQM13,MD01,3042.75,3043.,28,1,11:54:28.783,2013-05-22,52,11:54:12.753,T,T,T,,05/22/2013 06:54:30.066 PM 2,@NQM13,MD01,3042.75,3043.,28,2,11:54:28.783,2013-05-22,52,11:54:28.783,T,T,F,,05/22/2013 06:54:30.066 PM
Same time on the old application that uses the 4.9 protocol.
U,@EUM13,MD01,1.2879,1.288,15,44,11:54:13,2013-05-22,20,52,110,11:54:13,T,T,,05/22/2013 06:54:24.091 PM U,@EUM13,MD01,1.2879,1.288,14,44,11:54:13,2013-05-22,20,52,110,11:54:13,T,T,,05/22/2013 06:54:24.091 PM S,SERVER DISCONNECTED,05/22/2013 06:54:27.206 PM S,SERVER DISCONNECTED,05/22/2013 06:54:27.611 PM S,SERVER CONNECTED,05/22/2013 06:54:28.221 PM O,05/22/2013 06:54:28.221 PM TlJ~???1??q<??)U??q"qj?(??>?=z????????xk??2n<s?>???!1H??B??v? ????\????P{?wH??ve+???w?,05/22/2013 06:54:28.636 PM S,SERVER CONNECTED,05/22/2013 06:54:29.241 PM C,05/22/2013 06:54:29.241 PM U,@ESM13,MD01,1679.,1679.25,543,182,11:54:28,2013-05-22,20,52,43,11:54:13,T,T,,05/22/2013 06:54:30.066 PM
|
DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005
|
Posted: May 22, 2013 11:36 AM
Msg. 9 of 11
We have found the source of this. It is a bug in IQFeed 5.0.0.11 that will sometimes cause the data corruption after a disconnection from the Level 2 server has occurred. We will be releasing a new build to fix this issue. Until then, you will probably want to roll back to the 4.9 version (or restart the feed anytime you get a disconnection from the level 2 port).
Also, I noticed today you have a large number of reconnections within the last 45mins (roughly). If that isn't intentional from something on your end, it looks like you might be having connectivity issues to our servers (which will increase the likelihood of this issue happening) and you might want to contact your ISP to investigate the dropped connections.
|
Trender
-Interested User-
Posts: 16
Joined: Sep 22, 2009
|
Posted: May 22, 2013 11:43 AM
Msg. 10 of 11
Ok, thanks for the update. I am rolling back to 4.9 until a fixed is released. I will check the connectivity issues Regards, Trender
|
aQuant
-Interested User-
Posts: 49
Joined: Jul 20, 2012
|
Posted: May 22, 2013 11:57 AM
Msg. 11 of 11
Great you found the source of error. Do you have any idea when you will be releasing the new corrected version?
|
|
|
|