Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"As a past ******* customer(and not a happy one), IQ Feed by DTN is a much better and cheaper product with great customer support. I have had no problems at all since switching over." - Comment from Public Forum
"I use IQ Feed, Great stuff as far as data analysis information, storage and retrieval is concerned." - Comment from Public Forum
"Just a thank you for the very helpful and prompt assistance and services. You provided me with noticeably superior service in my setup compared to a couple of other options I had looked at." - Comment from John
"Thanks for following up with me. You guys do a great job in tech support." - Comment from Phelps
"After all the anxiety I had with my previous data provider it is a relief not to have to worry about data speed and integrity." - Comment from Eamonn
"I just wanted to say how happy I am with your service. I was able to download the API docs last week and I was able to replicate Interactive Brokers historical bar queries and realtime bar queries over the weekend. That was about one of the fastest integrations that I've ever done and it works perfectly!!!!" - Comment from Jason via Email
"Boy, probably spent a thousand hours trying to get ******* API to work right. And now two hours to have something running with IQFeed. Hmmm, guess I was pretty stupid to fight rather than switch all this time. And have gotten more customer service from you guys already than total from them… in five years." - Comment from Jim
"I am a hedge fund manager here. It’s funny, I have a Bloomberg terminal and a Bridge feed, but I still like having my DTN feed!" - Comment from Feras
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: mac
About Contact
Joined: Apr 6, 2017 08:50 PM
Last Post: May 20, 2023 10:17 AM
Last Visit: May 21, 2023 05:49 PM
Website:  
Location:
Occupation:
Interests:
Email: chapinmark@gmail.com
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
mac has contributed to 25 posts out of 21184 total posts (0.12%) in 2,567 days (0.01 posts per day).

20 Most recent posts:

Hi, I'm not sure if it's related to wine. I can't imagine is has anything to do with AWS. Hopefully I can test on windows next week.

MAC


Yes that would be fine. I'll send an email to support.

MAC


I occasionally see long delays receiving a response from a HIT 60 second request. I'm asking for the most recent interval bar (one bar). My requests are sent at 100-700 ms after the minute boundary. I send 35 symbol requests on 7 sockets in this time. I am not rate limited, I receive no "too many requests" error. I repeat this request every minute. In a 12 hour period, I saw 21 requests delays. The delay is always 15-16 seconds.

I'm running on linux/wine in AWS.

FROM CLIENT	LookupRequest	88	2520	2023-05-12 11:47:00	HIT,UNP,60,20230512 074600,20230512 074600,,,,,,,,0
FROM CLIENT LookupRequest 304 2521 2023-05-12 11:47:00 HIT,@YMM23,60,20230512 074600,20230512 074600,,,,,,,,0
FROM CLIENT LookupRequest 296 2523 2023-05-12 11:47:00 HIT,UNH,60,20230512 074600,20230512 074600,,,,,,,,0
TO CLIENT LookupData 360 2517 2023-05-12 11:47:00 LH,2023-05-12 07:47:00,57.7800,57.7500,57.7700,57.7500,14252,5167,0,
TO CLIENT LookupRequest 360 2517 2023-05-12 11:47:00 !ENDMSG!,
TO CLIENT LookupData 360 2517 2023-05-12 11:47:00 !ENDMSG!,
TO CLIENT LookupError 296 2523 2023-05-12 11:47:00 E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT LookupError 88 2520 2023-05-12 11:47:00 E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT LookupData 296 2523 2023-05-12 11:47:00 E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT LookupData 88 2520 2023-05-12 11:47:00 E,!NO_DATA!,,
!ENDMSG!,
TO CLIENT LookupData 288 2519 2023-05-12 11:47:00 LH,2023-05-12 07:47:00,176.0600,176.0000,176.0200,176.0000,1531419,7866,0,
TO CLIENT LookupRequest 288 2519 2023-05-12 11:47:00 !ENDMSG!,
TO CLIENT LookupData 288 2519 2023-05-12 11:47:00 !ENDMSG!,


Response 15-16 seconds later


TO CLIENT LookupData 304 2521 2023-05-12 11:47:15 LH,2023-05-12 07:47:00,33492.00,33489.00,33491.00,33489.00,24641,20,0,
TO CLIENT LookupRequest 304 2521 2023-05-12 11:47:15 !ENDMSG!,
TO CLIENT LookupData 304 2521 2023-05-12 11:47:15 !ENDMSG!,


MAC

IQFeed API Questions » Only send updates for dynamic fields subscribed May 8, 2023 02:18 PM (Total replies: 1)

If I set,

S,SELECT UPDATE FIELDS,Bid,Ask,Message Contents

and w@ESM23

Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,
Q,@ESM23,4152.50,4152.75,a,
Q,@ESM23,4152.50,4153.00,a,
Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,
Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,
Q,@ESM23,4152.75,4153.00,b,
Q,@ESM23,4152.50,4153.00,b,

I see only bid or ask price updates. However, when watching equities, I get mostly duplicate data. These are updates to Size and Market Data Center which are not part of my update fields. Is it possible to filter out these updates in iqconnect so I don't need to process duplicate data. It makes sense to get updates if market data center updates but not size if I'm not interested in this field.

wAAPL

Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,
Q,AAPL,173.5900,173.6000,ba,



MAC
Edited by mac on May 8, 2023 at 03:40 PM
Edited by mac on May 8, 2023 at 03:41 PM

IQFeed API Questions » Internal socket _IQIF_SBL1 hangs under wine May 8, 2023 01:21 PM (Total replies: 3)

The above approach to closing the socket is not working, iqconnect (or wine) is holding the internal iqconnect 5009 port open even after I disconnect my app.

This is causing me to repeatedly have to restart iqconnect (many times per week). I am currently developing an app to generate my own bars from level 1 data. Once I have this running I will stop connecting to the derivative port in production.

If I can create a docker file that reproduces this issue, could IQ support run this and see if it is reproducible on your end? I only see a problem under heavy market volume, so you might need to run for a few days before the internal port hangs.

MAC

IQFeed API Questions » Internal socket _IQIF_SBL1 hangs under wine Apr 28, 2023 09:44 AM (Total replies: 3)

I'm using the dotnet client so graciously provided here https://github.com/mathpaquette/IQFeed.CSharpApiClient

In this client, I see that the sockets are not explicitly disconnected, they are just disposed. It seems that this results in a quick release of the sockets on the iqconnect side in normal circumstances. I believe this causes the socket to be held onto by iqconnect when it enters this queued state. In the queued state, where the IQ derivative process in iqconnect no longer reads from the internal L1 socket, will hold onto this internal socket even though I am no longer connected to the derivative port. Data just piles up until it crashes when there is no more memory left.

On a positive note, if I explicitly disconnect from the derivative socket before disposing, iqconnect will eventually (30-60 seconds later) release the internal socket to L1. I have been able to reconnect to only the derivative socket without restarting iqconnect. All my sockets that are connected to L1 directly work fine through this problem.

Any insight as to why the derivative process would stop reading from its internal L1 socket would be helpful. I've had to write a bunch of code to circumvent this issue.

I will say, I don't believe this is an issue on Windows, only while running with wine.

MAC

IQFeed API Questions » Internal socket _IQIF_SBL1 hangs under wine Apr 24, 2023 03:49 PM (Total replies: 3)

Hi,

While running iqconnect under wine, I see that the internal _IQIF_SBL1 socket from a Derivative stream hangs causing subscribed bars to stop and for the _IQIF_SBL1 socket to start reporting queued data. This typically occurs during heavy market activity around open. However, I have seen it happen randomly through the day. I'm subscribing to 12 liquid equities and 23 liquid futures contracts. If the KBQueue reaches 10MB, I close all connected and open new ones. All sockets close except for the _IQIF_SBL1_***. These just keep piling up queued data until I restart iqconnect or eventually run out of memory. I'm running this in a docker container. I'm happy to share the Dockerfile if that helps.

I've had this issue for years now. My solution was to just kill the iqconnect process and restart it. This has become an issue for me and I can no longer afford to have interruptions in iqconnect.

In this first section of logs, you can see that the _IQIF_SBL1_*** connection stops delivering data(sent). It just piles up (kbqueued) I see no logged errors prior to this. I'm not sure what the level of logging is set to. If you can tell that debugging is not set, please let me know how to set this in a registry in 6.2 (64bit). I don't have easy access to the Diagnostics app

TO CLIENT Admin 404 4 2023-04-24 13:42:21 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.25,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104482.72,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33988.28,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009056.63,0.00,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35760.50,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030698.07,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:42:21 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.25,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104482.72,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33988.28,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009056.63,0.00,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35760.50,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030698.07,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:42:21 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264332.45,53.47,2.11,2425889.77,498.33,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:21 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264332.45,53.47,2.11,2425889.77,498.33,19.39,

TO CLIENT Admin 404 5 2023-04-24 13:42:21 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264332.45,53.47,2.11,2425889.77,498.33,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:22 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.39,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104483.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.13,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,150.54,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35775.04,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030935.49,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:42:22 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.39,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104483.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.13,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,150.54,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35775.04,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1030935.49,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:42:22 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264390.37,57.93,2.11,2426254.17,364.40,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:22 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264390.37,57.93,2.11,2426254.17,364.40,19.39,

TO CLIENT Admin 404 5 2023-04-24 13:42:22 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264390.37,57.93,2.11,2426254.17,364.40,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:23 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.53,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104484.43,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.99,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,362.70,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35792.81,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1031126.10,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:42:23 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16810.53,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104484.43,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,33989.99,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,362.70,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,35792.81,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1031126.10,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:42:23 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264440.68,50.31,2.11,2426464.41,210.23,19.39,

TO CLIENT Admin 404 4 2023-04-24 13:42:23 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264440.68,50.31,2.11,2426464.41,210.23,19.39,

TO CLIENT Admin 404 5 2023-04-24 13:42:23 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:42AM,Connected,6.2.0.25,465642,264440.68,50.31,2.11,2426464.41,210.23,19.39,



In this second section, you can see that some sockets disconnect while the _IQIF_SBL1 sockets remain connected.


TO CLIENT Admin 404 4 2023-04-24 13:43:04 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.30,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104519.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34025.14,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,9857.79,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36454.37,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039695.57,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:43:04 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.30,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104519.58,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34025.14,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,9857.79,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36454.37,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039695.57,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:43:04 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266633.01,86.73,2.13,2435771.50,394.70,19.46,

TO CLIENT Admin 404 4 2023-04-24 13:43:04 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266633.01,86.73,2.13,2435771.50,394.70,19.46,

TO CLIENT Admin 404 5 2023-04-24 13:43:04 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266633.01,86.73,2.13,2435771.50,394.70,19.46,

TO CLIENT Admin 404 4 2023-04-24 13:43:05 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.44,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104520.44,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34026.00,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,10052.97,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36467.51,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039871.30,0.00,

TO CLIENT Admin 404 5 2023-04-24 13:43:05 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.44,0.00,
S,CLIENTSTATS,0,4,TickIngestAdmin2023-04-22T21:06:23.6679584-06:00,20230423 030623,,,0.10,104520.44,0.00,
S,CLIENTSTATS,0,5,IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00,20230424 023332,,,0.11,34026.00,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,10052.97,
S,CLIENTSTATS,1,7,IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00,20230424 023331,36,0,1.32,36467.51,0.00,
S,CLIENTSTATS,1,5,TickIngestL12023-04-22T21:06:23.6683403-06:00,20230423 030623,36,0,29.16,1039871.30,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:43:05 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266678.29,45.29,2.13,2435962.22,190.71,19.46,

TO CLIENT Admin 404 4 2023-04-24 13:43:05 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266678.29,45.29,2.13,2435962.22,190.71,19.46,

TO CLIENT Admin 404 5 2023-04-24 13:43:05 S,STATS,12.36.218.206,60004,500,36,7,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266678.29,45.29,2.13,2435962.22,190.71,19.46,

This is where is have closed and disposed of all sockets connection (except #1). I have also started reconnecting. As you can see, the _IQIF_SBL1_IQMarketDataDeriv*** connections are not closed.
STATUS Information 404 7 2023-04-24 13:43:05 LEVEL1 SOCKET CLOSED - IQMarketDataL1-2023-04-23T20:33:41.3091165-06:00
STATUS Information 404 5 2023-04-24 13:43:05 ADMIN SOCKET CLOSED - IQMarketDataAdmin-2023-04-23T20:33:41.3085897-06:00
STATUS Information 404 11 2023-04-24 13:43:07 LEVEL1 SOCKET ACCEPTED -
STATUS Information 404 6 2023-04-24 13:43:07 ADMIN SOCKET ACCEPTED -
STATUS Information 404 5 2023-04-24 13:43:07 LEVEL1 SOCKET CLOSED - TickIngestL12023-04-22T21:06:23.6683403-06:00
STATUS Information 404 4 2023-04-24 13:43:07 ADMIN SOCKET CLOSED - TickIngestAdmin2023-04-22T21:06:23.6679584-06:00
STATUS Information 404 12 2023-04-24 13:43:08 LEVEL1 SOCKET ACCEPTED -
STATUS Information 404 11 2023-04-24 13:43:08 LEVEL1 SOCKET CLOSED -
STATUS Information 404 0 2023-04-24 13:43:08 LOOKUP SOCKET ACCEPTED 74 -
STATUS Information 404 7 2023-04-24 13:43:08 ADMIN SOCKET ACCEPTED -
STATUS Information 404 14 2023-04-24 13:43:09 LEVEL1 SOCKET ACCEPTED - iqconnect.exe
STATUS Information 404 12 2023-04-24 13:43:09 LEVEL1 SOCKET CLOSED -
STATUS Information 404 14 2023-04-24 13:43:09 LEVEL1 SOCKET CLOSED - _IQIF_SBL1

TO CLIENT Admin 404 6 2023-04-24 13:43:10 S,CURRENT PROTOCOL,6.2,

FROM CLIENT Admin 404 6 2023-04-24 13:43:10 S,SET PROTOCOL,6.2
TO CLIENT Admin 404 7 2023-04-24 13:43:10 S,CURRENT PROTOCOL,6.2,

FROM CLIENT Admin 404 7 2023-04-24 13:43:10 S,SET PROTOCOL,6.2
FROM CLIENT Admin 404 7 2023-04-24 13:43:10 S,SET CLIENT NAME,TickIngestAdmin2023-04-24T07:43:06.2700049-06:00
FROM CLIENT Admin 404 7 2023-04-24 13:43:10 S,CLIENTSTATS ON
TO CLIENT Admin 404 7 2023-04-24 13:43:10 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,3,74,_IQIF_SBLOOKUP,20230424 134308,,,0.05,0.02,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.58,0.00,
S,CLIENTSTATS,0,6,,20230424 134307,,,0.02,0.02,0.00,
S,CLIENTSTATS,0,7,TickIngestAdmin2023-04-24T07:43:06.2700049-06:00,20230424 134308,,,0.10,0.02,0.00,
S,CLIENTSTATS,1,14,_IQIF_SBL1,20230424 134308,0,0,1.57,1.05,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,10968.86,
S,CLIENTSTATS,1,12,TickIngestL12023-04-24T07:43:06.2701898-06:00,20230424 134307,36,0,2.98,705.04,0.00,
S,CLIENTSTATS,1,11,,20230424 134306,0,0,0.34,0.87,0.00,

TO CLIENT Admin 404 1 2023-04-24 13:43:10 S,STATS,12.36.218.206,60004,500,36,9,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266895.52,43.45,2.13,2436810.04,169.56,19.47,

TO CLIENT Admin 404 6 2023-04-24 13:43:10 S,STATS,12.36.218.206,60004,500,36,9,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266895.52,43.45,2.13,2436810.04,169.56,19.47,

TO CLIENT Admin 404 7 2023-04-24 13:43:10 S,STATS,12.36.218.206,60004,500,36,9,0,0,1,Apr 23 2:57AM,Apr 24 09:43AM,Connected,6.2.0.25,465642,266895.52,43.45,2.13,2436810.04,169.56,19.47,

STATUS Information 404 12 2023-04-24 13:43:10 LEVEL1 SOCKET CLOSED - TickIngestL12023-04-24T07:43:06.2701898-06:00
STATUS Information 404 7 2023-04-24 13:43:10 ADMIN SOCKET CLOSED - TickIngestAdmin2023-04-24T07:43:06.2700049-06:00
STATUS Information 404 17 2023-04-24 13:43:10 LEVEL1 SOCKET ACCEPTED -
STATUS Information 404 8 2023-04-24 13:43:11 ADMIN SOCKET ACCEPTED -
STATUS Information 404 17 2023-04-24 13:43:11 LEVEL1 SOCKET CLOSED -
TO CLIENT Admin 404 8 2023-04-24 13:43:11 S,CURRENT PROTOCOL,6.2,

FROM CLIENT Admin 404 8 2023-04-24 13:43:11 S,SET PROTOCOL,6.2
FROM CLIENT Admin 404 8 2023-04-24 13:43:11 S,SET CLIENT NAME,TickIngestAdmin2023-04-24T07:43:10.1673900-06:00
FROM CLIENT Admin 404 8 2023-04-24 13:43:11 S,CLIENTSTATS ON
TO CLIENT Admin 404 8 2023-04-24 13:43:11 S,CLIENTSTATS,3,37,_IQIF_SBLOOKUP_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,,,2.11,8.42,0.00,
S,CLIENTSTATS,3,74,_IQIF_SBLOOKUP,20230424 134308,,,0.05,0.02,0.00,
S,CLIENTSTATS,0,1,,20230423 025714,,,0.00,16816.72,0.00,
S,CLIENTSTATS,0,6,,20230424 134307,,,0.02,0.17,0.00,
S,CLIENTSTATS,0,8,TickIngestAdmin2023-04-24T07:43:10.1673900-06:00,20230424 134311,,,0.10,0.02,0.00,
S,CLIENTSTATS,1,14,_IQIF_SBL1,20230424 134308,0,0,1.57,1.07,0.00,
S,CLIENTSTATS,1,17,TickIngestL12023-04-24T07:43:10.1676316-06:00,20230424 134310,36,0,2.98,94.44,0.00,
S,CLIENTSTATS,1,9,_IQIF_SBL1_IQMarketDataDeriv-2023-04-23T20:33:41.3079511-06:00,20230424 023332,36,0,2.60,1009167.23,11084.92,
S,CLIENTSTATS,1,11,,20230424 134306,0,0,0.34,0.89,0.00,

MAC


Hi,

Any update on when the historical options file will be updated?

Thanks,

MAC


I just upgraded to IQConnect v6 and the bid is no longer frozen. I am still using 5.2 protocol. (see attached file)

@aQuant, which version of IQConnect are you running? If v6, I will assume the frozen bid I saw 2am on Feb 21 was version related.

I will continue to monitor with IQConnect v6.

I hope this finding will shed some light on the issue.

MAC


@quickTick,

It's interesting that you don't see this. The attached file is the raw text data from the socket. I've attached the full response to the HTT request run at 2018-02-21 21:20:00 ET. It's a bit longer but still shows the frozen bid.

I should note, I have yet to upgrade to IQConnect v6. I will do that now and see if it fixes the problem.

Below is the HTT command I issued.

HTT,@NQH19,20190221 015000,20190221 022000,0005000,000000,235959,1,REQ_HST_TCK0000000,0000500

Same result without including BeginFilterTime and EndFilterTime in the request

HTT,@NQH19,20190221 015000,20190221 022000,0005000,,,1,REQ_HST_TCK0000000,0000500

MAC


Attached is the HTT response for @NQH19.

Note that the bids do not change for 17 minutes.

MAC


aQuant,

I spoke too soon. I see the bid frozen from 2018-02-21 02:00:00 to 02:17:00 ET on all the products I'm monitoring except JY. It looks like my simple stats are able to catch this.

I haven't coded up the HT* command (request tick history). Will this show the same behavior? If so, surely IQFeed won't be able to refute the problem.

I will call them this morning.

MAC


aQuant,

I am now monitoring the following CME symbols (AD,BP,CD,JY,EC,NG,ES).

I didn't see anything out of the ordinary for trader date 2018-02-20. However, my monitoring is only gather stats and not saving the raw messages. There is a possibility I have a bug in my stats.

If it's not too much to ask, could you post here when you see one of these symbols get stuck or cross? I can then verify with my monitoring.

If I can verify, we can all push for a solution.

Thanks in advance.

MAC

IQFeed Developer Support » Latency of Q messages Feb 19, 2019 08:17 AM (Total replies: 3)

Thanks for the reply.

I have implemented the latest .net core 3.0 System.IO.Pipelines library. It "queues" and "dequeues" on separate threads while minimizing overallocation. With the help of the SequenceReader, you can access the buffer directly without having to copy to a string and then split. I have yet to test it with my mock server. I'll let you know if it performs better.

I also need to monitor KBQueued in code so I can log these infrequent latency events. Due to some logging changes, I was only able to see these events back in 2017. One of which coincided with an ECB announcement.

I will continue to monitor.

MAC


aQuant or quickTick,

Could you explain how you detect this? I want to start monitoring.

Is there a complete lack of Q messages arriving with 'b' or 'a' updates?

or

Are the Q messages coming in for 'b' or 'a' but the prices don't change?

I have disabled IQFeed from my production deployment and will run it in the background for monitoring only.

I was about to add NYMEX, COMEX, CBOT and EUREX to my subscriptions. I will wait until this has been resolved.

MAC

IQFeed Developer Support » Latency of Q messages Feb 11, 2019 08:00 PM (Total replies: 3)

Hi,

I've been trying to optimize my .net socket reader. First, I wrote my own server and pushed 1000 "Q," messages per second to help find any bottlenecks in my code. The mock server writes 1000 dummy Q messages in a loop. I've appended a high resolution time stamp to the messages (using interop GetSystemTimePreciseAsFileTime). I compare this time stamp on the reading side to see how fast I process messages. The writes of 1000 messages only takes a few milliseconds. I noticed that I was unable to keep up with this rate on the read side if I processed the message synchronously. I now push the messages to a BlockingCollection(ConcurrentQueue). Once I did this, my socket mostly reads one or two message per read.

I thought this would help reading the live feed. However, I still see many reads with numerous messages (>50). This is averaging 5, but sometime as many as 100+ messages are pulled in one read. Most of the time, these messages only differ in the Bid/Ask time stamp by a few milliseconds but sometimes are > 50ms (as high as 200 ms) from the first message to the last.

Is IQConnect batching the writes to the socket? Or are you batching the push from your servers? No matter what I do I cannot seem to reduce this.

I should note that I am collecting stats in an array and only dumping one line to the console every 10-15 second. I have timed the stat collection and display and I am confident that it is not skewing the results.

I need to sort this out not because I’m concerned about a few milliseconds but because I can get way behind (1-30 seconds!) in a fast market.

I am running on an AWS EC2 T2.Medium machine (2 cores).

My pings to the IQFeed server are constantly between 45-47 ms.

I am only watching CME products (@ES, @NQ, 5 major currencies)

MAC


Has this been resolved?

MAC

IQFeed Developer Support » Fast market probelm Jun 28, 2017 11:25 AM (Total replies: 3)

OK, my bad. I see 61,000 prints in the 8:17 minute.

That said, should a C# program be able to handle 1000+ prints per second? Is it possible?

MAC

IQFeed Developer Support » Fast market probelm Jun 28, 2017 10:43 AM (Total replies: 3)

I should note, the numbers quoted above (100 prints per second) were filtered prints. These were only logged if Bid or Ask price changed. I was not logging changes in size at the same price. It is likely much higher than 100 per second.

That said, I'm running a test right now that shows no latency issues with similar filtering. I'm getting fewer price changes since there isn't a fast market but only on the order of 30%-50%. I'm sampling every 10 seconds and see around 30-50 bid/ask price changes per second for ECU7. I am however subscribed to all 5 products. Average latencies are normal at around 75 milliseconds.

It's starting to look like my app can handle the throughput.

Any reported problems at IQFeed this morning?

MAC

IQFeed Developer Support » Fast market probelm Jun 28, 2017 09:44 AM (Total replies: 3)

I am relatively new to IQFeed and to socket programming in general. I’m coding in .Net. I’m basically doing the same socket reads as the example code, Level1Socket. I was naive to think I could keep up with the socket, I think.

I haven’t been monitoring the Admin socket so I don’t know how often I have queued data. I happened to be trading in a fast market at 08:17 - 08:18 ET June 28 when there was significant futures movement. I only watch 5 symbols; Globex currencies Euro, Japanese Yen, Australian Dollar, E-mini Nasdaq and AAPL. Euro currency had a very fast move at this time. I log level 1 update (bid/ask) latency during a trade. I was seeing 25-35 SECOND latencies.

After seeing this, I opened the Diagnostic program for market open at 9:30 ET to see if it reported any queued data. It showed none.

My machine clock is managed by a windows port of ntp.org software, so it is quite accurate (always within a few milliseconds of ntp time).

Is it possible for IQFeed to see if there were an abnormal amount of prints in this period relative to market open. I recorded around 6000 prints in the 08:18:00-08:18:59 minute for Euro Currency Sept 2017 (100 per second).

Unfortunately, having never seen this before, I didn’t spend too much effort optimizing my software. Every print, regardless of the product, fires an event with 5 subscribing class instances, one subscription per product. In the even handler, I check to see if the event args match the product I’m interest in. Maybe a better approach would be to open 5 different sockets, one per product. This could expand to 15 or 20 products in the future but not 100’s. This way I could read from the back of the socket? If a print is old, just throw it out. I just want the most recent info. If I had one socket per product, I could just grab the last update and throw the rest away. Also, this would enable me to fire an event with only one subscriber.

Also unfortunate, I don’t know where the bottleneck is.

Any thoughts?

MAC


Time: Mon April 15, 2024 6:37 AM CFBB v1.2.0 14 ms.
© AderSoftware 2002-2003