Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"If someone needs the best quality data and backfill beyond what their broker provides at a rate that is the best in the industry, I highly recommend IQFeed." - Comment from Josh via Public Forum
"Everything is working great with the API. I love it." - Comment from Calvin
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
"Its working FABULOUSLY for me!! Holy cow...there has been so much I've been missing lately, and with this feed and Linnsoft software...I'm in the game now." - Comment from Chris R.
"I cannot believe what a difference it makes trading with ProphetX!" - Comment from Bruce in Los Angeles
"I just wanted to tell you what a fine job you have been doing. While *******, from what I hear, has been down and out, off and on, IQ feed has held like a champ this week." - Comment from Shirin
"If you are serious about your trading I would not rely on IB data for serious daytrading. Took me a while to justify the cost of IQ Feed and in the end, it's just a 2 point stop on ES. Better safe than sorry" - Comment from Public Forum
"I ran your IQFeed DDE vs. my broker vs. a level II window for some slow-moving options. I would see the level II quote change, then your feed update instantaneously. My broker's DDE, however, would take as much as 30 seconds to update. I am not chasing milliseconds, but half a minute is unacceptable." - Comment from Rob
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"Just a quick one to say I'm very impressed so far :) The documentation for developers is excellent and I've quickly managed to get an app written to do historical downloads. The system is very robust and pretty quick considering the extent of data that's available. The support guys have been very helpful too, in combination with the forums it's been plain sailing so far!" - Comment from Adam
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »IQConnect.exe 5.2.1.0 (and 5.1.1.3) hangs on connect for multiple connections
Author Topic: IQConnect.exe 5.2.1.0 (and 5.1.1.3) hangs on connect for multiple connections (5 messages, Page 1 of 1)

dk_brad
-Interested User-
Posts: 12
Joined: May 11, 2014


Posted: Mar 30, 2016 03:54 AM          Msg. 1 of 5
I have come across a critical problem with the IQConnect software when establishing multiple connections in quick succession.

Steps to reproduce:
1 - launch client, login
2 - for i = 1 to 15 {
create new socket on port 9100
send set protocol command
sleep x
}
3 - wait
4 - close all sockets

For values of x < ~15ms, the threads for each socket in the iqconnect process seem to enter some sort of race condition, and ramp up to 100% CPU (core) usage. This will typically happen to all 15 connections, but is sometimes limited to fewer (e.g 3 IQConnect threads running at 100% on 3 cores).

During failures:
- IQConnect threads are running at 100% on all 8 cores of my machine (although sometimes fewer as mentioned above).
- There is no response on the socket to any commands sent, including the initial set protocol.
- There are no entries in the IQConnectLog after "LOOKUP SOCKET ACCEPTED i - " for each connection.

Log files are attached.

Configuration:

This occurs on versions 5.1.1.3 and 5.2.1.0 (didn't test others). I am connecting from Java using standard java.net / java.io libraries

OS tested:
- Debian 8 64-bit (kernel 4.5.0 x86_64)
- Debian 7 32-bit (kernel 4.5.0 x86),

JVM tested:
- 1.7.0_80-b15 (32 bit)
- 1.8.0_77 (32 and 64-bit)

Wine tested:
- 1.6.2
- 1.9.6

Interestingly, I couldn't reproduce the problem using wine 1.4.1, but I suspect that's because 1.4.1 is slower to create and connect the sockets rather than wine being the problem. This issue only became evident because I am staging newer wine versions.

I couldn't reproduce the issue with the same code on Windows 7.

I would guess that the most likely explanation is a subtle concurrency bug in the IQConnect.exe connection code that is only evident when multiple connections are made almost simultaneously on a machine with many cores, but it is surprising that it hasn't been seen before.

I can provide test code (Java) if required.
 
 



File Attached: IQConnectLogAll-fail.txt (downloaded 453 times)

dk_brad
-Interested User-
Posts: 12
Joined: May 11, 2014


Posted: Mar 30, 2016 04:12 AM          Msg. 2 of 5
Example screenshot showing partial (4 connections) hang.



File Attached: partial-lockup-example.png (downloaded 431 times)

DTN_Robert_Carrillo
-Interested User-
Posts: 6
Joined: Apr 25, 2007

-DTN Technical Support-


Posted: Mar 30, 2016 04:17 AM          Msg. 3 of 5
Thank you for your post. Developer Support will respond ASAP.

Robert Carrillo

support@iqfeed.net

support@dtniq.com

support@prophetx.com

Trading Markets Customer Service

1-800-397-7000

dk_brad
-Interested User-
Posts: 12
Joined: May 11, 2014


Posted: Mar 30, 2016 06:34 AM          Msg. 4 of 5
For reference, here's a log of the same code running when the issue doesn't occur.



File Attached: IQConnectLogAll-succeed.txt (downloaded 482 times)

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 30, 2016 08:36 AM          Msg. 5 of 5
This was a known issue in 5.1, but we have had good results with releases since and so far on 5.2, but we will take a look into it and see what we can come up with.

Tim
 

 

Time: Mon July 15, 2019 3:10 PM CFBB v1.2.0 0 ms.
© AderSoftware 2002-2003