Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I use IQ Feed, Great stuff as far as data analysis information, storage and retrieval is concerned." - Comment from Public Forum
"I have been using IQFeed now for a few years in MultiCharts and I have zero complaints. Very, very rare to have any data hiccups or anything at all go wrong." - Comment from Public Forum
"Can I get another account from you? I am tired of ******* going down so often" - Comment from George
"I've been using Neoticker RT with IQFeed for two months, and I'm very happy with both of the products (I've had IQFeed for two years with very few complaints). The service from both companies is exceptional." - Comment from Public Forum
"I have to tell you though that using the IQFeed API is about the easiest and cleanest I have seen for some time." - Comment from Jim
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"I ran your IQFeed DDE vs. my broker vs. a level II window for some slow-moving options. I would see the level II quote change, then your feed update instantaneously. My broker's DDE, however, would take as much as 30 seconds to update. I am not chasing milliseconds, but half a minute is unacceptable." - Comment from Rob
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
"With HUGE volume on AAPL and RIMM for 2 days, everyone in a trading room was whining about freezes, crashes and lag with *******, RealTick, TS and Cyber. InvestorRT with IQFeed was rock solid. I mean SOLID!" - Comment from Public IRC Chat
"Just a thank you for the very helpful and prompt assistance and services. You provided me with noticeably superior service in my setup compared to a couple of other options I had looked at." - Comment from John
Home  Search  Register  Login  Blogs Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Wish List »Multi-Threaded Decompression / Get Number of Ticks
Author Topic: Multi-Threaded Decompression / Get Number of Ticks (5 messages, Page 1 of 1)

gamozo
-Interested User-
Posts: 4
Joined: Apr 2, 2013


Posted: Sep 19, 2013 05:17 PM          Msg. 1 of 5
Is there any chance we could get multi-threaded decompression in IQConnect? I'm bottlenecking on CPU rather than download bandwidth.

Currently I just pull all my symbols at once, which means I bottleneck on bandwidth which makes me happy. However I would like to pull a single symbol at full network bandwidth.

Another thing that could allow me to implement this on my end is if you add a 'get number of ticks', so I could query how many ticks of history there is for a symbol. Once I get that number I could start at trades/threads offsets and do a multi-threaded pull.

Any chance this could happen soon?

-Brandon
Edited by gamozo on Sep 19, 2013 at 05:22 PM

DTN_Steve_S
-DTN Guru-
Posts: 2052
Joined: Nov 21, 2005


Posted: Sep 19, 2013 05:22 PM          Msg. 2 of 5
Assuming you are talking about historical data downloads, each socket connection to the lookup port is handled in it's own thread. Therefore, you can simply create multiple socket connections to the feed and balance your requests across the sockets to do a multithreaded pull.

gamozo
-Interested User-
Posts: 4
Joined: Apr 2, 2013


Posted: Sep 19, 2013 05:24 PM          Msg. 3 of 5
Yep. That's what I currently do. However that only works with multiple symbols. It would be nice to have one symbol at full speed. And the only way I could think of doing this is if I knew how many ticks there were historically, with that I could start multiple queries at different starting points. I guess I could do it in chunks of 100k trades or so and just guess for now, but it'd be nice to know how many trades to expect.

DTN_Steve_S
-DTN Guru-
Posts: 2052
Joined: Nov 21, 2005


Posted: Sep 20, 2013 09:12 AM          Msg. 4 of 5
How large of requests are you making? Have you looked into requesting data by timeframe instead of by X number of datapoints?

gamozo
-Interested User-
Posts: 4
Joined: Apr 2, 2013


Posted: Sep 20, 2013 10:09 AM          Msg. 5 of 5
Just tick-level on pretty much all the top 20 most traded futures. I think @ES# has about 55 million datapoints at tick level.

I pull at ticklevel as I postprocess all the data and frequently use tick data for simulated fills rather than n-length bars.

It's not a huge issue, I usually just pull once and then have the files around forever. I'm just requesting the feature if it's easy to add as it would be nice here and there.

-Brandon
 

 

Time: Sun November 18, 2018 9:11 PM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003