Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I'm satisfied with IQFeed. It's the most reliable and fastest quote feed I have ever used. Although I'm a resident in China, it's still very fast!" - Comment from Xiaofei
"Its working FABULOUSLY for me!! Holy cow...there has been so much I've been missing lately, and with this feed and Linnsoft software...I'm in the game now." - Comment from Chris R.
"For anyone considering using DTN.IQ for a data feed, my experience with the quality of data and the tech support has been very positive." - Comment from Public Forum
"I ran your IQFeed DDE vs. my broker vs. a level II window for some slow-moving options. I would see the level II quote change, then your feed update instantaneously. My broker's DDE, however, would take as much as 30 seconds to update. I am not chasing milliseconds, but half a minute is unacceptable." - Comment from Rob
"Everything is working amazing now. I'm already impressed with the true-tick feed of IQFeed and it's ability to support my 480 symbol layout." - Comment from Tyler via Email
"Everything is working great with the API. I love it." - Comment from Calvin
"IQ feed works very well, does not have all of the normal interruptions I have grown used to on *******" - Comment from Mark
"I've been using Neoticker RT with IQFeed for two months, and I'm very happy with both of the products (I've had IQFeed for two years with very few complaints). The service from both companies is exceptional." - Comment from Public Forum
"As a past ******* customer(and not a happy one), IQ Feed by DTN is a much better and cheaper product with great customer support. I have had no problems at all since switching over." - Comment from Public Forum
"I have been using IQFeed now for a few years in MultiCharts and I have zero complaints. Very, very rare to have any data hiccups or anything at all go wrong." - Comment from Public Forum
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Downloading Historical Intraday Future Ticks
Author Topic: Downloading Historical Intraday Future Ticks (4 messages, Page 1 of 1)

madman
-Interested User-
Posts: 2
Joined: Jul 16, 2020


Posted: Jul 16, 2020 11:37 PM          Msg. 1 of 4
Hey everyone,

I'm trying to download historical intraday CME futures tick data, but I'm running into issues when downloading the ticks.

I issue a request to download 1 day of data (July 1st), like so:
"HTT,@ESU20,20200701 000000,20200702 000000,,,,1\r\n"

Then I receive streaming tick data -- the data streams for a while, but hangs after I receive about 18,000 messages (or until about 02:17:00 on July 1st in the data stream).

Some questions:
1- is my request format correct?
2- should I be issuing smaller requests? (e.g: 15 minute chunks?)
3- any other tips or best practices when doing these downloads?

Cheers,
madman

DTN_Gary_Stephen
-DTN Guru-
Posts: 267
Joined: Jul 3, 2019


Posted: Jul 17, 2020 10:58 AM          Msg. 2 of 4
Hello, this is Gary with IQFeed API Support.

Your request is fine. You can request as much data as you want. It sounds like the data is queueing, which happens when IQFeed cannot send data that has been requested. In other words, IQFeed is sending data faster than your code is processing it, and the receive buffer is filling up. To monitor this, open Diagnostics/Client Stats tab, and observe the "KB Queued" column for the Lookup port during a request. You'll probably find that it starts low and slowly increases as data comes in.

How do you avoid this? Generally, DTN recommends using three different threads in your app: one to retrieve the data from the API, one to convert the information into the form you want, and a third thread to act on it (draw a chart, put it in a database, whatever your app does). Also: having multiple connections to the IQFeed, and having each connection retrieve part of the desired data, can improve performance.

I hope this helps! Let me know if you have any other questions or issues getting data from the API.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

madman
-Interested User-
Posts: 2
Joined: Jul 16, 2020


Posted: Jul 19, 2020 03:10 PM          Msg. 3 of 4
Hi Gary,

Thanks for the prompt and thorough response!

Good call: I checked the KB Queued column and saw the exact behavior you described. After tweaking my application with your tips, I can know maintain a KB Queued of 0 bytes.

Regards,
Greg

DTN_Gary_Stephen
-DTN Guru-
Posts: 267
Joined: Jul 3, 2019


Posted: Jul 20, 2020 07:58 AM          Msg. 4 of 4
Greg,

Excellent! I'm glad I could help.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
 

 

Time: Sun August 14, 2022 6:54 AM CFBB v1.2.0 47 ms.
© AderSoftware 2002-2003