Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I ran your IQFeed DDE vs. my broker vs. a level II window for some slow-moving options. I would see the level II quote change, then your feed update instantaneously. My broker's DDE, however, would take as much as 30 seconds to update. I am not chasing milliseconds, but half a minute is unacceptable." - Comment from Rob
"You have an excellent product !!!!!!" - Comment from Arely
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
"I will tell others who want to go into trading that DTN ProphetX is an invaluable tool, I don't think anyone can trade without it..." - Comment from Luther
"Interactive Brokers tick data was inconsistent, so I have switched to using DTN exclusively. It is great to no longer have to worry about my datafeed all day long." - Comment from Philippe
"DTN feed was the only feed that consistently matched Bloomberg feed for BID/ASK data verification work these past years......DTN feed is a must for my supply & demand based trading using Cumulative Delta" - Comment from Public Forum Post
"I like you guys better than *******...much more stable and a whole lot fewer issues." - Comment from Philip
"As a past ******* customer(and not a happy one), IQ Feed by DTN is a much better and cheaper product with great customer support. I have had no problems at all since switching over." - Comment from Public Forum
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
"I just wanted to tell you what a fine job you have been doing. While *******, from what I hear, has been down and out, off and on, IQ feed has held like a champ this week." - Comment from Shirin
Home  Search  Register  Login  Blogs Recent Posts

Information on Various DTN Products:
DTN IQFeed | DTN ProphetX | DTN Ag | NxCore
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Tick data into OHLCV
Author Topic: Tick data into OHLCV (23 messages, Page 1 of 1)

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 28, 2016 04:09 PM          Msg. 1 of 23
Hello,

I have a custom application that collects tick data (realtime and history) into OHLCV bars by different timeframes and analyses the price levels. As a result I want to get the same price and volume as in your history interval data. Currently I'm testing my application on history and realtime ticks and want to have a clear understanding how should I filter ticks to get the same volume (period volume) and prices (open, high, low, close) as your historical interval data?
For example, currently I do not use the following ticks:
- Realtime ticks with "Message Contents" field that is not equal "C" (Last Qualified Trade)
- History ticks with "Basis For Last" field that is not equal "C" (Last Qualified Trade)
Should I implement any additional filtering?

Thanks.

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Mar 29, 2016 01:25 AM          Msg. 2 of 23
Good morning, we include C & E trades in our interval data. So you would want to add those in as well.

Tim Walter

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 29, 2016 05:00 AM          Msg. 3 of 23
Thank you Tim.

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Mar 29, 2016 05:10 AM          Msg. 4 of 23
Anytime, while I am thinking about it, also make note our intervals are such that the data from 11:00:00 to 11:00:59.999999 is placed in the 11:01 minute bar. This can cause some people an off by one error when comparing data.

Let me know if you have any questions,

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 30, 2016 04:38 AM          Msg. 5 of 23
Hello Tim,

I'm checking the data matching between ticks and interval and found some inconsistencies.
Dealing with BD#C, timeframe is 1 minute, date - 19 November 2015, time 2:02.

Request of interval history data was the following:
HIT,BD#C,60,20150329 030000,20160329 160000,,,,1,HIT_BD#C

Interval data from response:
HIT_BD#C,2015-11-19 02:02:00,156.66,156.61,156.63,156.65,2242,2238,

Next, requested tick data for the interval above:
HTT,BD#C,20151119 020100,20151119 020159,,,,1,HTT_BD#C

Response and volume calculations are in the attached file (ticks_BD#C_20151119_0202.xlsx). There are tick fields in the columns from "A" to "J". Look at column "T" please, there is summary volume (sum of "last size" fields) of the tick data and and interval (period) volume of the interval data.

Could you tell me the reason why these volumes are not equal?

Thank you.
Simon

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 30, 2016 05:13 AM          Msg. 6 of 23
Sorry, xlsx attachments are not supported. Wrapped in a zip archive.



File Attached: ticks_BD#C_20151119_0202.zip (downloaded 77 times)

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 31, 2016 02:28 AM          Msg. 7 of 23
Hi guys, i'm finding numerous inconsistencies between interval data and tick data, could someone please get back to me on the above question today? Trying to get to the bottom of this!

Really appreciate your help

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Mar 31, 2016 09:03 AM          Msg. 8 of 23
Sorry, the file that you uploaded appears to have an issue. Could you re-upload your zip file for me to look over?

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 31, 2016 09:09 AM          Msg. 9 of 23
Zip-file is re-uploaded. Just in case, here is a link on dropbox: https://www.dropbox.com/s/c9t8cc08jc8p4k2/ticks_BD%23C_20151119_0202.zip?dl=0



File Attached: ticks_BD#C_20151119_02021.zip (downloaded 79 times)

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Mar 31, 2016 09:15 AM          Msg. 10 of 23
Thanks much. Looking it over now.

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Mar 31, 2016 09:31 AM          Msg. 11 of 23
What protocol are you requesting the data under?

Note: If you are not calling SET PROTOCOL anywhere on this socket, it would be protocol 4.9.

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 31, 2016 02:11 PM          Msg. 12 of 23
On startup my application connects to the Level1 and sets protocol v5.1.

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Mar 31, 2016 02:14 PM          Msg. 13 of 23
Hello,

Sorry for the delay, but this one took me a little while to track down. It appears this is related to an issue that occurred back in 2015 during our initial roll-out of microseconds. This caused the discrepancies that you are seeing in our minute data and we are looking into what it will take to get it resolved. The tick data is correct though in all of the cases that I researched, so in the interim, if you build your bars from the underlying tick data, you will have the correct information.

I will try to get more information on this in the days to come. Let me know if you have any questions beyond this though and I will be happy to impart whatever I can.

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Apr 3, 2016 12:38 PM          Msg. 14 of 23
Thank you very much Tim.

Currently I'm testing my program with realtime (Level1) ticks. To build bars from tick I'm using the following felds of the Q messages: "Last", "Bid", "Ask", "Last Size", "Last TimeMS" and "Last Date".
Checks showed that value of "Last TimeMS" has a large difference sometimes (more than 5 minutes) comparing to current time. Should I use "Most Recent Trade", "Bid", "Ask", "Most Recent Size", "Most Recent TimeMS" and "Most Recent Date" instead? I want to build bars in realtime and the build bars should match your history interval data.

Thank you.

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Apr 4, 2016 05:57 AM          Msg. 15 of 23
Most Recent TimeMS is what you would want. This time field only updates on a trade though, so having it go several minutes without changing, while the bid and ask values may be updating constantly can be completely normal. We would expect the time time value to update on each message that has a Message Contents of C, E, or O. Though, you should be filtering out trades with a message content value of 'O' if you want to match up to history.

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Apr 4, 2016 09:25 AM          Msg. 16 of 23
Thank you again for quick reply!

So, in the other words, your historical ticks (response for any of HT* command) according to realtime ticks in the following way:
Last (history) = Most Recent Trade (realtime)
Last Size (history) = Most Recent Trade Size (realtime)
Time Stamp (history) = Most Recent Trade Date + Most Recent Trade TimeMS (realtime)
Bid (history) = Bid (realtime)
Ask (history) = Ask (realtime)

Is it correct?

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Apr 4, 2016 09:49 AM          Msg. 17 of 23
Assuming you are filtering out "O" trades and that you are using protocol 5.1, the new protocol usees different time fields.

Last (history) = Most Recent Trade (realtime) = true
Last Size (history) = Most Recent Trade Size (realtime) = true
Time Stamp (history) = Most Recent Trade Date + Most Recent Trade TimeMS (realtime) = true

A snapshot of these values are taken at the time of the trade message. The individual messages are not saved.

Bid (history) = Bid (realtime)
Ask (history) = Ask (realtime)

Tim

Craig
-DTN Evangelist-
Posts: 212
Joined: Apr 16, 2010


Posted: Apr 4, 2016 12:45 PM          Msg. 18 of 23
'It appears this is related to an issue that occurred back in 2015 during our initial roll-out of microseconds'

Can you please expand on this...it kind of seems like a big deal.
How invalid is the post 2015 interval data?

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Apr 4, 2016 12:55 PM          Msg. 19 of 23
There were 2 days, on the 18th and 19th of November 2015, where we received trades with a second value of 60 or more from Eurex, so we are rebuilding those intraday bars, for that exchange.

Tim

Craig
-DTN Evangelist-
Posts: 212
Joined: Apr 16, 2010


Posted: Apr 4, 2016 01:19 PM          Msg. 20 of 23
Cheers.

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Apr 8, 2016 11:08 AM          Msg. 21 of 23
Hello,

Just wanted to let you know all relevant changes were made to the futures contracts affected.

Sorry for the inconvenience,

Tim

quickTick
-Interested User-
Posts: 31
Joined: Nov 17, 2013


Posted: Apr 8, 2016 02:10 PM          Msg. 22 of 23
By the way: Some minutes actually do have 61 seconds. The last one was apparently at "June 30, 2015 at 23:59:60 UTC".

https://en.wikipedia.org/wiki/Leap_second

How is this handled by IQFeed? Wouldn't want my app to crash in such a case.... :)

DTN_Tim Walter
-DTN Guru-
Posts: 1036
Joined: Apr 25, 2006


Posted: Apr 8, 2016 02:38 PM          Msg. 23 of 23
Yep, those happen every so often. They get rolled into the previous second, luckily they happen when most everything is closed.

Tim
 

 

Time: Sat September 23, 2017 2:22 AM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003