Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I just wanted to let you know how fast and easy I found it to integrate IQFeed into our existing Java code using your JNI client. In my experience, such things almost never go so smoothly - great job!" - Comment from Nate
"Its working FABULOUSLY for me!! Holy cow...there has been so much I've been missing lately, and with this feed and Linnsoft software...I'm in the game now." - Comment from Chris R.
"I am very pleased with the DTNIQ system for quotes and news." - Comment from Larry
"I just wanted to let u know that your data feed/service is by far the best!!! Your unfiltered tick data is excellent for reading order flow and none of your competitors delivers this quality of data!" - Comment from Peter via Email
"I am very happy I changed. I love the product, but more so I am thrilled with Tech Support. You are knowledgeable, polite, pleasant and professional." - Comment from Pat
"Interactive Brokers tick data was inconsistent, so I have switched to using DTN exclusively. It is great to no longer have to worry about my datafeed all day long." - Comment from Philippe
"Previously I was using *******. IQFeed is WAY more economical, and for my charting needs is just as good, if not better." - Comment from Public Forum Post
"Very impressed with the quality of your feed - ******* is a real donkey in comparison." - Comment from A.C. via Email
"DTN has never given me problems. It is incredibly stable. In fact I've occasionally lost the data feed from Interactive Brokers, but still been able to trade because I'm getting good data from DTN." - Comment from Leighton
"I use IQ Feed, Great stuff as far as data analysis information, storage and retrieval is concerned." - Comment from Public Forum
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Incorrect Historical Data Time Stamping
Author Topic: Incorrect Historical Data Time Stamping (6 messages, Page 1 of 1)

SierraChart
-Interested User-
Posts: 25
Joined: May 23, 2006


Posted: Jan 2, 2008 01:57 AM          Msg. 1 of 6
We have been getting many reports from our users of problems associated with incorrect timestamps for data records in your historical intraday data. We believe this affects the tick data. However it may affect minute data as well.

The problem is there is out of order time stamping in your data base. This causes problems for us. The first problem is that if there is a time stamp for a tick record which is ahead of the time that is for, then all tick records following that record that have a preceding timestamp will get filtered out by our program. This is a special filtering to maintain the integrity of our database. One reason we do this is that if there is an invalid or out of order time stamping in the chart, this will cause our chart drawing tools to not function properly. This filtering manifests as missing data to the user.

Apparently some of your support people have been telling our customers that this is a Sierra Chart problem. It is not. It is a problem that you need to resolve on your side. We do plan to remove this filtering because of this problem. However, when we do remove this filtering users are still going to have problems using drawing tools. It will be appreciated that you promptly correct this issue and acknowledge that there is a problem. If there is not a problem, then could you identify to us why we are having a problem.

We have been telling people that you are a reliable service, but the problems that people are having are just as bad as other services.

A prompt response on this issue is much appreciated.

Thank you,
Anthony

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Jan 2, 2008 09:02 AM          Msg. 2 of 6
Hello Anthony, are you reporting that the data is bad (incorrect timestamp) or are you saying that because the timestamps are out of order it causes problems for you?

Concerning the out of order trades:
Occasionally, the exchanges send us data out of order. You will see this in the Level 1 streaming data if you examine it closely. The IQFeed history servers do not reorder this data (it is served to your app in the same order it is received from the exchange).

On the other hand, if what you are reporting is a timestamp that is simply completely wrong and showing the wrong date, or something that is severely out of order (more than a few seconds), please give us an example of the data that we can investigate.

SierraChart
-Interested User-
Posts: 25
Joined: May 23, 2006


Posted: Jan 2, 2008 02:13 PM          Msg. 3 of 6
Incorrect timestamps definitely do cause problems for us and I would think they would cause problems for anyone. None of the data feeds we work with provide out of order timestamps. Your data feed has never done this until recently. I don't understand what is going on. You really need to provide timestamps that are always in order otherwise is going to be an ongoing problem. How is it that everything was OK before and now it is not? Therefore I don't really understand your answer. You are saying that the exchanges sometimes provide out of order data. However that was never the case in the past based upon what you have transmitted. Unless it was only off a few seconds.

It seems as though the time-stamp is simply incorrect but we have to look more closely.I think the timestamp is way off by more than a few seconds. But even a few seconds off is not very good. If you have been delivering data that has timestamps which are off by a few seconds, then that probably has gone unnoticed. But this problem definitely is vastly more severe than that.

Anthony
Sierra Chart

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Jan 2, 2008 03:45 PM          Msg. 4 of 6
This is not a new occurance. The exchanges have always (to my knowledge) transmitted quotes in this way. However, in the past, IQFeed did not transmit seconds in the timestamp. As a result, unless the discrepancy occured on the rollover of a minute, it was not noticable (since the trades had the same timestamp).

Assuming we are talking about out of order (chronologically) trades, it is important to clarify that the timestamps themselves are not incorrect (the trade occured at the time indicated).

It is not entirely uncommon for a trade to be out of order by a few seconds (less than 5). It is fairly rare (but not entirely unheard of) for it to be out of order by more than a few seconds (would require investigation by our market data team). If you are talking about timestamps that appear completely wrong (for example: a trade for midnight comming in the middle of the day or a trade with the wrong date), I am more inclined to chalk that up as something in our servers rather than data the exchange sent us bad but would still require investigation.

Whichever occurace we are talking about, at this point, I am not willing to say that this isn't an issue on our end. Right now, I am simply letting you know what you should expect from IQFeed currently. The issue requires more investigation (which requires some examples of the data in question).

SierraChart
-Interested User-
Posts: 25
Joined: May 23, 2006


Posted: Jan 2, 2008 04:36 PM          Msg. 5 of 6
Being that the problem is intermittent, I do not have any example now. We also have taken out the filtering, which means it has less of an effect on our software.

jpaladin
-Interested User-
Posts: 2
Joined: Jan 30, 2008


Posted: Feb 2, 2008 09:18 AM          Msg. 6 of 6
If I read these posts correctly, the datafeed sends the actual trades, with the correct time stamps, but they don't always arrive in time stamp order ?

I believe that my graphing code could tolerate this, since it just draws lines between the next two data points. However, this could throw a big wrench in my analysis routines which assume that ticks loaded to a vector are moving forward in time.

I could add a method to reorder the ticks, which would be fine for graphing at the end of the day. Reordering would to be to expensive to apply in realtime though.

I may need to add a filter that drops any back dated ticks before they get added to my tick stream Vector. This approach might also work for SierraChart.

Joe paladin
 

 

Time: Thu July 18, 2019 8:59 PM CFBB v1.2.0 0 ms.
© AderSoftware 2002-2003