Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)

"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"IQFeed version 4 is a real screamer compared to anything else I have seen." - Comment from Tom
"Version has been working well for me and I appreciate that it is now a much tighter client to work with. I feel I can go to press with my own application and rely on a stable platform" - Comment from David in IA.
"The people at Nirvana have very nice things to say about your company and I can see why! Price and service is a potent combination." - Comment from Ed
"I "bracket trade" all major news releases and I have not found one lag or glitch with DTN.IQ feed. I am very comfortable with their feed under all typical news conditions (Fed releases, employment numbers, etc)." - Comment from Public Forum
"Previously I was using *******. IQFeed is WAY more economical, and for my charting needs is just as good, if not better." - Comment from Public Forum Post
"This is an excellent value, the system is generous (allowing for 500 stocks) and stable (and really is tick-by-tick), and the support is fantastic." - Comment from Shirin via Email
"Everything is working great ! Very impressive client. The news refreshes better and is more pertinent than the ******* feed I paid $ 100/month for. I Also like the charts a lot." - Comment from Leon
"Thanks for all of your help. Great customer service deserves to be recognized which one the reasons I've been a customer of DTN for over 10 years!" - Comment from Stuart
"You have an excellent feed. Very few spikes for Spot Forex." - Comment from Public Forum Post
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Incorrect Historical Data Time Stamping
Author Topic: Incorrect Historical Data Time Stamping (6 messages, Page 1 of 1)

-Interested User-
Posts: 25
Joined: May 23, 2006

Posted: Jan 2, 2008 01:57 AM          Msg. 1 of 6
We have been getting many reports from our users of problems associated with incorrect timestamps for data records in your historical intraday data. We believe this affects the tick data. However it may affect minute data as well.

The problem is there is out of order time stamping in your data base. This causes problems for us. The first problem is that if there is a time stamp for a tick record which is ahead of the time that is for, then all tick records following that record that have a preceding timestamp will get filtered out by our program. This is a special filtering to maintain the integrity of our database. One reason we do this is that if there is an invalid or out of order time stamping in the chart, this will cause our chart drawing tools to not function properly. This filtering manifests as missing data to the user.

Apparently some of your support people have been telling our customers that this is a Sierra Chart problem. It is not. It is a problem that you need to resolve on your side. We do plan to remove this filtering because of this problem. However, when we do remove this filtering users are still going to have problems using drawing tools. It will be appreciated that you promptly correct this issue and acknowledge that there is a problem. If there is not a problem, then could you identify to us why we are having a problem.

We have been telling people that you are a reliable service, but the problems that people are having are just as bad as other services.

A prompt response on this issue is much appreciated.

Thank you,

-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005

Posted: Jan 2, 2008 09:02 AM          Msg. 2 of 6
Hello Anthony, are you reporting that the data is bad (incorrect timestamp) or are you saying that because the timestamps are out of order it causes problems for you?

Concerning the out of order trades:
Occasionally, the exchanges send us data out of order. You will see this in the Level 1 streaming data if you examine it closely. The IQFeed history servers do not reorder this data (it is served to your app in the same order it is received from the exchange).

On the other hand, if what you are reporting is a timestamp that is simply completely wrong and showing the wrong date, or something that is severely out of order (more than a few seconds), please give us an example of the data that we can investigate.

-Interested User-
Posts: 25
Joined: May 23, 2006

Posted: Jan 2, 2008 02:13 PM          Msg. 3 of 6
Incorrect timestamps definitely do cause problems for us and I would think they would cause problems for anyone. None of the data feeds we work with provide out of order timestamps. Your data feed has never done this until recently. I don't understand what is going on. You really need to provide timestamps that are always in order otherwise is going to be an ongoing problem. How is it that everything was OK before and now it is not? Therefore I don't really understand your answer. You are saying that the exchanges sometimes provide out of order data. However that was never the case in the past based upon what you have transmitted. Unless it was only off a few seconds.

It seems as though the time-stamp is simply incorrect but we have to look more closely.I think the timestamp is way off by more than a few seconds. But even a few seconds off is not very good. If you have been delivering data that has timestamps which are off by a few seconds, then that probably has gone unnoticed. But this problem definitely is vastly more severe than that.

Sierra Chart

-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005

Posted: Jan 2, 2008 03:45 PM          Msg. 4 of 6
This is not a new occurance. The exchanges have always (to my knowledge) transmitted quotes in this way. However, in the past, IQFeed did not transmit seconds in the timestamp. As a result, unless the discrepancy occured on the rollover of a minute, it was not noticable (since the trades had the same timestamp).

Assuming we are talking about out of order (chronologically) trades, it is important to clarify that the timestamps themselves are not incorrect (the trade occured at the time indicated).

It is not entirely uncommon for a trade to be out of order by a few seconds (less than 5). It is fairly rare (but not entirely unheard of) for it to be out of order by more than a few seconds (would require investigation by our market data team). If you are talking about timestamps that appear completely wrong (for example: a trade for midnight comming in the middle of the day or a trade with the wrong date), I am more inclined to chalk that up as something in our servers rather than data the exchange sent us bad but would still require investigation.

Whichever occurace we are talking about, at this point, I am not willing to say that this isn't an issue on our end. Right now, I am simply letting you know what you should expect from IQFeed currently. The issue requires more investigation (which requires some examples of the data in question).

-Interested User-
Posts: 25
Joined: May 23, 2006

Posted: Jan 2, 2008 04:36 PM          Msg. 5 of 6
Being that the problem is intermittent, I do not have any example now. We also have taken out the filtering, which means it has less of an effect on our software.

-Interested User-
Posts: 2
Joined: Jan 30, 2008

Posted: Feb 2, 2008 09:18 AM          Msg. 6 of 6
If I read these posts correctly, the datafeed sends the actual trades, with the correct time stamps, but they don't always arrive in time stamp order ?

I believe that my graphing code could tolerate this, since it just draws lines between the next two data points. However, this could throw a big wrench in my analysis routines which assume that ticks loaded to a vector are moving forward in time.

I could add a method to reorder the ticks, which would be fine for graphing at the end of the day. Reordering would to be to expensive to apply in realtime though.

I may need to add a filter that drops any back dated ticks before they get added to my tick stream Vector. This approach might also work for SierraChart.

Joe paladin


Time: Fri June 5, 2020 5:12 AM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003