Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"For anyone considering using DTN.IQ for a data feed, my experience with the quality of data and the tech support has been very positive." - Comment from Public Forum
"If you want customer service that answers the phone, your best bet is IQFeed. I cannot stop praising them or their technical support. They are always there for you, and they are quick. I have used ****** too but the best value is IQFeed." - Comment from Public Forum
"I'm very glad I switched to IQFeed. It's working perfectly with no lag, even during fast market conditions." - Comment from Andy via Email
"IQ feed is brilliant. The support is mind-bending. What service!" - Comment from Public Forum Post
"If you are serious about your trading I would not rely on IB data for serious daytrading. Took me a while to justify the cost of IQ Feed and in the end, it's just a 2 point stop on ES. Better safe than sorry" - Comment from Public Forum
"I just wanted to say how happy I am with your service. I was able to download the API docs last week and I was able to replicate Interactive Brokers historical bar queries and realtime bar queries over the weekend. That was about one of the fastest integrations that I've ever done and it works perfectly!!!!" - Comment from Jason via Email
"Just a quick one to say I'm very impressed so far :) The documentation for developers is excellent and I've quickly managed to get an app written to do historical downloads. The system is very robust and pretty quick considering the extent of data that's available. The support guys have been very helpful too, in combination with the forums it's been plain sailing so far!" - Comment from Adam
"Thanks for all of your help. Great customer service deserves to be recognized which one the reasons I've been a customer of DTN for over 10 years!" - Comment from Stuart
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"DTN feed was the only feed that consistently matched Bloomberg feed for BID/ASK data verification work these past years......DTN feed is a must for my supply & demand based trading using Cumulative Delta" - Comment from Public Forum Post
Home  Search  Register  Login  Blogs Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: jwelch
About Contact
Joined: Mar 15, 2018 10:13 PM
Last Post: Mar 15, 2018 10:38 PM
Last Visit: Mar 18, 2018 06:35 PM
Website:  
Location:
Occupation:
Interests:
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
jwelch has contributed to 1 posts out of 18704 total posts (0.01%) in 188 days (0.01 posts per day).

20 Most recent posts:
IQFeed Developer Support » Duplicate Q Messages Mar 15, 2018 10:38 PM (Total replies: 0)

TLDR; - Is it normal to expect the occasional duplicate Q message in the feed? Or are these meant to represent two genuine ticks that somehow end up with exactly the same data (including TickID?)

Details:
After doing some analysis on the feed data I receive, there are occasional duplicate tick messages (as in they have exactly the same crc64 value)

For Example, here is a summary of the duplicate lines found in a sample feed from 3/7/18.


Writing CRC Collisions to: IQFeed (3-7-2018).crc
Writing duplicate entries to: IQFeed (3-7-2018).dup
Processed 11734028 ticks
Found 2045 duplicate lines in the file
T: 0
Q: 1983
P: 2
S: 0
F: 60
CRC: 0 - No collisions

Ignoring the F and P duplicates as those might expected... here is a sample of some Q messages which had the same CRC64 value.


Duplicate Line Found on Hash [f612fb7a7fa49e97]
Line 1 [864989]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Line 2 [864991]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [cada7e85403bc991]
Line 1 [866805]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Line 2 [866807]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [3ee933e4adea4f37]
Line 1 [869514]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,
Line 2 [869516]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,


The number in square brackets e.g. 'Line 1 [#]' is a local message identifier. Clearly the messages are received very close to one another in time.

This is with the field parameters as follows:

S,CURRENT UPDATE FIELDNAMES,Symbol,Exchange ID,Last,Change,Percent Change,Total Volume,High,Low,Bid,Ask,Bid Size,Ask Size,Tick,Range,Open Interest,Open,Close,Spread,Settle,Delay,Restricted Code,Net Asset Value,Average Maturity,7 Day Yield,Extended Trading Change,Extended Trading Difference,Price-Earnings Ratio,Percent Off Average Volume,Bid Change,Ask Change,Change From Open,Market Open,Volatility,Market Capitalization,Fraction Display Code,Decimal Precision,Days to Expiration,Previous Day Volume,Open Range 1,Close Range 1,Open Range 2,Close Range 2,Number of Trades Today,VWAP,TickID,Financial Status Indicator,Settlement Date,Bid Market Center,Ask Market Center,Available Regions,Last Size,Last Time,Last Market Center,Most Recent Trade,Most Recent Trade Size,Most Recent Trade Time,Most Recent Trade Conditions,Most Recent Trade Market Center,Extended Trade,Extended Trade Size,Extended Trade Time,Extended Trade Market Center,Message Contents,Ask Time,Bid Time,Last Date,Extended Trade Date,Most Recent Trade Date


Note that TickID is included in the message, and is identical on lines which are found to have the same hash.

So, given the rather low occurrence of these duplicates (< 0.02%) this is probably not much of a concern for most analyses, but I was curious if this is expected behavior in the feed either way.

Thanks!


Time: Tue September 18, 2018 9:51 PM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003