Hi, a question about the timestamp in the update(Q) messages:

How is the timestamp in the update message been assigned? Where does it come from?

Please notice in the following updates, the timestamps seems following different timelines for different symbols. For each symbol, the timestamp keeps increasing. But between symbols, they could be seconds (sometimes more than minute) apart. It looks like some delay for some symbols.

Please note the following is the output of telnet with subscriptions to several symbols. The total active subscription to IQFeed is about 350 symbols. There is also another client connecting to IQFeed at the same time to consume the data. The machine is a 8-core linux server with 32G memory.


Q,@RTY#,2308.00,1,08:52:50.896622,43,5070,2307.80,3,2308.00,1,2304.50,2308.30,2298.60,2302.00,a,01,
Q,@NQ#,14706.00,1,08:53:00.823755,43,34156,14706.00,6,14706.50,1,14704.25,14721.75,14674.50,14713.75,C,01,
Q,@NQ#,14706.00,1,08:53:00.823755,43,34156,14706.00,5,14706.50,1,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,11,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,10,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,9,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,8,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,9,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,8,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,7,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14706.00,5,14706.50,1,14704.25,14721.75,14674.50,14713.75,C,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,3,14706.50,1,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,3,14706.50,2,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,3,14706.50,3,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,2,14706.50,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.50,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,1,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NIY#,28620,1,08:51:42.991782,110,9701,28620,21,28625,28,28755,28755,28485,28775,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,2,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,3,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.50,3,14706.25,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.50,2,14706.25,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.50,2,14706.00,1,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NIY#,28620,1,08:51:42.991782,110,9701,28620,15,28625,28,28755,28755,28485,28775,b,01,
Q,@NIY#,28620,1,08:51:42.991782,110,9701,28620,13,28625,28,28755,28755,28485,28775,b,01,

The timestamp is provided by the exchange. It is meant to indicate the timestamp that the tick occurred. It is possible for ticks with older timestamps to arrive later, as you point out. This is because the ticks come from many different sources, and they don't all do so on the same timeframe. So situations like in your example can occur, where the entire string of ticks is not in chronological order:

Q,@NQ#,14706.00,1,08:53:00.823755,43,34156,14706.00,5,14706.50,1,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@ES#,4342.75,2,08:52:53.296428,43,72578,4342.75,11,4343.00,29,4341.00,4343.75,4333.00,4342.75,b,01,

Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,1,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NIY#,28620,1,08:51:42.991782,110,9701,28620,21,28625,28,28755,28755,28485,28775,b,01,

Ticks for the same symbol will always be chronological, though.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

If the timestamps all come from the exchange, then it indicates some serious delay along the path.
Note all the symbols here are from Globex. The exchange clock should be the same/synchronized. Especially instruments like ES and NQ, they are within the same market data segment.

If an event happens a minute later than another, but received before that, that means there is at least a minute delay. Is there a way we can investigate this? How to check the backlog within IQFeed?

There's another detail at work in the data sample given in the original post.

You are correct that the delays are too long for the explanation I gave. The phenomenon I mentioned earlier is true, but should't make that much of a difference time-wise.

The dataset appears to be using the default set of fields from Protocol 6.1, which are:

S,CURRENT UPDATE FIELDNAMES,Symbol,Most Recent Trade,Most Recent Trade Size,Most Recent Trade Time,Most Recent Trade Market Center,Total Volume,Bid,Bid Size,Ask,Ask Size,Open,High,Low,Close,Message Contents,Most Recent Trade Conditions


With that in mind, let's look at just the @NQ# messages:


Q,@NQ#,14706.00,1,08:53:00.823755,43,34156,14706.00,6,14706.50,1,14704.25,14721.75,14674.50,14713.75,C,01,

The fields in bold face are Most Recent Trade Time and Message Contents. Message Contents will be a C, E, or O for a trade, and b or a for a bid or ask update. Most Recent Trade Time is the most recent time a trade occurred, and it includes both last-eligible and non-last eligible trades (C and O in Message Contents). Now look at the next @NQ# messages:

Q,@NQ#,14706.00,1,08:53:00.823755,43,34156,14706.00,5,14706.50,1,14704.25,14721.75,14674.50,14713.75,b,01,

This message is a bid update - not a sale. So "08:53:00.823755" is telling you the time of the most recent sale, not the timestamp of this message.

Next, we ahave:

Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14706.00,5,14706.50,1,14704.25,14721.75,14674.50,14713.75,C,01,

This is a sale, so the Most Recent Trade Time updates.

Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,3,14706.50,1,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,3,14706.50,2,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,3,14706.50,3,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,2,14706.50,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.50,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,1,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,2,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.75,1,14706.25,3,14704.25,14721.75,14674.50,14713.75,a,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.50,3,14706.25,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.50,2,14706.25,3,14704.25,14721.75,14674.50,14713.75,b,01,
Q,@NQ#,14706.00,5,08:53:00.824297,43,34161,14705.50,2,14706.00,1,14704.25,14721.75,14674.50,14713.75,a,01,

These are all bid or ask updates - not sales. The timestamp doesn't update because the most recent trade still occurred at 08:53:00.824297. These timestamps do not necessarily mean the timestamps of when this message was sent or it was received. These timestamps, as per the default protocol 6.1 setup, mean the time of the most recent trade, which could have occurred some time ago.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

That would make sense. Thank you, Gary.

So in order to get the most recent change time, I have to listen to most recent trade time, bid time and ask time, then check which one is the latest? Is there a field for the most recent change time?

If you want to know the most recent change time with no regard to what changed, then yes, you would need to watch all three of the fields.

The fields Most Recent Trade Time, Bid Time and Ask Time tell you the time of the most recent change to the price, Bid or Ask - which is not necessarily the current tick. If a Bid update occurs at 10:00:00 and a tick not containing a Bid update comes in at 10:00:01, the "Bid Time" will still be 10:00:00.

Side note: The "Most Recent" statistics include both last-eligible and non-last-eligible trades. "Last" includes only last-eligible. There is also "Extended" which covers after hours/Form T trades. So which "Time" fields you need to watch is dictated by which of those you're interested in.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist