Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)

"I have to tell you though that using the IQFeed API is about the easiest and cleanest I have seen for some time." - Comment from Jim
"I just wanted to let you know how fast and easy I found it to integrate IQFeed into our existing Java code using your JNI client. In my experience, such things almost never go so smoothly - great job!" - Comment from Nate
"DTN has never given me problems. It is incredibly stable. In fact I've occasionally lost the data feed from Interactive Brokers, but still been able to trade because I'm getting good data from DTN." - Comment from Leighton
"I would just like to say that IQFeed version 4 is running very well and I am very happy with its performance. I would also like to extend a big thanks for the fast and efficient help that I always receive. My questions and concerns are always addressed promptly. Way to go!" - Comment from Josh in CO.
"I "bracket trade" all major news releases and I have not found one lag or glitch with DTN.IQ feed. I am very comfortable with their feed under all typical news conditions (Fed releases, employment numbers, etc)." - Comment from Public Forum
"The service is great, I see a noticeable improvement in my volume profiles over [broker]'s data feed" - Comment from Larry
"The people at Nirvana have very nice things to say about your company and I can see why! Price and service is a potent combination." - Comment from Ed
"Thanks for following up with me. You guys do a great job in tech support." - Comment from Phelps
"I like you guys better than *******...much more stable and a whole lot fewer issues." - Comment from Philip
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
Home  Search  Register  Login  Blogs Recent Posts

Information on Various DTN Products:
DTN IQFeed | DTN ProphetX | DTN Ag | NxCore
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Streaming interval bars delays
Author Topic: Streaming interval bars delays (3 messages, Page 1 of 1)

-Interested User-
Posts: 6
Joined: Nov 22, 2016

Posted: Mar 29, 2017 05:20 PM          Msg. 1 of 3

I'm testing 1-min streaming bars (derivative data, "BW" requests) and I'm seeing couples issues:

1. Delays as long as 3 minutes long. In fact, I can get the same data much sooner by requesting historical data for the last 1-2 minutes.
For example, received at 17:35:54:
BC,SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,,

But I received the same data over 3 minutes earlier via history request, at 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

It's also possible to "cheat" by creating streaming bar watches and cancel them a minute later, then create them again and cancel them again - just to get the initial data on time... By doing so I'm able to get the initial BH history records sooner than the delayed watch records (BC), for example:
At 17:51:02: BH,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,0,
At 17:51:27: BC,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,,

My question is if there is a way to receive streaming data in the form of actual streaming, meaning immediately after each minute passes?
Note I'm testing this after-hours, which I'd assume would keep your servers less busy and able to provide data real-time?

2. In some cases both streaming bar and historical data arrive earlier than the time they indicate, for example:
Received at 17:35:59:
BC,SPY,2017-03-29 16:36:00,235.5100,235.5200,235.5100,235.5200,60566551,3660,,

Or another minute's history received 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

Subsequent data requests show that this 'early' data is final and no longer changes later, thus it seems to be finalized before the actual time covered by the specific period.
I'm guessing there may be some explanation to the minute's data being summarized and arriving early?

Note that my clock is synchronized with yours and generally correct, though above issues don't seem to be clock-related.

-Interested User-
Posts: 6
Joined: Nov 22, 2016

Posted: Mar 30, 2017 10:54 PM          Msg. 2 of 3
Correction to the 2nd issue:
After additional testing I see that streaming and history bars (like 60 seconds) are occasionally updated and resent with incremented data, mainly when the previous bar was sent a few seconds early.
I think it'd be better to receive a single bar once it's finalized, but the current approach is also acceptable once known.

The 1st issue with delayed "BC" stream still remains, though doesn't seem to happen as frequently during daytime when bars come pretty much within 1 second of each minute's ending, but still with some exceptions. The delays intensify after regular trading hours.

DTN_Tim Walter
-DTN Guru-
Posts: 1130
Joined: Apr 25, 2006

Posted: Apr 3, 2017 07:43 AM          Msg. 3 of 3
Hi, my apologies for the delayed reply.

Our goal with streaming bars was to make it so that you would always receive bars that would match our history if the two were later compared. The issue then became, how do you know when a bar has completed so that you can know to write it. The answer, due to the impossibility of syncing all clocks involved and the random latencies between connections, was to say that a bar complete message is only sent when a new message, with a time that is outside of the current bar, is received. So that is why BC messages are not timely.

We do have the update interval, that is discussed in our documentation, that attempts to help solve this. If you set your update interval to 60, you will always get a bar no later than 60 seconds after the last trade. For example, if you were requesting 1 minute bars and set your update interval to 15, a sample minute might go like this;

Note: Assuming this is the open and previous trades have not occurred for simplicity.

9:30:03 - A trade arrives - No message sent
9:30:06 - A trade arrives - No message sent
No trades arrive for 15 seconds after this, so we fire an update (BU) message at 9:30:21
9:30:46 - No trades since the last message, so do nothing
9:30:57 - A trade arrives - No message sent
9:31:12 - No trade for 15 seconds - another update (BU) message sent
9:31:15 - A trade arrives - This is the first trade outside of the 9:30 minute, so a bar complete (BC) message is sent for the 9:30 - 9:31 bar
9:31:29 - A trade arrives - no message
9:31:40 - A trade arrives - no message
9:31:53 - A trade arrives - no message
9:32:03 - A trade arrives - A bar complete message is sent here, since we never went 15 seconds without a message, no BU message was sent.

If the 15 second gap is too large for your app, just lower the update interval to whatever tolerance works for your need, but hopefully this will help illustrate the difficulties you are seeing and how you can help to resolve them.



Time: Sat March 24, 2018 10:37 PM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003