Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"There is no doubt that IQFeed is the best data provider. I am very satisfied with your services. And IQFeed is the only one that I would recommend to my friends. Now, most of them are using your product in China." - Comment from Zhezhe
"Boy, probably spent a thousand hours trying to get ******* API to work right. And now two hours to have something running with IQFeed. Hmmm, guess I was pretty stupid to fight rather than switch all this time. And have gotten more customer service from you guys already than total from them… in five years." - Comment from Jim
"IQ feed works very well, does not have all of the normal interruptions I have grown used to on *******" - Comment from Mark
"I just wanted to let u know that your data feed/service is by far the best!!! Your unfiltered tick data is excellent for reading order flow and none of your competitors delivers this quality of data!" - Comment from Peter via Email
"As a past ******* customer(and not a happy one), IQ Feed by DTN is a much better and cheaper product with great customer support. I have had no problems at all since switching over." - Comment from Public Forum
"I just wanted to let you know how fast and easy I found it to integrate IQFeed into our existing Java code using your JNI client. In my experience, such things almost never go so smoothly - great job!" - Comment from Nate
"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"DTN feed was the only feed that consistently matched Bloomberg feed for BID/ASK data verification work these past years......DTN feed is a must for my supply & demand based trading using Cumulative Delta" - Comment from Public Forum Post
"This is an excellent value, the system is generous (allowing for 500 stocks) and stable (and really is tick-by-tick), and the support is fantastic." - Comment from Shirin via Email
"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
Home  Search  Register  Login  Blogs Recent Posts

Information on Various DTN Products:
DTN IQFeed | DTN ProphetX | DTN Ag | NxCore
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Streaming interval bars delays
Author Topic: Streaming interval bars delays (3 messages, Page 1 of 1)

somebody
-Interested User-
Posts: 6
Joined: Nov 22, 2016


Posted: Mar 29, 2017 05:20 PM          Msg. 1 of 3
Hi,

I'm testing 1-min streaming bars (derivative data, "BW" requests) and I'm seeing couples issues:

1. Delays as long as 3 minutes long. In fact, I can get the same data much sooner by requesting historical data for the last 1-2 minutes.
For example, received at 17:35:54:
BC,SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,,

But I received the same data over 3 minutes earlier via history request, at 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

It's also possible to "cheat" by creating streaming bar watches and cancel them a minute later, then create them again and cancel them again - just to get the initial data on time... By doing so I'm able to get the initial BH history records sooner than the delayed watch records (BC), for example:
At 17:51:02: BH,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,0,
At 17:51:27: BC,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,,

My question is if there is a way to receive streaming data in the form of actual streaming, meaning immediately after each minute passes?
Note I'm testing this after-hours, which I'd assume would keep your servers less busy and able to provide data real-time?

2. In some cases both streaming bar and historical data arrive earlier than the time they indicate, for example:
Received at 17:35:59:
BC,SPY,2017-03-29 16:36:00,235.5100,235.5200,235.5100,235.5200,60566551,3660,,

Or another minute's history received 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

Subsequent data requests show that this 'early' data is final and no longer changes later, thus it seems to be finalized before the actual time covered by the specific period.
I'm guessing there may be some explanation to the minute's data being summarized and arriving early?

Note that my clock is synchronized with yours and generally correct, though above issues don't seem to be clock-related.

somebody
-Interested User-
Posts: 6
Joined: Nov 22, 2016


Posted: Mar 30, 2017 10:54 PM          Msg. 2 of 3
Correction to the 2nd issue:
After additional testing I see that streaming and history bars (like 60 seconds) are occasionally updated and resent with incremented data, mainly when the previous bar was sent a few seconds early.
I think it'd be better to receive a single bar once it's finalized, but the current approach is also acceptable once known.

The 1st issue with delayed "BC" stream still remains, though doesn't seem to happen as frequently during daytime when bars come pretty much within 1 second of each minute's ending, but still with some exceptions. The delays intensify after regular trading hours.

DTN_Tim Walter
-DTN Guru-
Posts: 1090
Joined: Apr 25, 2006


Posted: Apr 3, 2017 07:43 AM          Msg. 3 of 3
Hi, my apologies for the delayed reply.

Our goal with streaming bars was to make it so that you would always receive bars that would match our history if the two were later compared. The issue then became, how do you know when a bar has completed so that you can know to write it. The answer, due to the impossibility of syncing all clocks involved and the random latencies between connections, was to say that a bar complete message is only sent when a new message, with a time that is outside of the current bar, is received. So that is why BC messages are not timely.

We do have the update interval, that is discussed in our documentation, that attempts to help solve this. If you set your update interval to 60, you will always get a bar no later than 60 seconds after the last trade. For example, if you were requesting 1 minute bars and set your update interval to 15, a sample minute might go like this;

Note: Assuming this is the open and previous trades have not occurred for simplicity.

9:30:03 - A trade arrives - No message sent
9:30:06 - A trade arrives - No message sent
No trades arrive for 15 seconds after this, so we fire an update (BU) message at 9:30:21
9:30:46 - No trades since the last message, so do nothing
9:30:57 - A trade arrives - No message sent
9:31:12 - No trade for 15 seconds - another update (BU) message sent
9:31:15 - A trade arrives - This is the first trade outside of the 9:30 minute, so a bar complete (BC) message is sent for the 9:30 - 9:31 bar
9:31:29 - A trade arrives - no message
9:31:40 - A trade arrives - no message
9:31:53 - A trade arrives - no message
9:32:03 - A trade arrives - A bar complete message is sent here, since we never went 15 seconds without a message, no BU message was sent.

If the 15 second gap is too large for your app, just lower the update interval to whatever tolerance works for your need, but hopefully this will help illustrate the difficulties you are seeing and how you can help to resolve them.

Tim
 

 

Time: Sat November 18, 2017 5:06 PM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003