Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
"After all the anxiety I had with my previous data provider it is a relief not to have to worry about data speed and integrity." - Comment from Eamonn
"Can I get another account from you? I am tired of ******* going down so often" - Comment from George
"Thank you so much - awesome feed, awesome service!" - Comment from Greg via Email
"If you want customer service that answers the phone, your best bet is IQFeed. I cannot stop praising them or their technical support. They are always there for you, and they are quick. I have used ****** too but the best value is IQFeed." - Comment from Public Forum
"If you are serious about your trading I would not rely on IB data for serious daytrading. Took me a while to justify the cost of IQ Feed and in the end, it's just a 2 point stop on ES. Better safe than sorry" - Comment from Public Forum
"You have an excellent feed. Very few spikes for Spot Forex." - Comment from Public Forum Post
"Boy, probably spent a thousand hours trying to get ******* API to work right. And now two hours to have something running with IQFeed. Hmmm, guess I was pretty stupid to fight rather than switch all this time. And have gotten more customer service from you guys already than total from them… in five years." - Comment from Jim
"The service is great, I see a noticeable improvement in my volume profiles over [broker]'s data feed" - Comment from Larry
Home  Search  Register  Login  Blogs Recent Posts

Information on Various DTN Products:
DTN IQFeed | DTN ProphetX | DTN Ag | NxCore
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Streaming interval bars delays
Author Topic: Streaming interval bars delays (3 messages, Page 1 of 1)

somebody
-Interested User-
Posts: 6
Joined: Nov 22, 2016


Posted: Mar 29, 2017 05:20 PM          Msg. 1 of 3
Hi,

I'm testing 1-min streaming bars (derivative data, "BW" requests) and I'm seeing couples issues:

1. Delays as long as 3 minutes long. In fact, I can get the same data much sooner by requesting historical data for the last 1-2 minutes.
For example, received at 17:35:54:
BC,SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,,

But I received the same data over 3 minutes earlier via history request, at 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

It's also possible to "cheat" by creating streaming bar watches and cancel them a minute later, then create them again and cancel them again - just to get the initial data on time... By doing so I'm able to get the initial BH history records sooner than the delayed watch records (BC), for example:
At 17:51:02: BH,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,0,
At 17:51:27: BC,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,,

My question is if there is a way to receive streaming data in the form of actual streaming, meaning immediately after each minute passes?
Note I'm testing this after-hours, which I'd assume would keep your servers less busy and able to provide data real-time?

2. In some cases both streaming bar and historical data arrive earlier than the time they indicate, for example:
Received at 17:35:59:
BC,SPY,2017-03-29 16:36:00,235.5100,235.5200,235.5100,235.5200,60566551,3660,,

Or another minute's history received 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

Subsequent data requests show that this 'early' data is final and no longer changes later, thus it seems to be finalized before the actual time covered by the specific period.
I'm guessing there may be some explanation to the minute's data being summarized and arriving early?

Note that my clock is synchronized with yours and generally correct, though above issues don't seem to be clock-related.

somebody
-Interested User-
Posts: 6
Joined: Nov 22, 2016


Posted: Mar 30, 2017 10:54 PM          Msg. 2 of 3
Correction to the 2nd issue:
After additional testing I see that streaming and history bars (like 60 seconds) are occasionally updated and resent with incremented data, mainly when the previous bar was sent a few seconds early.
I think it'd be better to receive a single bar once it's finalized, but the current approach is also acceptable once known.

The 1st issue with delayed "BC" stream still remains, though doesn't seem to happen as frequently during daytime when bars come pretty much within 1 second of each minute's ending, but still with some exceptions. The delays intensify after regular trading hours.

DTN_Tim Walter
-DTN Guru-
Posts: 1102
Joined: Apr 25, 2006


Posted: Apr 3, 2017 07:43 AM          Msg. 3 of 3
Hi, my apologies for the delayed reply.

Our goal with streaming bars was to make it so that you would always receive bars that would match our history if the two were later compared. The issue then became, how do you know when a bar has completed so that you can know to write it. The answer, due to the impossibility of syncing all clocks involved and the random latencies between connections, was to say that a bar complete message is only sent when a new message, with a time that is outside of the current bar, is received. So that is why BC messages are not timely.

We do have the update interval, that is discussed in our documentation, that attempts to help solve this. If you set your update interval to 60, you will always get a bar no later than 60 seconds after the last trade. For example, if you were requesting 1 minute bars and set your update interval to 15, a sample minute might go like this;

Note: Assuming this is the open and previous trades have not occurred for simplicity.

9:30:03 - A trade arrives - No message sent
9:30:06 - A trade arrives - No message sent
No trades arrive for 15 seconds after this, so we fire an update (BU) message at 9:30:21
9:30:46 - No trades since the last message, so do nothing
9:30:57 - A trade arrives - No message sent
9:31:12 - No trade for 15 seconds - another update (BU) message sent
9:31:15 - A trade arrives - This is the first trade outside of the 9:30 minute, so a bar complete (BC) message is sent for the 9:30 - 9:31 bar
9:31:29 - A trade arrives - no message
9:31:40 - A trade arrives - no message
9:31:53 - A trade arrives - no message
9:32:03 - A trade arrives - A bar complete message is sent here, since we never went 15 seconds without a message, no BU message was sent.

If the 15 second gap is too large for your app, just lower the update interval to whatever tolerance works for your need, but hopefully this will help illustrate the difficulties you are seeing and how you can help to resolve them.

Tim
 

 

Time: Tue January 16, 2018 1:21 PM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003