Hi,

I'm testing 1-min streaming bars (derivative data, "BW" requests) and I'm seeing couples issues:

1. Delays as long as 3 minutes long. In fact, I can get the same data much sooner by requesting historical data for the last 1-2 minutes.
For example, received at 17:35:54:
BC,SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,,

But I received the same data over 3 minutes earlier via history request, at 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

It's also possible to "cheat" by creating streaming bar watches and cancel them a minute later, then create them again and cancel them again - just to get the initial data on time... By doing so I'm able to get the initial BH history records sooner than the delayed watch records (BC), for example:
At 17:51:02: BH,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,0,
At 17:51:27: BC,SPY,2017-03-29 17:51:00,235.5200,235.5200,235.5200,235.5200,61804158,956,,

My question is if there is a way to receive streaming data in the form of actual streaming, meaning immediately after each minute passes?
Note I'm testing this after-hours, which I'd assume would keep your servers less busy and able to provide data real-time?

2. In some cases both streaming bar and historical data arrive earlier than the time they indicate, for example:
Received at 17:35:59:
BC,SPY,2017-03-29 16:36:00,235.5100,235.5200,235.5100,235.5200,60566551,3660,,

Or another minute's history received 17:32:22:
60-SPY,2017-03-29 17:33:00,235.4800,235.4800,235.4800,235.4800,61039289,10000,0,

Subsequent data requests show that this 'early' data is final and no longer changes later, thus it seems to be finalized before the actual time covered by the specific period.
I'm guessing there may be some explanation to the minute's data being summarized and arriving early?

Note that my clock is synchronized with yours and generally correct, though above issues don't seem to be clock-related.

Correction to the 2nd issue:
After additional testing I see that streaming and history bars (like 60 seconds) are occasionally updated and resent with incremented data, mainly when the previous bar was sent a few seconds early.
I think it'd be better to receive a single bar once it's finalized, but the current approach is also acceptable once known.

The 1st issue with delayed "BC" stream still remains, though doesn't seem to happen as frequently during daytime when bars come pretty much within 1 second of each minute's ending, but still with some exceptions. The delays intensify after regular trading hours.

Hi, my apologies for the delayed reply.

Our goal with streaming bars was to make it so that you would always receive bars that would match our history if the two were later compared. The issue then became, how do you know when a bar has completed so that you can know to write it. The answer, due to the impossibility of syncing all clocks involved and the random latencies between connections, was to say that a bar complete message is only sent when a new message, with a time that is outside of the current bar, is received. So that is why BC messages are not timely.

We do have the update interval, that is discussed in our documentation, that attempts to help solve this. If you set your update interval to 60, you will always get a bar no later than 60 seconds after the last trade. For example, if you were requesting 1 minute bars and set your update interval to 15, a sample minute might go like this;

Note: Assuming this is the open and previous trades have not occurred for simplicity.

9:30:03 - A trade arrives - No message sent
9:30:06 - A trade arrives - No message sent
No trades arrive for 15 seconds after this, so we fire an update (BU) message at 9:30:21
9:30:46 - No trades since the last message, so do nothing
9:30:57 - A trade arrives - No message sent
9:31:12 - No trade for 15 seconds - another update (BU) message sent
9:31:15 - A trade arrives - This is the first trade outside of the 9:30 minute, so a bar complete (BC) message is sent for the 9:30 - 9:31 bar
9:31:29 - A trade arrives - no message
9:31:40 - A trade arrives - no message
9:31:53 - A trade arrives - no message
9:32:03 - A trade arrives - A bar complete message is sent here, since we never went 15 seconds without a message, no BU message was sent.

If the 15 second gap is too large for your app, just lower the update interval to whatever tolerance works for your need, but hopefully this will help illustrate the difficulties you are seeing and how you can help to resolve them.

Tim