Hi Support,

I am using 4.1.1.1 and request history 5 min bar for 6 days around 9:40PM PST 2/22/2006.

HM,C,6,5; But I only got one day data
2006-02-21 16:05:00,46.17,46.17,46.17,46.17,13681100,252700
to
2006-02-21 09:35:00,46.24,46.18,46.20,46.18,732700,724400

HM,INTC,6,5; I only got 2 days data
2006-02-22 16:05:00,20.1600,20.1420,20.1600,20.1480,136855393,481536
to
2006-02-21 09:35:00,20.7400,20.6100,20.6600,20.6600,3583206,3420856

Could you let me know if anything is wrong?

Thank you.

The Minute and Tick history requests do not account for days the exchanges are closed.

HM,C,6,5; is going to return the last 6 days of data which will be Thursday, Wednesday, Tuesday, Monday (closed holiday), Sunday (closed), Saturday (closed).

However this doesn't account for what you were seeing last night since at that time, 6 days of data should have included friday as well.

I spoke with our support department and they didnt recieve any other reports of this behavior.

It appears everything is working currently. Are you still experiencing problems?

Steve,

Thank you. I will try tongiht in the same time frame.

Thanks.

Hi Steve,

I tried the same command to request "C" and "INTC" 6 days 5Min bar data at 9:20PST. I only got bars for the two days 2/21 and 2/22, the bars for the day 2/23 is missing.

Please check it.

Thanks.

Ok, I have a little more information as to what is going on. I missed in your original post that you are PST.

All of the servers run on exchange time (EST). So when you request 6 days at 9:20PM PST, this is 12:20AM EST. The server includes the current day of trading with minute and tick requests even if the symbol has not traded on that day. So, the 2 days of data you were recieving on Wednesday night (thursday morning server time) is correct.

This still doesnt explain why you may have been missing data for 2/23 last night (this morning server time).

After doing a little digging into this, there is a process that runs on the server just after midnight EST that will cause a few contracts to be unavailable at times however, if this is the case it should not be returning data at all for the contracts affected.

We will continue to look into this and monitor the data as well as monitor other related customer issues however, I would recommend that if possible, you change the time that your program runs.

Hi Steve,

I sent the 5Min bar request last night 2/23/2006 at 9:20PM PST. It should return the bar data for the 2/23/2006, right? But it only give me 2/21 and 2/22.

Thanks.

Correct. It should have returned data for 2/23 as well.

Hi Steve,

We still can't see the today's 5M bar data in the time period from 9 PM PST to 10PM PST. Most of our customers are from West Coast which is in Pacific timezone. We got a lot of complains about the missing data in the period. If there are jobs running in 9 PM PST(12AM EST), could you reschedule them to run in the 12 AM (PST) which is 3 AM(EST).

Thanks.

Rescheduling seems to be a viable solution, however there are many more factors to consider before we make any kind of changes. Primarily, how it will adversly affect the rest of our customers throughout the world.

Reading back through this thread, I want to make sure that your issue is actually being caused by the midnight processing.

Your issue seems to be slightly different from other reports which makes me believe that it may be something else entirely. Can you verify a few bits and pieces from this post for me?

Is it All symbols or is it only a few symbols?
Is it exactly 9PM-10PM every night? or does it vary from night to night?
Does it happen every night?
During this time, you are only missing the current day of data for these requests correct?
Can you see the 5min bars for the current day prior to 9PM?
Is it only 5min bars or is it all minute data or is it all historical data (ticks, daily, etc)?
Edited by DTN_Steve_S on Apr 20, 2006 at 04:41 PM

Hi Steve,

The following are the answers to your questions

1. Is it All symbols or is it only a few symbols?
From 9PM PST, all symbols for 5Min bars will start missing bars for current day.
But gradually, the missing bars for current day will show in the sequence from Z to A.
For example,
for VLO, from 9 PM PST to 9:15 PM PST will miss bar data for current day
for INTC, from 9 PM PST to 9:40 PM PST will miss bar data for current day
for AAPL, from 9 PM PST to 10:10 PM PST will miss bar data for current day

2. Is it exactly 9PM-10PM every night? or does it vary from night to night?
Not exactly ending with 10PM, but the issue starting from 9PM. No, it doesn't very from night to night.

3. Does it happen every night?
Yes, it happens every night.

4. During this time, you are only missing the current day of data for these requests correct?
Yes, only current day is missing. If I request bars for 4 days, it will return 3 days except current day.

5. Can you see the 5min bars for the current day prior to 9PM?
Yes, I can see 5 Min bars for current day prior to 9PM

6. Is it only 5min bars or is it all minute data or is it all historical data (ticks, daily, etc)?
The issue is only for 1Min and 5Min, I didn't try ticks data. Daily bars data are OK.

I hope those info will help.

Thanks.

Hello, It does appear that this should affect all (and only) minute data and is related to the server midnight processes. We will continue to look into a better method of doing this processing so that it minimises user impact.

Thank you for the very detailed report.
Edited by DTN_Steve_S on Apr 26, 2006 at 08:40 AM

Hi Steve,

Is there any update for the issue? It is 10 months from when I reported the issue.

Our clients are complaining about it consistently. Most of our clients are in west coast and busy to use our application around the 9PM PST(12AM EST).

Thanks,
Jim