Are you guys aware of any reason why history requests could be slow?

A particular app grabs a few days of 1-min data for it's symbols on startup, today that was by 8:30 NYSE time.

A lot of days it will blow through a list of around 400 symbols in a few minutes, but days like today, it gets to 125 or so and just sits there. Every once in a while it will trickle in another 1 or 2, and takes FOREVER. This is usually a one-time thing and only needs the data once, I make the rest myself from quotes in order NOT to hammer the servers. UPDATE: once the market opened today, it blew through the remaining 200 symbols.... weird. You'd think stuff would slow down then. This login is on 66.112.156.110

Data speeds to other locations are still fine, CPU hardly used, and have plenty of free bandwidth here. Is this common during high traffic market days? The history has been very intermittent.

Eric

For me, it's quite common to see history requests time out. I have an end-of-day history fetch running right now, and it looks like the timeouts are occurring every 300 to 500 symbols.

Oddly, this started suddenly about three years ago. Before then I never had any problems.

I just modified my apps to retry after ten seconds have passed without receiving any history data, and everything seems fine.

Allen

I currently just request the entire batch at once, each with a unique ID. Most of... er... much of the time, they just stream in fine. However sometimes there will be a huge pause, sits at, say, 150 received of 500 requested.... then a minute later, they start to stream in again. Occasionally, there will even be an error for a symbol near the pause, although I'll have to check the exact error next time it happens.

If there IS some kind of throttle, it would just be good to know so I can at least stop trying to find a problem. Otherwise I could try something like that re-request.

I've been seeing similar behavior for about 5-6 weeks. This happened a year or two ago, then the problem stopped happening. Re-requesting eventually gets the feed going again, but this problem has really been slowing my app's startup down.

Re the slowdown: Yes, that's a problem for me, too. I work around it by issuing history requests from multiple threads, so that if one stalls the app as a whole continues to make progress. It's extremely rare to have a stall on more than one thread at the same time.

Allen

Hello everyone. I just wanted to chime in here and let you guys know that we are looking into this currently and it is a high priority to make sure the history servers are performing up to par.

History speeds are TERRIBLE today... most opportunity comes in these big market days, and we're stuck not trading because 2 days of 1-min data are flowing in a couple symbols PER MINUTE. Apps not even close to loading what they need for the day. Last week was pretty darn decent, but then all the trading opportunity will probably be today, as usual.

Hello, we are not seeing any obvious issues on our end currently (nor were we seeing anything obvious from our logs around the time you posted this). Can you send me your loginID that you are using and some examples of the requests that were slow?

Feel free to send this via a PM on the forums if you don't want to post this information on the public forum.

Sorry I didn't reply to this; I didn't have the data easily accessible. Historical requests have been very slow for me all morning today; not sure if you can see anything on your side.

Jim

Hello Jim, we made some adjustments on our end that should have aleviated the slowness. Please let us know if you continue to have problems.

Steve, it's been good, but today again historical requests are very slow.

Jim

The last couple days have definitely been much slower.

We usually budget about 3 minutes for the initial history load before the open. Lately it's been more like 15-20+ minutes.

Was there some kind of throttle put on HIT (and similar) requests?

Thanks for the reports. We are aware of current issues with the history system being slower than usual. Yesterday we had to take one of our history systems offline due to data corruption caused upstream from the IQFeed servers. We are working on getting that system restored as quickly as possible at which time we should be back to full operating capacity.