Joined: |
Apr 11, 2016 05:38 PM |
Last Post: |
Apr 5, 2018 08:07 AM |
Last Visit: |
Nov 26, 2018 04:24 AM |
Website: |
NA |
Location: |
Australia |
Occupation: |
Programmer |
Interests: |
Trading / Software |
Avatar: |
Trade on
|
|
|
lgwaustralia has contributed to 8 posts out of 18926 total posts
(0.04%) in 1,042 days (0.01 posts per day).
20 Most recent posts:
Thanks for the quick reply. Will try out tSymbol.
Lance
Hello.
Thanks for the reply. I see now that we can match realtime with historical by not processing realtime data that has Message Content that's not C.
Is there a way in realtime to just get Message Contents that are C so we don't have to filter on our end?
Thanks.
Lance
Hi there,
We have a system where we use Realtime tick data from IQFeed and then we download all the data for the day at the end of the day as Historical data from IQFeed.
We are seeing that they do not match. For example:
Here is data from a time period we got in realtime: (using command: wXG#) Q,XG#,04/04/2018,07:57:20.072000,11838.50,11839.50,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.50,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.00,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.50,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.00,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.50,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.00,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.00,11839.50,11838.50, Q,XG#,04/04/2018,07:57:20.072000,11838.50,11839.50,11838.50,
Here is the data from the same time period we got as historical data: (using command: HTT,XG#,20180404 000000,20180409 170500,,,,0,,) 2018-04-04 07:57:20.072000,11838.50,1,57349,11838.50,11839.50,3106867,C,68,01,
It seems that the historical data is consolidated some how. Is it possible to get the same resolution as realtime when downloading historical data?
Regards.
Lance
Tim,
Thanks for the response. How come IQFeed is not fixing these issues before we get the data.
Are you saying that Ninjatrader is adjusting the data in real-time as it comes in from your servers?
How do they know to do this?
Is there a flag in the data that says the following tick timestamps will all be out by X amount and you need to adjust it?
What other instruments does this happen with?
I was expecting that the data we got from IQFeed was high quality data, and did not contain issues like this that needed to be fixed by end users.
Regards.
Lance Edited by lgwaustralia on Mar 6, 2018 at 04:17 AM
Hi again.
I checked the dates in question above ninjatrader using iqfeed as source and loaded up the date range in question. FYI, the times are shown in localtime (PHT) in ninjatrader.
I found that the start times shown in ninjatrader seem to follow the daylight savings time. It changes start of open from 15:00 / 3pm PHT on March 24, 2017 (Friday) to a 14:00 / 2pm PHT on March 27, 2017 (Monday).
It then goes back to starting at 15:00 / 3pm PHT on October 30, 2017.
So how do we deal with this? Should we: - update the historical data we have in database to follow DST conventions (changes to 3am EST on march then goes back to 2am EST past last sunday of october) - OR change them so they will all start 2am EST.
Thanks.
Lance
Okay I see now. US goes, with dsts, 2nd sunday of march while they do last sunday of march and in between those dates it will appear as 3am. The same happens with the dates during transition back, last sunday of october for europe and first sunday of november in US.
Thanks.
Lance
Thanks for replying.
We are using symbol XG# with method HTT.
> but since markets don't trade Sunday morning, DST doesn't make us much difference. Ah since dst change happens on a sunday, then monday comes and should still open at 2am ET is that correct?
Thanks again.
Lance
I wanted to ask about the timestamps of some historical data. Also would like to ask about the timezone if it's still EST and if timestamps observer daylight savings time.
So i have this set of timestamps and trade opens at 2am EST, changes to 3am around november so that's the daylight savings, then goes back to 2am on nov 11. I thought it would have stayed at 3am until march when it reverts back to 2am: (Edit: pm to am)
(shows only specific dates where transition occurs) (transtion from 2p to 3p on nov2016) "2016-09-01 02:00:04.381" "2016-09-15 02:00:03.884" "2016-09-20 02:00:06.72" "2016-10-20 02:00:05.545" "2016-10-25 02:00:37.324" "2016-10-31 03:00:05.133" "2016-11-01 03:00:04.63" "2016-11-02 03:00:05.146" "2016-11-03 03:00:03.583" "2016-11-04 03:00:03.148" "2016-11-07 02:00:03.947" "2016-11-10 02:00:06.397" "2016-11-15 02:00:06.662" "2016-11-24 02:00:03.19" "2016-11-25 02:00:06.682" "2016-11-28 02:00:03.653" "2016-11-29 02:00:05.423" "2016-11-30 02:00:05.156"
(transition from 2p to 3p in mar2017) "2017-03-01 02:00:04.376" "2017-03-10 02:00:06.082" "2017-03-15 03:00:04.078" "2017-03-20 03:00:04.517" "2017-03-27 02:00:07.559" "2017-03-28 02:00:04.393" "2017-03-29 02:00:03.89" "2017-03-30 02:00:05.692"
(transition from 2p to 3p in nov2017) "2017-10-05 02:00:10.811" "2017-10-10 02:00:07.772" "2017-10-20 02:00:05.902" "2017-10-25 02:00:07.685" "2017-10-26 02:00:05.272" "2017-10-27 02:00:04.593" "2017-10-30 03:00:07.905" "2017-11-01 03:00:08.247" "2017-11-10 02:00:05.471" "2017-11-15 02:00:04.3" "2017-11-20 02:00:09.104" "2017-11-27 02:00:10.51" "2017-11-28 02:00:16.352" "2017-11-29 02:00:13.293"
Edited by lgwaustralia on Feb 27, 2018 at 07:34 AM
|
|