Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"And by the way, have to say this. I love the IQFeed software. It's rock solid and it has a really nice API." - Comment from Thomas via RT Chat
"Thanks for following up with me. You guys do a great job in tech support." - Comment from Phelps
"Version 4.0.0.2 has been working well for me and I appreciate that it is now a much tighter client to work with. I feel I can go to press with my own application and rely on a stable platform" - Comment from David in IA.
"Thanks for all of your help. Great customer service deserves to be recognized which one the reasons I've been a customer of DTN for over 10 years!" - Comment from Stuart
"My broker in Davenport suggested I give you a try as he uses your service and says its the best." - Comment from Bill via RT Chat
"After all the anxiety I had with my previous data provider it is a relief not to have to worry about data speed and integrity." - Comment from Eamonn
"Previously I was using *******. IQFeed is WAY more economical, and for my charting needs is just as good, if not better." - Comment from Public Forum Post
"I am very happy I changed. I love the product, but more so I am thrilled with Tech Support. You are knowledgeable, polite, pleasant and professional." - Comment from Pat
"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"I am a hedge fund manager here. It’s funny, I have a Bloomberg terminal and a Bridge feed, but I still like having my DTN feed!" - Comment from Feras
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »NEW IQFEED FORUMS »Data Questions »Only TRADES for "HIT" historical data requests?
Author Topic: Only TRADES for "HIT" historical data requests? (15 messages, Page 1 of 1)

bbmat
-Interested User-
Posts: 30
Joined: Mar 31, 2014


Posted: Dec 22, 2019 06:39 AM          Msg. 1 of 15
Am I correct in my assumption that HIT historical data requests are built from trades?

My question then is whether I can request historical bar/ohlc data that are not comprised of trades but of bid and ask prices? For example, a 1-minute bar of bids would contain the open bid in that 1-minute time span, the highest bid, the lowest bid and the closing bid. How can I obtain ohlc data based on bids or offers?

The historical tick data requests are not really suitable to my need as I sometimes need to request data that are older than what tick data allow access to.

altmany
-Interested User-
Posts: 73
Joined: Jul 30, 2018

IQML - IQFeed-MATLAB connector


Posted: Dec 22, 2019 06:56 AM          Msg. 2 of 15
It is my understanding that only historic trades are stored/reported by IQFeed. The amount of data that would be required to store non-trading ticks would be simply enormous and so AFAIK IQFeed does not offer it.

You can create your own database of bid/ask OHLC bars by streaming all quotes using the w[symbol] (don't use t[symbol] because this streams only trades).

Yair Altman
IQML - IQFeed-MATLAB connector
https://UndocumentedMatlab.com/IQML

I am not a DTN employee; my post reflects my personal opinion

bbmat
-Interested User-
Posts: 30
Joined: Mar 31, 2014


Posted: Dec 22, 2019 07:46 AM          Msg. 3 of 15
I guess you are referring to tick data requests (HTT) requests. I just tested and yes, HTT tick data requests only return trades.

But I was specifically referring to open-high-low-close bar data at a minimum granularity of 1-minute. It is not an enormous data requirement and would only require 8 additional time series per symbol on their server to store, 4 for the bid (o/h/l/c) and 4 for the offer (o/h/l/c).

If the IQFeed API does not allow for the retrieval of bids and offers then the historic data function are completely worthless for options. Most options trade at most once even per hour.

I need the bid and offer to align with the bid and offer of the underlying in order to generate a volatility surface. Only being able to obtain historical option trades makes absolutely no sense as most option trades are stale.

altmany
-Interested User-
Posts: 73
Joined: Jul 30, 2018

IQML - IQFeed-MATLAB connector


Posted: Dec 22, 2019 08:25 AM          Msg. 4 of 15
I don't want to answer in DTN's stead, but I think you are underestimating the required effort to have HIT support for non-trading ticks. Remember that HIT enables specifying an interval of not just 60 secs but even a couple of secs, as well as any number of ticks (i.e., not a fixed-duration interval) and volume. To support all of these combinations with a custom interval size by the users, IQFeed would be required to store and process every single tick. This may not seem much for out-of-the-money options, but it would be an enormous amount of data for Forex options or S&P 500 equities (for example). In short, except for certain asset classes, it is an unrealistic expectation to support. And if IQFeed were to support non-trading ticks with only these certain asset types, users would naturally complain that all other asset types are not similarly supported.

Note that I am not a DTN employee and this is just my own personal opinion.

Yair Altman
IQML - IQFeed-MATLAB connector
https://UndocumentedMatlab.com/IQML

I am not a DTN employee; my post reflects my personal opinion

bbmat
-Interested User-
Posts: 30
Joined: Mar 31, 2014


Posted: Dec 22, 2019 08:36 AM          Msg. 5 of 15
Well, I never argued that quote data are not resource intense. Hence there is no point to further argue about this. I just need to know whether historical option bid and ask prices are anywhere available, via additional premium subscription or not. If not I will need to cancel my service and api subscription as I need those data now.

Other data providers seem to offer it. Interactive Brokers does but it has too restrictive data request throttles and pacing. But livevol pro does offer historical option quotes,not just trades.

So again, my question to dtn staff is: are historical option quotes (bid/ask) available, and I don't mean the bid and ask quotes that are stored alongside trades in HTT responses.

DTN_Gary_Stephen
-DTN Guru-
Posts: 394
Joined: Jul 3, 2019


Posted: Dec 22, 2019 02:51 PM          Msg. 6 of 15
BBMat,

Currently, the HIT command will return only data from trades, not bids/asks. It sounds like you want a history of bid/ask data, correct? The forthcoming IQFeed 6.2 will have expanded book capabilities, but I don't know the details of yet.

Sincerely,
Gary Stephen
DTN IQFeed API Support

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

bbmat
-Interested User-
Posts: 30
Joined: Mar 31, 2014


Posted: Dec 22, 2019 08:04 PM          Msg. 7 of 15
Hi Gary,

So, can you confirm that iqfeed as of right now does not allow for the retrieval of any historical bid/ask quotes?

DTN Todd
-Interested User-
Posts: 74
Joined: Mar 24, 2010


Posted: Dec 22, 2019 08:09 PM          Msg. 8 of 15
Yes that is correct, bid/ask historical data is not currently available

Thanks

bbmat
-Interested User-
Posts: 30
Joined: Mar 31, 2014


Posted: Dec 22, 2019 08:10 PM          Msg. 9 of 15
Thanks, Todd

rob2
-Interested User-
Posts: 1
Joined: Nov 24, 2020


Posted: Nov 24, 2020 04:59 PM          Msg. 10 of 15
It looks like this historic bid/ask interval data is not part of the 6.2 release. Are there any future plans for it? Even limited data (e.g. bid/ask on wider intervals or HD* messages) would be great.

As others have said, historic option trade prices are not very useful for detailed analysis.

thanks

andrewm
-Interested User-
Posts: 52
Joined: Feb 23, 2015


Posted: Apr 7, 2021 10:52 PM          Msg. 11 of 15
Being able to see historical bid/ask changes only when a trade happens makes intraday analysis of thinly traded products impossible.

It forces users who need this data to stream subscriptions and capture them in real time locally. That's an unreliable solution over the internet and something users should not be wasting their time building.

IQ Feed could snapshot the BBO once per second and make that historical data available via the API or better as an EOD FTP file for subscribers. It could be a replay of a realtime feed that coalesces ticks into buckets sent at a maximum update frequency of 1 sec. Even 5 second snapshots would be ok.

For now I call a python script every 5 mins to dump the 5MS output for CBOE to a file and compress it. If anyone wants that script, let me know.
Edited by andrewm on Apr 7, 2021 at 10:53 PM

stargrazer
-DTN Guru-
Posts: 302
Joined: Jun 13, 2005

Right Here & Now


Posted: Apr 7, 2021 11:49 PM          Msg. 12 of 15
Quote: It forces users who need this data to stream subscriptions and capture them in real time locally. That's an unreliable solution over the internet and something users should not be wasting their time building.


I would say that many have done so, including me.

Quote: IQ Feed could snapshot the BBO once per second


You'll find that when you use that mechanism for liquid instruments, the quality of data becomes lacking due to the high frequency aspects of the data.

FWIW, I believe DTN's choice of quote at trade storage is a reasonable trade-off.

Collecting data real-time during the day allows you to catch the stream as it happens with all its brutality and uneveness. Which is required for proving any sort of trading system 'in the natural'.

https://github.com/rburkholder/trade-frame
Edited by stargrazer on Apr 7, 2021 at 11:52 PM

andrewm
-Interested User-
Posts: 52
Joined: Feb 23, 2015


Posted: Apr 8, 2021 12:27 AM          Msg. 13 of 15
> You'll find that when you use that mechanism for liquid instruments
> the quality of data becomes lacking due to the high frequency aspects of the data.

Agreed that it's not perfect however it's much better than nothing which is effectively what we have now for historical intraday quotes on thinly traded products such as nearly all equity option contracts.

> Collecting data real-time during the day allows you to catch the stream as it happens

Except:

You can only collect as many symbols as your account is permissioned for.
If you decide to test a new strategy on 6 months of historical data you must first spend 6 months collecting it.
If you want to know how your new strategy would have performed during some historical event when you weren't collecting data on that particular symbol, too bad.

> I believe DTN's choice of quote at trade storage is a reasonable trade-off.

It may work for @ES and SPY but for a product that rarely trades it's mostly useless. The data storage requirements for snapshotting the BBO would be minimal since snapshots would only need to include changed quotes. Even if the symbol's quote changed every second, 23 hours a day and 100 byte records were stored that's only 8MB/day/symbol. Actual sizes would be far smaller.

DTN_Gary_Stephen
-DTN Guru-
Posts: 394
Joined: Jul 3, 2019


Posted: Apr 8, 2021 08:57 AM          Msg. 14 of 15
DTN has received several requests to add more historical bid/ask data to IQFeed. There are no immediate plans to do this, but we are aware of demand for it.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

Seb
-Interested User-
Posts: 1
Joined: May 11, 2023


Posted: May 11, 2023 04:53 AM          Msg. 15 of 15
Upvoting demand for this 2 years on, during my trial. As suggested above, best bid ask changes would suffice to keep track of illiquid securities like most option contracts. Now I am likely also looking for alternative data sources (IB, QC) or also record these eventually.
 

 

Time: Fri April 19, 2024 4:36 AM CFBB v1.2.0 14 ms.
© AderSoftware 2002-2003