Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"I use IQ Feed, Great stuff as far as data analysis information, storage and retrieval is concerned." - Comment from Public Forum
"I was on the phone with a friend who uses CQG and right after the Fed announcement, CQG was as much as 30 seconds behind DTN.IQ. Some quotes were off by as much as 15-18 cents. Your feed never missed a beat." - Comment from Roger
"DTN has never given me problems. It is incredibly stable. In fact I've occasionally lost the data feed from Interactive Brokers, but still been able to trade because I'm getting good data from DTN." - Comment from Leighton
"For anyone considering using DTN.IQ for a data feed, my experience with the quality of data and the tech support has been very positive." - Comment from Public Forum
"Very impressed with the quality of your feed - ******* is a real donkey in comparison." - Comment from A.C. via Email
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
"Previously I was using *******. IQFeed is WAY more economical, and for my charting needs is just as good, if not better." - Comment from Public Forum Post
Home  Search  Register  Login  Blogs Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »No way to query historical (expired) equity option chain symbols?
Author Topic: No way to query historical (expired) equity option chain symbols? (59 messages, Page 2 of 2)

AK786
-Interested User-
Posts: 21
Joined: Jul 17, 2018


Posted: Aug 17, 2018 06:24 PM          Msg. 31 of 59
Tim,

This message is with reference to your post from:
***
Posted: Aug 2, 2018 07:35 AM Quote Post Reply To Post Msg. 18 of 30
There is some work being done on our website currently and that bumped the automation back, but I did place a file up a couple weeks ago for everything that was expired up to that date if you want to take a look at it.

ftp://www.dtniq.com/beta/IEOPTION.zip
***

Goal:
Get Options Tick Data for Expired Options and Current Options for the 6 months.

I downloaded this file and parsed the file. However the last trade date is 20180713

Q1. I am assuming that ftp://www.dtniq.com/beta/IEOPTION.zip contains only Option Symbols that have expired and the last date they traded? For example, 20180710 has only 24103 symbols as you can see below and
http://www.dtniq.com/product/mktsymbols_v2.zip file contains approx 900k option symbols which are all currently trading symbols.

Can you please verify this assumption is accurate

Q2. Can you kindly update this file ftp://www.dtniq.com/beta/IEOPTION.zip until 20180817 (The last date is 20180713)

After this - i can get latest symbols daily from http://www.dtniq.com/product/mktsymbols_v2.zip and request IQFeed data.

Your help is very appreciated. Thank you.

Please see attached file: (IEOptionsFile_DateSym.xlsx)
Date Symbols
20180713 603
20180711 130
20180710 24159
20180709 112
20180707 385
20180706 403
20180703 27661
20180702 87
20180701 386
...






AK786

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Aug 18, 2018 09:15 AM          Msg. 32 of 59
Good morning,

Yes, your assumption is correct. I will try to get a new file created Monday morning for you when I get back in.

Tim

AK786
-Interested User-
Posts: 21
Joined: Jul 17, 2018


Posted: Aug 18, 2018 10:35 AM          Msg. 33 of 59
Thank you very much, your help is very much appreciated. Very satisfied with IQFeed Service.

AK786

AK786
-Interested User-
Posts: 21
Joined: Jul 17, 2018


Posted: Aug 19, 2018 04:49 PM          Msg. 34 of 59
Tim,

This is WRT the post from:
***
Posted: Aug 7, 2018 08:32 AM Quote Post Reply To Post Msg. 22 of 33
Actually, the expiration date is the blank field, the other date is the last traded date.

I'll correct the email above to be accurate. Expiration can be inferred from the symbol, but we will look into that prior to release as well.

Tim
***

I parsed out Sample Data from the IEOptions File from 20150101 to 20180713

Somehow, for the a Large Number of Options, The Last Traded Date is after the Expiration Date.

Can you kindly explain how that might have happened.

Please see attached. I know you will be generating the latest IEOptions FIle tomm, so I wanted to make sure your efforts do not go to waste.

Thank you

AK786

AK786
-Interested User-
Posts: 21
Joined: Jul 17, 2018


Posted: Aug 19, 2018 04:59 PM          Msg. 35 of 59
Also another question is how Far Options History do you carry, i tried requesting Options Tick Data between
<OptionsBeginDate value= "20180501"/>
<OptionsEndDate value= "20180713"/>

for all options in (From the above attached file) for Equity Symbol AAPL and i did not get a single data tick data history

AK786

J
-Interested User-
Posts: 24
Joined: Jun 27, 2017


Posted: Aug 19, 2018 05:12 PM          Msg. 36 of 59
@AK786,

Assuming you are paying for the OPRA feed, there is probably something wrong with how you are querying. The typical problem I observed has to do with naming the symbol correctly.

Here is a sample query that worked for me just now. I picked one of the first AAPL symbols from the ftp file mentioned above.

HDT,AAPL1001J210,20090101,,,1,AAPL1001J210_HISTORY_c7gxv4myzxvlt99c8kw6,

(Obviously the query identifier -- the string that has HISTORY in it-- is optional and can be omitted).
The reply I got was one line:

AAPL1001J210 2010-09-28T17:05:54 76.34 68.34 68.34 73.69 20 0

J
-Interested User-
Posts: 24
Joined: Jun 27, 2017


Posted: Aug 19, 2018 05:16 PM          Msg. 37 of 59
Oh right. You mentioned tick data. Try pulling 1 minute bars to validate the query first, then try for tick.

AK786
-Interested User-
Posts: 21
Joined: Jul 17, 2018


Posted: Aug 19, 2018 05:18 PM          Msg. 38 of 59
Thank you J - you are correct,

Tim,

Regarding this - I was making a mistake at my end.
This issue is resolved.

Please let me know about the other situation related to last trade date (after expiration) - its still not a problem, if its a system issue at your end.

Thanks

AK


****
Also another question is how Far Options History do you carry, i tried requesting Options Tick Data between
<OptionsBeginDate value= "20180501"/>
<OptionsEndDate value= "20180713"/>

for all options in (From the above attached file) for Equity Symbol AAPL and i did not get a single data tick data history
***

AK786

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Aug 20, 2018 01:35 PM          Msg. 39 of 59
Hi guys,

Well, evidently there is more work to be done on that file creation process. This file is all I have ready access to give you though, so while the symbols are up to date, these last trade dates do appear to be wrong. We will get that addressed before we go live of course. In the meantime, you could do an HDD request on each to get the last traded date. The expiration is in the symbol itself if you wanted to parse it out for the ones it is blank on, but otherwise when it is there, it appears to be correct.

The current file is on the FTP now.

Tim

mathpaquette
-Interested User-
Posts: 12
Joined: May 18, 2018


Posted: Aug 23, 2018 10:18 AM          Msg. 40 of 59
Thank you Tim,
I think we should keep the file option.. For me this is a more convenient way to retrieve expired options. Can you expose to us the long term goal get them at the end?

Thank you.
Mathieu

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Aug 27, 2018 07:32 AM          Msg. 41 of 59
Having the file is part of the long term view, we just need to finish it up and get it QA'd of course, so that when you are receiving it, it is correct.

Tim

mathpaquette
-Interested User-
Posts: 12
Joined: May 18, 2018


Posted: Aug 27, 2018 07:39 AM          Msg. 42 of 59
Excellent news Tim, I'm in the process of adding support for this file in the IQFeed.CSharpiApiClient wrapper. Will be very easy to use. Ill post back on this thread once available.

Mathieu

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Aug 27, 2018 08:03 AM          Msg. 43 of 59
We plan on putting in a header at the top as well, so you might want to consider for that as well.

Once I know where the file will live I will let everyone know.

Tim

AK786
-Interested User-
Posts: 21
Joined: Jul 17, 2018


Posted: Aug 27, 2018 11:26 AM          Msg. 44 of 59
To all IQFeed Forum Readers,

i just wanted to share that I have been using Matt P. (IQfeed Csharp Client) and its phenomenal. Tick Data download for all US Equities (Approx 8000) can get done within 15 mins / each day. Options data download which can have over a million queries for entire dataset - lightning fast even. I downloaded all US Equity Options for 6 months which has "Expired" + "Recently Traded" Options.

The Live Data Client runs on super low CPU and memory. Check out his code, he has saved me tons of time. I am very grateful to him for responding to my queries almost immediately and resolving minor bugs if any.

Thank you Matt P. and all his IQFeed github contributors.

AK786

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Sep 27, 2018 12:06 PM          Msg. 45 of 59
FYI a new file was posted today if anyone wanted to update their lists.

Tim

mathpaquette
-Interested User-
Posts: 12
Joined: May 18, 2018


Posted: Sep 27, 2018 08:27 PM          Msg. 46 of 59
Thank you so much Tim.
BTW guys, in the newest version 1.4.2 of IQFeed.CSharpApiClient, I've added a very simple way to retrieve those expired options and market symbols.

Code sample:
var lookupClient = LookupClientFactory.CreateNew();
var expiredOptions = lookupClient.Symbol.GetAllExpiredOptions()
.Where(x => x.EquityOption.Expiration >= DateTime.Now.AddMonths(-6)).ToList();

var marketSymbols = lookupClient.Symbol.GetAllMarketSymbols().ToList();

This will take some time the first time but the code will reuse the cached file and check for new update every time.

Hope this helps.
Mathieu

mathpaquette
-Interested User-
Posts: 12
Joined: May 18, 2018


Posted: Oct 4, 2018 09:52 AM          Msg. 47 of 59
I added more examples including one with Expired Options:
https://github.com/mathpaquette/IQFeed.CSharpApiClient/tree/master/src/IQFeed.CSharpApiClient.Examples

Thank you,
Mathieu

veng1
-Interested User-
Posts: 6
Joined: Oct 8, 2018


Posted: Oct 8, 2018 09:49 AM          Msg. 48 of 59
In using a modification of Mathieu's excellent client (Thanks, Mathieu) I'm seeing symbols that are in IEOPTION but appear to have no data. Could this be verified and, preferably, the data added or the symbols removed from the file.

SPY1018L180 !NO_DATA!
SPY1018L185 !NO_DATA!
SPY1030I71 !NO_DATA!
SPY1228L130 !NO_DATA!
SPX1018L550 !NO_DATA!
SPX1117L525 !NO_DATA!
SPX1118F550 !NO_DATA!
SPXW1531H1795 !NO_DATA!
VXX1228L23 !NO_DATA!
VXX1228X24 !NO_DATA!
AAPL1703K110 !NO_DATA!

Thanks.
Edited by veng1 on Oct 8, 2018 at 09:50 AM

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Oct 8, 2018 10:09 AM          Msg. 49 of 59
Daily data does exist for the ones I checked. They were likely an implied trade or something along those lines, since we did not create a minute bar, but that is why these are in the files.

I will check into how this is filtered with regards to symbols that receive no trading at all. But, I believe they are filtered.

Tim
Edited by DTN_Tim Walter on Oct 8, 2018 at 10:10 AM

veng1
-Interested User-
Posts: 6
Joined: Oct 8, 2018


Posted: Oct 8, 2018 12:02 PM          Msg. 50 of 59
Does the daily data actually include at least one trade?

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Oct 8, 2018 12:32 PM          Msg. 51 of 59
The ones I checked did, but I need to clarify something with regards to this file. It is not an official offering for us. The file and it's creation are not finalized. It has known bugs and we still have work to do on them, work that does not have a current ETA for completion.

We certainly want to be able to help and provide information for everyone to use. That is why we are here and Mathieu does a fantastic job with his project, staying up to date and in this case, ahead of our release schedule. But, I need to be clear that this is a process that in an alpha phase, not production, so please take everything in them with a few grains of salt.

Tim

veng1
-Interested User-
Posts: 6
Joined: Oct 8, 2018


Posted: Oct 8, 2018 01:42 PM          Msg. 52 of 59
I appreciate that this is an alpha product. My code traps an error to a file but does not attempt to save the trade to csv if there is "NO DATA" returned.

My post was intended to be constructive in an effort to help find bugs.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Oct 8, 2018 01:48 PM          Msg. 53 of 59
Absolutely, and there are a couple places that I put in research tasks towards that end as well based upon your notes. I just didn't want someone new to the thread coming in and misunderstanding what these files contain. But, at this time the plan is to have symbols be in the file if they have a minute, or daily data available. I did find that I cannot find any data on the AAPL symbol you reported, so I will add that to the task for additional research.

I say files, because eventually we plan to have this for all exchanges and security types, not just IEOPTION, but that is the primary one requested so we are starting with that.

veng1
-Interested User-
Posts: 6
Joined: Oct 8, 2018


Posted: Oct 8, 2018 02:24 PM          Msg. 54 of 59
I am particularly looking forward to expired futures contracts. My requirement is to calculate contango and backwardation. In an ideal world, there would be a continuous front month and continuous back months so the the term structure would be simple math.

If there are at least all the past symbols it would be easier.
Edited by veng1 on Oct 8, 2018 at 02:44 PM

stargrazer
-DTN Evangelist-
Posts: 249
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 8, 2018 02:47 PM          Msg. 55 of 59
Quote: In using a modification of Mathieu's excellent client (Thanks, Mathieu) I'm seeing symbols that are in IEOPTION but appear to have no data. Could this be verified and, preferably, the data added or the symbols removed from the file.


I am wondering what your expectations are. Have you followed options on the real time feeds? You'll note that there are continuous quotes available, but there are very infrequent trades happening. And if history only has the trades stored, the trades (if they are available), may not be indicative of the open, the close, the min, the max or anything in between. And that may be the reason why you see NO_DATA, because, well, the option was available, but no one traded anything. Ie, repeating myself, trades are very sparse in the options world.
Edited by stargrazer on Oct 8, 2018 at 03:04 PM

stargrazer
-DTN Evangelist-
Posts: 249
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 8, 2018 03:03 PM          Msg. 56 of 59
Quote: I am particularly looking forward to expired futures contracts. My requirement is to calculate contango and backwardation. In an ideal world, there would be a continuous front month and continuous back months


Due to the fact that trades can be sparse on back months on even popular symbols, I've found it of more value to capture and store daily feeds, keep the quotes as well as the trades, and then perform algorithm analysis based upon the quotes captured. I build my own daily summaries from the quotes, and typically use quote mid-points for rough approximations. This allows me to deal with open, close, and any time in between without worrying about the lack of trades.

You will also note that, as various sites say in the footers, past history may not be a good prognosticator of current movements. Which suggests even more value in collecting your own current data, complete with both quote and trade info, to use in your modeling and algorithm development.

veng1
-Interested User-
Posts: 6
Joined: Oct 8, 2018


Posted: Oct 8, 2018 03:36 PM          Msg. 57 of 59
After the demise of XIV, I'm looking at option trades for backtesting, primarily on EOD, with a vertical on VXX. I believe that the term structure needs, among other things that are needed, to do a synthetic short, exit or hold cash. The reason for 1 minute data is to look for block trades, not so much as price action.

The term structure for VIX can be calculated on daily basis. Whether there are suitable options trades remains to be seen. I don't much care to simply short VXX.

I agree in the value of collecting one's own data but that can only be done one day at a time. However, it is a fast trip to the poor house to trade based on past history without forward testing.

And yes, I've watched the tape on options, specifically on Livevol for literally years.
Edited by veng1 on Oct 8, 2018 at 03:38 PM

stargrazer
-DTN Evangelist-
Posts: 249
Joined: Jun 13, 2005

Right Here & Now


Posted: Oct 8, 2018 04:03 PM          Msg. 58 of 59
Quote: I'm looking at option trades for backtesting, primarily on EOD


I am curious. I gather you have already pulled down historical option tick data and/or historical option 1 minute bar data for your symbols of interest for what they supply currently. Do you get coverage through out the day, or is it pretty sparse? Do you see ticks or bars falling at EOD regularily?

veng1
-Interested User-
Posts: 6
Joined: Oct 8, 2018


Posted: Oct 9, 2018 08:03 AM          Msg. 59 of 59
Picking a random expiration, the most popular strike, by file size, SPXW1708L2570 has 40 EOD prices.

The same strike has 288 1Min quotes starting at 2017-10-12 10:55:00 and ending at 2017-12-08 15:43:00.

Interestingly, SPXW1708L2650_Min has 1866 quotes.

I'm not pulling tick data.
 
Page 2 of 2 Go to page: · 1 · [2] · Prev

 

Time: Tue October 23, 2018 5:53 PM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003