Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: ChainsawDR
About Contact
Joined: Jan 4, 2019 08:08 PM
Last Post: Nov 29, 2022 08:29 PM
Last Visit: Nov 29, 2022 08:29 PM
Website:  
Location: Seattle
Occupation: Product Manager
Interests:
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
ChainsawDR has contributed to 29 posts out of 21251 total posts (0.14%) in 2,165 days (0.01 posts per day).

20 Most recent posts:

Hi,

I'm aware that the standard account has a maximum of 500 stocks that can be subscribed to at any given time. Please can I ask if individual options (such as CPRI2209L56, CPRI2209L57, CPRI2209L57.5 etc) each count against the 500 limit?

Asking as I'm currently subscribing to ~400 stocks at any given time, and have started to watch option chains for some of the stocks in Excel via XLQ at the same time and wanted to understand if I need to ensure I don't go over the 500 overall limit.

Many thanks in advance

ChainsawDR

New IQFeed Forum » Full options chain command? Nov 22, 2022 01:31 PM (Total replies: 10)

Wow! Thank you both very much for your help!

@AndrewM - I was expecting to spend the next ~2 days chewing through this problem and you generously posting your code has just solved it completely. Thank you so much

New IQFeed Forum » Full options chain command? Nov 22, 2022 09:46 AM (Total replies: 10)

Hi Gary,

I came across this thread while trying to obtain latest bid/ask offers on equity options (ideally a snapshot of latest bid/ask on all option chains for all S&P500 component stocks). I've tried looking into the 5MS report but haven't been able to retrieve equity options data - do you have an example 5MS request please that would return a list of options with bid/ask?

Currently a request for "5MS,2,5" (which is I think '2,IEOPTION,Index/Equity Option,' & '5,NASDAQ,Nasdaq,5,NASDAQ') is returning a 'E,50007,No file available' error.

Otherwise, if there isn't an overall report that shows option chains I'll likely proceed down the OP's original path of retrieving the whole chain for various stock tickers, then iterate over them - subscribing and unsubscribing to each option.

Many thanks in advance

ChainsawDR

Data Questions » Historical Pre-Market Volume Jun 18, 2022 01:36 PM (Total replies: 14)

Thanks altmany. Yes understood. I originally had an older client, however updated to 6.2 prior to my recent posts. Unfortunately I'm still getting the issue when setting 6.2 or not setting a protocol at all, however specifying 6.1 seems to be working absolutely fine.

Data Questions » Historical Shares Outstanding Jun 17, 2022 08:26 PM (Total replies: 1)

Hi,
Please could you let me know if there is a way of retrieving the recorded shares outstanding data for a given symbol on a given day in the past? E.g. what was the AAPL Shares Outstanding on 2022/01/28? If not available, are there any equivalent/similar data points for this (e.g. Market Cap on 2022/01/28)?
Many thanks in advance
ChainsawDR

Data Questions » Historical Pre-Market Volume Jun 17, 2022 05:22 PM (Total replies: 14)

Firstly, thanks for all of your help with this Gary. Secondly, love your last update - great idea! To confirm, will it always choose the latest tick in the range? Also, am I correct in thinking that for a multi-day request like 06/15 to 06/16, it will only pick the latest across all days, or would it return the latest per day? (if the former I can just use python to cycle through each day).

Coming back to the main problem, here's the summary of what I'm seeing...
If I update my message to send an OLD protocol like 5.1...

message = "S,SET PROTOCOL,5.1\r\nHTT,%s,20220615 000000,20220616 130000,,092900,092959\n" % sym 


... then I get a good response back: S,CURRENT PROTOCOL,5.1 - plus I get all of the info for GOOG (still testing, but specifying an old protocol seems to be working). However if I take this exact same code and the only change I make is to update the protocol from 5.1 to 6.2, then it gets weird. A) I get this error response: E,6.2 is not a valid protocol. B) I now get 1 row of data for GOOG back (before specifying protocol, I used to get 0 rows back). This feels super weird - sending 6.2 is getting an error, but at the same time it is meaning I get 1 row of data for GOOG vs 0 rows of data.

All in all, one request I'd love to make if possible would be to add a basic sample code for Python to the IQ Developer Support section. I know a lot of experienced devs use C++ & Java, but Python is quite a common entry level language so hoping it could be supported to a MVP level like opening a socket, setting protocol, submitting a request (to take off the table situations like this when I'm wondering if its my coding or the system - for this it looks like a problem with the system, both with GOOG data responding differently and with accepting the latest protocol incorrectly).

Will crack on with trying to get it working using the old 5.1 approach, and loop back if no joy (but please shout if you already know there'll be an issue with using the old protocol).

Thanks again for your help Gary, really appreciate it!

ChainsawDR
Edited by ChainsawDR on Jun 17, 2022 at 05:24 PM

Data Questions » Historical Pre-Market Volume Jun 17, 2022 11:00 AM (Total replies: 14)

In case it helps. I tried setting the protocol as shown below, which generated the error further below:

Code:

# Construct the message needed by IQFeed to retrieve data
message = "S,SET PROTOCOL,6.2\nHTT,%s,20220616 000000,20220616 130000,,092900,092959\n" % sym
#also tried multiple variations such as message = "S,SET PROTOCOL,6.2<CR><LF>\nHTT,%s,20220616 000000,20220616 130000,,092900,092959\n" % sym

# Open a streaming socket to the IQFeed server locally
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))

# Send the historical data request
# message and buffer the data
sock.sendall(message.encode())
data = read_historical_data_socket(sock)
sock.close
print("First part of data:" + data[0:100])


Error:

Downloading symbol: SPY...
First part of data:E,6.2 is not a valid protocol.,
2022-06-16 09:29:59,370.4600,100,3609350,370.4300,370.4600,80847523


Data Questions » Historical Pre-Market Volume Jun 16, 2022 06:01 PM (Total replies: 14)

Thanks Gary. So the steps I'm currently performing are:
1) In Windows, open "IQFeed Applications" > Charts (which opens the connection to iqfeed from my computer)
2) In my Python IDE, I run the script shown above.

I have downloaded and installed the latest 6.2 version of the client and tried my existing code again, but no joy for GOOG again. TBH I'm not an experienced developer and I adapted the above script from a tutorial I found online. Do you know please if:
A) there is a way to set the protocol to 6.2 in step #1 above? Or
B) there's a method for adapting the above Python script to achieve this?

I've tried a few methods opening a socket to send that message, but the script stalls. I understand if not able to provide coding support (tried looking for sample files but unfortunately python isn't one of the languages listed).

Many thanks again

ChainsawDR

Data Questions » Historical Pre-Market Volume Jun 16, 2022 03:52 PM (Total replies: 14)

or to simplify the question, any ideas why requests for "HTT,GOOG,20220609 000000,20220610 130000,,092900,092959\n" do not work, but requests for others like "HTT,AMZN,20220609 000000,20220610 130000,,092900,092959\n" do work?
Edited by ChainsawDR on Jun 16, 2022 at 03:53 PM

Data Questions » Historical Pre-Market Volume Jun 15, 2022 09:08 PM (Total replies: 14)

Hi Gary,

Hoping this may be a quick one... I'm trying to execute in Python the command you shared. During testing I'm trying to download for 4 stock symbols, however requests for GOOG are not returning. Any ideas if I'm doing something wrong here, such as missing specifying the exchange or similar when making the HTT request?

# iqfeed.py

import sys
import socket


def read_historical_data_socket(sock, recv_buffer=4096):
"""
Read the information from the socket, in a buffered
fashion, receiving only 4096 bytes at a time.

Parameters:
sock - The socket object
recv_buffer - Amount in bytes to receive per read
"""
buffer = ""
data = ""
while True:
data = sock.recv(recv_buffer).decode('utf-8')
buffer += data

# Check if the end message string arrives
if "!ENDMSG!" in buffer:
break

# Remove the end message string
buffer = buffer[:-12]
return buffer

# iqfeed.py

# iqfeed.py

if __name__ == "__main__":
# Define server host, port and symbols to download
host = "127.0.0.1" # Localhost
port = 9100 # Historical data socket port
#syms = ["SPY", "AAPL", "GOOG", "AMZN"]
#syms = ["SPY", "AAPL", "GOOG", "AMZN"]
syms = ["AMZN"]

# Download each symbol to disk
with open("all.csv", mode='a+', encoding="utf8", newline='') as f:
for sym in syms:
print("Downloading symbol: %s..." % sym)

# Construct the message needed by IQFeed to retrieve data
message = "HTT,%s,20220609 000000,20220610 130000,,092900,092959\n" % sym

# Open a streaming socket to the IQFeed server locally
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))

# Send the historical data request
# message and buffer the data
sock.sendall(message.encode())
data = read_historical_data_socket(sock)
sock.close

# Remove all the endlines and line-ending
# comma delimiter from each record
#data = "".join(data.split("\r")) note: this is the original that doesn't include the symbol per line.
#data = data + ","
data = ("%s" % sym).join(data.split("\r"))
data = data.replace(",\n","%s \n" % sym)[:-1]

# Write the data stream to disk
# f = open("%s.csv" % sym, "w") note: this is the original that saves each in an individual file

# Move read cursor to the start of file.
f.seek(0)
# If file is not empty then append '\n'
countdata = f.read(100)
if len(countdata) > 0:
f.write("\n")
# Append text at the end of file
f.write(data)


Data Questions » Historical Pre-Market Volume Jun 15, 2022 09:40 AM (Total replies: 14)

Thank you very much Gary. Yes option 2 looks like it will solve the problem. Thanks again for your help!

Data Questions » Historical Pre-Market Volume Jun 8, 2022 09:16 AM (Total replies: 14)

I still have this use case that I'm trying to address - it looks like I will need to pull the data from the API. Does anyone have any advise on how to structure the iqfeed request? for example, if wishing to pull the MSFT pre-market volume on 6/7/22, is there a way of requesting the volume for 9.29am & 59seconds?

Thanks in advance for any help (wishing to avoid making too many requests to the API)

Data Questions » Historical Pre-Market Volume Dec 27, 2021 04:27 PM (Total replies: 14)

Hi,

My objective is to understand for the last 90-180+ days, which stocks had the highest pre-market trading volume. I'm currently using TradingView's screener of 4000+ nasdaq stocks and downloading it each day, building it up over time - which is obviously taking a lot of time...

I was hoping to ask please if there is an efficient way of obtaining this via iqfeed? I thought about using the API to pull the data for all NASDAQ stocks over the last 90-180 days, but wasn't sure if there was a better approach (and if I'd get flagged for pinging so many stock tickers).

Many thanks in advance for any guidance you can provide

ChainsawDR

New IQFeed Forum » Different Data on Tick vs 1min Nov 12, 2021 06:51 PM (Total replies: 10)

In case anyone else has this issue in future, there's more info on workaround steps on the NT8 forum thread here: https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1178301-corrupted-tick-data-iqfeed-downloads

Thanks

New IQFeed Forum » Different Data on Tick vs 1min Nov 12, 2021 02:44 AM (Total replies: 10)

Thanks for looking into this Todd

New IQFeed Forum » Different Data on Tick vs 1min Nov 11, 2021 09:36 PM (Total replies: 10)

In case it helps, these are the stock symbols (all nasdaq or nyse) that were corrupted during the overnight download:
IOVA
GPS
SAVE
EQT
DISCK
KEY
SM
NWSA
EQH
IVZ
HUN
DVN
SPWR
AES
NI
NVTA
VRT


I then deleted all historical data and redownloaded for these symbols, which fixed the issue for all except the following 5:
GPS
DISCK
KEY
IVZ
OXY

I've just noticed that for GPS, the corrupted data on the original download was for different days (not 9/13) so I do think it'll be a NT8 issue (but if you could please confirm that'd be great). Many thanks

ChainsawDR

New IQFeed Forum » Different Data on Tick vs 1min Nov 11, 2021 09:22 PM (Total replies: 10)

Hi Stephen,

Re-Posting on this topic as its causing some frustration. When downloading historical data from iqfeed using NinjaTrader 8, I'm frequently seeing issues with the tick data (unsure if its a problem with iqfeed, NT, or user error). Yesterday I set off a download of ~6 months of tick data for ~500 stock symbols, and today when testing using that data there were tick data issues with ~50 of them (obvious issues such as having 1MM trade volume for the same microsecond time stamp). What I have been doing as a workaround is to delete the data for the affected symbols and redownloading them. However when I just did this a moment ago, I'm still seeing the same issue for some of the symbols. A good example is the stock GPS...

I setoff a download for GPS Tick, Minute & Day data for Bid/Ask/Last from 4/1/21 to 9/30/21 (in NinjaTrader>Import Historical Data>Download.
The tick data for 9/13/2021 looks corrupted (minute data looks fine).
The screenshot attached shows tick data and 1min data side by side for GPS. In the tick data there are ~2MM+ trades in GPS with the timestamp 3:57:00:438, which 1min data for 3:57 has ~88k volume.

As I mentioned earlier, I'm not sure if the issue is at iqfeed's end or NT8's end, but hoping you can please look into the data from iqfeed's side to rule out a data issue please.

Many thanks in advance

ChainsawDR
Edited by ChainsawDR on Nov 11, 2021 at 09:23 PM

New IQFeed Forum » Different Data on Tick vs 1min Aug 20, 2021 03:23 AM (Total replies: 10)

Thank you both. I was able to recreate the issue for another stock at a later time, and clicking 'reload historical data' updated the tick data. Stephen your comment on timezone is likely an apt one. I'm on pacific time and at one point I changed it to Eastern time. I suspect the data I'm looking at may have an issue because of this. I will purge and reload. Thanks for your help with this.

New IQFeed Forum » Different Data on Tick vs 1min Aug 18, 2021 07:49 PM (Total replies: 10)

adding tick screenshot

New IQFeed Forum » Different Data on Tick vs 1min Aug 18, 2021 07:49 PM (Total replies: 10)

Hi,
I use Iqfeed with NinjaTrader 8. I'm seeing material differences between tick and 1 min data, and unsure if I should contact Iqfeed or NinjaTrader (presume Iqfeed as its a data issue).

Please see attached screenshots for VUZI on 1/20/21. On a 1min bar the High reaches $9.69 at 11:47, On tick data the high reaches $10.13. If you look at the 1min chart for 15mins either side, it gets nowhere close to $10. It's clear that for the same time and the same stock, pulling tick and 1min data is telling different stories. Please could you take a look at the data for VUZI to see if there is an issue, and which source of truth is correct?

I am building an algorithm that auto-trades with tick and 1min data, so obviously very nervous about this issue and guarding against future instances. Do you have any advice please on how to guard against repeat issues?

Many thanks

ChainsawDR


Time: Sat December 7, 2024 12:14 AM CFBB v1.2.0 7 ms.
© AderSoftware 2002-2003