Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"You have an excellent product !!!!!!" - Comment from Arely
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
"And by the way, have to say this. I love the IQFeed software. It's rock solid and it has a really nice API." - Comment from Thomas via RT Chat
"Previously I was using *******. IQFeed is WAY more economical, and for my charting needs is just as good, if not better." - Comment from Public Forum Post
"I've been using Neoticker RT with IQFeed for two months, and I'm very happy with both of the products (I've had IQFeed for two years with very few complaints). The service from both companies is exceptional." - Comment from Public Forum
"Thanks for the great product and support. During this week of high volume trading, my QuoteTracker + IQ Feed setup never missed a beat. Also, thanks for your swiftness in responding to data issues. I was on ******* for a few years before I made the switch over early this year, and wish I had done it a long time ago." - Comment from Ken
"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"I noticed that ******* quotes locked up shortly after the interest rate announcement yesterday while yours stayed stable." - Comment from Ron in Utah
"Thank God for your Data Feed as the only Zippers I see are on my pants (LOL), and no more 200 pip spikes to mess up charts." - Comment from Spiro via Email
"IQ feed is brilliant. The support is mind-bending. What service!" - Comment from Public Forum Post
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »NEW IQFEED FORUMS »New IQFeed Forum »iqconnect Out of Memory
Author Topic: iqconnect Out of Memory (3 messages, Page 1 of 1)

beauchamp117
-Interested User-
Posts: 1
Joined: May 25, 2018

EMH


Posted: Oct 18, 2023 03:14 PM          Msg. 1 of 3
Running out of Memory when looking at 400 equities level 1 or 76 futures contracts level 2. I am using the Python library which works great. I am having trouble determining what this means. It looks like my data feed stays up but this notification is concerning.



File Attached: Screenshot 2023-10-18 161301.png (downloaded 205 times)

mkvalor
-Interested User-
Posts: 24
Joined: Oct 6, 2020

Keep your tools sharp.


Posted: Feb 23, 2024 01:58 AM          Msg. 2 of 3
I'm a user like yourself, but I've seen a few posts in the past dealing with memory issue problems.

One thing: do you happen to know if your app makes a separate connection to IQConnect.exe for each symbol? Especially using python, I could see this adding up to a lot of CPU and memory overhead. Even though this strategy can be more convenient (because each program only has to deal with data for a single symbol), it can be a lot more memory and CPU efficient to open multiple symbol watches (say, 50 at a time) on a single connection to IQConnect and then use logic in your applicatoin to send each line to the right place based on the ticker symbol at the beginning (such as to a separate data file, to a data stream queue like Kafka, or to some other logic handler to calculate moving averages, etc).

For example, you could use a python dict where the key is the IQF symbol name and the value is an open file handle for that symbol (if you don't find it in the dict, you create a new file and then add it there) -- then check the start of each line to look up the right file to write it to. This is more complex because of the logic I mentioned but also because you'd need to do something like read a config file per process with the list of symbols you want to watch per process. If you got fancy, you could have a single file with all symbols, one per line, and then start the program by adding command line params that tell it which line to start at in the symbols file and how many of them to read for that particular process.

Anyway -- enough advice based on guessing. If you're already reading multiple symbols per process, then it must be something else.



-Mark D. Valor
Edited by mkvalor on Feb 23, 2024 at 02:01 AM
Edited by mkvalor on Feb 23, 2024 at 02:03 AM

DTN_Gary_Stephen
-DTN Guru-
Posts: 393
Joined: Jul 3, 2019


Posted: Feb 23, 2024 04:29 PM          Msg. 3 of 3
It is always advisable to keep the number of connection to a minimum, and keep them open rather than frequently closing/reopening. IQFeed will allow you to collect to make multiple connections to the same socket from the same machine, and this has its uses. (Making 2/4 connections, and dividing the workload among them, can help overall speed in handling a large number of symbols.) But having a very large number of connections, or frequently disconnecting and reconnecting, are counterproductive and may contribute to this kind of situation happening.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
 

 

Time: Sun February 25, 2024 6:49 PM CFBB v1.2.0 12 ms.
© AderSoftware 2002-2003