Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"If you are serious about your trading I would not rely on IB data for serious daytrading. Took me a while to justify the cost of IQ Feed and in the end, it's just a 2 point stop on ES. Better safe than sorry" - Comment from Public Forum
"Just a quick one to say I'm very impressed so far :) The documentation for developers is excellent and I've quickly managed to get an app written to do historical downloads. The system is very robust and pretty quick considering the extent of data that's available. The support guys have been very helpful too, in combination with the forums it's been plain sailing so far!" - Comment from Adam
"I am very pleased with the DTNIQ system for quotes and news." - Comment from Larry
"I was on the phone with a friend who uses CQG and right after the Fed announcement, CQG was as much as 30 seconds behind DTN.IQ. Some quotes were off by as much as 15-18 cents. Your feed never missed a beat." - Comment from Roger
"Thanks for following up with me. You guys do a great job in tech support." - Comment from Phelps
"You are much better than lawyers or the phone company because you answer the phone when I call! I just love your customer service." - Comment from Isreal
"Thanks for the great product and support. During this week of high volume trading, my QuoteTracker + IQ Feed setup never missed a beat. Also, thanks for your swiftness in responding to data issues. I was on ******* for a few years before I made the switch over early this year, and wish I had done it a long time ago." - Comment from Ken
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"I'm satisfied with IQFeed. It's the most reliable and fastest quote feed I have ever used. Although I'm a resident in China, it's still very fast!" - Comment from Xiaofei
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »NEW IQFEED FORUMS »New IQFeed Forum »iqconnect Out of Memory
Author Topic: iqconnect Out of Memory (6 messages, Page 1 of 1)

beauchamp117
-Interested User-
Posts: 1
Joined: May 25, 2018

EMH


Posted: Oct 18, 2023 03:14 PM          Msg. 1 of 6
Running out of Memory when looking at 400 equities level 1 or 76 futures contracts level 2. I am using the Python library which works great. I am having trouble determining what this means. It looks like my data feed stays up but this notification is concerning.



File Attached: Screenshot 2023-10-18 161301.png (downloaded 436 times)

mkvalor
-Interested User-
Posts: 26
Joined: Oct 6, 2020

Keep your tools sharp.


Posted: Feb 23, 2024 01:58 AM          Msg. 2 of 6
I'm a user like yourself, but I've seen a few posts in the past dealing with memory issue problems.

One thing: do you happen to know if your app makes a separate connection to IQConnect.exe for each symbol? Especially using python, I could see this adding up to a lot of CPU and memory overhead. Even though this strategy can be more convenient (because each program only has to deal with data for a single symbol), it can be a lot more memory and CPU efficient to open multiple symbol watches (say, 50 at a time) on a single connection to IQConnect and then use logic in your applicatoin to send each line to the right place based on the ticker symbol at the beginning (such as to a separate data file, to a data stream queue like Kafka, or to some other logic handler to calculate moving averages, etc).

For example, you could use a python dict where the key is the IQF symbol name and the value is an open file handle for that symbol (if you don't find it in the dict, you create a new file and then add it there) -- then check the start of each line to look up the right file to write it to. This is more complex because of the logic I mentioned but also because you'd need to do something like read a config file per process with the list of symbols you want to watch per process. If you got fancy, you could have a single file with all symbols, one per line, and then start the program by adding command line params that tell it which line to start at in the symbols file and how many of them to read for that particular process.

Anyway -- enough advice based on guessing. If you're already reading multiple symbols per process, then it must be something else.



-Mark D. Valor
Edited by mkvalor on Feb 23, 2024 at 02:01 AM
Edited by mkvalor on Feb 23, 2024 at 02:03 AM

DTN_Gary_Stephen
-DTN Guru-
Posts: 400
Joined: Jul 3, 2019


Posted: Feb 23, 2024 04:29 PM          Msg. 3 of 6
It is always advisable to keep the number of connection to a minimum, and keep them open rather than frequently closing/reopening. IQFeed will allow you to collect to make multiple connections to the same socket from the same machine, and this has its uses. (Making 2/4 connections, and dividing the workload among them, can help overall speed in handling a large number of symbols.) But having a very large number of connections, or frequently disconnecting and reconnecting, are counterproductive and may contribute to this kind of situation happening.

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

trulyunstable
-Interested User-
Posts: 1
Joined: Apr 4, 2024


Posted: Apr 4, 2024 03:37 AM          Msg. 4 of 6
I'm wondering if keeping it on all the time will drain the battery?

hersanchez
-Interested User-
Posts: 1
Joined: May 1, 2024


Posted: May 1, 2024 10:42 PM          Msg. 5 of 6
It sounds like your system is encountering memory issues when processing a large amount of data for 400 equities at level 1 or 76 futures contracts at level 2. This notification indicates that your system's memory resources may be insufficient to handle the volume of data you're trying to process. It would be advisable to review your system's memory allocation and consider optimizing your code to handle large datasets more efficiently.

edprofess
-Interested User-
Posts: 2
Joined: May 26, 2024


Posted: May 26, 2024 10:56 PM          Msg. 6 of 6
thank https://geometrydashbreeze.net
Edited by edprofess on Jun 11, 2024 at 11:03 PM
 

 

Time: Sun June 16, 2024 12:33 AM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003