Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
Viewing User Profile for: taa_dtn
About Contact
Joined: May 7, 2004 01:04 PM
Last Post: Oct 16, 2023 07:10 PM
Last Visit: Aug 4, 2024 12:39 PM
Website:  
Location:
Occupation:
Interests:
AIM:
ICQ:
MSN IM:
Yahoo IM:
Post Statistics
taa_dtn has contributed to 154 posts out of 21251 total posts (0.72%) in 7,457 days (0.02 posts per day).

20 Most recent posts:
Data Questions » Out-of-order trades during history fetch Oct 16, 2023 07:10 PM (Total replies: 8)

Everything looks good today. Thanks for the fix!

Data Questions » Out-of-order trades during history fetch Oct 12, 2023 09:16 PM (Total replies: 8)

MUCH improved, though I still see out-of-order data for these symbols:

KYMR TRUP TMDX TPIC URGN TNDM TLT UPLD HLMN TTD WEST STRO FOLD ZLAB IBB VAXX

Data Questions » Out-of-order trades during history fetch Oct 10, 2023 07:09 PM (Total replies: 8)

Any progress on this? I see data for 2023-10-04 is still out-of-order.

Data Questions » Out-of-order trades during history fetch Oct 5, 2023 09:12 PM (Total replies: 8)

I fetch historical data (ticks) for a large number of stocks at end-of-day (8PM Eastern). Yesterday I noticed several symbols for which trades were returned out-of-order chronologically. It's been a number of years since I've seen this behavior.

Examples: IAS WTW FTDR TTMI WOOF JD XP

My code hasn't changed since I upgraded to protocol 6.2 shortly after it was released.

Has anyone else noticed this? Or is it expected behavior that just didn't occur for years?

System Status Announcements » Connection issue 12-Mar-2023 Mar 13, 2023 10:31 AM (Total replies: 9)

Thanks, Steve. I've upgraded.

System Status Announcements » Connection issue 12-Mar-2023 Mar 12, 2023 02:13 PM (Total replies: 9)

I can log into IQFeed successfully at the moment.

I've seen the problem you reported occasionally over the past month. I have confirmed physical disconnects, though, and restarting my app always solved the IQFeed issues, so I had guessed the disconnects were responsible. Given your evidence, I'm reconsidering.

System Status Announcements » Connection issue 12-Mar-2023 Mar 12, 2023 01:51 PM (Total replies: 9)

I've seen this, too, but I thought it was due to local connection issues.

DTNIQ Client Software Wish List » Intraday split-adjusted data for stocks Feb 22, 2023 11:24 AM (Total replies: 2)

For some indicators you want the split-adjusted prices, for others you want the historical prices and volumes exactly as transmitted on that day. There's no one-size-fits-all solution.

I like the current IQFeed approach, where I can get the tick data as-transmitted, plus the splits. I store both so I can construct split-adjusted data when I need it.

That said, it's often the case that splits are described incorrectly either as to date or as to ratio. This problem is not unique to IQFeed; you can observe it on practically any online source of split history. As a consequence I run a sanity-check on splits in my database every night. Things to watch for include multiple splits within a few days of one another, splits with implausibly large or small ratios, and split information that's delivered for the first time after the date of the split.

My split-adjustment code includes a table of known corrections that are applied automatically when split-adjusting. That table has to be maintained manually, though.

If anyone has a better solution, I'd love to hear about it!

New IQFeed Forum » Empty result from HTT request Dec 11, 2022 01:24 PM (Total replies: 12)

Same here; each history-request thread has its own socket. I actually did this in reverse of the order jayqwi used; I started with each requester being a separate process, then switched to separate threads because load balancing is easier that way. I'm using tags, but only to make sure that no stale data arrives from previous requests; the intent is to run each request to completion before starting another.

Probably irrelevant, but while history-requests are being processed I do have at least two other independent processes with their own connections to the Level 1 port.

And I should emphasize that I haven't seen this misbehavior at all this month. Since my software hasn't changed (or even been recompiled), and iqconnect.exe hasn't been changed, I think that points to either Wine or the software running on the servers. I know there was a Wine update last month, so I suspect something got fixed in that release.

New IQFeed Forum » Empty result from HTT request Dec 11, 2022 01:09 AM (Total replies: 12)

I'm running client version 6.2.0.23 currently. I've been seeing this behavior for years, though, with many different versions of the client and a few different versions of the protocol and with two versions of my app. That's why I went to the trouble of implementing the workaround, and among the reasons why I suspect the problem isn't in the app or iqconnect.

In my case, each thread (of 8) issues a request and processes it to completion before issuing another request. Anything unexpected is logged, so I'm pretty sure there's nothing leftover that hasn't been processed. I simply get no response (and I've also tried a wide range of timeouts with no success).

New IQFeed Forum » Empty result from HTT request Dec 8, 2022 05:57 PM (Total replies: 12)

My app is written in C++ rather than Python, but I've often seen equivalent behavior. I believe this to be due to a bug in Wine. I work around it by closing the socket, opening a new one, and retrying the request. This works always, so far as I can tell.

Coincidentally, I haven't seen this happen in over a week, so I was wondering if a recent Wine update had fixed it. My current Wine version is 7.22 (Staging), the default version for Fedora 36.

IQFeed Developer Support » Unable to connect this morning Sep 13, 2022 01:41 PM (Total replies: 1)

False alarm. Turned out to be due to an update elsewhere in the system that cause network connectivity issues.

IQFeed Developer Support » Unable to connect this morning Sep 13, 2022 12:58 PM (Total replies: 1)

Using Wine 7.12 under Feodora 36. No updates to Wine or to my software in the past several weeks. No problems until this morning. Here's what's in the log. Any ideas?

=== IQConnect Log File Opened On Tue Sep 13 10:43:38 ===
Current Log Levels,Connectivity,Information,Admin
Current IQFeed Version,6.2.0.23
STATUS Connectivity 464 0 2022-09-13 10:43:38 Initializing the login thread
STATUS Connectivity 456 0 2022-09-13 10:43:38 Creating trader account verification thread. Status idle
STATUS Information 460 0 2022-09-13 10:43:38 Unable to start Authentication Server. Error (107): Connection refused
STATUS Connectivity 452 0 2022-09-13 10:43:42 Waiting for authentication threads. Result:102 Code:183.

New IQFeed Forum » Missing symbol guides? May 10, 2022 05:34 PM (Total replies: 3)

Thanks! It looks like that will work nicely.

New IQFeed Forum » Missing symbol guides? May 9, 2022 10:04 PM (Total replies: 3)

Any suggestions?

New IQFeed Forum » Missing symbol guides? May 6, 2022 03:20 PM (Total replies: 3)

Hi, folks. The lists of symbol additions/changes/deletions at http://iqfeed.net/symbolguide are no longer updating. Have those moved, or is this a glitch?

Miscellaneous Messages » Machine learning and tick-by-tick data Jan 31, 2022 11:10 AM (Total replies: 1)

Yes, I experimented with this a few years ago. Your questions are relevant and insightful, but I don't have much useful information to offer in reply.

I haven't seen many published papers on the subject in recent years. Take that with a grain of salt, though, because I'm not looking actively enough. Possibly if the technique has been applied successfully, it hasn't been discussed in public for the obvious reasons. Hopefully someone else will reply with better information.

In general, I hit the same roadblocks you did. It's hard to choose the right network architecture (financial data isn't statistically stationary, so I wasn't able to design either recurrent or convolutional networks that were consistently successful). Raw tick-by-tick data has so much variability along so many dimensions that I suspect feature engineering is necessary, but that's a major research project in its own right. Techniques currently being used for natural language processing are probably where I'd start if I were to look at this again today.

Possibly the most fundamental problem I ran into is that it doesn't seem workable to use a scalar value to measure outcomes, so anything based on simple gradient descent is problematic. I think a practical outcome measurement must be at least three-dimensional -- it needs to include return, risk, and capital management. Arguably more, but the need for those three is easy to understand.

New IQFeed Forum » iqfeed crashed with latest winhq version on ubuntu Dec 1, 2021 11:56 AM (Total replies: 4)

Hmm. I haven't run into this problem on my system (Wine 6.21 on Fedora 34). However, I'm still having issues with iqconnect failing to shut down after the last client closes, so I'm killing it explicitly and restarting early each day. Do you have any unexpected iqconnect, explorer, etc. processes lying around?

IQFeed Developer Support » Possible problem under Wine? Nov 9, 2021 12:27 PM (Total replies: 3)

Thanks for the reply, Gary.

I've written a toy app that exhibits all the correct (expected) behavior, so I'll try to nail down what's different between that and my real apps. I don't have much time available this week, so my next update here might take a while longer.

Now that I have an example working, it's clear that the problem is unrelated to the protocol and client software upgrades.

For reference, I start iqconnect.exe using fork() and execl(), passing a path to the executable and all the appropriate arguments. I wait a few seconds for it to initialize and then I connect. This has worked well for years. CrossOver has never been necessary.

My concern is that iqconnect is still running, sending data to a socket that it thinks is still connected to a client even though the client process is long gone. This is very likely to cause problems somewhere down the line. At the very least, iqconnect's CPU demands are going to get progressively larger over time as clients connect and disconnect. At the worst, it could block and be unable to accept new connections. Plus, the widget tray fills up with multiple connection manager icons, some of which don't respond to mouse clicks, which is ugly and confusing.

IQFeed Developer Support » Possible problem under Wine? Nov 8, 2021 07:39 PM (Total replies: 3)

I still have no solution to this, though a few things I've observed suggest the problem might be related to the way I start iqconnect. I've tried several different methods, and they all misbehave, but some in different ways than others. The common behavior for all of them is that iqconnect thinks it's still connected to a client even after the client (a) closes the only socket it has open to iqconnect, or (b) exits entirely.

FWIW, this problem doesn't exist when using protocol 4.9, but does with 6.2.

So, what is the recommended way to start iqconnect under Wine?


Time: Sat October 5, 2024 8:18 AM CFBB v1.2.0 7 ms.
© AderSoftware 2002-2003