I wonder what I am doing wrong but my Lookup socket only receives one single line (1 datapoint) no matter to what I set DataPointsPerSend when submitting the request. What is wrong and how can this be solved? I would expect a much larger load to be forwarded by the IQConnect gateway as described in the developer documentation.

Thanks
Matt

Hello Matt,

Are you seeing the ENDMSG coming through the socket after your request? The rest of the data may be on the socket waiting to be read. If not, send us an email with your ID and I will check the logs to see what the requests are and how much data we are sending you.

Tim

Tim Walter,

I do see the ENDMSG coming through but its the last message. Each single tick is reported in a separate message. I would expect to see 500 or 2500 data points to arrive at my socket at once but that is not the case. Each single message contains only one single data point , such as

myquestId,2014-03-31 05:00:00,1.37701,1.37540,1.37570,1.37660,81355,12575,0,

the last message contains ENDMSG at the end.

What I would expect to see are multiple datapoints in one packet.

Bbmat,

I can find no instances where we returned only one datapoint to any of your requests, they are all 25, 500, 1000, or 0. Have you tried to turn on logging of the history socket so you can compare what you are seeing with what IQFeed is returning? This is starting to feel like a parsing/socket issue.

Tim

OK, could you please help me locate where to turn on loggin in the gateway?
thanks

Right click the IQFeed icon in the system tray, under support, launch diagnostics you will see a logging tab where you can enable and disable as you choose. I would just turn on all Lookup items for while you are testing. Just don't forget to turn it off when your done.

Tim

Here is what I am seeing:

FROM CLIENT LookupRequest 9296 2014-03-31 17:51:46 HIX,EURUSD.COMP,3600,500,0,159c0467-29b0-4402-a945-f9ea09132d7f,5000,s

I attached the logFile could you please take a look, it still looks like as if the gateway sends each message (each data point) individually, or is this to be understood as one single response message?

I just wonder, if this was a socket issue then why do I get each message containing exactly one full datapoint with ending CF flag? If the buffer was set too low in my socket then I would understand but then each message should be truncated which it is not.

Edited by bbmat on Mar 31, 2014 at 04:56 AM
Edited by bbmat on Mar 31, 2014 at 04:57 AM
Edited by bbmat on Mar 31, 2014 at 04:58 AM

Ok, well the data is being returned, the line above is the request, and

the TO CLIENT LookupData 9296 lines below that are the data being returned.

As an example, from the server we will say we receive the string

ABC<CR><LF>DEF<CR><LF>GHI<CR><LF><!ENDMSG!><CR><LF>

In your first buffer you could get everything through the first letter G.

ABC<CR><LF>DEF<CR><LF>G

You would store the G, everything to the right of the terminating characters to a temp var.

Then you would process the complete lines that you have,

ABC<CR><LF>
DEF<CR><LF>

Then you will read from the buffer again, you would receive
HI<CR><LF><!ENDMSG!><CR><LF>

You would want to prepend the G you saved off to get your full messages and then clear your temp var.

GHI<CR><LF>
<!ENDMSG!><CR><LF>

If the message was larger you would continue your loop until you process the ENDMSG Line.

You should be able to step through your debugger and see this as well. Also, if you are using C#, make sure your BeginReceive and EndReceive logic is lined up right, so that the socket is not tossing data. I know some people have had trouble in that area in the past.

Tim

Tim Walter, thanks for your effort but I guess you misunderstand; My issue is that my socket only receives incoming data packets with the EXACT size and content of 1(!!!) single data point, one line (ending with "\r"). That is all each packet contains. I expect to receive hundreds of data points in one packet. It still looks to me that the gateway sends out line by line rather than all 500 requested data points at once.

So, currently I do not need to store anything in temp variables because I receive exactly one data point and it is easy to parse that into a POCO. My problem, however, is that I want to get many datapoints (optimally) the whole request in one or few large packets. I double checked and my socket logic does not impose any limits on incoming data sizes.

My socket logic seems perfectly fine. I receive exactly what I request, not one single data point is dropped. But my socket receives each single data point in an individual packet rather than having many data points lumped together in one packet. And right now I still have to suspect that it is the Gateway IQConnect which is the speed throttle here -> Sending each data point individually slows down things to a large degree.


Edited by bbmat on Mar 31, 2014 at 05:33 AM
Edited by bbmat on Mar 31, 2014 at 05:36 AM

Ok, so if you open the C# sample code for history socket and set a breakpoint on the HistorySocketForm.cs and request GOOG, what do you get? I got 24 bytes the first pull, one record as you mentioned, but about 25k on subsequent hits of this breakpoint with tick requests being closer to 250k bytes received.

Does your behavior match this?

Tim

Line 156 for the breakpoint, I was checking the received bytes values.

will need to double check and get back to you on this, but good idea to isolate whether the gateway is involved or not.

My apologies,

problem solved! I wrote an AsyncSocket class some time ago and completely forgot that I internally already manage potentially split messages. Sorry to occupy your time with this issue. I verified and the Gateway indeed sends messages containing multiple data points, constrained by the "dataPointsPerSend" parameter.

Great, glad we got it figured out.

Tim