I have been having an issue with getting the news story back from a request. Most of the time the story related to the headline does come back. However, on some requests there is a big delay and the following result comes back:

15:10 ..NONE..
15:10
15:10 !ENDMSG!
(the time mark is mine)

Then on the next request for a story for a different headline, I get the story for the last request that gave the above message result. It is like the story is in the queue but did not get out in the first request. The interesting thing is that for a given news story id the return always comes back and repeats successfully, and one that shows this behavior is repeatable too. It is a systematic problem with some headlines. I am posting the beginning and end of the story that is an example of the problem - I first got the ..NONE.. message and then on a subsequent request for a different news story got the story for the first request.

15:11 <?xml version='1.0'?>
15:11 <news_stories>
15:11 <news_story>
15:11 <is_link>N</is_link>
15:11 <story_text>
15:11 TO BUSINESS, MEDICAL AND HEALTH EDITORS:
15:11 Management to Host a Conference Call Today At 5:00 PM Eastern Standard Time
15:11
15:11 ST. LOUIS, Nov. 11 /PRNewswire-FirstCall/ -- Stereotaxis, Inc. (Nasdaq: STXS)
15:11 today reported third quarter 2004 revenue of $5.7 million, a significant
. (center text left out )
.
15:11 jstolze@stereotaxis.com ernie.knewitz@eurorscg.com
15:11
15:11 212-845-4269
15:11 Brian Ritchie (investors)
15:11 brian.ritchie@eurorscg.com
15:11
15:11 SOURCE Stereotaxis, Inc.</story_text>
15:11 <symbols>:STXS:</symbols>
15:11 </news_story>
15:11 </news_stories>
15:11
15:11
15:11 !ENDMSG!

The story id for retrieval for this story is: 6422938

Because this is a repeatable event it should show on other software that uses a news tcp/ip interface.

It is causing me problems in my news presentations as first it gives nothing, and then the story when another request is made. It can be confusing as the next request may be for a different one. This has been causing me delays until I found the culprit. It is frustrating as it is nothing that I can fix and reflects on my software. Once out of balance it stays that way. The only thing I noticed is it seems to occur on longer stories.

I use the tcp/ip connection to retrieve news and news headlines.

Thanks,

David

---

IQXP Software
http://www.iqxp.com

LiveWire Update Service
PO Box 1417
Fairfield, IA 52556
641-472-8393
http://www.livewire-cablesoft.com/

David,

Can you give me full line that you are entering to make this request? I will try to recreate the issue here with you exact request. Thanks!

Natalie Hannan

---

Natalie Hannan DTN Market Access, LLC.

Natalie:

I write to the tcp/ip port with the following expression which follows the specification:

NN:(ID):(Type):(Date);

My exact statement:
Write( 'NN:' + SId + ':' + NS + ':' + SDate + ';');

Where SID is the story ID, NS is the news source, and SDate is the date. For the date I use '20040101'

This returns the correct story in almost all cases except for some noted in the first post. It is repeatable in that for a given story it always comes back, and I can recall it again and again. For the story that gives the ..NONE.. response it is also repetitive in that it never comes back on the first try, and is stuck in the queue until the next requesst. I use a thread that is always looking at the port output so it is not dormant between requests.

The story id for retrieval for this story was: 6422938
I don't recall the news source but here is a line from the story:

"15:11 ST. LOUIS, Nov. 11 /PRNewswire-FirstCall/ -- Stereotaxis, Inc. (Nasdaq: STXS) "
Search for headlines for STXS and you will get the exact headline.

I will do some testing this afternoon to find other candidates and give you the story id and the news source.

Thanks,

David

---

IQXP Software
http://www.iqxp.com

LiveWire Update Service
PO Box 1417
Fairfield, IA 52556
641-472-8393
http://www.livewire-cablesoft.com/

David,

Thanks for the additional information. I'll look into this on Monday.

Natalie Hannan

---

Natalie Hannan DTN Market Access, LLC.

David,

In testing, I have found that the "..NONE.." message gets returned if the previous message hasn't been fully read from the TCP port. This isn't necessarily a flaw in your logic for reading the port but rather an inherant behavior of the TCP transfer protocol. If a TCP packet is received out of order, the protocol waits until the proper packet is received(transparent to the upper level of the protocol). In other words, when you read the socket to retrieve the story, not all of it may be there. This gives you an incomplete story which you may or may not notice. The next time you make a request and try to read the port, the rest of the story is still there. This begins to mess up the port and so IQConnect gets invovled, flushes the port, and sends the ..NONE.. message instead.

Now, how to get around this. The key is to make sure all of every message has been retrieved. You can do this by making sure you have received the "!ENDMSG!" token or at the very least, make sure the last byte you receive is a new-line character. If you haven't received all of the message, read the port again and append the buffer. The rest of the message may not instantly be there so it's better to be doing this inside of a worker thread.

This problem really can't be resolved by increasing your TCP buffer size since if all of the packets aren't there then they can't be read.

I hope this helps.

---

Glenn Benes
-------------------------------------------------------
Glenn Benes DTN Market Access, LLC.

Glenn:

Thanks for the feedback. However I think there is a problem in IQConnect - in the example I gave the stories come back ok for certain news story id's. For the story id I gave, I re-tried it the other day and it still initially came back ..NONE.. Prior to that I had received the !ENDMSG! on the last request - however you haved raised a point in that I should also look for a new-line character. I have not examined this or added it to my code.

I do all of my news story and symbol headline searches in separate program threads. I do not do a time out on requests - the ..NONE.. comes back after quite a delay. I do not look for a subsequent 'LF' to complete the request return but will examine this in the near future.

The confusion for me is that for a given story, it always comes back ok, never a ..NONE.. - my software is consistent in that it is not changing between requests.

This leads to another question: How many other undocumented messages like ..NONE.. can come back from requests? This one I had not trapped so it initially caused some problems for me. I'd like to know of any others so I can handle them properly. Also, is there a time out message return for any request, bar history, news or otherwise? I do get 'not available' for some history requests. That helps greatly.

Thanks for your help!

David

---

IQXP Software
http://www.iqxp.com

LiveWire Update Service
PO Box 1417
Fairfield, IA 52556
641-472-8393
http://www.livewire-cablesoft.com/

David,

I was having a hard time reproducing this error until I tried sending the request manually. The only way I was able to get a ..NONE.. result was by having a space between the request story token and the story ID. Instead of "NN:6422938:YYY:20040101;" I sent "NN: 6422938:YYY:20040101;", causing the retrieval to fail. Could you log the actual text of your request and let me know what is being sent? BTW, as long as the correct story ID is sent, the provider type and date are ignored.

Glenn Benes

---

Glenn Benes DTN Market Access, LLC.
Edited by DTN_Glenn_B on Dec 1, 2004 at 10:37 AM
Edited by DTN_Glenn_B on Dec 1, 2004 at 10:39 AM
Edited by DTN_Glenn_B on Dec 1, 2004 at 10:43 AM

Glen:

I have run some tests on the news story retrieval and here are the results. First I request the same story several times to illustrate that all is well and working. In this case I did it 15 times in a row - I only show the first and last request for brevity in the log below.

Then I requested the story that illustrates the "NONE" issue. The response is ..NONE.. for the first try and then comes back ok (full story) on the next. On the third try it comes back NONE again. Then I did a request for the first story that I got back OK for 15 requests in a row. Now I get ..NONE.. for the first try, No data available for the second, and then the story on the third request.

I have enclosed the string I send to request the story in >> request<< so you can see what I am sending each time. My news retrieval is run in a separate thread and is not suspended or terminated at the end of response. Any additional data would show. The log is from the output of the tcp/ip port with time marks that I add.

18:30 News Request: >>NN:6516425:RTB:20040101;<<
18:30 0
18:30 <?xml version='1.0'?>
18:30 <news_stories>
18:30 <news_story>
18:30 <is_link>N</is_link>
18:30 <story_text>
18:30 (RealTimeTraders.com) - Siebel Systems announces its market entry strategy for
18:30 Russia and the Commonwealth of Independent States, opens own office in Moscow
18:30
18:30 Copyright(c) 2004 RealTimeTraders.com, Inc. All Rights Reserved</story_text>
18:30 <symbols>:SEBL:</symbols>
18:30 </news_story>
18:30 </news_stories>
18:30 !ENDMSG!
.
15 more reqests for the same story were done... all returned ok - I show the first and last results
.
18:31 News Request: >>NN:6516425:RTB:20040101;<<
18:31 0
18:31 <?xml version='1.0'?>
18:31 <news_stories>
18:31 <news_story>
18:31 <is_link>N</is_link>
18:31 <story_text>
18:31 (RealTimeTraders.com) - Siebel Systems announces its market entry strategy for
18:31 Russia and the Commonwealth of Independent States, opens own office in Moscow
18:31
18:31 Copyright(c) 2004 RealTimeTraders.com, Inc. All Rights Reserved</story_text>
18:31 <symbols>:SEBL:</symbols>
18:31 </news_story>
18:31 </news_stories>
18:31 !ENDMSG!

Now I request the story that gives the NONE result and get:
18:31 News Request: >>NN:6422938:CPR:20040101;<<
18:31 ..NONE..
18:31 !ENDMSG!

Then I request the same story again and actuall get it! ( I am leaving out the center section of the story)
18:31 News Request: >>NN:6422938:CPR:20040101;<<
18:31 0
18:31 <?xml version='1.0'?>
18:31 <news_stories>
18:31 <news_story>
18:31 <is_link>N</is_link>
18:31 <story_text>
18:31 TO BUSINESS, MEDICAL AND HEALTH EDITORS:
18:31 Management to Host a Conference Call Today At 5:00 PM Eastern Standard Time
18:31 <symbols>:STXS:</symbols>
. . . . snip out centter
18:31 </news_story>
18:31 </news_stories>
18:31 !ENDMSG!

Then I request it again and get the NONE again:
18:31 News Request: >>NN:6422938:CPR:20040101;<<
18:32 ..NONE..
18:32 !ENDMSG!

Then I request it again and it comes back! (I snip out the story for shortness)
18:32 News Request: >>NN:6422938:CPR:20040101;<<
18:32 0
18:32 <?xml version='1.0'?>
18:32 <news_stories>
- - -
18:32 </news_stories>
18:32 !ENDMSG!

Now I rereques it again and get NONE
18:32 News Request: >>NN:6422938:CPR:20040101;<<
18:32 ..NONE..
18:32 !ENDMSG!

Then I try the first news story that I got 15 times in a row without any problems (shown in the firt part of this post)
18:33 News Request: >>NN:6516425:RTB:20040101;<<
18:33 ..NONE..
18:33 !ENDMSG!
Hmmm... says NONE now

Now I retry and get "No data found"
18:33 News Request: >>NN:6516425:RTB:20040101;<<
18:33 No data was found.
18:33 !ENDMSG!

then I try it again, and the story comes back
18:33 News Request: >>NN:6516425:RTB:20040101;<<
18:33 0
18:33 <?xml version='1.0'?>
18:33 <news_stories>
18:33 <news_story>
18:33 <is_link>N</is_link>
18:33 <story_text>
18:33 (RealTimeTraders.com) - Siebel Systems announces its market entry strategy for
18:33 Russia and the Commonwealth of Independent States, opens own office in Moscow
18:33
18:33 Copyright(c) 2004 RealTimeTraders.com, Inc. All Rights Reserved</story_text>
18:33 <symbols>:SEBL:</symbols>
18:33 </news_story>
18:33 </news_stories>
18:33 !ENDMSG!

18:33 Closing ::IQXP:: , 12/01/04, 18:33:47

There is something wrong in River City...

I also do not understand the "0" (zero) that appears at the top start of each response except
for those with NONE in the response. Also, the 'No data found ' was unexpected and I have now added it to my response parsing. Any other messages like this I need to know about?

Thanks!

David

---

IQXP Software
http://www.iqxp.com

LiveWire Update Service
PO Box 1417
Fairfield, IA 52556
641-472-8393
http://www.livewire-cablesoft.com/

David,

Thanks for the log detail. I was able to verify that the server isn't the source of the problem. However, I no longer can reproduce the problem on my side. Do you have a log file from IQFeed? Set the log detail to 4 and send the log file to
Thanks.

Glenn Benes

---

Glenn Benes DTN Market Access, LLC.
Edited by DTN_Steve_S on Sep 19, 2011 at 10:59 AM