20050125: Latencies on our redundant HDS feed (cont.)

>From: "Dr. Charles Graves" <address@hidden>
>Organization: SLU
>Keywords: 200501211730.j0LHUiv2002639 LDM IDD NOAAPORT

Hi Chuck,

I have to tell you that each time I see an email from Dr. Charles Graves
I have a mind hiccup until I remind myself that it is you, Chuck :-)

>As you suggested, I increased the size of our product queue.  I did
>it in two steps:
>
>  1) Around 24 Jan at 13Z (24/13 on graphs) I created a new 800 MB queue 
>     (after removing the old one first).  As shown in the latency
>     graphics there was no real change.  The only thing I noticed was an
>     increase in the number/amount of products coming from our local NOAAPORT
>     feed relative to papagayo.

OK.  I thought that this should do the trick.  Evidently not.

>  2) Around 14Z today (25 Jan; 25/14 on graphs) I created a new 1200 MB
>     queue (again after removing the old one first).  Again I found no 
>     real changes in latencies.

Ditto comment above.

>So I doubled and then tripled the product queue with out much success
>(FYI: according to the cumulative stats we peak at about 600 MB/hr).

Rats.

>I used "ldmadmin watch" of the HDS stream for about an hour and
>consistently found proucts like:
>
>Jan 24 17:49:27 pqutil:    40714 20050124162532.225     HDS 11025250
>YHZM80 KWBC 241200 /mAVN !grib /ncep/AVN/#225/200501241200/F096/HGT/800 mb/ 
>
>to have the largest latencies.  I'm not sure if this is relevant or
>simply the random products that were slow over this time period.
>
>Any other suggestions?

I will check with Chiz for further ideas.

>DVB-S:
>I have advised the department about the DVB-S broadcast and the need for
>new hardware/software.  I was waiting to see if Unidata was going to post
>"their" solution on the web before I actually place the hardware order.
>I was hoping to get the system ordered this spring.

We (Chiz) are very close to finishing our (Chiz's) NOAAPORT upgrade to
work robustly with the test DVB-S broadcast the NWS is conducting.
I am excited about how well this is now working...

I am hoping that the software mods that are being put in place can be
made available by the end of this week (doubtful) or sometime next week
(more likely).  One of my primary goals in this development was to get
a system working on as _slow_ a machine as possible, mainly for folks
that find themselves in the position of having to upgrade their
existing NOAAPORT ingest installations.  We are currently running our
tests on a dual 500 Mhz PIII box running SMP Fedora Core 1 Linux that
was put together from parts that were being excessed.  After the next
round of testing, I will be rebooting the machine into single kernel
mode to test if that configuration can keep up with the data currently
being sent.  FYI: the amount now being sent is more than is included in
all of the current NOAAPORT streams and promises to grow substantially
over time.

If I have not already done so, I can tell you that SSEC has adopted our
code AS IS for their NOAAPORT SDI users (i.e., you among others).

As far as hardware you may need to purchase to upgrade your system,
I can say that you will definitely need:

- a Novra S75 DVB-S broadband receiver (www.novra.com; order specifying
  that the unit will be used to ingest NOAAPORT data from the NWS).
  There are other solutions being tested in the greater community, but
  we followed the NWS lead and adopted use of the Novra S75.

- second Ethernet adapter for each ingest box you want to run.  The
  reason for this is that we think that it is better to connect the
  UDP stream from the Novra into a private interface.

>Again many thanks for all of your help!

No worries.  I will chat with Chiz to see if he has any ideas of why
you are seeing the high latencies from papagayo.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.

>From address@hidden  Mon Jan 31 09:13:45 2005

Tom,

On my "To Do List" this morning was to ask if you had any additional
ideas concerning our latencies here at SLU from papagayo...but before 
sending my email I decided to look at the current latencies for our HDS 
feed (the one giving us the most trouble) and as of around 2 UTC on 30
January our latencies have flat lined!  I have no explanation...no one
here has taken credit.  I'll keep an eye on it...hopefully this is not a
glitch but the real deal.  

I will leave things as they stand for the next week or two...then I will
try requesting other feeds (i.e. CONDUIT) to see how that goes.

If things go south again, you will probably here from me.  Again
thanks for all your help!

Chuck