Re: RE; Apparent Latency Problem Solution

On Fri, 1 Dec 2000, Jim Koermer wrote:

> Hello All,
> 
> Last week, we had a good discussion on the latency problems that our
> school (Plymouth State) and others were experiencing over the FOS feed
> because of the injection of NOAAPORT NWSTG data into the stream. It
> seemed like more often than not that we were experiencing latencies > 60
> minutes and we were losing a great deal of data or getting it too late.
> 
> Russ said a few things in his e-mail that got me thinking and
> experimenting, especially relating to the fact that some machines may
> not be able of handling all the rpc calls. In fact, our primary IDD
> machine was an 1996 vintage RS-6000 with ~67MHz processor and 256MB of
> RAM. It had been working well until the NOAAPORT (including ETA) data
> expansion.
> 
> As a result, I had my feed sites, allow a much faster Dell PowerEdge
> server with dual PII-MHz processors and 1 GB RAM, to request data. I
> kept the IDS|DDPLUS feeds on the RS6000 and have MCIDAS and HDS data
> going into the Dell machine. 
> 
> Since doing this, my FOS latencies for DDPLUS|IDS are down to usually
> less than 2 minutes. I don't know the latency stats for the Dell, yet,
> since the stats perl script uses what I think is a non-ASCII option with
> "ls" (i.e. -x - we are working on an alternative). However, for about

Jim,

Glad you found some solutions to your data reception problems.  

For the problem with mailpqstats, I believe the -x flag in the ls command
can be omitted without a problem. The important flag is the -r flag, it
causes the files to show with the latest one first, thus the most recent
data is sent to UPC.  Let me know if this doesn't work.

Robb...




> the first time in recent memory, we are getting complete ETA grib files
> (> 200mb) in very timely fashion, so I think the latencies are probably
> quite low on the Dell. This performance has continued for several days,
> since making the transition.
> 
> The lesson seems to be that the pipe may be fine, but the ingestor may
> need to be upgraded to greatly improve IDD data reception and
> performance.
> 
> BTW, this came together at just the right time, since our NOAAPORT NWSTG
> receiver has gone south over the last few days and needs some work.
> 
>                                 Jim
> -- 
> James P. Koermer             E-Mail: address@hidden
> Professor of Meteorology     Office Phone: (603)535-2574
> Natural Science Department   Office Fax: (603)535-2723
> Plymouth State College       WWW: http://vortex.plymouth.edu/
> Plymouth, NH 03264
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================