Re: 20050207: CONDUIT data stream timing/contents/etc.

Pete,

I noticed the missed products as well. There was a switching of
systems at NWS that was at that time (not the LDM host, but
the systems that notify the LDM host). It sounds like this
was a coincidental problem and not related to the other
problems. We'll continue to monitor the stream.

Steve Chiswell

On Tue, 8 Feb 2005, Pete Pokrandt wrote:

>
> Just a note, the 18 UTC ETA on the 212 grid and the 104 grid
> were incomplete this afternoon. We missed the 27 hour forecast
> on the 212 grid, and 24 and 27 hour forecast on the 104 grid.
>
> Our latencies to tgsv32 were no more than about 40 seconds.
>
> Pete
>
> In a previous message to me, you wrote:
>
>  >Pete et al.,
>  >
>  >The primary limitation at the LDM source of CONDUIT is narrowing in on
>  >the size of the product queue and the amount of system memory. The
>  >backlog is caused by the amount of time it takes to "insert" a file into
>  >the queue, which when this exceeds the rate at which files are being
>  >posted to the NWS servers, builds the delay for which the data will
>  >appear into the LDM queue. This is normally a process that takes about
>  >30 seconds to acquire and insert the data, but has been taking 2-3
>  >minutes to accomplish as the volume being inserted has increased.
>  >
>  >At 16Z today, the ldm product queue at the LDM source for CONDUIT was
>  >decreased to half its previous size. At this size, the amount of system
>  >paging should be decreased so that backlogs are decreased. The tradeoff
>  >is in buffering to top tier LDMs in the event of network trouble,
>  >which would be 20-40 minutes.
>  >
>  >At 22:04Z, the onedeg GFS fh.0057 grid was arriving via LDM, while the
>  >FTP server showed the file posted at 21:44Z, which is an improvement-
>  >hopefully this is noticable. At this point, the LDM server is idling
>  >waiting for the file server to offer it data.
>  >
>  >A note about GRIB2- NWS is beginning to remove certain GRIB1 data sets
>  >from NOAAPORT as GRIB2 products have replaced their use (ETA 218 and
>  >ETA242 have been scheduled thus far).
>  >
>  >Steve Chiswell
>  >Unidata User Support
>  >
>
> --
> +>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>+<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<+
> ^ Pete Pokrandt                    V 1447  AOSS Bldg  1225 W Dayton St^
> ^ Systems Programmer               V Madison,         WI     53706    ^
> ^                                  V      address@hidden       ^
> ^ Dept of Atmos & Oceanic Sciences V (608) 262-3086 (Phone/voicemail) ^
> ^ University of Wisconsin-Madison  V       262-0166 (Fax)             ^
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<+>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>+
>