[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[TIGGE #FDX-319560]: latency values

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [TIGGE #FDX-319560]: latency values
  • Date: Tue, 31 Oct 2006 11:01:55 -0700
Doug,

> The clock was not being reset every 18 hours, but now is being set
> daily.  The latency values have now
> dropped, with more variation.
> 
> http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?EXP
> +ultrazone.ucar.edu+LOG

That looks better.

> Another question I have regards queue structure.  ECMWF is sending
> data at a rate of about
> 10 GB/hour, but only has a queue size of 4 GB with 1,000,000 slots.
> Ultrazone is using a queue of 20GB with 400,000 slots.  Do these need
> to match up to be more effective?

The best thing to do is to periodically execute the pqmon(1) utility from a
crontab script, save the parameters in a file, and then plot the age of the
oldest product versus time using something like gnuplot(1).  This will show
you the minimum residency time and allow you to decide if the queue-size
should be increased.  You can also plot the number of products
in the queue versus time to see if the number of slots needs to be 
increased.

That's what we do here.

> Also,  Is it best to have the queue built on the same disk as pqact
> writes the products on?

I think it would be better to use a different disk for the filed or
decoded data than the disk that the product-queue is on.  This should
allow the OS to perform the two categories of I/O in parallel.

> We've
> been having some problems writing all products from ldm.pq to their
> final location before they age
> off.

You guys are pushing a lot of data.  At some point, you just might
exceed the capacity of the network and/or the computers.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: FDX-319560
Department: Support IDD TIGGE
Priority: Normal
Status: Closed