[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[TIGGE #LGY-600646]: Re: Missing fields from CMA

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [TIGGE #LGY-600646]: Re: Missing fields from CMA
  • Date: Wed, 25 Apr 2007 11:16:55 -0600
Manuel,

> I am sorry, but ... where do you get 12 GB/h from ?

The 12 GB/h value was just an example.

> Let's use the real numbers I presented. We said we currently exchange
> 204 GB/day. We want to reserve enough bandwidth for resends, so we can
> say we exchange 204 GB/12 hours or 408GB/day:
> 408 / 24 = 17 GB/h
> 
> So we are looking at a minimum queue size of 17 GBytes, to be reviewed
> each time a new partner starts using LDM or when any model increases
> resolution.
> Please, correct me if I am wrong...

You are right.

The important concept is that the maximum rate of data arrival 
over the minimum residency interval is used to ensure that the
age of the oldest data-product in the product-queue never falls
below the minimum residency time.  This value is most-easily
determined from the "rtstats" volume plots --- assuming that the
desired minimum residency time is one hour.  For example, I
would choose the value 19 GB/h from the plot

http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?EXP+tigge-ldm.ecmwf.int

to ensure that the age of the oldest data-product in the
product-queue was always greater than one hour.

The alternative is to accept a smaller minimum residency time
and, hence, permit a smaller queue.  If this is done, however,
then the "ldmadmin" "start" and "restart" commands must 
alway use the "-m" option to specify the maximum latency
allowed; otherwise, the LDM is likely to accept duplicate
data-products (as we believe we see with CMA due to the
small queue size).

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: LGY-600646
Department: Support IDD TIGGE
Priority: Normal
Status: Open