20040212: Data product Expiration

>From: "Schaffert, Lowell" <address@hidden>
>Organization: Lockheed Martin Corporation
>Keywords: 200402122011.i1CKBbp1029946 LDM queue pqexpire

Lowell,

>THE UNIDATA LDM SYSTEM: RECENT IMPROVEMENTS FOR SCALABILITY (paper)
>
>States:
>
>3. THE LDM PRODUCT QUEUE
>Each LDM system uses a local data store called the
>product queue, which is shared among the LDM programs
>that get data from upstream hosts, read and process
>the data, and send data on to downstream hosts.
>Old products are expired out of the product queue when
>they are no longer needed. Product queues typically
>hold about an hour's worth of data from each data
>stream. This buffer of data provides the elasticity needed
>to support reliable data delivery in the face of outages
>and congestion. The LDM is designed so that a relay site
>can be shut down and restarted, then catch up from
>where it left off by getting the products it missed from the
>upstream host so that downstream sites will still receive
>all the subscribed products.
>
>Steve,
>
>What if anything do we need to control the lifetime of products in the Product
>  Queue? Should we be using pqexpire?

You need take no action to age old products out of an LDM-6 queue.
'pqexpire' can be run, but there is no reason to since the oldest
products will be replaced by newly ingested ones automatically.

>Thanks,

No worries.

>Lowell Schaffert
>Lockheed Martin Corporation
>1111 W. Betteravia Rd.
>Santa Maria, CA 93455
>
>Phone (805)348-2227
>e-mail address@hidden

Tom Yoksas
--