20041212: LDM question

>From: "Edwin Pryor" <address@hidden>
>Organization: DOC/NOAA/NWS - National Weather Service
>Keywords: 200412121835.iBCIZalI029494 LDM queue check

Hi Ned,

>I originally tried to reach Jeff Weber but learned he is on travel.
>Any assistance you may provide on my LDM question would be appreciated.

Please send all inquiries directly to Unidata User Support
<address@hidden>.  Since multiple folks here at the UPC read
the support inbox, you are less likely to have to wait for a reply than
when sending questions directly to a UPC staff member who, like Jeff,
may be on travel.

>My name is Ned Pryor and I work for the NWS Middle Atlantic River
>Forecast Center in State College, PA.
>
>I spoke to you recently as we installed LDM version 5.2.2 on a Linux
>machine that is part of our AWIPS network.

Hopefully, Jeff strongly advised you to upgrade your LDM 5.x installation
to the latest LDM-6 offering.  The primary reasons for this are:

- we no longer support LDM-5
- LDM-6 is much more efficient at relaying data than LDM-5
- LDM-6 fixed several bugs that existed in LDM 5.x

>We have used various
>versions of LDM  over the years and I was never quite sure about storing
>incoming products in the event LDM stopped running.  We have had times
>when LDM has died for one reason or another but we continue to queue
>products using "pqinsert" from a separate data handling script.

The issues you note about unexpected LDM-5 stops are a good reason to
upgrade to the latest LDM-6 distribution.  Our experience with LDM-6
is that it runs for months without any interruption.  In fact, the
LDMs running on our NOAAPORT ingest machines only get restarted when
we do something like upgrade the OS or software that processes the
NOAAPORT offerings into the various IDD datastreams or there is
an power outage.

>Ideally
>we would like LDM (once restarted) to work its way through any queue
>that has built up over time rather than blasting the queue or paring the
>queue.  Do you have any ideas or info on this topic?

Depending on why your LDM stops, there may or may not be a need to
delete and remake the queue.  The automatic startup script that we
recommend folks run to start the LDM at machine bootup checks the
ingegrity of the queue so as to not lose anything in it if it is not
damaged.  If the queue is damaged, there is no option but to delete and
remake the queue and then start the LDM.

You could/should adopt the same approach of checking your queue
integrity before deleting and remaking it.  There are two ways of doing
this:

LDM-6.1.x:

- use 'pqcheck -q <LDM_product_queue>' to check the product count in
  the queue.  If the count is not zero, the queue may be damaged

LDM-5.2.x, LDM-6.X:

- use 'pqcat -s' to check the queue

If you are running LDM-6.1.x and the 'pqcheck' test passes, you should
be able to safely start your LDM.  If the 'pqcheck' check fails, you
should run the 'pqcat' check to see if the queue is, in fact, in an
inconsistent state.  If it is not, you should be able to start the LDM
safely.  If the 'pqcat' check fails or hangs, you will need to delete
and remake your queue before restarting.

PLEASE upgrade to use of the latest LDM-6 distribution.  The NWS
adopted this distribution for use in the NEXRAD Level II gathering and
redistribution project, so it is not like it is not being used in an
operational setting.

Cheers,

Tom Yoksas
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.