[LDM #ZOC-298338]: Missing data problems

Hi Mark,

re:
> Tom, I just received work from the NWS that they are running v6.6.3 of
> LDM,

OK, so there should not be a version-related problem.

> so i don't know where that leaves us with finding a reason for the
> files being dropped between here and there.
> 
> Any ideas?

Since we are not receiving statistics from the NWS site who is indicating
a loss of products, we can not verify that they are, in fact, not getting
all of the products that they should.  The reasons that I emphasize this
are:

- there is no end-to-end comparisons possible for number of products and
  volume of data being delivered

- we have seen numerous instances of sites reporting that they are not
  receiving all of the products that they should when they are, in fact,
  receiving all of the products.  In a number of cases, the perception
  that not all of the products are being received is caused by the pqact.conf
  processing of products out of the downstream queue not being fast enough
  to process products before they are overwritten by newly received products.
  This can happen when the downstream site is using a single 'pqact' to
  process lots of products (the processing of pqact.conf actions is
  serialized) and/or when the downstream machine's LDM queue is too
  small to hold the received products for a period long enough for pqact.conf
  processing to occur fully

So, we must ask the following question again:

- How is the NWS site determining that they are not receiving all of the
  products that they believe that they should be getting?

If the answer is that they are not seeing all of the products being processed
out of the queue, then the solution might be that they need to breakup their
pqact.conf processing into mutually-exclusive pieces that are each handled
by a separate instance of 'pqact'.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: ZOC-298338
Department: Support LDM
Priority: Emergency
Status: Closed