[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #TWQ-411566]: Successful pqinsert but files missing downstream

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #TWQ-411566]: Successful pqinsert but files missing downstream
  • Date: Wed, 14 Jun 2023 15:34:07 -0600
Hi Lee,

> I've first tried the easiest suggested and keeping what's inserted into
> the queue on the same machine, and I'm finding that I'm not getting all
> files that are inserted even on the same machine.  It looks like it may
> be a residence time issue and queue size issue combined, and on both the
> server and requesting machine.  I'm getting zero for a status value for
> the pqinsert, so it's at least terminating correctly.  However, I've
> thrown the debug flag in as well, and I'm seeing several of the
> following at random times:
> 
> 20230614T190316.649827Z pqinsert[2792931]
> pq.c:rpqe_mkspace:4945              DEBUG pq.c:rpqe_mkspace(): Deleting
> oldest to make space for 97362528 bytes

This is normal and to be expected.

> Of course the byte size listed is changing depending on the products
> sent.  I've read some literature speaking of balancing the queue for
> residence time and size, but it doesn't seem to be totally clear to me. 
> Do you have any resource that explains better how to balance the LDM
> queues and why it would need to be set as such?

Basically, the minimum residence time of a product in the queue must be greater 
than the maximum latency parameter in the LDM registry. The residence time is 
best found via the "ldmadmin plotmetrics" command. If your installation doesn't 
support that, then you can periodically execute the command "pqmon -v"; the 
last output field will be the age of the oldest product in the queue in 
seconds. The maximum latency parameter can be found via the command "regutil | 
grep max".

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: TWQ-411566
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.