[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #HCO-891524]: Unidata Performance Question

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #HCO-891524]: Unidata Performance Question
  • Date: Mon, 23 Mar 2015 09:41:13 -0600
Mike,

I'm convinced that your product-queue was corrupted somehow.

Please keep me apprised.

> Steve, The same thing happened again yesterday morning but this time I have 
> log captures to demonstrate what happened.  Pqinsert fails when the max 
> number of slots is reached.  I estimated a file rate for the queue and set a 
> value along with a size in the registry.xml file.  I figured it might make 
> the file management effective to have a product counter that wasn’t much 
> larger than what I expected for the number of products it would take to fill 
> the queue.
> 
> In yesterdays case we were receive a larger ingest of smaller files so the 
> product queue had grown to about 16GB but the count of 65536 slot was reached 
> and the log file I’m attaching shows that pqinsert failed, then the process 
> hung up backed up all the other pqinserts and corrupted the queue.  This was 
> the behavior I was expected.  I was under the presumption that if the max 
> queue slots was reached the oldest product was deleted to make room for the 
> next one.
> 
> To mitigate the problem in the mean time I quadrupled the number of slots in 
> the registry.xml configuration, rebuilt the queue and will watch and see in 
> about 4 more days when the queue fills up how LDM does.
> 
> -Mike
> 
> This is line 67486 of the attached log file:
> pqinsert ERROR: fb layer ran out of product slots, too many products in queue
> 
> 
> 
> 
> Just an FYI for you.
> -Mike

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: HCO-891524
Department: Support LDM
Priority: Normal
Status: Closed