[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 200004271735: LDM pq_size

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: 200004271735: LDM pq_size
  • Date: Fri, 28 Apr 2000 08:39:47 -0600
Collin Daly wrote:

> Hi,
>
> We're building an LDM (5.0.8) on a Red Hat Linux 6.1 and have gotten to the
> point of changing our pq_size.  It appears that data is not outputted until
> the queue is filled as we've increased our queue from 35MB to 500MB.  Is
> this what is actually happening?  What are the pro's and con's of having a
> larger queue?  Also it seem that the theoretical limit for the buffer of the
> queue is 1-1.6GB is this more or less true?
>
> Thanks,
> Collin
>
> Collin Daly
> Meteorological Systems Engineer
> Alden Electronics, Inc.
> 40 Washington Street
> Westborough, Ma 01581
> Voice: 508.366.8851 x.2268
> Fax:    508.898.2427
> e-mail: address@hidden
>  visit us at:
> http://www.alden.com

Hi Collin,

I'm not sure what you mean by "data is not outputted until the queue is
filled".  Your products should be acted upon by pqact almost immediately after
they've been inserted into the queue.  The queue size does not affect the
actions on the products.

We're assuming that you're receiving NOAAPort data.  We've determined that a
queue size of 150MB should be plenty for that stream.

Especially on a Linux box, the size of the queue could affect the performance
of the machine depending on how much RAM you have.  This is because the queue
is a memory mapped file.    (It used to be the case that Linux didn't handle
the memory mapped files well - I need to check whether or not that problem has
been addressed.)  Having enough RAM to hold your entire queue is ideal.  If the

queue is larger than the amount of RAM, portions will be swapped in and out of
memory, incurring the related overhead.   The larger the queue to memory ratio,

the greater the swapping.

Yes, you are right about the limit on queue size being 1 - 1.5GB.  We've made
queues up to 2GB, but the performance wasn't satisfactory.  Right now we're
investigating whether we might be able to make some changes and possibly double

the max queue size.

One other thing.  Please send support questions to address@hidden
rather than to me directly.  This way, if I'm unavailable for some reason then
someone else will answer your question rather than having it languish in my
mail box.  Thanks!

Anne

--
***************************************************
Anne Wilson                     UCAR Unidata Program
address@hidden                  P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************