[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20040612: large queue support under Linux

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: 20040612: large queue support under Linux
  • Date: Mon, 14 Jun 2004 09:08:12 -0600
Tom,

This is good, actionable information.  I'll start work on it today.

Offhand, I don't know why the "4000000000" specification didn't work.

--Steve

> >From: Unidata User Support <address@hidden>
> >Organization: Unidata Program Center/UCAR
> >Keywords: LDM Linux large queue
>
> 
> Steve,
> 
> While trying to upgrade the queue a Fedora Core 1 Linux machine (it was
> using a 2 GB queue, and the age of the oldest product in the queue was
> less than one hour since it is getting everything in the IDD), I learned
> out that you can set CPPFLAGS so that large file support is include in
> LDM routines:
> 
> cd ~ldm/ldm-6.0.14/src
> make distclean
> export CPPFLAGS="-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
> ./configure && make && make install && sudo make install_setuids
> 
> I then edited ~ldm/bin/ldmadmin and set $pq_size to 4 GB:
> 
> $pq_size = 4000000000;
> 
> and then used ldmadmin to delete and make a new queue:
> 
> ldmadmin delqueue
> ldmadmin mkqueue -f
> 
> No matter what numeric value greater than 2 GB I set in ldmadmin, a 2
> GB queue was all that was created.  I even tried quoting the numeric
> size value:
> 
> $pq_size = "4000000000";
> 
> But I still got only a 2 GB queue.
> 
> When I tried to set the queue size using a format that is supported by
> pqcreate, 4000M:
> 
> $pq_size = 4000M;
> 
> I got an error.
> 
> Quoting the queue size in the 'M' format, on the other hand:
> 
> $pq_size = "4000M";
> 
> did result in a properly sized queue.
> 
> So, two recommendations for the next LDM release are:
> 
> - quote the pq_size field in ldmadmin and add instructions for
>   how to specify a queue size greater than 2 GB
> 
> - update the web page that talks about building large file support
>   into the LDM on Linux to tell folks that then not only have to
>   set CPPFLAGS, but they also have to change the format of
>   pq_size in ldmadin:
> 
> http://my.unidata.ucar.edu/content/software/ldm/ldm-6.0.14/basics/huge-queues.html#Linux/Intel
> 
> I just wanted to get this "down on paper" before the meeting Monday.
> 
> Cheers,
> 
> Tom
> --
> +-----------------------------------------------------------------------------+
> * Tom Yoksas                                             UCAR Unidata Program 
> *
> * (303) 497-8642 (last resort)                                  P.O. Box 3000 
> *
> * address@hidden                                   Boulder, CO 80307 *
> * Unidata WWW Service                            http://www.unidata.ucar.edu/ 
> *
> +-----------------------------------------------------------------------------+