20040612: large queue support under Linux

>From: Unidata User Support <address@hidden>
>Organization: Unidata Program Center/UCAR
>Keywords: LDM Linux large queue

Steve,

While trying to upgrade the queue a Fedora Core 1 Linux machine (it was
using a 2 GB queue, and the age of the oldest product in the queue was
less than one hour since it is getting everything in the IDD), I learned
out that you can set CPPFLAGS so that large file support is include in
LDM routines:

cd ~ldm/ldm-6.0.14/src
make distclean
export CPPFLAGS="-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
./configure && make && make install && sudo make install_setuids

I then edited ~ldm/bin/ldmadmin and set $pq_size to 4 GB:

$pq_size = 4000000000;

and then used ldmadmin to delete and make a new queue:

ldmadmin delqueue
ldmadmin mkqueue -f

No matter what numeric value greater than 2 GB I set in ldmadmin, a 2
GB queue was all that was created.  I even tried quoting the numeric
size value:

$pq_size = "4000000000";

But I still got only a 2 GB queue.

When I tried to set the queue size using a format that is supported by
pqcreate, 4000M:

$pq_size = 4000M;

I got an error.

Quoting the queue size in the 'M' format, on the other hand:

$pq_size = "4000M";

did result in a properly sized queue.

So, two recommendations for the next LDM release are:

- quote the pq_size field in ldmadmin and add instructions for
  how to specify a queue size greater than 2 GB

- update the web page that talks about building large file support
  into the LDM on Linux to tell folks that then not only have to
  set CPPFLAGS, but they also have to change the format of
  pq_size in ldmadin:

http://my.unidata.ucar.edu/content/software/ldm/ldm-6.0.14/basics/huge-queues.html#Linux/Intel

I just wanted to get this "down on paper" before the meeting Monday.

Cheers,

Tom
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                            http://www.unidata.ucar.edu/ *
+-----------------------------------------------------------------------------+