[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: ldmadmin mkqueue problem - solved, further file locking questions

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: ldmadmin mkqueue problem - solved, further file locking questions
  • Date: Thu, 24 May 2007 07:34:21 -0600
Alan,
For now, this leaves us with the only option of running LDM pointing to a local disk -- not ideal for sharing with our cluster systems.
We can work around this by staging data out via the network to a cluster-mounted file system, but this is also not ideal. Before doing this, I wanted to know whether the file locking features in pq.c are essential, and if not, whether they could be relaxed if running LDM under controlled conditions. Do you have any suggestions?
File-locking of the product-queue is critical to the correct operation of the LDM system.
If you are collecting data from several upstream sites, then you could have one computer running an LDM that performs the collection and another computer that also runs an LDM that receives the data from the first, collecting computer. This second computer could run a "pqact" process that writes data-products to Lustre files.
Alternatively, only the first LDM system could write data-products to Lustre files via "pqact" processes.
In any case, you should drop any ideas of sharing an LDM product-queue amongst multiple computers. Multiple processes on the same computer, yes; multiple computers, no. 

Regards,
Steve Emmerson