[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20020709: LDM5-Can I force a pqinsert of duplicate files?

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: 20020709: LDM5-Can I force a pqinsert of duplicate files?
  • Date: Tue, 09 Jul 2002 17:41:57 -0600
Bunny Pfau wrote:
> 
> 
> Well, I'm using LDM to transfer large data images from an
> observatory we support.  IN addition, I'm transferring some
> log files and such--just daily administrative stuff.
> SOMEtimes the contents of the files *could* (it's rare, but it
> happens every few months) turn out to
> be identical.  Yeah, I know--I could insert a timestamp
> into the file to make it unique, but then on the downstream
> side I just have to remove the timestamp.  ..So I thought I'd
> ask if there was a flag or variable to set that would allow
> me to override  PQUEUE_DUP ..
> 
> Bunny

Bunny, 

So, is it necessary to keep products in the queue for a long time? 
There's only a conflict if the identical product is still in the queue.

If you don't need to actually have the product in the queue twice, one
possibility is that before inserting a product you could try to delete
it first using pqexpire.  In general for the latest versions of the ldm
we advise against using pqexpire.  But it may work well for a situation
like this.  Assuming you're calling pqinsert from a script, you could
call pqexpire (with no interval so it doesn't loop) first.

The other way to keep products from languishing (assuming that's what
they're doing) in the queue is to have a smaller queue.  Space in the
queue is reused as it is needed, so if you have a smaller queue the
liklihood of the product being a duplicate would decrease.

Just brainstorming...  Anyway, there is no way to override PQUEUE_DUP. 
But, if you modify the code and you show us the changes we'll consider
incorporating your changes in future distributions.  If we were to
include the changes it would alleviate having to make the same
modifications for each upgrade.

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************