This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
HI Karen, > We have been using ldm6.0.14 to move some data (algorithm products) > between machines for quite a while, but recently started using it for a > "new" set of products and we seem to be having some problems. > > Only a small subset of the data is getting in appropriately, and we are > getting messages from pqinsert about "Product already in queue" even > though the name of the products are unique. > > An example is: > Jun 21 11:06:02 nmqxrt-12 pqinsert: Product already in queue: > 8a6ab6685c1f80411e1de1a9232ac46c 9009 20060621160602.253 DDS > 000 /home/qpesums/data/realtime/vpr/KUDX/vpr/VPR_INST.20060621.1605.dat > Jun 21 11:06:02 nmqxrt-12 pqinsert: 9009 20060621160602.259 DDS > 000 /home/qpesums/data/realtime/vpr/KUDX/vpr/VPR_AVG.20060621.1605.dat > Jun 21 11:06:02 nmqxrt-12 pqinsert: Product already in queue: > 8a6ab6685c1f80411e1de1a9232ac46c 9009 20060621160602.445 DDS > 000 /home/qpesums/data/realtime/vpr/KVNX/vpr/VPR_INST.20060621.1605.dat > Jun 21 11:06:02 nmqxrt-12 pqinsert: Product already in queue: > 8a6ab6685c1f80411e1de1a9232ac46c 9009 20060621160602.451 DDS > 000 /home/qpesums/data/realtime/vpr/KVNX/vpr/VPR_AVG.20060621.1605.dat > Jun 21 11:06:02 nmqxrt-12 pqinsert: Product already in queue: > 8a6ab6685c1f80411e1de1a9232ac46c 9009 20060621160602.624 DDS > 000 /home/qpesums/data/realtime/vpr/KVTX/vpr/VPR_INST.20060621.1605.dat > Jun 21 11:06:02 nmqxrt-12 pqinsert: Product already in queue: > 8a6ab6685c1f80411e1de1a9232ac46c 9009 20060621160602.630 DDS > 000 /home/qpesums/data/realtime/vpr/KVTX/vpr/VPR_AVG.20060621.1605.dat > > As you can see we are giving the full directory path for pqinsert, as > some of the file names are common, but paths are not. Yet each of the > products is being given the same product ID according to pqinsert. > Could you tell me how it determines whether a product is unique? Is it > possible our string is too long -- and the first parts are identical? > Or is it that each of these is getting inserted every 5 minutes with > only the very end of the filename (the .hhmm.data part being modified? > And yet we are still getting *some* of them into the queue.... I think that the rejected data-products are identical. pqinsert(1) computes the data-product "signature", which is an MD5 checksum, of the data portion of the data-product. It doesn't use the product-identifier in computing the signature. If the data in the data-products that you're inserting is the same, then the resulting signatures will be the same and the data-product will not be allowed in the product-queue. If you really want the same data to be assigned to multiple data-products, then you'll have to vary them somehow. One way would be to insert a counter somewhere in the data-product. Another way would be to add a timestamp to the data-product. Anything that will make the data portion of the data- product unique will work. Regards, Steve Emmerson Ticket Details =================== Ticket ID: PAK-752919 Department: Support LDM Priority: Normal Status: Closed