This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Mike, pqact(1) uses the signature (i.e., MD5 checksum) of the last, successfully-processed data-product from a "*.state" file to determine where to start processing data-products from the product-queue. If the corresponding product exists, then pqact(1) will start just after that product; otherwise, pqact(1) will start with the product whose age in the product-queue in seconds is given by the argument of the "-o" option. The default is zero, i.e., start with the most recent product. On another issue, specifying a time-offset of 24 hours for the LDM system will only be effective if the product-queue of the upstream LDM holds at least 24 hours worth of data. > Hello Steve, > > I was wondering if you could help me clear up one more point of > understanding with LDM. > > The way UNAVCO treats data is that it's timeless and needs to be collected > regardless of delays. I know this is different than the LDM model, > but I think I have the tuning set to work for us. After an outage I was > using the fields time-offset and max-latency in the registry.xml file to > have the downstream LDM request old queued files for up to 24 hours past > current time. This however was not producing the results I expected. > I found the *.info files in the ldm home directory and tried to clear > those so that there were no indications of the last file processed by > the downstream LDM. > > This still didn't help. Next I cleared the downstream product queue and > still had unexpected results. What I finally discovered is that in the > ldmd.conf file where pqact gets forked, needed to have add the -o flag > set with a value to match the max-latency value. Now that I have all the > settings I need I believe I understand how the LDM process was working. > The downstream was requesting the data from upstream correctly but then > pqact was considering the data too old because the default offset for > pqact was only around 10 minutes. When pqact saw old data it sent a > 'reclass' message I believe to tune the data request with another > feedme request to the upstream LDM starting at the current time minus > ten minutes. This is what I gathered from the entries in the log file. > > I think I now have everything appropriately tuned so that after an > outage our downstream LDM should go back in time up to 24 hours to > retrieve products starting with the last file processed as logged in the > .info file. Performance is good under this configuration so I'm happy > with the version of LDM I am running. What I was most curious about is > could you confirm my understanding of the behavior of LDM and can you > give me a reason why it's advantageous for pqact to run on it's own flags > in the ldmd.conf file and not take it's offset from the registry.xml file. > > Thanks > > -Mike Regards, Steve Emmerson Ticket Details =================== Ticket ID: AZV-974943 Department: Support LDM Priority: Normal Status: Closed