This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi Brian, I apologize for not getting back to you before now... re: > We request nogaps gribs via ldm from usgodae3.fnmoc.navy.mil, and for the > past week we have been noticing some missing fields. For example, last > night the 2006061400 nogaps is missing the PRMSL (slp) field at f00, f12, > and f36. We have not changed anything on our end, so I wonder if you can > please check your nogaps file, or contact fnmoc to see if they have had > some issues. I notice that you are receiving alot of data on the machine that is requesting FNMOC data: thunder.msrc.sunysb.edu http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?thunder.msrc.sunysb.edu It is possible that you are not processing the data out of your LDM queue fast enough to ensure that data in the queue does not get overwritten by new data being received during peak ingest periods. In order to determine if you are losing data out of the LDM queue, please check the following: - the size of your LDM queue. Your peak ingest rate is just about 2GB/hour. If your queue is undersized (many sites never increase their queue size above the 400 MB default), you can easily lose data (not process, that is) that you have successfully received - you can get more information from the 'pqact' invocation(s) that are processing data out of your LDM queue by putting it(them) into verbose logging mode by sending the process ID a 'kill -USR2' signal. The first 'kill -USR2' puts 'pqact' into verbose logging mode; the second puts it into debug logging mode; the third returns the process to normal (notice) logging mode. Put the process(es) into debug mode when the FNMOC data is being received and see how long it is taking to process the data out of the queue. NOTE: do not forget to leave the debug logging mode! This mode will generate LOTS of output in your ~ldm/logs/ldmd.log file!! - if you are trying to process all of the data out of your LDM queue using a single 'pqact' process, then I can all but guarantee that this is the cause of your data loss. One instance of 'pqact' has little chance of processing 2 GB/hour of data As far as our checking our receipt of the FNMOC data, please alert us to the next instance where you see data loss (after checking the things I listed above), and we will see if we missed data. We can not go back to the 14th to check since we don't keep the data around for that long. > Thanks for your time. Again, I apologize for not being able to get back to you on your inquiry before now. Cheers, Tom **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: GFZ-680741 Department: Support IDD Priority: High Status: Closed