This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi Felix, re: re: I assume that a downstream LDM is running on Chinkapin and is not receiving NAM data from an upstream LDM somewhere. > As I understand it, this is the problem. We are indeed a downstream > LDM server and we are not receiving our NAM "conduit" data, which is > typically delivered via IDD/LDM. OK. re: What is the fully-qualified hostname on which the upstream LDM is running? > idd.cise-nsf.gov > idd.unidata.ucar.edu OK. These toplevel IDD relay nodes both have all of the data available in CONDUIT. If the data you are requesting is available, they will send them to you if you are allowed. > We have determined that these are our upstream LDM servers by > examining ldmd.conf. These are the lines from ldmd.conf where these > hostnames appear: > > request WMO ".*" idd.cise-nsf.gov > request WMO ".*" idd.unidata.ucar.edu > request NNEXRAD ".*" idd.cise-nsf.gov > request NNEXRAD ".*" idd.unidata.ucar.edu > request CRAFT ".*" idd.cise-nsf.gov > request CRAFT ".*" idd.unidata.ucar.edu > request CONDUIT > "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)!" > idd.cise-nsf.gov > request CONDUIT > "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)!" > idd.unidata.ucar.edu > request NMC "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) > !(.*)!" idd.cise-nsf.gov > request NMC "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) > !(.*)!" idd.unidata.ucar.edu The request lines for 'NMC' are redundant: NMC == GPSSRC|CONDUIT|FNEXRAD You can remove them from your ~ldm/etc/ldmd.conf file. NOTE: - Remember that after making changes to ldmd.conf, you will need to stop and restart your LDM for the changes to take effect: <as 'ldm'> -- edit ~ldm/etc/ldmd.conf and remove the REQUEST lines for NMC ldmadmin stop ldmadmin start re: Is there anything untoward in the LDM log file on Chinkapin? The log file is, typically, ~ldm/logs/ldmd.log. > The log file is empty. If ~ldm/logs/ldmd.log has zero length, it means that logging is not setup/setup correctly on your machine. The LDM log file is an invaluable resource in troubleshooting problems, so this is something that needs fixing. re: Is there anything untoward in the LDM log file on the upstream host? > I don't have access to these hosts presently. Should I? No. Steve asked this to see if you had contacted the LDM administrator on your upstream host(s). Your reply indicated that we are the administrator for both machines. I can assure you that your machine is allowed to request CONDUIT data on both of these toplevel relays. re: What does the notifmye command do when executed by the LDM user on Chinkapin: > Note here that setting the NAM_PATTERN in bash seemed impossible since > the pattern contains an exclamation mark, and bash won't let you escape > exclamation marks normally (an escaped excl. mark still prints the > backslash, which messes up the regex). I used zsh to issue this > command, but even then I had to add an additional space at the end of > the NAM_PATTERN string. This may have broken my regexp's > functionality. (?) I think that your regular expression pattern is responsible for you not receiving the data you want. > UPSTREAM_HOST set to: idd.unidata.ucar.edu > NAM_PATTERN set to: > ^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! > > RESULT: > > [ldm@chinkapin]~/ldmhome/etc% notifyme -vl- -h $UPSTREAM_HOST -f CONDUIT -o > 3600 -p $NAM_PATTERN > Mar 05 18:20:58 notifyme[11687] NOTE: Starting Up: idd.unidata.ucar.edu: > 20090305172058.353 TS_ENDT {{CONDUIT, > "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}} > Mar 05 18:20:58 notifyme[11687] NOTE: LDM-5 desired product-class: > 20090305172058.353 TS_ENDT {{CONDUIT, > "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}} > Mar 05 18:20:58 notifyme[11687] INFO: Resolving idd.unidata.ucar.edu to > 128.117.140.3 took 0.000269 seconds > Mar 05 18:20:58 notifyme[11687] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK Very good. This demonstrates that you have been allowed to request CONDUIT data from idd.unidata.ucar.edu. Question: - did you let this run for enough time to see if there were any products that match your pattern? I ask this because I just ran the a similar 'notifyme' from my workstation here at the UPC and got results: % notifyme -vl- -f CONDUIT -h idd.unidata.ucar.edu -p "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! " -o 100000 Mar 06 20:44:11 notifyme[9131] NOTE: Starting Up: idd.unidata.ucar.edu: 20090305165731.681 TS_ENDT {{CONDUIT, "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}} Mar 06 20:44:11 notifyme[9131] NOTE: LDM-5 desired product-class: 20090305165731.681 TS_ENDT {{CONDUIT, "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}} Mar 06 20:44:11 notifyme[9131] INFO: Resolving idd.unidata.ucar.edu to 128.117.140.3 took 0.000424 seconds Mar 06 20:44:11 notifyme[9131] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK Mar 06 20:44:16 notifyme[9131] INFO: 7645 20090306193739.826 CONDUIT 002 data/nccf/com/nam/prod/nam.20090306/nam.t18z.awip3d00.tm00.grib2 !grib2/ncep/NAM_84/#000/200903061800F000/VSBY/0 - NONE! 000002 Mar 06 20:44:16 notifyme[9131] INFO: 15420 20090306193739.827 CONDUIT 007 data/nccf/com/nam/prod/nam.20090306/nam.t18z.awip3d00.tm00.grib2 !grib2/ncep/NAM_84/#000/200903061800F000/AVOR/1000 Pa PRES! 000007 Mar 06 20:44:16 notifyme[9131] INFO: 10586 20090306193739.827 CONDUIT 012 data/nccf/com/nam/prod/nam.20090306/nam.t18z.awip3d00.tm00.grib2 !grib2/ncep/NAM_84/#000/200903061800F000/UREL/10 m HGHT! 000012 ... Notes: - you can sorround the regular expression pattern in double quotes so that the exclamation points do not get interpreted by your shell - I included the '-o 10000' flag to look back for products that match the specified pattern over the past 10000 seconds > UPSTREAM_HOST set to: idd.cise-nsf.gov > > RESULT: > [ldm@chinkapin]~/ldmhome/etc% notifyme -vl- -h $UPSTREAM_HOST -f CONDUIT > -o 3600 -p $NAM_PATTERN > Mar 05 18:29:10 notifyme[11932] NOTE: Starting Up: idd.cise-nsf.gov: > 20090305172910.568 TS_ENDT {{CONDUIT, > "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}} > Mar 05 18:29:10 notifyme[11932] NOTE: LDM-5 desired product-class: > 20090305172910.568 TS_ENDT {{CONDUIT, > "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}} > Mar 05 18:29:10 notifyme[11932] INFO: Resolving idd.cise-nsf.gov to > 192.12.209.62 took 0.00027 seconds > Mar 05 18:29:10 notifyme[11932] NOTE: NOTIFYME(idd.cise-nsf.gov): OK Ditto for idd.cise-nsf.gov: your data request was allowed. I took a look at the real time statistics that your machine is reporting back to us: Unidata HomePage http://www.unidata.ucar.edu Tools -> IDD http://www.unidata.ucar.edu/software/idd Real-Time IDD statistics http://www.unidata.ucar.edu/software/idd/rtstats/ Statistics by Host http://www.unidata.ucar.edu/cgi-bin/rtstats/siteindex chinkapin.cs.indiana.edu http://www.unidata.ucar.edu/cgi-bin/rtstats/siteindex?chinkapin.cs.indiana.edu From the last page, click on the CONDUIT 'volume' link to get a graphical representation of the CONDUIT data you have received recently: http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+chinkapin.cs.indiana.edu This plot shows that you _are_ getting CONDUIT data. Since your CONDUIT request is a smallish fraction of the full CONDUIT datastream, it is most likely that you are receiving all of the data that you are requesting. This, in turn, implies that you are not _processing_ the data you are receiving. The place to look now is your LDM pattern-action file entry(ies). Please send us the pattern-action file you are using to process the CONDUIT data you are requesting. In addition, please send us the ldmd.conf file also. It is likely that there is a simple mistake in your pattern action file that is preventing the CONDUIT data you are receiving from being processed. Side comment: I recommend that you upgrade your LDM installation to the latest version avaiable: LDM-6.7.0. The reason for this is that this version runs a sanity check on all pattern-action files being used to see if there are mistakes. Ordinarily, mistakes would be logged into the ~ldm/logs/ldmd.log file, but since your LDM logging is apparently not setup/setup correctly, we have no way of seeing the error(s). Cheers, Tom -- **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: BXH-661016 Department: Support IDD Priority: Normal Status: Closed