This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
>From: "Patrick L. Francis" <address@hidden> >Organization: BGSU >Keywords: 200510312116.j9VLGv7s008531 IDD re: I do not understand what you are asking for. >i am sorry... i have students in and out today and was thinking >faster than my fingers are keeping up... as soon as I try to >work I am 'interrupted' .. 420 students is too many... Wow, that is a heavy load! >is why I >keep saying verbal conversations via phone / internet >would be more conducive to support instead of sending >emails back and forth.. I strongly disagree. With verbal communications there is no "paper trail". Since we make email transactions accessible to search engine robots, others are able to learn from transactions on subjects of interest. If we conducted support by phone, we would be spending lots more time documenting conversations so that the information would not be lost. We have had good feedback over the years that use of email in our support process is extremely valuable to the community. >anywho ;) > >a sample of how to adjust LDMD.CONF to specify 1,2,or 3 products to >stream though instead of an entire feed. OK, how I understand. The easiest way to learn how to limit requests is use the LDM facility 'notifyme' to list out the headers of products in a datastream. Here is a simple example for the NEXRAD Level III feed NNEXRAD: <as 'ldm'> notifyme -vxl- -f NNEXRAD -o 3600 Nov 01 22:25:30 notifyme[48640] NOTE: Starting Up: localhost: 20051101212530.200 TS_ENDT {{NNEXRAD, ".*"}} Nov 01 22:25:30 notifyme[48640] NOTE: LDM-5 desired product-class: 20051101212530.200 TS_ENDT {{NNEXRAD, ".*"}} NOTIFYME(localhost) returns OK Nov 01 22:25:30 notifyme[48640] NOTE: NOTIFYME(localhost): OK Nov 01 22:25:31 notifyme[48640] INFO: 503d9768c3deccb88328b8c2b609c559 4954 20051101213631.138 NNEXRAD 45155629 SDUS53 KAPX 012128 /pNTPAPX Nov 01 22:25:31 notifyme[48640] INFO: 07fd74d856b0c6ce5c5eff84159e48f1 4479 20051101213631.140 NNEXRAD 45155630 SDUS58 PAFC 012132 /pNTPAIH Nov 01 22:25:31 notifyme[48640] INFO: a96a1f5d80f121c2162433cf22b0216f 3540 20051101213631.143 NNEXRAD 45155631 SDUS33 KAPX 012128 /pN1PAPX Nov 01 22:25:31 notifyme[48640] INFO: 28127883454545bfcd5d199e42847ea8 3860 20051101213631.144 NNEXRAD 45155632 SDUS25 KABQ 012135 /pN1RFDX Nov 01 22:25:31 notifyme[48640] INFO: d9b1de11ca3eeefc074e4f79d712f6bb 856 20051101213631.145 NNEXRAD 45155633 SDUS53 KMQT 012130 /pNCRMQT ... Notice that there is a field in the header that says what the product is and what the station is: ... 856 20051101213631.145 NNEXRAD 45155633 SDUS53 KMQT 012130 /pNCRMQT ^ ^__ station |_____ product Next use 'notifyme' to list out _only_ the NCR products from MQT: notifyme -vxl- -f NNEXRAD -o 3600 -p NCRMQT Nov 01 22:27:47 notifyme[48693] NOTE: Starting Up: localhost: 20051101212747.043 TS_ENDT {{NNEXRAD, "NCRMQT"}} Nov 01 22:27:47 notifyme[48693] NOTE: LDM-5 desired product-class: 20051101212747.043 TS_ENDT {{NNEXRAD, "NCRMQT"}} NOTIFYME(localhost) returns OK Nov 01 22:27:47 notifyme[48693] NOTE: NOTIFYME(localhost): OK Nov 01 22:28:03 notifyme[48693] INFO: d00abacf421b08fe5573a5ee46a8c103 857 20051101214222.845 NNEXRAD 45158813 SDUS53 KMQT 012136 /pNCRMQT Nov 01 22:28:37 notifyme[48693] INFO: b48478115722b93280dd3e7844ae3af3 824 20051101214808.384 NNEXRAD 45161498 SDUS53 KMQT 012141 /pNCRMQT Once you have figure out the pattern for just the products you want, you can modify your ldmd.conf 'request' line(s) to request just those products from the datastream. Here is an example of requesting just the N0R (base reflectivity at tilt 1) images for all stations in the NNEXRAD feed: request NNEXRAD "/pN0R" idd.unidata.ucar.edu and so on. re: your machines are essentially idling >yes > >ldm@unidata3:~/runtime/bin$ uptime > 15:32:27 up 8 days, 21:27, 1 user, load average: 0.01, 0.11, 0.07 > >ldm@unidata2:~/runtime/bin$ uptime > 15:33:18 up 70 days, 22:52, 1 user, load average: 0.11, 0.19, 0.24 > >[ldm@unidata bin]$ uptime > 15:33:02 up 8 days, 9:21, 1 user, load average: 0.14, 0.16, 0.16 > >[ldm@weather bin]$ uptime > 3:33pm up 223 days, 22:50, 1 user, load average: 0.19, 0.15, 0.12 This is as I expected. I can't recall if we have ever seen machine loading intefere with the ability of an LDM to ingest data into its queue; the intest/relay component of the LDM uses very little system resource. Site-reported data problems typically fall into three categories: - high latencies caused by external factors such as packet shapers, slow network links, and firewall configurations - slow processing caused by the user having one 'pqact' invocation process all of the data being ingested - the user machine ingesting LOTS of data and their LDM queue not being large enough to hold the received data long enough for it to get processed by pqact(s) The first problem is mitigated by finding the source of the latency and working with the appropriate contact (e.g., IT, etc.) to resolve it. The second problem is mitigated by configuring ldmd.conf to run more than one pqact invocation while making sure that each invocation processes a portion of the jobs to be done (i.e., not have any overlap in processing duties). The third problem is mitigated by implementing the same fix for the second problem AND by increasing the size of the queue. Cheers, Tom -- NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.