Hi Bei, re: > This morning I use the ldm watch, OK, I assume you mean 'ldmadmin watch'... re: > On 11:15, data flow is shown, and after I restart the ldm, WAIT FOR 45 > MINUTES, on 12:00, the data flow is shown again. So, I guess when I > can't see the data flow, it's because RO event is over? Yes, I believe so. The data does not flow continuously, it is episodic. It is likely that you have been unlucky enough to hit the end of a transmission sequence. re: > When I can't see data flow by ldm watch, I tried notifyme -vl- -f ANY, > the output is: > [ldm@Bei ~]$ notifyme -vl- -f ANY > Feb 28 19:26:59 notifyme[3352] NOTE: Starting Up: localhost: > 20110228192659.250 TS_ENDT {{ANY, ".*"}} > Feb 28 19:26:59 notifyme[3352] NOTE: LDM-5 desired product-class: > 20110228192659.250 TS_ENDT {{ANY, ".*"}} > Feb 28 19:26:59 notifyme[3352] INFO: Resolving localhost to 127.0.0.1 > took 0.009469 seconds > Feb 28 19:26:59 notifyme[3352] NOTE: NOTIFYME(localhost): OK OK, this shows two things: - 'notifyme' was able to contact the LDM top level process on the local machine (this is a GOOD thing :-) - there was no data being received at the time you ran 'notifyme' Comment: - 'notifyme' is an incredible useful utility. It can not only show what you are receiving "now"; it can also show what you received for some period in the past (the length of time you can see "into to past" depends on how large your LDM queue is. For instance, to see all of the products received by your LDM in the past hour\ (i.e., in the past 3600 seconds), you would run: <as 'ldm'> notifyme -vl- -f ANY -o 3600 - 'notifyme' can be used to see what the upstream host you are trying to ingest data from is receiving now, or for 'n' seconds in the past. For instance: Show what products the upstream host idd.unidata.ucar.edu is receiving in the EXP datastream: <as 'ldm'> notifyme -vl- -f EXP -h idd.unidata.ucar.edu Show what Cosmic products the upstream idd.unidata.ucar.edu has received in the EXP feed type in the past hour (3600 seconds): <as 'ldm'> notifyme -vl- -f EXP -h idd.unidata.ucar.edu -o 3600 - 'notifyme' accepts a "pattern" through the '-p' flag. This allows you to narrow the list of products you want reporting on. For instance, <as 'ldm'> notifyme -vl- -f EXP -h idd.unidata.ucar.edu -o 3600 -p cosmicrt tells 'notifyme' to report on the products in the EXP feedtype whose product IDs match the regular experssion "cosmicrt" (i.e., that have 'cosmicrt' in the product IDs). The pattern for the '-p' flag can be specified to be very simple (like the example just stated) or much more complex. It is the latter capability that is of use to the end-user who wants to figure out how to request a specific subset of products in a data feed, and/or to fashion an extended regular expression for a pattern-action file action. So, I recommend using 'notifyme' to learn what the products look like that have been received in a particular datastream: <as 'ldm'> notifyme -vl- -f EXP -p cosmicrt -o 3600 Feb 28 21:46:24 notifyme[15425] NOTE: Starting Up: localhost: 20110228204624.941 TS_ENDT {{EXP, "cosmicrt"}} Feb 28 21:46:24 notifyme[15425] NOTE: LDM-5 desired product-class: 20110228204624.941 TS_ENDT {{EXP, "cosmicrt"}} Feb 28 21:46:24 notifyme[15425] INFO: Resolving localhost to 127.0.0.1 took 0.00068 seconds Feb 28 21:46:24 notifyme[15425] NOTE: NOTIFYME(localhost): OK Feb 28 21:46:25 notifyme[15425] INFO: 447096 20110228211011.551 EXP 000 /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C001.2011.059.19.08.G15_0001.0001_nc Feb 28 21:46:25 notifyme[15425] INFO: 448716 20110228211011.589 EXP 000 /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C001.2011.059.19.08.G28_0001.0001_nc Feb 28 21:46:25 notifyme[15425] INFO: 498096 20110228211011.639 EXP 000 /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C001.2011.059.19.27.G22_0001.0001_nc Feb 28 21:46:25 notifyme[15425] INFO: 544076 20110228211011.753 EXP 000 /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C001.2011.059.19.58.G28_0001.0001_nc Feb 28 21:46:26 notifyme[15425] INFO: 415096 20110228211011.818 EXP 000 /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C001.2011.059.19.58.G31_0001.0001_nc Feb 28 21:46:26 notifyme[15425] INFO: 466996 20110228211011.838 EXP 000 /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C001.2011.059.20.09.G11_0001.0001_nc ... We can see from this (abbreviated) listing that Cosmic product IDs look like: /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C005.2011.059.17.22.G10_0001.0001_nc /pub_out/cosmicrt/level2/atmPrf/2011.059/atmPrf_C005.2011.059.17.32.G16_0001.0001_nc ... Using this information, we can fashion an extended regular expression to test: notifyme -vl- -f EXP -o 1000000 -p '([12][0-9][0-9][0-9])\.([0-3][0-9][0-9])/(bfr|ion|wet)Prf(.*_nc)' Feb 28 21:48:36 notifyme[15635] NOTE: Starting Up: localhost: 20110217080156.706 TS_ENDT {{EXP, "([12][0-9][0-9][0-9])\.([0-3][0-9][0-9])/(bfr|ion|wet)Prf(.*_nc)"}} Feb 28 21:48:36 notifyme[15635] NOTE: LDM-5 desired product-class: 20110217080156.706 TS_ENDT {{EXP, "([12][0-9][0-9][0-9])\.([0-3][0-9][0-9])/(bfr|ion|wet)Prf(.*_nc)"}} Feb 28 21:48:36 notifyme[15635] INFO: Resolving localhost to 127.0.0.1 took 0.000678 seconds Feb 28 21:48:36 notifyme[15635] NOTE: NOTIFYME(localhost): OK Feb 28 21:48:37 notifyme[15635] INFO: 11264 20110228212018.886 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.17.32.G17_0001.0001_nc Feb 28 21:48:37 notifyme[15635] INFO: 11912 20110228212018.910 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.17.35.G04_0001.0001_nc Feb 28 21:48:37 notifyme[15635] INFO: 13040 20110228212018.919 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.17.49.G20_0001.0001_nc Feb 28 21:48:37 notifyme[15635] INFO: 11624 20110228212019.032 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.17.57.G31_0001.0001_nc Feb 28 21:48:37 notifyme[15635] INFO: 13160 20110228212019.095 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.17.57.G32_0001.0001_nc Feb 28 21:48:37 notifyme[15635] INFO: 11312 20110228212019.112 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.18.12.G16_0001.0001_nc Feb 28 21:48:37 notifyme[15635] INFO: 11696 20110228212019.139 EXP 000 /pub_out/cosmicrt/level2/ionPrf/2011.059/ionPrf_C005.2011.059.18.23.G03_0001.0001_nc ... OK, now we have a useful extended regular expression that can be useful in a pattern-action file action: ([12][0-9][0-9][0-9])\.([0-3][0-9][0-9])/(bfr|ion|wet)Prf(.*_nc) So, a pattern-action file action that should work to capture Cosmic data in the EXP feed type would be: EXP ([12][0-9][0-9][0-9])\.([0-3][0-9][0-9])/(bfr|ion|wet)Prf(.*_nc) FILE -close data/Cosmic/\1\2/\3Prf/\3Prf\4 NOTE: make sure that all whitespace in this pqact.conf entry are tabs, not spaces! This should create files in the ~ldm/data/Cosmic/2011059/ionPrf/ directory, and the file names should look like: ionPrf_C005.2011.059.18.23.G03_0001.0001_nc re: > I guess it's normal, and I think this problem is shot.... I am not sure I understand this, but no worries... re: > So till now the only problem is data can not be saved and logging. The pattern I sent you earlier was incorrect -- it assumed that there was a '.' between the 'nc' at the end of the Product IDs when the character right before the 'nc' is an '_' (underscore). You should use the new pattern-action file entry above in replacement of the one I sent you earlier. Again, remember that anytime you make a modification to a pattern-action file action, you should check to make sure that there were no obvious/gross/big errors: <as 'ldm'> ldmadmin pqactcheck ldmadmin pqactHUP <- if no errors were reported from 'ldmadmin pqactcheck' re: > I attached all the important conf files to you. And the 'ldm' > permission for all these conf files and ldmd.log is rw. And also > hupsyslog is setuid to root: > > -rwsr-xr-x. 1 root ldm 11246 Feb 18 17:12 /home/ldm/bin/hupsyslog Very good. I have edited the pqact.conf file and updated the regular expression pattern in the action designed to FILE Cosmic data. The updated pqact.conf file is attached to this reply. re: > I don't know if there is anything else I can let you know, but > do you mind remote logon my computer to check all the configurations?... > My IP is 136.129.122.134 ... I can not SSH to your machine. I suspect that either a firewall is blocking SSH access from the outside, or the SSH daemon (sshd) is not running. Please replace your current ~ldm/etc/pqact.conf file with the copy I attached to this email and then stop and restart your LDM: <as 'ldm'> cd cd etc -- replace pqact.conf cd ldmadmin stop ldmadmin start -- OR -- ldmadmin restart Please let me know if you continue to have problems saving the Cosmic data to disk. Now, on the topic of LDM logging, please try the following: <as 'ldm'> cd ls -alt <- send me the output listing ls -alt /home/ldm/var/logs/ldmd.log <- send me the output listing logger -p local0.debug 'test of LDM logging' ls -alt /home/ldm/var/logs/ldmd.log <- send me the output listing Cheers, Tom -- **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: PGJ-802823 Department: Support IDD Priority: Normal Status: Closed
Attachment:
pqact.conf
Description: Binary data