[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20010530: LDM ingest of NLDN data; ldm-mcidas decoding of same
- Subject: 20010530: LDM ingest of NLDN data; ldm-mcidas decoding of same
- Date: Thu, 31 May 2001 18:36:14 -0600
>From: alan anderson <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200105302052.f4UKq2p15034 ldm-mcidas LDM pqact.conf
Alan,
>Hope summer is treating you ok.
It has been pretty hectic for me so far (too many things going on at
the moment).
>I am trying to get the NLDN lightning
>data into our ldm ingester and seem to be stuck. I have obtained
>permission from Albany, and they have added us with an 'allow'
>statement on their server. When our ldm starts, I see a request to
>striker.atmos.albany.edu and then a feedme OK so I think that
>part is working ok.
OK. This sounds promising. A quick check of your LDM ingest status
using 'notifyme' shows that you are receiving the data from SUNYA:
/local/ldm% notifyme -vxl- -f NLDN -o 3600 -h waldo.stcloudstate.edu
Jun 01 00:03:36 notifyme[21880]: Starting Up: waldo.stcloudstate.edu:
20010531230336.975 TS_ENDT {{NLDN, ".*"}}
NOTIFYME(waldo.stcloudstate.edu) returns OK
Jun 01 00:03:38 notifyme[21880]: NOTIFYME(waldo.stcloudstate.edu): OK
Jun 01 00:03:38 notifyme[21880]: 36b03f8fa3cd0004a735249bd245a29b 18228
20010531230636.292 NLDN 000 2001151230005
Jun 01 00:03:39 notifyme[21880]: 61101470ff0fda9de7ec141981ffc230 17388
20010531231236.837 NLDN 000 2001151230611
Jun 01 00:03:39 notifyme[21880]: 517b0b1a7f2ca77aaedc1f9d1c0c0fd3 16632
20010531231835.657 NLDN 000 2001151231217
Jun 01 00:03:40 notifyme[21880]: 076aa5b7085a23e7b977eed59c85f742 16828
20010531232437.165 NLDN 000 2001151231823
Jun 01 00:03:40 notifyme[21880]: 93d04775d5df866f536858eef0ddce6d 13692
20010531233036.820 NLDN 000 2001151232429
Jun 01 00:03:41 notifyme[21880]: a2a5e0e65e543b6360efec398da4aaf3 14308
20010531233639.239 NLDN 000 2001151233035
Jun 01 00:03:41 notifyme[21880]: 8b16ba07c7f7a2de30993c45a75f7b88 13412
20010531234236.203 NLDN 000 2001151233641
Jun 01 00:03:41 notifyme[21880]: e4b17c492bbc52d9506eda83e1eb0e5d 11872
20010531234837.751 NLDN 000 2001151234247
Jun 01 00:03:41 notifyme[21880]: 693fd0aef2dba30f84d02499c8971c36 12460
20010531235437.415 NLDN 000 2001151234853
^CJun 01 00:03:42 notifyme[21880]: Interrupt
Jun 01 00:03:42 notifyme[21880]: exiting
>My pqact entry for the NLDN data is just below; basically just a copy
>from what has been there since last year some time. I did change the
>first 2 entries as I assume I need to list something for
>
> YYJJJHHMbMe as listed in the pqact.conf discussion on your web
> pg.
>
>My leading items in the pattern are 0 and 1-9 to match year 01 -
>09 (looking ahead) I note that the example on your web site starts
>with [12][0-9][0-9][0-9] | which I don't understand for a 2 digit
>listing of year YY.
This pattern will match 1???, 2???, and ??: examples are 1999, 2000, 2001,
99, etc.
>Anyway, I am not getting any data; no MDXX0071, which is the file for
>the data. Also, doing a Route List in MCIDAS shows 'none' for NLDN
>data received.
The existence of a lightning MD file and an entry in ROUTE.SYS showing
such a file are two different ways of looking at the same thing.
You are getting data, but it isn't being decoded. More on that below.
>One more question about my LDM. Looking today, I see lots of error
>lines regarding a broken pipe in my ldmd.log. Samples listed at
>bottom. I did not see these yesterday when I was editing pqact.conf,
>but maybe I created another error. I did run pqactcheck and it said
>ok.
OK. The errors seem to indicate a problem in FSL wind profiler decoding.
Did you stop and restart the LDM after doing the pqact.conf modifications?
I am thinking that the answer is yes, and further that the environment
from which you stopped and started the LDM had a PATH that did not include
the ~ldm/decoders directory where proftomd and nldn2md live.
>The machine is waldo, and Tom knows how to get in if needed.
Got it.
>pqact.conf
>NLDN ^(0[1-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
> PIPE -close
> nldn2md -d /var/data/mcidas 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CCN
This should be:
NLDN
^([12][0-9][0-9][0-9]|[0-9][0-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
PIPE -close
nldn2md -d /var/data/mcidas 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CCN
>May 30 20:32:43 waldo pqact[1126]: pipe_prodput: trying again
>May 30 20:32:43 waldo pqact[1126]: pbuf_flush (6) write: Broken pipe
>May 30 20:32:43 waldo pqact[1126]: pipe_dbufput:
>-closeproftomd-v-d/var/data/mci
>dasU6WPR691 write error
>May 30 20:32:43 waldo pqact[1126]: child 6205 exited with status 127
>May 30 20:32:43 waldo pqact[1126]: child 6203 exited with status 127
This is failing for proftomd for some reason. I will login and poke around.
<later>
OK, based on my hunch the the environment in which you stopped and restarted
your LDM did not have a PATH that contained the ~ldm/decoders directory,
I did the following:
o login to waldo as ldm
o edit pqact.conf to change the NLDN entry
o stopped the LDM
o restarted the LDM
I now see that both FSL wind profiler and NLDN lightning data are being
decoded:
Jun 01 00:32:42 waldo proftomd[14176]: Starting up
Jun 01 00:32:43 waldo proftomd[14176]: Making /var/data/mcidas/MDXX0092; may
take some time...
Jun 01 00:32:47 waldo proftomd[14176]: Decoding 2001152.0018 data into
/var/data/mcidas/MDXX0092
Jun 01 00:32:47 waldo proftomd[14176]: Exiting
Jun 01 00:33:58 waldo nldn2md[14178]: NLDN2MD -- BEGIN
Jun 01 00:33:58 waldo nldn2md[14178]: PRODUCT CODE=LD 2001152
002400
Jun 01 00:33:58 waldo nldn2md[14178]: Created MD file: 72
Jun 01 00:33:58 waldo nldn2md[14178]: NLDN2MD - Flash events in file: 385
Jun 01 00:33:59 waldo nldn2md[14178]: NLDN2MD -- DONE
The corresponding entries in the McIDAS routing table are in agreement with
this:
<as 'mcidas'>
cd workdata
route.k LIST
...
LD NLDN Lightning Flashes 71-71 MDXX0072 01152 33 none 3
...
U6 FSL2 6-minute Wind profil default MDXX0092 01152 32 none 3
...
route.k: Done
Looks like things are working. Please let me know if you find anything
else that looks amiss.
Tom