20010601: LDM ingest of NLDN data; ldm-mcidas decoding of same (cont.)

>From: alan anderson <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200105302052.f4UKq2p15034 ldm-mcidas LDM pqact.conf

Alan,

>Thanks Tom, but just a couple of questions.

NLDN    
^([12][0-9][0-9][0-9]|[0-9][0-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])

>1. I thought I understood the format of the pattern-action statement
>   listed above, but how is it that the leading [12] can be interpreted
>   as either  a 1 or a 2, followed by the digits in the next series of
>   [  ]?   To me,  [12] could only be matched by  12.

The values inside of the brackets mean any number from the specified
list or range of numbers.  The list [12] means either 1 or 2; the range
[0-9] means any number between 0 and 9 inclusively.  Given this,
[12][0-9] can mean 10, 11, ..., 19, 20, 21, ..., 29.

>   Also,  is part of this to allow for either a yyyy  or yy  format ?

Yes.  At one point there was a debate between myself and Dave Knight of
SUNY Albany on whether or not we should change the year portion in the
data header from YY to CCYY; we eventually did.  We came up with the
pattern above so that either format would be supported (e.g., 99, 00,
1999, 2000, etc.).

>2. Regarding the environment from which I stopped and started the ldm,  I
>   know I was logged in as user ldm,  so am puzzled about the PATH problem.

I worried about this as I was driving home yesterday (this was after I
sent you my email).  It didn't seem likely that you were logged in as a
different user since the LDM processes were running as 'ldm' when I
logged on.  This fact didn't sink in until I was half way home, and
then I forgot to send you a follow-up discussing my probably incorrect
assumption.  Sorry...

>   The profiler stuff has been working before, and I have occasionally had
>   to stop and restart the ldm.  I try to remember to do this only as user
>   ldm.  Could you comment, and is there something about user ldm's path I
>   need to check?

No, the PATH looked OK.  I just couldn't figure out why there was a
write error to proftomd and nldn2md.  The typical way that this can
happen is that the routines can't be found (the LDM pqact.conf action
simply tries to run the named program and send it input through
standard in).  The PATH that would be used to find these routines would
be the PATH that was in effect when the LDM was started.

When I modified your NLDN entry in pqact.conf, I did an 'ldmadmin
pqactcheck' (just as you did) and then sent a HUP signal to pqact.
After this, I noted that neither proftomd nor nldn2md were decoding
data and errors were showing up in the LDM log file (as seen by an
'ldmadmin tail').  I checked the PATH of the user 'ldm' and verified
that both of these routines could be found (in the ~ldm/decoders
subdirectory), so I was really puzzled.  I then decided to stop and
restart the LDM so I could work from a point which I felt was correct
(i.e., logged in as 'ldm' and after having verified the PATH).  At that
point everything started running fine, so I figured that the only way
for this to have happened was the PATH in effect the last time the LDM
was started had to be wrong.  That I why I sent you the comment that I
did.

>Thanks

Sorry for the confusion.

Tom