[LDM #BXH-661016]: NAM conduit data not arriving via chinkapin.cs.indiana.edu LDM

Hi Felix,

re:
> Thanks for replying.  Sure enough, the email that you sent last
> Friday was sitting in my junk folder!  D'oh!!!

Been there :-)

re: you are properly ALLOWed by the toplevel CONDUIT relays
> That's good ...

re: The request lines for 'NMC' are redundant
    NMC == GPSSRC|CONDUIT|FNEXRAD

> Ok, I have removed these lines and restarted LDM.

Very good.

re: LDM logging

> It might be worth mentioning that I do see an error message when I run
> 'ldmadmin start':
> 
> [ldm@chinkapin bin]$ ./ldmadmin start
> The product-queue is OK.
> /home/ldm/etc/pqact.conf is syntactically correct
> hupsyslog: couldn't open /var/run/syslogd.pid
> Starting the LDM server...

Ah Ha!  This is likely related-to/the cause-of your LDM logging problem.

> The inevitable question which I need to ask you -- since my logging is
> set up incorrectly, how can I set it up correctly? :)

To answer this properly, I need to know some things:

- what kind of operating system you are using
- is the system logger daemon, syslogd, running

These two questions are best answered by sending back the results
of the following:

uname -a
ps -eaf | grep syslog

If you are running Fedora Linux, please send the results of:

cat /etc/fedora-release

re: Steve asked this to see if you had contacted the LDM administrator
on your upstream host(s).

> Ok.  Makes sense.

re: your 'notifyme' invocation shows that you are ALLOWed to request
CONDUIT data from the upstreams you tried

> Ok.  I'm a bit confused by this

If you were _not_ allowed by the upstream you were connecting to with
'notifyme', you would have gotten an indication that your connection
was being denied.

> but I'm moving ahead.  My regular
> expression pattern is incorrect but I'm allowed to request CONDUIT
> data.  Seems like those two statements might contradict each other?

Not at all.

> Or maybe my notifyme pattern was wrong but what I have in ldmd.conf is
> right?

Exactly.

> I see from below that maybe we are receiving but not PROCESSING
> our data. ...

Correct.  The real-time statistics information your machine is sending
us shows that you are receiving some CONDUIT data.  If the request
you are using is all of the CONDUIT data you are asking for, then the
volume plots of data received indicate that you are getting all that
you are asking for.  This, in turn, means that the problem is in
processing the data out of your LDM queue.

re: did you let 'notifyme' run for long enough

> Yeah, I let notifyme run for quite a while.  The notifyme for
> idd.cise-nsf.gov ran all weekend and looked the same when I got back:
> 
> Mar 11 11:46:37 notifyme[11932] NOTE: NOTIFYME(idd.cise-nsf.gov): OK
> Mar 11 11:52:06 notifyme[11932] NOTE: LDM-5 desired product-class:
> 20090311084405.541 TS_ENDT {{CONDUIT, 
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
> Mar 11 11:52:06 notifyme[11932] INFO: Resolving idd.cise-nsf.gov to> 
> 192.12.209.62 took 1.2e-05 seconds
> Mar 11 11:52:06 notifyme[11932] NOTE: NOTIFYME(idd.cise-nsf.gov): OK

The last line shows that idd.cise-nsf.gov has an ALLOW line that matches
your machine.  If it did not, you would not get the 'OK' indication.


re: I used a slightly different 'notifyme' invocation and got the kind of 
listing
that is expected:

% notifyme -vl- -f CONDUIT -h idd.unidata.ucar.edu -p 
"^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! " -o 
100000

> In bash and zsh... this still doesn't work.  The strings were
> double-quoted originally.  Bash forces the execution of ! even within
> double-quotes, and if you use a backslash to escape the !, it comes out
> looking like "\!"  instead of just giving you the "!" without the "\" --
> thus changing the regexp.  With zsh as well, I was forced to insert the
> final space character even using double quotes.
> What shell are you using to do this at the command line?

Try using Csh or Tcsh.  The example I provided worked exactly as written
on my Fedora 10 Linux system.

re: real-time stats pages

> Ok.  These are good info pages which our more experienced admin has
> shown me a couple of times -- I'm bookmarking them for future reference.

Very good.  These are very useful when troubleshooting problems.

> I'm attaching both pqact.conf and ldmd.conf.

Thanks.  The active pqact lines from your ldmd.conf file are:

exec "pqact /usr/local/ldm/etc/pqact.conf"
exec "pqact -f DDPLUS|IDS|HDS|CRAFT /usr/local/ldm/etc/pqact.conf_file_open"
exec "pqact -f NIMAGE|NNEXRAD /usr/local/ldm/etc/pqact.conf_file_close"

exec "pqact -f NEXRD2 /usr/local/ldm/etc/pqact.conf_nexradII_raw"

exec "pqact -f CONDUIT /usr/local/ldm/etc/pqact.conf_conduit"

exec "pqact -f NMC2 /usr/local/ldm/etc/pqact.conf_conduit_dc"

The operative lines are the last two:

exec "pqact -f CONDUIT /usr/local/ldm/etc/pqact.conf_conduit"

exec "pqact -f NMC2 /usr/local/ldm/etc/pqact.conf_conduit_dc"

Given this, the pattern-action files we need to see are:

~ldm/etc/pqact.conf_conduit
~ldm/etc/pqact.conf_conduit_dc

Note:

- I hope you are now aware that CONDUIT and NMC2 are the same feed,
  so you have two different 'pqact' invocations acting on the same
  set of data

It is not clear to me that the pattern-action file you sent was either
pqact.conf_conduit or pqact.conf_conduit_dc.  Please let me know if it
was, or send us those files if it wasn't.
re: I recommend upgrading to LDM-6.7.0

> Upgrade to LDM-6.7.0 sounds like a good idea, for sure.  I'll do it.
> I'll check the manual and see if I can figure out anything about logging
> configuration, but if you have any ideas about why it's not working, I'm
> glad to hear them.

If you are running on a newer Fedora system, it is possible that your
system logger daemon is 'rsyslogd', not 'syslogd'.  We have seen on
our own Fedora 10 systems that the system logger daemon is named one
thing, and the system logger daemon PID file is named the other.  Until
a new LDM version is made available that takes care of this muddled
situation, our approach was to make a symbolic link to the needed file
in /var/run.  For instance:

If /var/run/syslogd.pid does not exist, but /var/run/rsyslogd.pid
does AND 'rsyslogd' is the system logger facility AND it is running,
do the following:

<as 'root'>
cd /var/run
ln -s rsyslogd.pid syslogd.pid

Again, this is on a Fedora Linux 9 or 10 system.  in the beginning of this
reply I asked what OS you are running to help clear-up this situation
so we can get LDM logging working correctly.

Final comments for this email:

- I deleted your CC to address@hidden as this address is an
  email list that Unidata User Support does not belong to.  The last
  message I sent had the CC, and the CC bounced back to us

- we are getting two copies of your email.  Please delete the To: line
  entry that is to support-ldm.  Thanks for this; it makes our life
  a bit easier

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: BXH-661016
Department: Support IDD
Priority: Normal
Status: Closed