[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IDD #LRB-829354]: Issues with product-queue
- Subject: [IDD #LRB-829354]: Issues with product-queue
- Date: Thu, 21 Aug 2014 15:58:34 -0600
Hi Tim,
re:
> Okay, this logging seems to be the case.
I interpret this to mean that logging is not working.
re:
> How would I properly set this up?
I logged back into your system, and I saw that that the things
that need to be setup are, in fact, setup for LDM logging to
work. I then noticed that you are running Ubuntu Linux, and
I understood what is going on. For some reason in Ubuntu
rsyslogd to write to your LDM log file; this is because the
read/write/execute permissions on ldmd.log are typically setup
as 664 (-rw-rw-r--), so the file not being writable by world
is the problem.
A workaround for this is to set the 'umask' for 'ldm' to be
'000' (it is currently 002, which is normal). I just implemented
this by adding:
umask 000
to the top of ~ldm/.bashrc. I tested the change by logging off
and then logging back on to your machine. After doing this, I
deleted and remade the LDM log file:
<as 'ldm'>
rm logs/ldmd.log && touch logs/ldmd.log
and then restarted the LDM:
ldmadmin restart
After doing this LDM logging is working; this can be see by the
non-zero size of the LDM log file:
:~$ ls -alt ~ldm/logs/ldmd.log
-rw-rw-rw- 1 ldm ldm 7568 Aug 21 15:41 /home/ldm/logs/ldmd.log
The next thing I did was to run 'notifyme' to see if:
1) you are ALLOWed to REQUEST NEXRAD2 data from the upstream hosts
you want to REQUEST from
You are ALLOWed.
2) if you were getting any data
You are not getting any data.
The next thing I did was to examine the ~ldm/etc/ldmd.conf REQUEST
lines you have for NEXRAD2 data:
:~$ grep -i ^request etc/ldmd.conf
REQUEST NEXRAD2 "/p...(UDX|ABR|FSD)" idd.tamu.edu
REQUEST NEXRAD2 "/p...(UDX|ABR|FSD)" idd.meteo.psu.edu
REQUEST NEXRAD2 "/p...(UDX|ABR|FSD)" idd.unidata.ucar.edu
If you look at the LDM/IDD Product IDs for the NEXRAD2 data:
:~$ notifyme -vl- -f NEXRAD2 -h idd.tamu.edu -o 600
Aug 21 21:44:33 notifyme[9273] NOTE: Starting Up: idd.tamu.edu:
20140821213433.705 TS_ENDT {{NEXRAD2, ".*"}}
Aug 21 21:44:33 notifyme[9273] NOTE: LDM-5 desired product-class:
20140821213433.705 TS_ENDT {{NEXRAD2, ".*"}}
Aug 21 21:44:33 notifyme[9273] INFO: Resolving idd.tamu.edu to 128.194.76.175
took 0.002042 seconds
Aug 21 21:44:33 notifyme[9273] NOTE: NOTIFYME(idd.tamu.edu): OK
Aug 21 21:44:34 notifyme[9273] INFO: 40786 20140821213433.740 NEXRAD2 502065
L2-BZIP2/KILX/20140821213042/502/65/I/V06/0
Aug 21 21:44:34 notifyme[9273] INFO: 47088 20140821213433.733 NEXRAD2 40066
L2-BZIP2/KDGX/20140821213043/40/66/I/V06/0
Aug 21 21:44:34 notifyme[9273] INFO: 65527 20140821213433.736 NEXRAD2 222050
L2-BZIP2/KUDX/20140821213138/222/50/I/V06/0
Aug 21 21:44:35 notifyme[9273] INFO: 37829 20140821213433.809 NEXRAD2 88006
L2-BZIP2/KDOX/20140821213400/88/6/I/V07/0
Aug 21 21:44:35 notifyme[9273] INFO: 51051 20140821213433.973 NEXRAD2 941017
L2-BZIP2/KGJX/20140821213349/941/17/I/V06/0
Aug 21 21:44:35 notifyme[9273] INFO: 76064 20140821213433.795 NEXRAD2 541057
L2-BZIP2/KCYS/20140821213133/541/57/I/V06/0
...
You will see that the pattern you are using in your REQUEST lines will never
be matched since there is no '/p' in any NEXRAD2 Product ID.
I think that what you are trying to REQUEST is the set of NEXRAD Level 2
stations that have (UDX|ABR|FSD) as part of their name. Given this
assumption, I went ahead and redid your ldmd.conf REQUEST lines and then
restarted your LDM:
#request Nexrad level 2 data
#REQUEST NEXRAD2 "/p...(UDX|ABR|FSD)" idd.tamu.edu
#REQUEST NEXRAD2 "/p...(UDX|ABR|FSD)" idd.meteo.psu.edu
#REQUEST NEXRAD2 "/p...(UDX|ABR|FSD)" idd.unidata.ucar.edu
REQUEST NEXRAD2 "(UDX|ABR|FSD)" idd.tamu.edu
REQUEST NEXRAD2 "(UDX|ABR|FSD)" idd.meteo.psu.edu
REQUEST NEXRAD2 "(UDX|ABR|FSD)" idd.unidata.ucar.edu
After editing ldmd.conf and restarting, LOTs of Level 2 data is
being received:
:~$ ldmadmin watch
(Type ^D when finished)
Aug 21 21:50:43 pqutil INFO: 127516 20140821213134.410 NEXRAD2 419038
L2-BZIP2/KABR/20140821212927/419/38/I/V06/0
Aug 21 21:50:43 pqutil INFO: 59277 20140821213136.329 NEXRAD2 887021
L2-BZIP2/KFSD/20140821213040/887/21/I/V06/0
Aug 21 21:50:43 pqutil INFO: 58197 20140821213138.364 NEXRAD2 887022
L2-BZIP2/KFSD/20140821213040/887/22/I/V06/0
Aug 21 21:50:43 pqutil INFO: 10142 20140821213139.391 NEXRAD2 222001
L2-BZIP2/KUDX/20140821213138/222/1/S/V06/0
Aug 21 21:50:43 pqutil INFO: 60988 20140821213140.401 NEXRAD2 887023
L2-BZIP2/KFSD/20140821213040/887/23/I/V06/0
Aug 21 21:50:43 pqutil INFO: 129924 20140821213139.486 NEXRAD2 419039
L2-BZIP2/KABR/20140821212927/419/39/I/V06/0
Aug 21 21:50:43 pqutil INFO: 141280 20140821213142.474 NEXRAD2 222002
L2-BZIP2/KUDX/20140821213138/222/2/I/V06/0
Aug 21 21:50:43 pqutil INFO: 54057 20140821213143.439 NEXRAD2 887024
L2-BZIP2/KFSD/20140821213040/887/24/I/V06/0
Aug 21 21:50:43 pqutil INFO: 135551 20140821213143.561 NEXRAD2 419040
L2-BZIP2/KABR/20140821212927/419/40/I/V06/0
Aug 21 21:50:43 pqutil INFO: 62913 20140821213145.483 NEXRAD2 887025
L2-BZIP2/KFSD/20140821213040/887/25/I/V06/0
Aug 21 21:50:43 pqutil INFO: 101175 20140821205101.110 NEXRAD2 876028
L2-BZIP2/KFSD/20140821204946/876/28/I/V06/0
Aug 21 21:50:43 pqutil INFO: 194473 20140821213145.571 NEXRAD2 222003
L2-BZIP2/KUDX/20140821213138/222/3/I/V06/0
...
The final thing was to see if any products were being written to disk; they
were not. I then took a look at your pattern-action file action to see
what you are trying to do. The only uncommented action I found was:
NEXRAD ^SDUS5. .... ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(...)(...) FILE
-close data/nexrad/\5/\4/\4_(\1:yyyy)(\1:mm)\1_\2\3
This will not match anything, since the feed type you are specifying to
check, 'NEXRAD', is not what you are REQUESTing. The feed type for
NEXRAD Level 2 data is 'NEXRAD2', not NEXRAD which an equivalent name
for NEXRAD3 which is the feed of NEXRAD Level III products.
So, the question is which NEXRAD data do you want/need, Level II or
Level III?
If you want/need Level II data (full volume scan data), we need to change
the action entry in your ~ldm/etc/pqact.conf file.
If you want/need Level III data (NEXRAD products), we need to change the
REQUEST lines in your ~ldm/etc/ldmd.conf file.
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: LRB-829354
Department: Support IDD
Priority: Normal
Status: Closed