[IDD #JNM-107366]: Configuration Files

Hi Massoud,

re: 
> I just wanted to check to see if anyone has received my email from Tuesday
> night regarding about the configuratoin files?

Yes, we received your email regarding the configuration files.

re:
> I edited the data request entries in ~ldm/etc/ldmd.conf and it seems to
> be running fine. I copied and pasted a small section of this file into
> a Wordpad attachment, ldmd.conf.rtf, for you to look at. I also pasted
> the same section in this email in case your having trouble opening the
> attachment.

OK.  In the future, please send attachments in ASCII (we are not a Windows
shop :-).

Also, just so you know you should not edit the LDM configuration files
using Windows applications as they use different conventions for the
end of line than Unix/Linux.  This small detail has caused many strange
problems over the years that are hard to track down since a simple listing
of the file(s) contents looks OK.

> By the way, what is the primary name for the NCEP model
> data's for GFS-CONUS_80km and NAM-CONUS_12 km?

I believe that they are both contained in HDS.  I will need to check with
our model data guru to make sure that the 12 km NAM data is in the same
stream as the 80 km GFS; it is possible that one or both of these are
in the NGRID or CONDUIT feeds.

> ~ldm/etc/ldmd.conf
> #REQUEST WMO ".*" initial-primary-host.some.domain:388
> #REQUEST WMO ".*" initial-secondary-host.another.domain
> #REQUEST WMO ".*" another-secondary-host.yet_another.domain:3152
> REQUEST EXP "NCWF*" *awl.rap.ucar.edu* <http://awl.rap.ucar.edu/>
> REQUEST UNIWISC ".*" *aeolus.ucsd.edu* <http://aeolus.ucsd.edu/>
> REQUEST FNEXRAD ".*" *aeolus.ucsd.edu* <http://aeolus.ucsd.edu/>

Your request lines need to be modified to not use regular expressions
for the machine name you want to request data from.  I would change
the last three entries to look like:

REQUEST EXP "NCWF*" awl.rap.ucar.edu
REQUEST UNIWISC ".*" aeolus.ucsd.edu
REQUEST FNEXRAD ".*" aeolus.ucsd.edu

Also please note:

- capitalization of 'REQUEST' is not needed; 'request' is equivalent to 
'REQUEST'
- the regular expression "NCWF*" is better stated as "NCWF"; the trailing '*' 
  is not needed and it actually slows down the regular expression parsing 
routines
- the volume in the UNIWISC and FNEXRAD feeds is low enough that your two 
requests
  can be turned into one.  For example, you could:

change:

REQUEST UNIWISC ".*" aeolus.ucsd.edu
REQUEST FNEXRAD ".*" aeolus.ucsd.edu

to:

REQUEST FNEXRAD|UNIWISC ".*" aeolus.ucsd.edu

> One of things we are still having difficulties with is editing
> ~ldm/etc/pqact.conf. We have looked over the online tutorial, but we're
> not sure what needs to be changed or added. Again, I copied and pasted
> a small section of this file into a Wordpad attachment, pqact.conf.rtf,
> and in this email as well. Could you explain to us briefly what the
> first two lines mean exactly

~ldm/etc/pqact.conf actions are regularized as follows:

feed_name<tab>regular_expression_that_matches_product_header(s)
<tab>action

It is _most_ important to make sure that the whitespace between the feed name
and regular expression that matches one or more product headers is a TAB, not
a space!

Also, it is most important to realize that no line in ~ldm/etc/pqact.conf can
begin with a space.


> ~ldm/etc/pqact.conf
> EXP
> ^NCWF6_PROB_FCST_NCWD2_([0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9])_([0-2][0-9][0-5][0-9][0-9][0-9])
> FILE -close -overwrite data_test/ncwf6/grib/NCWD2/\1/\2\3.grib

Let's consider the above action:

1) the feed type, EXP, should be the first characters on the action line
2) the regular expression that starts with '^NCWF6' needs to be separated from 
the feed
   type by a TAB
3) the action will only be run for product headers that match the regular 
expression that
   you specify
4) the portions of the regular expression that are enclosed by parentheses, "(" 
and ")",
   are referenced in the action portion of the entry by the '\n' selectors as 
follows:

   \1 <-> ([0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9])
   \2 <-> ([0-2][0-9][0-5][0-9][0-9][0-9])

   NOTE: your FILE action line uses a '\3' reference, but there is no 
corresponding
   match in the regular expression specified.  This is an error that will 
prevent
   the action from being executed
5) the FILE action line must begin with a TAB
6) the whitespace between FILE and '-close' and between '-close' and 
'-overwrite'
   must be TABs
7) the object of FILE, data_test/ncwf6/grib/NCWD2/\1/\2\3.grib, says to file the
   product whose product identifier matches the regular expression in
   ~ldm/data_test/ncwf6/grib/NCWD2/\1/\2/\3.grib.

   NOTE: as I mentioned in 4), your regular expression does not have a third
   clause sorrounded by parentheses, so this action is invalid.  Given the
   regular expression specified, the following would be valid:

   data_test/ncwf6/grib/NCWD2/\1/\2.grib

The best way to check the regular expression(s) you want to use in 
~ldm/etc/pqact.conf
is by using the LDM utility 'notifyme'.  Here is an example:

<as 'ldm'>
notifyme -vxl- -f EXP -o 3600 -p 
'NCWF6_PROB_FCST_NCWD2_([0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9])_([0-2][0-9][0-5][0-9][0-9][0-9])'

If the regular expression is valid AND matches a product header being received 
by your
machine, 'notifyme' will list out the product.  If it is invalid or does not 
match,
nothing will be listed.

What I do when creating a new pqact.conf action is:

1) use 'notifyme' to list out the products in the datastream for which I want 
to create
   an action:

notifyme -vxl- -f EXP -o 3600

2) if I see products that I want to match, I make a note of their product ids

3) I create a regular expression that will match the product id(s) that I am 
interested
   in and test them using 'notifyme'

4) when I have a regular expression that matches the product id(s) I am 
interested in,
   I use it to create a ~ldm/etc/pqact.conf action.  I do this using a standard
   Unix/Linux editor.  NB: never use a Windows editor to create/maintain either 
your
   ~ldm/etc/ldmd.conf or ~ldm/etc/pqact.conf files!!!!!

5) after entering a new ~ldm/etc/pqact.conf action, I use 'ldmadmin pqactcheck' 
to 
   run a coarse check on the action to see if I have made any typos.  If the 
output
   from 'ldmadmin pqactcheck' shows errors, I go back into the editor and 
correct
   them.  Continue the iteration until 'ldmadmin pqactcheck' shows reports no 
errors.

6) after I have a valid ~ldm/etc/pqact.conf file with the new action, I use 
'ldmadmin pqactHUP'
   to send a HUP signal to pqact telling it to reread ~ldm/etc/pqact.conf.  I 
do not
   have to stop and restart the LDM to activate the new action(s)!

> and how can we add UNIWISC, FNEXRAD, and NCEP model data's into 
> ~ldm/etc/pqact.conf?

Follow the procedure outline above using UNIWISC or FNEXRAD for the feed name
in the '-f' part of the 'notifyme' invocation.

After you gain confidence in creating new ~ldm/etc/pqact.conf actions, and if 
you
still have problems, please send us your ~ldm/etc/pqact.conf file as an 
attachment.
Remember, do not convert it into another format like RTF/DOC/etc.  We will need
to see the exact file contents to make sure that you do not have disallowed
characters in your actions.

Also, please send along the specifics of the machine on which you are running 
the
LDM: operating system and release, amount of disk space, amount of RAM, etc.
This is most easily done by sending the output of:

<as 'ldm'>
cd ~ldm
uname -a
df -k
cat /proc/meminfo  <- assumes your system is running Linux
ls -alt

> Thanks for your time.

No worries.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: JNM-107366
Department: Support IDD
Priority: Normal
Status: Closed