[Datastream #NVL-219514]: NGRID headers

Hi Justin,

re:
> I was trying to filter the LDM feed in ldmd.conf with the following
> entries:
> 
> REQUEST    CONDUIT    "(prod/gfs.*pgrbf.*|prod/gfs.*pgrb2f.*)" ldm.tamu.edu
> REQUEST    CONDUIT    "(prod/nam*.awip12|prod/nam*.grbgrd)"  ldm.tamu.edu
> REQUEST    CONDUIT    ".*rap.*awp252.*"    ldm.tamu.edu
> REQUEST    CONDUIT    
> "(prod/sref.*pgrb212.*WRF_NMM.*|prod/sref.*prgb212.*WRF_EM|prod/sref.*SREF_113)"
>  ldm.tamu.edu

First, a couple of comments on how best to write extended regular expressions:

- leading and trailing ".*" constructs are not needed/desired in extended
  regular expressions

  The first and third REQUESTs are better written as:

REQUEST    CONDUIT    "(prod/gfs.*pgrbf|prod/gfs.*pgrb2f)" ldm.tamu.edu
REQUEST    CONDUIT    "rap.*awp252"    ldm.tamu.edu

- it looks like you have a typo on the second REQUEST; it seems like you meant:

REQUEST    CONDUIT    "(prod/nam.*awip12|prod/nam.*grbgrd)"  ldm.tamu.edu

- invariant pieces of extended regular expressions are best moved
  outside of patterns as this improves readability:

REQUEST    CONDUIT    "prod/gfs.*(pgrbf|pgrb2f)" ldm.tamu.edu
REQUEST    CONDUIT    "prod/nam.*(awip12|grbgrd)"  ldm.tamu.edu
REQUEST    CONDUIT    
"prod/sref.*(pgrb212.*WRF_NMM.*|prgb212.*WRF_EM|SREF_113)" ldm.tamu.edu

With these changes, your REQUESTs would like like:

REQUEST    CONDUIT    "prod/gfs.*(pgrbf|pgrb2f)"   ldm.tamu.edu
REQUEST    CONDUIT    "prod/nam.*(awip12|grbgrd)"  ldm.tamu.edu
REQUEST    CONDUIT    "rap.*awp252"                ldm.tamu.edu
REQUEST    CONDUIT    
"prod/sref.*(pgrb212.*WRF_NMM.*|prgb212.*WRF_EM|SREF_113)" ldm.tamu.edu

NB: I am not saying that your extended regular expressions are correct, but
written in this way they are more readable and cause less work for the
regular expression parser.

re:
> However, when I added the entries, I was no longer receiving the CONDUIT
> data.
> 
> Would you be able to assist me in revising these entries to filter out the
> data?

The best thing for you to do at this point is to learn how to use the LDM 
utility
'notifyme'.  'notifyme' can be used for a variety of things like:

- see if the upstream has ALLOWed you to REQUEST data
- see what products your upstream is receiving
- see what products you are receiving
- test extended regular expressions for use in the LDM configuration
  file, ~ldm/etc/ldmd.conf, and in pattern-action file actions

Here is one example of testing an extended regular expression:

<as 'ldm'>
notifyme -vl- -f CONDUIT -h ldm.tamu.edu -o 3600 -p "prod/nam.*(awip12|grbgrd)"

NB: you have to make sure to enclose your extended regular expression in shifted
quotes so that your *nix shell does not expand the regular expression!

The output from this command when I just ran it is:

[ldm@gale ~]$ notifyme -vl- -f CONDUIT -h ldm.tamu.edu -o 3600 -p 
"prod/nam.*(awip12|grbgrd)"
Oct 22 20:24:11 notifyme[4985] NOTE: Starting Up: ldm.tamu.edu: 
20141022192411.238 TS_ENDT {{CONDUIT,  "prod/nam.*(awip12|grbgrd)"}}
Oct 22 20:24:11 notifyme[4985] NOTE: LDM-5 desired product-class: 
20141022192411.238 TS_ENDT {{CONDUIT,  "prod/nam.*(awip12|grbgrd)"}}
Oct 22 20:24:11 notifyme[4985] INFO: Resolving ldm.tamu.edu to 128.194.76.175 
took 0.087426 seconds
Oct 22 20:24:11 notifyme[4985] NOTE: NOTIFYME(ldm.tamu.edu): OK
Oct 22 20:24:12 notifyme[4985] INFO:   134871 20141022200931.990 CONDUIT 000  
data/nccf/com/nam/prod/nam.20141022/nam.t18z.awip1233.tm00.grib2 
!grib2/ncep/NAM_84/#000/201410221800F033/EMSL/0 - NONE! 000000
Oct 22 20:24:12 notifyme[4985] INFO:   125479 20141022200932.399 CONDUIT 009  
data/nccf/com/nam/prod/nam.20141022/nam.t18z.awip1233.tm00.grib2 
!grib2/ncep/NAM_84/#000/201410221800F033/HGHT/0 - NONE! 000009
Oct 22 20:24:12 notifyme[4985] INFO:   109090 20141022200932.402 CONDUIT 010  
data/nccf/com/nam/prod/nam.20141022/nam.t18z.awip1233.tm00.grib2 
!grib2/ncep/NAM_84/#000/201410221800F033/TMPK/2 m HGHT! 000010
Oct 22 20:24:12 notifyme[4985] INFO:     5533 20141022200932.732 CONDUIT 019  
data/nccf/com/nam/prod/nam.20141022/nam.t18z.awip1233.tm00.grib2 
!grib2/ncep/NAM_84/#000/201410221800F033/SWEM03/0 - NONE! 000019
Oct 22 20:24:12 notifyme[4985] INFO:     1138 20141022200932.732 CONDUIT 020  
data/nccf/com/nam/prod/nam.20141022/nam.t18z.awip1233.tm00.grib2 
!grib2/ncep/NAM_84/#000/201410221800F033/WXTS/0 - NONE! 000020
Oct 22 20:24:12 notifyme[4985] INFO:    89058 20141022200932.799 CONDUIT 029  
data/nccf/com/nam/prod/nam.20141022/nam.t18z.awip1233.tm00.grib2 
!grib2/ncep/NAM_84/#000/201410221800F033/HLCY/3000-0 m HGHT! 000029
 ...

The next job is to decide if the extended regular expression being tested is the
one you want (i.e., is the list of products the ones that you are interested 
in).
If yes, you can use the extended regular expression; if not, make a modification
and see if the new extended regular expression works.

I hope that this helps!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: NVL-219514
Department: Support Datastream
Priority: Normal
Status: Closed