20030926: Confused about profiler entry...

>From: Gilbert Sebenste <address@hidden>
>Organization: NIU
>Keywords: 200309261537.h8QFbGk1010504 LDM FSL2 pqact.conf

Gilbert,

>OK. This profiler discussion on LDM-users has left me confused. I'm not 
>filing any profiler data, but am receiving the feed. My entries are these:
>
>FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(.*)\..* 
>        FILE    profiler/\1%m%d.hr
>#       
>FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min\.(.*)\..*
>        FILE    profiler/\1%m%d.six
>
>What do I need to change here to get them to work, and why?

When in doubt, always drop back to looking at the headers of the products
that come across in the datastream.  The easiest way to do this is
to use the LDM utility 'notifme':

% notifyme -vxl- -f FSL2 -o 3600
Sep 26 16:00:26 notifyme[927]: Starting Up: localhost: 20030926150026.750 
TS_ENDT {{FSL2,  ".*"}}
Sep 26 16:00:26 notifyme[927]: Connected to upstream LDM-5
        NOTIFYME(localhost) returns OK
Sep 26 16:00:26 notifyme[927]: NOTIFYME(localhost): OK
Sep 26 16:00:27 notifyme[927]: a4372c0977b5a11195583d5ab901becc   172188 
20030926151043.235    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691500
Sep 26 16:00:28 notifyme[927]: 141470bbc96f20fa9dbe178c59f65d63   182896 
20030926151812.805    FSL2 000  FSL.NetCDF.NOAAnet.windprofiler.01hr.20032691500
Sep 26 16:00:28 notifyme[927]: 703661560cb47aa00d98035fd40df9e3   172188 
20030926151855.586    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691506
Sep 26 16:00:29 notifyme[927]: fabc5759fffb56d471045d89ab2bcfd5   182868 
20030926152134.001    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691512
Sep 26 16:00:30 notifyme[927]: 85c96bc30dcbeacf9d813cb69080e547   177528 
20030926152740.969    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691518
Sep 26 16:00:31 notifyme[927]: c23ea51e827cf46db74c3045d4b4596b   177528 
20030926153348.144    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691524
Sep 26 16:00:32 notifyme[927]: 722ebd3ad630fb0cf422ccec532b63ac   182868 
20030926153957.617    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691530
Sep 26 16:00:33 notifyme[927]: 722b72a93da739edf073683a98046825   182868 
20030926154537.689    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691536
Sep 26 16:00:33 notifyme[927]: 30bdbc35cccea5fe5cc3e2faee3816f6   182868 
20030926155142.446    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691542
Sep 26 16:00:34 notifyme[927]: b10e223fa196bf772e03707477b8bf31   182868 
20030926155732.521    FSL2 000  
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691548
^CSep 26 16:00:41 notifyme[927]: Interrupt
Sep 26 16:00:41 notifyme[927]: exiting

From this listing, the headers for the hourly profiler data look like:

FSL.NetCDF.NOAAnet.windprofiler.01hr.20032691500

The headers for the six minute profiler data look like:

FSL.NetCDF.NOAAnet.windprofiler.06min.20032691500
FSL.NetCDF.NOAAnet.windprofiler.06min.20032691506
 ...

The pattern you are using for the hourly summary:

^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(.*)\..

fails at the end since it is expecting one more field separated from the
others by a '.' (period):

FSL.NetCDF.NOAAnet.windprofiler.01hr.20032691500

^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr - matches 
FSL.NetCDF.NOAAnet.windprofiler.01hr
\.(.*)                                    - matches .20032691500
\..                                       - does not match anything

Your patterns should, therefore, at a minimum be changed to:

FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(.*)
        FILE    profiler/\1%m%d.hr
#       
FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min\.(.*)
        FILE    profiler/\1%m%d.six


The \1 in your FILE action will be the date as 'CCYYJJJHHMM'.  It
looks like you want to file your profiler data as something that
uses the month and day instead of the Julian day.

If you want to file things using names following 'CCYYmmddHHMM'
your patterns should change to:


FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(....)...(....)
        FILE    profiler/\1%m%d\2.hr
#       
FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min\.(....)...(....)
        FILE    profiler/\1%m%d\2.six

The pqact.conf actions for the ldm-mcidas profiler decoder 'proftomd'
can easily be changed by dropping everything after the hour/6min indicator:

FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr
        PIPE    -close
        decoders/proftomd -l logs/ldm-mcidas.log
        -d data/mcidas -v U2 WPRO 81

FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min
        PIPE    -close
        decoders/proftomd -l logs/ldm-mcidas.log
        -d data/mcidas -v U6 WPR6 91

Remember to _always_ check the integrity of your pqact.conf changes
after doing any editing:

ldmadmin pqactcheck

If there are not errors indicated, send pqact a HUP signal:

ldmadmin pqactHUP

Tom