[IDD #PLF-256397]: Pqact help

Hi Phil,

First, I moved your post from the noaaport email list to our inquiry tracking
system.  That is why you are receiving a reply from Unidata Support instead
of folks subscribed to our noaaport email list.  In the future, questions
like the one you pose are best sent to Unidata User Support using one of
the following two email addresses:

address@hidden    <- questions about implementation
address@hidden    <- questions specific to the LDM

I put your inquiry into the address@hidden category as it is
less of an LDM-specific question and more of an implementation question.
To tell the truth, either department would have been appropriate...

re:
> The way our LDM is currently set up is we receive most of our files via
> the LDM and then a script is run via the crontab to post data (such as
> MOS-DATA and Nexrad data) to our website. I'm sure using the EXEC
> command, our scripts could run as the data is received... thereby
> getting the data on our site immediately.

Yes, running your processing from an action in an LDM pattern-action file
would insure that the data get processed as soon as it is received.  We
feel that this is the best approach as it takes advantage of the 
event-notification
nature of the LDM.

> Here is a snippet from our
> pqact file (below), but it doesn't seem to be working.  Any help is
> appreciated --

> # MRF MOS
> WMO ^...... .... ([0-3][0-9])([0-2][0-9]).*/pMEX
>      FILE data/nwx/mos/mrfnew/(\1:yyyy)(\1:mm)\1\2.mrfmos
>      EXEC /usr/local/gempak/scripts/mmos/mmos.pl
> WMO ^FOX[CESW].* KWBC ([0-3][0-9])([0-2][0-9])
>      FILE data/nwx/mos/mrf/(\1:yy)(\1:mm)\1\2.mrfmos
>      EXEC /usr/local/gempak/scripts/mmos/mmos.pl
> WMO ^FOAK2[0-4] K... ([0-3][0-9])([0-2][0-9])
>      FILE data/nwx/mos/mrf/(\1:yy)(\1:mm)\1\2.mrfmos
>      EXEC /usr/local/gempak/scripts/mmos/mmos.pl
> WMO ^FECN21 KWBC ([0-3][0-9])([0-2][0-9])
>      FILE data/nwx/mos/mrf/(\1:yy)(\1:mm)\1\2.mrfmos
>      EXEC /usr/local/gempak/scripts/mmos/mmos.pl

> I'm still a bit new with the LDM/Gempak, but understand
> the basics.

No worries.

> mmos.pl is a simple script that goes through the .mrfmos data file and
> parces out each site's MRF-MOS data.

The problem I see is one of understanding how the actions in a pattern-action
file work:  there can be one action in each pattern-action block, but you
are trying to specify two:

# MRF MOS
WMO     ^...... .... ([0-3][0-9])([0-2][0-9]).*/pMEX
        FILE    data/nwx/mos/mrfnew/(\1:yyyy)(\1:mm)\1\2.mrfmos
        EXEC    /usr/local/gempak/scripts/mmos/mmos.pl

The FILE action in the above will be executed; the EXEC
action will not be.

There are two approaches you can take to fix this entry so
it will work in the way that you want:

- replicate the pqact.conf entry and have the second instance
  run the EXEC action

- modify your mmos.pl script to write the product to the desired
  output file and then do whatever processing you want.

The first case entry would change from:

# MRF MOS
WMO     ^...... .... ([0-3][0-9])([0-2][0-9]).*/pMEX
        FILE    data/nwx/mos/mrfnew/(\1:yyyy)(\1:mm)\1\2.mrfmos
        EXEC    /usr/local/gempak/scripts/mmos/mmos.pl

to:

# MRF MOS
WMO     ^...... .... ([0-3][0-9])([0-2][0-9]).*/pMEX
        FILE    -close     data/nwx/mos/mrfnew/(\1:yyyy)(\1:mm)\1\2.mrfmos


# MRF MOS
WMO     ^...... .... ([0-3][0-9])([0-2][0-9]).*/pMEX
        EXEC    /usr/local/gempak/scripts/mmos/mmos.pl

Note that I added the '-close' flag to the FILE action.  Without this, there
is no guarantee that the product will have been written to disk before the
EXEC action is run.  This approach will work because actions in a pattern-action
file are processed serially.  It will only work the way that you want if the
EXEC of your Perl script is placed after the writing of the product to disk.

In the second case, your mmos.pl script would need to read the entire product
from STDIN and write it to the desired output file as its very first action.
The pqact.conf entry for a rewritten action would look something like:

# MRF MOS
WMO     ^...... .... ([0-3][0-9])([0-2][0-9]).*/pMEX
        PIPE    /usr/local/gempak/scripts/mmos/mmos.pl 
        data/nwx/mos/mrfnew/(\1:yyyy)(\1:mm)\1\2.mrfmos

Here the product is PIPEed to your Perl script (sent via STDOUT from pqact)
and the fully qualified pathname for the desired output is passed as
a command line parameter to your Perl script.  mmos.pl would need to read
the product from STDIN, write it to the file specified by the parameter
passed to it (data/nwx/mos/mrfnew/(\1:yyyy)(\1:mm)\1\2.mrfmos), and then
do whatever processing it currently is setup to do.


Please be aware that pqact.conf actions are processed serially.  This means
that any actions after the invocation of your Perl script(s) will not be
processed until your Perl script(s) finishes and exits.  This can cause problems
if your Perl script is slow or somehow gets into an infinite loop.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: PLF-256397
Department: Support IDD
Priority: Normal
Status: Closed