[LDM #PHE-874806]: LDM and filing of GOES-17 NIMAGE data

Hi Mike,

re:
> Thanks again for the feedback.  My gmail doesn't agree with inline
> responses so I'll include all here.  I will take your advice and post
> something to the gembud list, but I wanted to provide a bit more follow-up
> here as well.


OK, sounds good.

re:
> I was able to get the L2ProdFile.sh to work intact when with two important
> edits to the pqact entry:
> 
> 1) remove the initial "/data/ldm/pub..." path because my ldm had no
>    permission to write there

Yes, the directory to which one wants to file the products needs to
match the local setup, so having to change this is entirely understandable.

re:
> 2) I needed to remove the spaces between arguments to supply all in the
>    first argument $1
> i.e.
> NIMAGE
> ^(.*)/(GOES1.)/(Products)/(.*)/((CONUS|FullDisk|Mesoscale-1|Mesoscale-2|PuertoRico|Alaska|Hawaii).*)/(20......)/(OR_...-L2.*.nc)
> PIPE    -close
> /usr/local/ldm/util/L2ProdFile.sh images/sat/\2/\3/\4/\5/\7/\8
> logs/L2ProdFile.log

This should not have been necessary.  Here is the pattern-action file action
we are running on our server machines verbatim:

NIMAGE  
^(.*)/(GOES1.)/(Products)/(.*)/((CONUS|FullDisk|Mesoscale-1|Mesoscale-2|PuertoRico|Alaska|Hawaii).*)/(20......)/(OR_...-L2.*.nc)
        PIPE    -close
        util/L2ProdFile.sh \1 \2 \3 \4 \5 \7 \8 logs/L2ProdFile.log

On both of our servers, we are using the old LDM setup of:

- having ~ldm/data and ~ldm/logs directories

- setting the LDM registry so that the current working directory for pqact
  invocations is the HOME directory of the user 'ldm'

re:
> The suggested pqact entry on this page:
> https://www.unidata.ucar.edu/data/pqact.conf_nimage.html#L2_ACTION
> 
> shows spaces which mean different arguments, and L2ProdFile.sh expects only
> two arguments, then file path/name and the log .
> 
> i.e this:  util/L2ProdFile.sh \1 \2 \3 \4 \5 \7 \8 logs/L2ProdFile.log
> 
> does not work because of the spaces.

It shouldn't.  Please check to see that your copy of L2ProdFile.sh
looks like:

SHELL=/bin/bash
export SHELL

# Date format for LOG file follows: 20170901T000001.826060Z, and default
# number of files to keep in 'current' directories
dfmt="%Y%m%dT%H%M%S"

# Home directory of 'ldm'
LDMHOME=`echo ~ldm`

# Program name and time hack
program="`date -u +$dfmt`.`date -u +'%N' | cut -b 1-6`Z `basename $0`[$$]:"

# Set log file name
if [ $# -ge 8 ]; then
  logfile=$8
else
  logfile=$LDMHOME/logs/L2ProdFile.log
fi

# Make sure that the log directory exists
dname=`dirname $logfile`
if [ ! -d "$dname" ]; then
  mkdir -p $dname >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    message="ERROR: unable to create LOG directory $dname, exiting"
    echo $program $message
    cat > /dev/null
    exit 1
  fi
fi

# Send all messages to the log file
exec >>$logfile 2>&1

# Top level output directory
topdir=$1/$2/$3/$4/$5
dname=$topdir/$6

# Create output directory
if [ ! -d "$dname" ]; then
  mkdir -p $dname >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    message="ERROR: unable to create output directory $dname"
    echo $program $message
    cat > /dev/null
    exit 1
  fi
fi


Notice in this listing snippit that 'topdir' is set to use the 1st through
fifth parameters in the invocation command line.

Hmm... You wouldn't be editing any files using Windows would you?

re:
> As far as plotting sat imagery, I'm not able to get anything to plot with
> NMAP2, but i have a got a few crumbs to plot via gpmap.    Where I am now
> is that the GAREA= entry will not accept standard latlon entries such as:
> GAREA=25;-150;50;-110
> but it will accept something like GAREA=US.
> BUT, no matter what I put in for GAREA,the plot always looks like this over
> Hawaii:
> 
> http://www.met.sjsu.edu/weather/images/test/goes-vis.gif
> 
> Hopefully this is something as simply as a table entry tweak.  I will
> expand on this and post to gembud.

I think that there will be at least 2 and possibly more gembud subscribers
that can help you MUCH better than I can.

re:
> Perhaps I should switch over to McIDAS?

That is, of course, up to you.  I can say, however, that I can actually help
you with McIDAS questions :-)

re:
> I wonder how the learning curve is
> there and how complicated the env/config would be to set up.

McIDAS is much different than GEMPAK, so there would be a learning
curve.  But, I can help.

re:
> I imagine if
> I had some help and a sample script I could get something going.  Any
> advice?   Will McIDAS be support going forward?

McIDAS will be supported by Unidata for as long as I am here.  What happens
after I retire is not known to me right now.

re:
> thanks as always,

No worries.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: PHE-874806
Department: Support GEMPAK
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.