20020610: McIDAS AREA image ranges

>From: "Neil R. Smith" <address@hidden>
>Organization: Texas A&M
>Keywords: 200206102047.g5AKl8J05338 Unidata-Wisconsin ldm-mcidas

Neil,

>One of our McIdas users, being a guru of the earlier vintages,
>noticed that the image product ranges are 9-10hrs, whereas 
>he remembers 48hr product ranges; eg.,
>GOES 8 IR: 1500-1547
>GOES 8 VIS: 1400-1447
>where those are currently 150-159 & 210-219, respectively.
>
>Does this reflect a change in the McIdas feed product offering
>or is this a local ldm-mcidas config issue?

This is a local McIDAS routing table, ROUTE.SYS, configuration issue.

>If the later, please expound.

Yes, there is.  The LDM-McIDAS decoder for Unidata-Wisconsin satellite
imagery, pnga2area, can be told to write the products out to files
whose names differ from the time honored (i.e., old) AREAnnnn
convention.  The pqact.conf action that you undoubtedly have in place
right now most likely looks like:

# CIMSS and UW Products decoded into AREA files using McIDAS routing table
MCIDAS  ^pnga2area Q. (..) (.*) (.*) (.*) (.*) (........) (....)
        PIPE    -close
        pnga2area -vl /usr/local/ldm/logs/ldm-mcidas.log
        -d /data/ldm/mcidas -r \1,\2

Your action may differ in the output log file:

'-vl /usr/local/ldm/logs/ldm-mcidas.log'

and/or the directory in which you are writing the data files:

'-d /data/ldm/mcidas'

The '-r' flag to pnga2area says to use the McIDAS routing table,
ROUTE.SYS, for information on how to name output AREA files.  In order
to be used, ROUTE.SYS has to exist in the /data/ldm/mcidas (or whatever
your -d directory is) directory AND be readable and writable by the
user running the LDM (typically the user 'ldm').

The fact that you are now seeing 10 of each kind of image product being
decoded tells me that someone at TAMU replaced the existing copy of
ROUTE.SYS in the '-d' directory.  You can get back to your former
configuration by restoring the previously used version of ROUTE.SYS to
that output directory.

>If the former, is 
>there a cute way to rename the files so they are not overwritten
>and thus hold at least 48hrs worth?

Yes, but it may affect other things in operation at your site.
'pnga2area' can easily create output file names _not_ in the AREAnnnn
format.  (I added this feature specifically for GEMPAK users).
For instance, the following pqact.conf entries will write out VIS, IR
and WV Unidata-Wisconsin images into subdirectories of
/data/ldm/gempak/iamges/sat:

# UW Products decoded into GEMPAK file hierarchy
#
# Unidata-Wisconsin PNG compressed Products
#              \1  \2   \3          \4      \5    \6     \7
#           Qx pd anum SAT         BAND    RES CCYYMMDD HHMM
# pnga2area Q0 CF 0015 GOES-10_SND UNKBAND 1km 20000525 2000
#
#
# Standard VIS, IR and WV product actions
#
MCIDAS  ^pnga2area Q1 (U[^ACXY1]) (.*) (.*)_IMG (0.65)um (.*) (........) (....)
        PIPE    -close
        pnga2area -vl /usr/local/ldm/logs/ldm-mcidas.log
        /data/ldm/gempak/images/sat/\3/\5/VIS/VIS_\6_\7

MCIDAS  ^pnga2area Q1 (U[^ACXY1]) (.*) (.*)_IMG (6.8)um (.*) (........) (....)
        PIPE    -close
        pnga2area -vl /usr/local/ldm/logs/ldm-mcidas.log
        /data/ldm/gempak/images/sat/\3/\5/WV/WV_\6_\7

MCIDAS  ^pnga2area Q1 (U[^ACXY1]) (.*) (.*)_IMG (10.7)um (.*) (........) (....)
        PIPE    -close
        pnga2area -vl /usr/local/ldm/logs/ldm-mcidas.log
        /data/ldm/gempak/images/sat/\3/\5/IR/IR_\6_\7

In the examples above, the naming of the subdirectories is controlled
by inforation from the product headers.  Some examples of the output
names are:

VIS:

/data/ldm/gempak/images/sat/GOES-8/4km/VIS/VIS_20020609_2215
/data/ldm/gempak/images/sat/GOES-8/4km/VIS/VIS_20020609_2315
/data/ldm/gempak/images/sat/GOES-8/4km/VIS/VIS_20020610_0015
 ...

IR:

/data/ldm/gempak/images/sat/GOES-8/4km/IR/IR_20020609_2215
/data/ldm/gempak/images/sat/GOES-8/4km/IR/IR_20020609_2315
/data/ldm/gempak/images/sat/GOES-8/4km/IR/IR_20020610_0015

WV:

/data/ldm/gempak/images/sat/GOES-8/8km/WV/WV_20020609_2215
/data/ldm/gempak/images/sat/GOES-8/8km/WV/WV_20020609_2315
/data/ldm/gempak/images/sat/GOES-8/8km/WV/WV_20020610_0015

  NB: There are other entries for the other images in the Unidata-Wisconsin
  datastream.  Examples of these can be found in the ldm-mcidas-pqact.conf
  example include with the ldm-mcidas release and in:

  LDM-McIDAS HomePage
  http://www.unidata.ucar.edu/packages/mcidas/mcidd/
    Using LDM-McIDAS Decoders
    http://www.unidata.ucar.edu/packages/mcidas/mcidd/ldm-mcidas-use.html

The upside of using the different naming capabilities of pnga2area are:

o you can keep as many data files online as you like (and have disk space for)

o if you choose your output directories correctly, you can use the files
  directly with the GEMPAK GUIs, GARP and NMAP

The downside of doing this are:

o you may have one or more scripts that run that expect the file
  names to be in the form AREAnnnn.  These scripts would need to be
  changed.

o if you are using old, non-ADDE McIDAS applications like DF to display
  images, then this will not work.  You would need to upgrade your old
  use of McIDAS to use of ADDE concepts.

o if you are using ADDE applications to display/manipuate the images, you
  will need to redo your ADDE datasets for the images.  This is not hard
  and will only have to be done once.

o you will need to implement a data scouring scheme.  The old McIDAS
  AREAnnnn convention depended on a "circular" naming convention:

  AREA1500
  AREA1501
  ...
  AREA1547

  If you had enough disk space to hold an entire set, you would be OK
  since you would never need more disk space.

  To help in the data scouring, we provide a couple of example C shell
  scripts that are designed to be FTPed by you, put in a directory in
  the PATH of the 'ldm', and edited by you to set how many images you
  want to keep online, and finally kicked off by cron.  The scripts
  can be found in the pub/ldm5/scour directory of anonymous FTP on
  our FTP server, ftp.unidata.ucar.edu.  The scripts are all very
  similar to each other.  In scour_bynumber.csh you need to set the KEEP
  value to the number of files you want to keep in a particular
  directory.  In the other files, you need to set KEEP and PATH and
  'areadir'.

  The advantage of scour_bynumber.csh is that it is simple to configure
  and run.  The advantage of the 'prune_*' scrits is that they can
  scour down a directory hierarchy.

Please let me know if you have additional questions about the above.

Tom Yoksas