20050207: GeoTIFF output from McIDAS ADDE server (cont.)

>From: address@hidden
>Organization: GMU
>Keywords: 200501201830.j0KIUJv2013893 McIDAS AREA GeoTIFF ADDE

Hi Ying,

re: using ADDE to convert images from AREA to GeoTIFF formats

>Thanks! It's now working. You are quite right, the path was not defined
>correctly in LSSERVE.BAT file.

Sounds good.

>I can have the GEOTIFF files stored under ~mcidas/workdata, named like
>GOES0003.tif. 

OK.

>The question is, how to write script in pqact.conf to make the remapping
>automatic? 
>
>That is, to remap the stored data, put it under certain folder, and name
>the new files clearly, i.e. VIS_20050204_0115?

I would create a pqact.conf entry for each kind of image that you want
to convert to GeoTIFF.  I don't know exactly what you are doing
right now: presumably you are using pqact.conf actions that were
provided either in the ldm-mcidas package's example actions file,
ldm-mcidas-pqact.conf.all, or are using actions as recommended by
GEMPAK.  In either case, you will want to create actions that
EXEC a script for the individual type of image you are interested in.

For instance, if you are following the pqact.gempak_images example,
you should have an action like:

# Visible images

MCIDAS  ^pnga2area Q. (U[^ACXY1]) (.*) (.*)_IMG (0.65)um (.*) (........) (....)
        PIPE    -close
        pnga2area -vl /usr/local/ldm/logs/ldm-mcidas.log
        /data/ldm/gempak/images/sat/\3/\5/VIS/VIS_\6_\7

If you are only converting the GOES-East VIS images to GeoTIFF, you
should add an action something like:

# Convert UNWISC GOES-East Visible images to GeoTIFF

MCIDAS  ^pnga2area Q. (UV) (.*) (.*)_IMG (0.65)um (.*) (........) (....)
        EXEC    your_script options_to_your_script

The header contains a two letter product code (UV in this case) that
uniquely identify what kind of image it is.  Here is the list of
product codes for the UNIWISC images:

 Pcode    Format   Image type
 -------+--------+------------------------------------
   CA      AREA    CIMSS Cloud Top Pressure           
   CB      AREA    CIMSS Precipitable Water           
   CC      AREA    CIMSS Sea Surface Temperature      
   CD      AREA    CIMSS Lifted Index                 
   CE      AREA    CIMSS CAPE                         
   CF      AREA    CIMSS Ozone                        
   CG      AREA    CIMSS Wildfire ABBA Nth Hemishere  
   CH      AREA    CIMSS Wildfire ABBA Sth Hemishere  
   UA      AREA    Educational floater I              
   UB      AREA    GOES-W Western US Water Vapor      
   UC      AREA    Educational floater II             
   UI      AREA    GOES-E North America Infrared      
   UV      AREA    GOES-E North America Visible       
   UW      AREA    GOES-E North America Water Vapor   
   UX      AREA    Global Mollweide Infrared Composite
   UY      AREA    Global Mollweide Water Vapor Composite
   U1      AREA    Antarctic composite                
   U3      AREA    Manually Digitized Radar           
   U5      AREA    GOES-W Western US Infrared         
   U9      AREA    GOES-W Western US Visible          

After you know how to setup pqact.conf entries to process the images
you are interested in, you next need the script to run.

In the McIDAS distribution are two example shell scripts that are
intended to be copied to different directories, renamed, and edited
by the end user.  The scripts in the distribution are named 'mcrun.sh'
and 'mcbatch.sh'; both will be found in the ~mcidas/data directory.

The structure of 'mcrun.sh' is:

- set needed environment variables needed to run McIDAS programs
- setup an output log file
- run a sequence of McIDAS commands

The structure of 'mcbatch.sh' is:

- set needed environment variables needed to run McIDAS programs
- setup an output log file
- run the sequence of McIDAS commands contained in a McIDAS BATCH
  file

Either script can be used for your purposes, but I am assuming
that you might be more comfortable using 'mcrun.sh' as it is
more UNIXlike.

** NOTE **

Make sure to make your own local copy of 'mcrun.sh' renaming it to
something that makes sense to you and put it in a directory that is in
the PATH of the user running your LDM!

The part of 'mcrun.sh' that runs the McIDAS programs looks like:

#
# Now run 'mcenv' to create a McIDAS-X environment in which the McIDAS
# binary commands will run.
#
# It is the user's responsibility to put the commands they want to run
# in the following.
#

cd $MCDATA
mcenv << EOF

# put McIDAS-X commands you want to run here, one command per line.

# Example (note that these lines are commented out!!):
#
# dataloc.k ADD GINIEAST adde.ucar.edu
# eg.k
# imgdisp.k GINIEAST/GE1KVIS STA=KMIA MAG=-2 EU=IMAGE SF=YES
# map.k H
# bar.k
# frmsave.k 1 miamivis.gif

# done
  exit

EOF

Your job is to add the lines you need to do the IMGREMAP and IMGCOPY.

** NOTE **
You use the names of the actual McIDAS executables in your copy of
the 'mcrun.sh' script:

dataloc.k  -> DATALOC
imgremap.k -> IMGREMAP
etc.

As you noticed, the output of the final IMGCOPY will be in the first
directory in the MCPATH that is in scope.  Since you will be running
the script from a pqact.conf action running as the user 'ldm', the
first directory of the MCPATH will be ~ldm/mcidas/data, not
~mcidas/workdata.  You must ensure that ~ldm/mcidas/data exists and is
readable and writable by the user running your LDM:

<login as 'ldm'>
cd ~ldm
mkdir mcidas
mkdir mcidas/data

The last part of the question you posed was how to "name the new files
clearly, i.e. VIS_20050204_0115".  For this you will need to learn how
to use the IMGLIST command; trap the output from IMGLIST; parse the
output from IMGLIST to extract the date and time of the image; and then
use that information to rename the output file that gets created by the
IMGCOPY command.

As an example, if you are converting GOES-East VIS images, you would
get an IMGLIST listing for the most recent RTIMAGES/GE-VIS image
and send its output into an ASCII text file.  Here is an example:

IMGLIST RTIMAGES/GE-VIS FORM=ALL DEV=T GE-VIS.TXT

This will create the ASCII file 'GE-VIS.TXT' in the first writable
directory of your MCPATH.  In this example, this would be
~ldm/mcidas/data/GE-VIS.TXT.  The contents of this file will look
like:

Image file directory listing for:RTIMAGES/GE-VIS
 Pos Satellite/         Date       Time      Center      Res (km)   Image_Size
     sensor                                 Lat  Lon    Lat   Lon
 --- -------------  ------------  --------  ---- ----  ----- ----- ------------
   3  G-12 IMG       8 FEB 05039  21:15:00     0   72
   Band: 1    0.65 um VIS Cloud and Surface Features    4.03  4.59  2600 x 1732
     proj:    0 created: 2005039 213425  memo: RT GVAR
     type:VISR     cal type:BRIT
     offsets:  data=    2816 navigation=  256 calibration=    0 auxiliary=    0
     doc length:   0   cal length:   0   lev length:   0 PREFIX=   0
     valcod:          0 zcor:  0 avg-smp: N
     start yyddd: 2005039  start time:211513  start scan:  306
     lcor: 2677  ecor:  9032  bytes per pixel: 1  ss: 78
     Resolution Factors (base=1):   Line=    4.0   Element=    8.0
SEE: Done...EOF Encountered.

Your job is to read in this file and extract the information you want
to use in renaming your output file.

As you noted above, the IMGCOPY command is creating files named like
GOES0003.tif.  I suggest that it is easiest if you keep the output file
name/ADDE dataset member the same so that the output name is
GOES0001.tif for VIS conversions, GOES0002.tif for IR conversions,
etc.  That way you can add a simple 'mv' of the file created to rename
it using the naming convention you want.

I believe that this should be enough to get you going on this project.
Please let me know if anything does not make sense.

>Appreciate your help and happy Chinese New Year by the way!

Thank you very much!

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.