20000413: Running McIDAS batch files from other than /home/mcidas/workdata

>From: Gilbert Sebenste <address@hidden>
>Organization: NIU
>Keywords: 200004131753.LAA27350 McIDAS-X MCPATH REDIRECT BATCH

Gilbert,

>Quick question for you.
>
>I want to run some McIDAS batch files from both the /home/mcidas/workdata
>and /home/mcidas/cod directories on my machine. How can I tell McIDAS to
>check BOTH directories to see if the desired batch file to run is in
>either location?

As long as the BATCH files in the different directories don't have the
same name, the best way to do this is use the MCPATH searching of McIDAS.
The /home/mcidas/workdata directory should already be the first MCPATH
directory for your 'mcidas' user.  The next two directories should be
/home/mcidas/data and /home/mcidas/help.  To tell McIDAS to also search
/home/mcidas/cod, I would add this directory to the end of MCPATH:

setenv MCDATA /home/mcidas/workdata
setenv MCPATH ${MCDATA}:/home/mcidas/data:/home/mcidas/help:/home/mcidas/cod

These definitions should be put in your .cshrc file (I am assuming that
you are using the C or Tcsh shell).  The advantage of the MCPATH route
is that the MCPATH directories are searched in order (left to right) for
McIDAS data or ancillary data (e.g., scripts like BATCH files).  What
you need to make sure of, however, is that you do _not_ have REDIRECTions
for any of the BATCH files that you want to use in this way.  In McIDAS,
a REDIRECTion takes precedence over MCPATH searches.  If you specify that
you want to find a particular BATCH file in a certain directory by
using a REDIRECTion, that is the _only_ place that McIDAS will look
for that file.

>Thanks for your help!

No problem.

>LOVE the ADDE stuff.

Me too.  By the way, Jim Koermer has allowed me to setup the McIDAS
remote ADDE server on a machine at Plymouth State that has access to
the GOES-East/West GINI imagery.  He has also given me permission to
announce that his server is open to the Unidata community for ADDE
access to this imagery.  I am searching for additional sites with
NOAAPORT reception systems that would be willing to act as ADDE servers
for GINI imagery, so if you know of anyone, pass the info along.

>COD will be getting a McIDAS
>machine up and running soon, and when that happens, they'll want to use
>the satellite stuff for their classrooms, I'm sure.

Sounds good.

>Take care!

On a different topic, I am in the process of putting together a survey
regarding what satellite imagery Unidata sites want to see in the
Unidata-Wisconsin datastream.  I have been gathering statistics about
performance of different compression algorithms, and have found that
the numbers that I bounced off of you some time back were overly
optimistic (sigh).

Using the best overall compression approach for imagery in McIDAS AREA
format, I now find that I can get the following savings (all sizes are
averages over about a week (1000 products represented):

  Current         AREA         Delta encoded  PNG compressed    PNG
Image Sector    file size      product size   product size    Savings
---------------+--------------+--------------+---------------+--------
GOES-West VIS    2439056        2145447        1525458         619989
GOES-West IR     2439056        1309143         990694         318449
GOES-West H2O     613426         225347         156641          68706
GOES-East VIS    2422256        2142377        1545774         596603
GOES-East IR     2422256        1388884        1068211         320673
GOES-East H2O     608371         231733         167498          64235
Floater I         607776         397478         307993          89485
Floater II        607776         399856         311895          91863
---------------+--------------+--------------+---------------+--------
Sub Total       12159973        8240265        6074164        2179003

Mollweide IR      225088         163798         104013          59785
Mollweide H2O     225088         138423          70699          67724
Antarctic IR      308509         187221         135232          51989
MDR               225088          47667           4655          43012
---------------+--------------+--------------+---------------+--------
Sub Total         983773         537109         314599         222510

---------------+--------------+--------------+---------------+--------
Totals          13143746        8777374        6388763        2401513

The 'AREA file size' column is the size of the product after it has
been turned into an AREA file.  The 'Delta encoded product size' column
represents the current size of the product sent over the IDD.  The 'PNG
compressed product size' column represents the size the product would
have in the IDD using PNG compression.  The 'PNG Savings' column
represents the number of bytes saved by switching to PNG compression.

As you can see, the current Delta encoding of VIS images is pretty
poor.  PNG compression of the same images results in quite a bit of IDD
product size reduction, but is not as startling as I had hoped that it
would be (previous email).


In the options presented below, it is assumed that PNG compression will
be used on the Unidata-Wisconsin datastream image products.  Furthmore,
numbers presented assume that the spatial and temporal resolution of
the Global Mollweide IR, Global Mollweide H2O, Antarctic IR, and MDR
products remain as they are now.

Datastream Options:

1) Status quo: same spatial and temporal resolution of products we have
   now

   For a typical daylight hour, we would expect to see a reduction of
   about 2.3 MB in the hourly volume of imagery.

2) Doubling temporal resolution of current GOES-East/West VIS, IR, H2O,
   Floater I, and Floater II products

   We would expect to see the hourly volume of imagery rise to about
   11.9 MB.  This is an increase of about 3.5 MB over what is being
   sent now.

3) Doubling the spatial resolution of GOES-East/West VIS sectors while
   keeping all other products the same

   We would expect to see the hourly volume of imagery rise to about 15
   MB.  This is an increase of about 7 MB over what we are currently
   sending.  The compressed GOES-East/West VIS sector products sent
   through the IDD would each be about 6 MB.  After the VIS sectors are
   converted to AREA files, they would each be just over 9.6 MB in size.

4) Doubling temporal resolution of current GOES-East/West VIS, IR, H2O,
   Floater I, and Floater II products, AND doubling the spatial
   resolution of the GOES-East/West VIS products

   We would expect to see the hourly volume of imagery rise to about 29
   MB.  This is an increase of about 21 MB over what we are currently
   sending.

The numbers above are worst case scenarios.  They assume that every
product would be sent every hour.  In reality, the GOES-East/West VIS
sectors are only sent during the day, and the Mollweide IR, and H2O and
Antarctic IR products are only sent every three hours.  The total
volume in the datastream is, however, heavily dominated by the
GOES-East/West sectors especially the VIS ones.

What do you think about these numbers?

Tom