20010731: Problems at UPRM (cont.)

>From: Unidata Mcidas Account <address@hidden>
>Organization:  University of Puerto Rico
>Keywords:  200106202036.f5KKab115440 McIDAS LDM pqact

Luis,

>   With the old McIDAS installation I could see all the images by using
>the f-key menu(~/data/UNIDATA.MNU).  Karli, whom you may remember, was
>saving images by replicating the commands run by the f-key menu and then
>using batch.k to save them to disk.

I am curious about why this was being done IF the images that she was
saving were the ones that are being conveyed in the IDD.  The reason I
would be suprised by this is that the images are being sent to
atmos.uprm.edu hourly.  The exist in the /var/data/mcidas directory as
I noted in earlier emails about setting up this machine.

>Everything I tried out was donde
>using the "mcidas" account.  My plan was to replicate the work Karli had
>done earlier.  I first tried to display the images from within the mcidas
>program.  I'm a novice McIDAS user and I don't know all the commands/data
>sets by heart.  When the menu didn't work I guessed it was because that
>was a generic menu script that comes with the distribution, so I then
>tried to rewrite the same commands but with different arguments.  No luck
>again.

I am now guessing that the images that Karli was saving were the high
resolution NOAAPORT GINI VIS and IR images that she was getting by
ADDE.  If this is the case, then the reason that the menu is not
working is that you have not set up the access to a machine that will
serve you the data.  The GINI images are divided into three datasets
so that a site can get different images from different servers:

GINICOMP - composite images
GINIEAST - GINI images from GOES-East
GINIWEST - GINI images from GOES-West

I am betting that the stuff that Karli had setup was IMGCOPYs from
the GINIEAST dataset.  You have a choice of machines that you
can poing at to get these images:

snow.plymouth.edu    - in New Hampshire
cacimbo.ggy.uga.edu  - in Georgia
papagayo.unl.edu     - in Nebraska
stratus.al.noaa.gov  - in Colorado
adde.ucar.edu        - in Colorado

To setup access to one of these machines, you would run DATALOC commands
from a McIDAS session in the account you are using ('mcidas' in your case).
I would suggest something like:

<start a McIDAS session as 'mcidas'>
DATALOC ADD GINICOMP snow.plymouth.edu
DATALOC ADD GINIEAST cacimbo.ggy.uga.edu
DATALOC AAD GINIWEST papagayo.unl.edu

After "pointing" to a machine for ADDE service (by running DATALOC as
indicated above), you should be able to use the Fkey menu to display
the images and IMGCOPY to copy them from the ADDE server to your local
machine.

>I then went to the webpage and tried out some other commands, some
>of them worked(the basic ones in the "testing mcidas" section of the
>installation instructions, commands like EG, MAP NA..etc).  Once those
>worked, I narrowed the problem down to either the images are not being
>downloaded correctly or (the likely case) I'm doing something wrong.

Again, since I am betting on you trying to get at the GINIEAST images,
I can add that the images are not being sent to you.  You have to go
get them from an ADDE server that has the images and allows you to
access them.  The list above does just this.

>   We want to be able to archive most of the images we're receiving, not
>just for 24-48 hours but rather all year long and then save them to backup
>media on a monthly or regular basis.  I thought that the images we were
>receiving were encoded and that every time mcidas wanted to access them,
>they were run through the decoders.

The images in the Unidata-Wisconsin datastream (MCIDAS LDM feed type) are
PNG compressed.  The "decoder" pnga2area run by a pqact.conf action 
uncompresses the images back into their original AREA form.  By the way,
extensive tests here at Unidata showed that the PNG compression we are
using is the most efficient for these images.  Given this, and if you
want to archive the images that you are receiving through the IDD, you
may want to add a pqact.conf entry to simply file the images as received.
This way, you wouldn't have to recompress them before saving them to
your archive.  If you have existing images in AREA format, you can use
the 'area2png' encoder that is also distributed with the ldm-mcidas
decoders that you have loaded on atmos.uprm.edu.  You can find area2png
in the ~ldm/decoders directory along with pnga2area.  If you run
area2png with no arguments, you will get an idea of how to run it:

mcidas@atmos:/home/ldm/decoders$ ./area2png
Usage: ./area2png -f AREAfile <-a SATANNOT> <-b SATBAND> <-vxl-> PNGfile
Where:
    -a SATANNOT  McIDAS SATANNOT pathname (def: /home/mcidas/data/SATANNOT)
    -b SATBAND   McIDAS SATBAND pathname (def: /home/mcidas/data/SATBAND)
    -f AREAfile  Input AREA pathname (no default)
    -h           Print the help file, then exit the program
    -n           Echo the output file name to stdout (def=no)
    -l logname   Log file name (default uses local0; use '-' for stdout)
    -v           Verbose logging
    -x           Debug logging
Positionals:
    PNGfile      Output PNG pathname mask (no default)

                 'PNGfile' may contain 'replaceables':
                    \BAND - band information from SATBAND
                    \RES  - resolution information from SATBAND
                    \SAT  - satellite platform info from SATANNOT

                 and 'strftime' style day/time specifiers:
                    %Y    - image year       [CCYY]
                    %m    - image month      [MM]
                    %d    - image day        [DD]
                    %j    - image Julian day [JJJ]
                    %H    - image hour       [HH]
                    %M    - image minute     [MM]
                    etc.

Example Invocation:
    ./area2png -l - -f /data/ldm/mcidas/AREA0170 \SAT_\RES_\BAND_%Y%m%d.%H%M

might expand to the equivalent:
    ./area2png -l - -f /data/ldm/mcidas/AREA0170 
GOES-8_IMG_8km_6.8um_20000511.2000

>In any case, my #1 priority right now
>is to have access to the images, If I have access to them via the
>filesystem even better then.

The Unidata-Wisconsin images are on your machine.  They can be found in
/var/data/mcidas.

>I'll use a conversion program to compress
>them for archiving.  Once I can archive them, I'll have spare time to go
>through all the turorials.  I'm on a vicious circle right now.  I'm in a
>hurry to have all these things done as soon as possible, because of this I
>don't have time to work with the tutorials... I can't do things
>effciently, because I'm not familiar with the program and its commands...

I understand.  It is too bad that you don't have enough time to work through
the tutorial since your understand of how things work would make what
you are trying to do so much easier.

>Was I clear on what I need to do?

Yup.

>It's just copy the images from
>wherever they are on the disk and compress them.  I just need an
>easy/efficient way to access them.  Actually, I just need to know where
>they are...

Remote files are on ADDE serves and need to be IMGCOPYied to your system.
Unidata-Wisconsin images are in the /var/data/mcidas directory.

>   Once again, thanks a lot for your help.  You're very helpful.  And I am
>_VERY_ sorry for being ambiguous, it was not intentional.

No problem.

So as to get you really setup to use the data you are already getting,
do the following:

o edit ~mcidas/data/LOCDATA.BAT and make it look like:

DATALOC ADD CIMSS     atmos.uprm.edu
REM DATALOC ADD RTGINI    136.145.165.17
DATALOC ADD GINICOMP  snow.plymouth.edu
DATALOC ADD GINIEAST  cacimbo.ggy.uga.edu
DATALOC ADD GINIWEST  papagayo.unl.edu
DATALOC ADD RTGRIDS   papagayo.unl.edu
DATALOC ADD RTIMAGES  atmos.uprm.edu
REM DATALOC ADD RTNIDS    136.145.165.17
DATALOC ADD RTNEXRAD  atmos.uprm.edu
REM DATALOC ADD RTNOWRAD  136.145.165.17
DATALOC ADD RTPTSRC   atmos.uprm.edu
DATALOC ADD RTWXTEXT  atmos.uprm.edu
DATALOC ADD TOPO      atmos.uprm.edu

After making these editorial changes, make them active in the 'mcidas'
account by doing:

<start a McIDAS session>
BATCH LOCDATA.BAT

Verify that the settings you just made were accepted:

DATALOC LIST

After you setup a DATALOC to GINIEAST images, try the following to
verify that things work:

DSINFO I GINIEAST
IMGLIST GINIEAST/GPR1KVIS
IMGDISP GINIEAST/GPR1KVIS STA=TJSJ EU=IMAGE SF=YES REFRESH='EG;MAP H'

>Let me know if the above helped.

Tom