19990824: decoding grib by XCD (cont.)

>From: "Jennie L. Moody" <address@hidden>
>Organization: UVa
>Keywords: 199907281535.JAA02200 McIDAS GRIB DMGRID

Jennie,

>But first I wanted to say, I am around
>this afternoon, if you want to try to logon to aeolus and look at 
>the poke problem. 

You probably figured out that I did not have the time to jump on
your machine the day you wrote this message.

re: coming up with a better way to do the grib conversion process
>You will want to be user mcidas on ...

OK, I just logged in.

>The spool and decoder grid output are placed in
>
>mcidas: /incoming/data/mcidas/xcd $ ls
>#chmod.scr~    ../            GRID5035       IDXALIAS.DAT   chmod.scr
>./             GRID5015       HRS.SPL        SYSKEY.TAB     svgrid/
>
>the grib files we are working with are in:
>
>mcidas: /incoming/data/grib $ ls -al *05*
>-rw-r-----   1 ldmadmin sys       216307 Aug 20 14:51 us008_gf078_97090500_YxA
> Ax
>-rw-r-----   1 ldmadmin sys       226589 Aug 20 14:51 us008_gf078_97090500_YxA
> Cx
>-rw-r-----   1 ldmadmin sys       225531 Aug 20 14:51 us008_gf078_97090500_YxA
> Ex
>
>Woops, actually, these are not the eta grids, these are the mrf grids.

I see lots of files in the /incoming/data/grib directory.

>I will move some eta grids back in here, primarily because (totally
>separate problem, which I haven't looked into yet, sigh) we cannot
>get the decoder to get *anything* out of these files which we got
>from Dolores Kneissling (sp?) at COMET, they should have global mrf
>grids, but I haven't checked to see if we might be blocking the
>decoding of the grids in these grib files...need to look at NOGRIB
>and RTMODELS to see what's up with that....

OK.

>Anyway, this other stuff is overkill if you first want to just
>identify why we cannot poke (and change) the value of GRIBDEC.PRO
>from within mcidas.  

Firt things first.

The very first thing that I did was to verify that your version of LWU
could not update word 0 of GRIBDEC.PRO like it should.  It couldn't.
I then went into the source directory for McIDAS 7.1
(which was /unidata/mcidas/mcidas7.1/mcidas7.1/src (weird)) and
took a look at lwu.pgm (the source of LWU) and see that your copy
does not have a bugfix that we included made available.  The code
that we modified was just that needed to get around the inability
to change a value in a file that was only 1 4-byte word long:

snippit from your version of lwu.pgm

      IF (POS.LT.0.OR.POS.GE.MAX_WORD) THEN
           CALL EDEST('Word number specified is past end of file.',0)
           CALL EDEST('Maximum word number for LW file '//
     &     cfile(first_char:last_char)//' is: ',MAX_WORD)
           GOTO 98
      END IF


same snippit from our master distribution version:

      IF (POS.LT.0.OR.POS.GT.MAX_WORD) THEN ! <<<<< UPC mod 960130 >>>>>
           CALL EDEST('Word number specified is past end of file.',0)
           CALL EDEST('Maximum word number for LW file '//
     &     cfile(first_char:last_char)//' is: ',MAX_WORD)
           GOTO 98
      END IF


You can see that the modification was the change of the .GE. to a .GT.
With this change, your 7.1 version of LWU will work as it should.  I
am betting that Clay never picked up any bugfixes for your 7.1 
distribution.  To get things rolling. I updated your lwu.pgm and remade
lwu.k.  I then tested the new version by copying GRIBDEC.PRO to
GRIBDEC.NEW in the 710/workdata directory and then running the
LWU POKE invocation that was failing:

!cp GRIBDEC.PRO GRIBDEC.NEW
DMAP GRIB
PERM      SIZE LAST CHANGED FILENAME              DIRECTORY
---- --------- ------------ --------------------- ---------
-rw-      1878 Dec 30  1996 GRIBDEC.CFG           /home/mcidas/710/wor>
-rw-         4 Aug 26 17:22 GRIBDEC.NEW           /home/mcidas/710/wor>
-rw-     20252 Aug 10 16:30 GRIBDEC.OUT           /home/mcidas/710/wor>
-rw-         4 Aug 20 15:09 GRIBDEC.PRO           /home/mcidas/710/wor>
-rw-         4 Jul 27 15:42 GRIBDEC.PRO.bak       /home/mcidas/710/wor>
-rw-         4 Aug 10 15:59 GRIBDEC.PRObeforepoke /home/mcidas/710/wor>
22146 bytes in 6 files

LWU POKE GRIBDEC.NEW 4096 0
LWU:  Value was: 11845604

OK, one hurdle out of the way.

>By the way, you might notice that over here on aeolus, we had
>set up mcidas using the ldm-like version control, with a runtime
>directory, and links, etc.

Yes, I see that.  It doesn't look complete, however.  One thing that is
missing is a /unidata/mcidas/workdata directory.  I do see links to
bin, data, help, but not to doc, workdata, savedata, etc.

>There is actually only one full version of
>mcidas on this machine right now (7.1), but Clay had set this up
>this way long ago.  So, I was familiar with the idea (which you
>also have mentioned before), and I will attempt to set up the
>new version of 7.6 this way on windfall, since I think it will
>give us the most flexibility in trying to retain our use of 
>the mcidas rt-model code under 7.4 which (as you recall) was not 
>trivial to get working with our distribution, and eventually try to 
>get it working with 7.6.

OK.  When you do this, please don't use the setup on aeolus as a
complete guide for what to do.  I can provide more information if/when
you need it.

>It is an important part of Tony's thesis
>work, and he is hustling to finish it all up now, with the 
>intention of defending his thesis in October or so.  

Short fuse.  I wish him luck.

>Also, thanks for reminding me that netcdf gets bundled with
>7.6....I obviously read this, but then in the process of 
>feeling overwhelmed I managed to forget what I had read!

No problem.

>Thats the extent of my response so far....more later.  If you
>get on aeolus and have questions or comments, you can try to catch
>me on windfall or aerial with a *talk* or give me a ring
>(804) 924-0592.  I will be around until about 4PM today.

I poked around on aeolus some to see how things run.  I changed the
pointer in GRIBDEC.PRO as I indicated above and then started DMGRID
from a McIDAS session.  DMGRID dutifully worked its way though HRS.SPL
adding onto a couple of GRID files that existed in
/incoming/data/mcidas/xcd (I hope that this was OK).  Boy, is aeolus
slow! (But you knew that already.)

It seems to me that a script could be developed that could look at the
beginning, ending, and fill pointer in HRS.SPL and the location pointer
in GRIBDEC.PRO to ascertain if DMGRID had worked its way through a
everything in a spool that was to be decoded.  If it had, new data
could be written to the spool and the decoder would continue.  If it
hadn't, the script would sleep for awhile and then check later.  This
script could be either a Unix shell script that ran some McIDAS
commands or a McBASI script that was told what directory to look in for
a list of grib files to decode.  Given such a script and given that you
have enough disk space for the decoded output, then I could imagine
that one would:

o grab some grib files to decode and stick them into a staging directory
o write a file whose contents would be a listing of the files that
  are to be decoded, one per line
o run the script passing the name of the file containing the list of
  files to decode

Watching for DMGRID to go inactive by use of 'ps' doesn't seem to be
very useful.  I have been watching as DMGRID worked its way through
HRS.SPL and the 'ps' indication of whether or not it was active didn't
change even when its decoding was finished.  Watching the pointer in
GRIBDEC.PRO (word 0) as compared to the end of data pointer in HRS.SPL
(word 0) in conjunction with the beginning and end of data pointers in
HRS.SPL (words 1 and 2, respectively) would seem to be the thing to
do.


Tom