20000808: ADDE access to NOAAPORT GINI imagery at U. Georgia (cont.)

>From: "Thomas L. Mote" <address@hidden>
>Organization: University of Georgia
>Keywords: 200007172022.e6HKMuT11816 UGA McIDAS-X ADDE NOAAPORT GINI imagery

Tom,

>I am back in the office and I think we can make some 
>progress. I have put a new hard disk on the server this 
>morning so that I have about 11GB of disk space for current 
>data.

Sounds good.

>I have also made sure the LDM is running (it died while I 
>was gone).
>
>I started the GRIB decoder. A question here... a "DECINFO 
>LIST" shows the GRIB decoder active and the .SPL file shows 
>that it is being updated (time almost matches current 
>time). However, the statmon still shows the GRIB decoder in 
>red. I don't know whether everything is really correct here.

By the time I logged on, GRID file decoding was apparently working.
This was evidenced by the new GRID files in /data/mcidasd:

cacimbo% ls -alt /data/mcidasd/GRID5*
-rw-rw-r--   1 ldm      apps     25568108 Aug  8 18:24 /data/mcidasd/GRID5261
-rw-rw-r--   1 ldm      apps     254739376 Aug  8 18:14 /data/mcidasd/GRID5031
-rw-rw-r--   1 ldm      apps     27913964 Aug  8 15:53 /data/mcidasd/GRID5251
-rw-rw-r--   1 ldm      apps     24362752 Aug  8 15:39 /data/mcidasd/GRID5081
-rw-rw-r--   1 ldm      apps     28693668 Aug  8 15:35 /data/mcidasd/GRID5071
-rw-rw-r--   1 ldm      apps     16955144 Aug  8 15:10 /data/mcidasd/GRID5431
-rw-rw-r--   1 ldm      apps     83189420 Aug  8 14:45 /data/mcidasd/GRID5001
-rw-rw-r--   1 ldm      apps     3111424 Aug  8 13:58 /data/mcidasd/GRID5241
-rw-rw-r--   1 ldm      apps     171766980 Aug  8 13:41 /data/mcidasd/GRID5041
-rw-rw-r--   1 ldm      apps      416628 Aug  8 05:51 /data/mcidasd/GRID5141
-rw-rw-r--   1 ldm      apps     9134544 Aug  8 05:42 /data/mcidasd/GRID5021
-rw-rw-r--   1 ldm      apps     8375368 Aug  8 05:13 /data/mcidasd/GRID5011
-rw-rw-r--   1 ldm      apps      222992 Aug  8 03:30 /data/mcidasd/GRID5051
-rw-rw-r--   1 ldm      apps       80760 Aug  8 01:15 /data/mcidasd/GRID5401
-rw-rw-r--   1 ldm      apps      180004 Aug  8 00:32 /data/mcidasd/GRID5061

>I also followed the directions in the online manual to make 
>sure the ADDE server is running. I don't currently have 
>another machine running mcidas (working on upgrading the 
>instructional lab this week) to check to see if I can 
>remotely access data.

From my workstation here at Unidata, I was unable to do the following
(the first and simplest test for checking on a remote ADDE
installation):

telnet cacimbo.ggy.uga.edu 503

ADDE does not use the telnet protocol, but this is a "quick and dirty"
method for finding out if one can even get to the port in question.

I am, however, able to do this from cacimbo.  This tells me that
outside machines are blocked from port 500 and 503 services, but
that machines at UGA might not be.

>Finally, I know you have told me in the past how to save 10 
>hours of imagery rather than the default four. I cannot 
>find your old e-mail, even though I usually do save all of 
>my correspondence with UNIDATA. Can you remind me how I do 
>this.

This is done through the McIDAS routing table, ROUTE.SYS.  I logged on
and took a quick look at this.  ROUTE.SYS was in the correct directory
(/data/mcidasd) and appeared to have the correct read/write permissions
(664), but it had not been updated since January.  I tried changing
permissions on ROUTE.SYS to be 666 (rwrwrw), and its entries started to
update.

I next verified that the users 'ldm', 'mcidas', and 'mcadde' (ADDE
remote server account) were in the same group:

cacimbo% id ldm
uid=201(ldm) gid=2000(apps)
cacimbo% id mcidas
uid=206(mcidas) gid=2000(apps)
cacimbo% id mcadde
uid=209(mcadde) gid=2000(apps)

All appears to be as it should.

The next thing I did was take a look at the output data directory, /data:

cacimbo% ls -alt /data
total 200348
-rw-rw-r--   1 ldm      apps     102354944 Aug  8 17:32 ldm.pq
drwxrwxr-x   2 ldm      other       9728 Aug  8 17:00 mcidasd
-rw-rw-r--   1 ldm      apps      274432 Aug  8 11:26 pqsurf.pq
drwxrwxrwx  24 root     root         512 Aug  8 11:26 .
drwxr-xr-x  30 ldm      root        1024 Aug  8 11:19 ..
drwxrwxr-x   3 ldm      apps         512 Aug  8 10:02 safe
drwxrwxr-x   2 ldm      apps       17920 Aug  8 09:07 contest
drwxrwxr-x   2 ldm      apps         512 Jun 30 11:35 GRIB
drwxr-xr-x  11 gempak   pdinrs      1024 Jun 27 00:02 goeseast
drwxrwxr-x  23 ldm      apps         512 Jan 18  2000 weather
drwxrwxr-x   2 ldm      apps        5120 Jan 18  2000 nldn
drwxrwxr-x   2 ldm      apps        1536 Jan 18  2000 upperair
drwxrwxr-x   2 ldm      apps         512 Jan 18  2000 severe
drwxrwxr-x   3 ldm      apps        1024 Jan 18  2000 forecasts
drwxrwxr-x   2 ldm      apps         512 Jan 18  2000 summary
drwxr-xr-x   2 ldm      other       1024 Jan 18  2000 logs
drwxrwxr-x   9 ldm      apps         512 Jan 10  2000 wxr
drwxrwxr-x  14 ldm      apps         512 Jun 25  1999 gempak
drwxrwxr-x   3 ldm      apps         512 Dec 19  1997 wxp
drwxrwxr-x   2 ldm      apps         512 Nov 20  1997 gif
drwxrwxr-x   2 ldm      apps         512 Nov 20  1997 raw
drwxrwxr-x   2 ldm      apps         512 Nov 20  1997 convert
drwxrwxr-x   2 ldm      apps         512 Nov 20  1997 text
drwxrwxr-x   2 ldm      apps         512 Nov 20  1997 grid
drwxrwxr-x   8 ldm      apps         512 Oct 22  1997 surface
drwx------   2 root     root        8192 Nov 18  1994 lost+found

Notice that the mcidasd and logs directories indicate that they are
owned by the user 'ldm' in the group 'other'.  I would guess that at
some time in the past you changed the group of 'ldm' on your
system(s).  This would account for 'mcidas' not being able to update
/data/mcidasd/SYSKEY.TAB and for 'ldm' not being able to update
/data/mcidasd/ROUTE.SYS.  We really should get to the bottom of this so
that the permissions on ROUTE.SYS and SYSKEY.TAB can be set to be 664.
I would say that you should change the owner and group on thes two
directories to 'ldm' and 'apps', respectively, by 'root'.  After
doing this, you should be able to change the ROUTE.SYS and SYSKEY.TAB
permissions so that they are owner and group readable/writable, but
only readable by world.

I found that SYSKEY.TAB can't be updated since I tried to update your
routing table to be able to ingest the CIMSS products that were added
to the Unidata-Wisconsin datastream at the end of July.  Following
along this line of thinking, I noticed that you had not downloaded
the latest ldm-mcidas distribution.  I further noticed that your
system is running Solaris SPARC 5.5.1, and I don't have a binary
installation of ldm-mcidas for that platform.  So, I downloaded
the ldm-mcidas-7.6.4 source into ~mcidas and built the latest decoders:

cd ~mcidas
ftp ftp.unidata.ucar.edu
  <user: ftp>
  <pass: address@hidden>
  cd pub/ldm-mcidas
  binary
  get ldm-mcidas-7.6.4.tar.Z
  quit
zcat ldm-mcidas-7.6.4.tar.Z | tar xvf -
rm ldm-mcidas-7.6.4.tar.Z

<edit ~mcidas/.cshrc and define environment variables needed for ldm-mcidas
build (e.g., CC, FC, CPP_MCIDAS, etc.)>
source .cshrc

cd ldm-mcidas-7.6.4/src
./configure
make
make test
make install
make distclean

This ends up creating the ~mcidas/ldm-mcidas-7.6.4/(bin|etc|lib|man)
directories and installing 7.6.4 files in them.  I noticed that your
LDM pqact.conf file references ldm-mcidas decoders by the use of their
location, /unidata/home/mcidas/ldm-mcidas, and that ldm-mcidas is a
link to an ldm-mcidas.x.x.x distribution node.  You should change this
link to point at the new ldm-mcidas binaries:

cd ~mcidas
rm ldm-mcidas
ln -s ldm-mcidas-7.6.4 ldm-mcidas

Your LDM pqact.conf lwtoa3 entry will now point at the latest build
of lwtoa3.  However, I recommend that you switch to the new ldm-mcidas
decoder for PNG compressed UW stream images, pnga2area.  This will be
really simple:

comment out:

MCIDAS  ^(LWTOA3 .*)
        PIPE
        -close /unidata/home/mcidas/ldm-mcidas/bin/lwtoa3 -d /data/mcidasd -v

add:

MCIDAS  ^pnga2area Q. (..) (.*) (.*) (.*) (.*) (........) (....)
        PIPE    -close
        /unidata/home/mcidas/ldm-mcidas/bin/pnga2area -v
        -d /data/mcidasd -r \1,\2

(mind the tabs!)

After making mods to pqact.conf, you should always verify its integrity:

ldmadmin pqactcheck

If you get no errors, then you need to send a HUP signal to pqact:

kill -HUP <process id of pqact>

Once you have verified that the new decoder has successfully replaced
the old one, you should probably change your LDM ldmd.conf request for
MCIDAS data:

change:

request MCIDAS ".*" pluto.met.fsu.edu

to:

request MCIDAS "^pnga2area Q[01]" pluto.met.fsu.edu

Of course, after changing ldmd.conf, you will have to stop and restart
the LDM for the mods to take effect:

ldmadmin stop
<wait until all LDM processes exit cleanly>
ldmadmin start

Of course, you can only do this since you are an IDD leaf node, not a
relay (relays have to pass the old delta encoded images (the ones that
get decoded with lwtoa3) along to downstream sites.

OK, since I was on a mini-roll, I decided that your McIDAS installation
needed to come up to the current bugfix revision (Version 7.612; you
were at 7.604).  Here is what I did:

cd ~mcidas/mcidas7.6/update
ftp ftp.unidata.ucar.edu
  <user: ...>
  <pass: ...>
  cd unix/760/bugfix
  binary
  get mcupdate.tar.Z
  quit
zcat mcupdate.tar.Z | tar tvf -
rm mcupdate.tar.Z
cd ../src
make all
make install.all

This is taking awhile (just passing a couple of hours) since your machine
is not screamingly fast :-(  I may have to finish it (the make install.all)
part tomorrow.

The other thing I did was setup the configuration files for the GINI
ADDE server.  This involved:

/unidata/home/mcidas/workdata/GINI.CFG
/unidata/home/mcidas/RESOLV.SRV

Take a look to see what was done and if it makes sense to you.

>The NOAAport server is dumping the files into 
>/data/goeseast, which is NSF mounted to the same point on 
>cacimbo. I can send the data via the LDM, but I have 
>been told the LDM has a hard time keeping up.

It might, esepcially on your machine (a SPARC 20) which seems mighty
slow.

>It might be 
>interesting to try it again. I have the pqact set up to 
>accept it. It would no more than a half hour to make the 
>switch.

Where are you going to send it to, and do you have enough disk space?

>Let me know what you need me to do to get this functional.

To be accessible from the outside world, your remote ADDE setup needs
something, but I am not exactly sure what.  I think that this might
have something to do with a security perimeter of some kind since I can
exercise the ADDE remote server from the 'mcidas' account on cacimbo.

Got to run...

Tom