20010205: McIDAS: NEXRAD Config File Read Error (cont.)

>From: Michael Keables <address@hidden>
>Organization: Geography Dept, University of Denver
>Keywords: 200102021959.f12JxiX16383 McIDAS-X NEXRAD DATALOC MCTABLE_READ

Mike,

>Still having problems with the mkeables account.

OK.

re: MCTABLE_WRITE="/home/user/mcidas/data/MCTABLE.TXT"

>Checked this out and modified the code in .cshrc to specify the directory as
>/export/home/mkeables/mcidas/data/MCTABLE.TXT

re: If this is how your MCTABLE_WRITE is setup, then you should CD to the
~mkeables/mcidas/data directory and see if the file MCTABLE.TXT exists.

>It does (and did before I changed the line in .cshrc).

Strange...

re: read/write permissions for ~mkeables/mcidas/data/MCTABLE_WRITE
>Checked this as well; I can read and write to it (permissions are u+rwxd,
>g+rw, o+r)

OK.

>I copied the CONEXRAD.CFG file to $MCDATA and can now display the images.

OK.  This means that ADDE services will be provided "locally", i.e.,
by ADDE routines running in your session, not from the ADDE remote server.

>I
>don't think the ADDE server is behaving as it should, because DATALOC ADD
>RTNEXRAD LOCAL-DATA still shows no groups or server IP addresses.

This needs investigation.

re: the options
>> o access the RTNEXRAD data from a remote ADDE server; this would be
>>   cyclone.natnet.du.edu in your case
>>
>> o setup your own account so that you can access the RTNEXRAD data as
>>   LOCAL-DATA
>>
>> The first option is the easiest for the average user.  It avoids that
>> user having to setup all of the supporting configurations in his/her
>> own account.  Let's pursue this avenue.  Given this, your problem now
>> boils down to the inability of your McIDAS-X session to write to
>> your MCTABLE.TXT file.  Again, you should verify that you can write
>> the file in your mcidas/data subdirectory.

>I would prefer the first option as well (and want to set up additional
>accounts this way as well.) I want to be sure that things are configured
>properly before creating new user accounts, however.

Right.  I understand and agree.

re: login access to your account denied

>Sorry about that. I changed the password for temporary access to my account
>for another issue down here and I guess I didn't reset the password to the
>same as the one I gave you (thought I did but guess not.)
>
>I'd appreciate it if you could look at the mkeables account and see what
>gives.

OK.  I did just that.  The following is rather lengthy, so hang in there.
There were a number of problems that needed to be corrected.

Here is the blow-by-blow of what I did and what I changed:

<login as you>

cd mcidas/data
ls

 ...
COUNTRY.DAT       GW6.GIF           KEITH9.GIF        TERMCHAR.001
 ...
FRAMENH.001       GW9.GIF           KJ2.GIF           VIRT9001
 ...
GE8.GIF           ISAAC26.GIF       MCTABLE.TXT       WV7.GIF
 ...
GEW10.GIF         ISAAC29.GIF       RESOLV.SRV        WXADV.BAT
 ...
GEW6.GIF          ISAAC6.GIF        SAT4.GIF          core
 ...

I noted the following problems:

o FRAMENH.001 and TERMCHAR.001 should only be found in a subdirectory
  of ~/.mctmp; they are supposed to be created on a per session basis
  and deleted when the session exits.  I deleted these files.

  The core file was caused by 'dsserv.k'.  It undoubtedly was happening
  because your MCTABLE_READ was setup incorrectly.  -- See below --

  Based on finding these, I then did the following:

dmap.k Frame
/export/home/mcidas/workdata% dmap.k Frame
PERM      SIZE LAST CHANGED FILENAME  DIRECTORY
---- --------- ------------ --------- ---------
-r--      3072 Sep 30  1999 Frame1.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame10.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame11.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame12.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame13.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame14.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame15.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame16.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame17.0 /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame2.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame3.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame4.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame5.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame6.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame7.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame8.0  /export/home/mcidas/help
-r--      3072 Sep 30  1999 Frame9.0  /export/home/mcidas/help
52224 bytes in 17 files

   These files, FrameN.M, are also supposed to be created on a per-session
   basis in subdirectories of each user's ~/.mctmp directory.  Notice
   that the PERM on these files is read only.  This would cause you
   problems when trying to run McIDAS from your (and any other user's)
   McIDAS session.  Also note that these files are _old_!

   Since the files need to be deleted, I logged in as 'mcidas' and basically
   repeated the steps above.  I also found copies of FRAMENH.001 and
   TERMCHAR.001 in the ~mcidas/workdata directory which is also not
   good.

   As 'mcidas' I deleted FRAMENH.001 and TERMCHAR.001 from ~mcidas/workdata
   and Frame*.* from ~mcidas/help.
   
   I am amazed that I was able to run a McIDAS-X session even as McIDAS
   and successfully display images and draw maps on them.  The fact
   that the FrameN.M files were read only should have prevented this.
   I am at a total loss to explain this!!

  after exiting the 'mcidas' Unix session, I went back to troublshooting
  in your account.

o the next thing I did as you was list out the Unix environment:

> env
HOME=/export/home/mkeables
 ...
UMASK=077
MCHOME=/export/home/mcidas
MCDATA=/export/home/mkeables/mcidas/data
MCPATH=/export/home/mkeables/mcidas/data:/export/home/mcidas/data:/export/home/mcidas/help
MCGUI=/export/home/mcidas/bin
MCTABLE_READ=/export/home/mkeables/mcidas/data/MCTABLE.TXT:/export/home/mcidas/data/ADDESITE.TXT
MCTABLE_WRITE=/export/home/mkeables/data/MCTABLE.TXT

  There are several problems here:

  -> I recommend that umask be 002, but this is not crutial

  -> MCTABLE_READ is incorrectly set.  The problem is the colon separating:

     /export/home/mkeables/mcidas/data/MCTABLE.TXT

     from:

     /export/home/mcidas/data/ADDESITE.TXT

     This should be a semi-colon

  -> MCTABLE_WRITE is incorrect.  It should be

     /export/home/mkeables/mcidas/data/MCTABLE.TXT

  To correct the problems above, I edited ~mkeables/.cshrc and changed
  the McIDAS definition block to:

# C-shell environment variable definitions for a general user

# MCHOME - the HOME directory of the user 'mcidas'
setenv MCHOME /export/home/mcidas

# NOTE: conditional definition is needed for C-shell users
if ( ! ${?MCPATH} ) then
  setenv MCDATA $HOME/mcidas/data
  setenv MCPATH ${MCDATA}:$MCHOME/data:$MCHOME/help
  setenv MCGUI  $MCHOME/bin
  setenv MCTABLE_READ "$MCDATA/MCTABLE.TXT;$MCHOME/data/ADDESITE.TXT"
  setenv MCTABLE_WRITE "$MCDATA/MCTABLE.TXT"
  if ( ! ${?PATH} ) then
    setenv PATH $MCGUI
  else
    setenv PATH ${MCGUI}:$PATH
  endif
endif

# Limit ADDE transfers to compressed ones
#setenv MCCOMPRESS TRUE

  Note that the colon in MCTABLE_READ has been replaced by a colon, and
  MCTABLE_WRITE was changed to point at 
/export/home/mkeables/mcidas/data/MCTABLE.TXT

  One question, however.  Why is the MCCOMPRESS line commented out?  I
  makes more sense to set MCCOMPRESS so that ADDE transfers be done in
  compressed mode.  This really helps speed things up when going out
  to remote servers.  Since I was already in the file, I uncommented
  this setting.

  To make the changes in .cshrc active, I then did:

  unsetenv MCPATH
  source ~/.cshrc

o after making the above changes, I went back to your ~/mcidas/data
  directory to see if the DATALOC invocation would work correctly:

> cd mcidas/data
> dataloc.k LIST

Group Name                    Server IP Address
--------------------         ----------------------------------------
BLIZZARD                     144.92.109.163
MYDATA                       <LOCAL-DATA>
RTGRIDS                      CYCLONE.NATNET.DU.EDU
RTIMAGES                     CYCLONE.NATNET.DU.EDU
RTNEXRAD                     <LOCAL-DATA>
RTNIDS                       CYCLONE.NATNET.DU.EDU
RTNOWRAD                     CYCLONE.NATNET.DU.EDU
RTPTSRC                      CYCLONE.NATNET.DU.EDU
RTWXTEXT                     CYCLONE.NATNET.DU.EDU
TOPO                         CYCLONE.NATNET.DU.EDU

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done

As you can see, it does.  Since you really want to access RTNEXRAD data
from your remote ADDE server, I then ran a DATALOC command to pint at it:

dataloc.k ADD RTNEXRAD cyclone.natnet.du.edu

Group Name                    Server IP Address
--------------------         ----------------------------------------
RTNEXRAD                     CYCLONE.NATNET.DU.EDU

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done

Now, the output shows that the command worked.  A full list verifies this:

> dataloc.k LIST

Group Name                    Server IP Address
--------------------         ----------------------------------------
BLIZZARD                     144.92.109.163
MYDATA                       <LOCAL-DATA>
RTGRIDS                      CYCLONE.NATNET.DU.EDU
RTIMAGES                     CYCLONE.NATNET.DU.EDU
RTNEXRAD                     CYCLONE.NATNET.DU.EDU
RTNIDS                       CYCLONE.NATNET.DU.EDU
RTNOWRAD                     CYCLONE.NATNET.DU.EDU
RTPTSRC                      CYCLONE.NATNET.DU.EDU
RTWXTEXT                     CYCLONE.NATNET.DU.EDU
TOPO                         CYCLONE.NATNET.DU.EDU

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done

o I tried exercising the remote server by issuing an IMGLIST asking for a
  list of NEXRAD IDs:

> imglist.k RTNEXRAD/N0R ID=LIST
Image file directory listing for:RTNEXRAD/N0R
imglist.k: Image Directory READ failed -- Server Error

  This is the same failure as you were reporting. This is puzzling since
  I note that the VERSION.TXT file in ~mcidas/data shows that you are up
  to addendum 7.704, the current update.

  I decided to log back in as 'mcidas' and troubleshoot there.  I found
  the same problem with MCTABLE_READ in ~mcidas/.cshrc as I found
  in ~mkeables/.cshrc.  I made the same correction and made the
  changes active by unsetting MCPATH and sourcing ~/.cshrc.

  I then went into the workdata directory and checked out the DATALOCs
  for 'mcidas'.  They included access to the NEXRAD data as LOCAL-DATA:

/export/home/mcidas/workdata% dataloc.k LIST

Group Name                    Server IP Address
--------------------         ----------------------------------------
BLIZZARD                     144.92.109.163
MYDATA                       <LOCAL-DATA>
RTGRIDS                      CYCLONE.NATNET.DU.EDU
RTIMAGES                     CYCLONE.NATNET.DU.EDU
RTNEXRAD                     <LOCAL-DATA>
RTNIDS                       CYCLONE.NATNET.DU.EDU
RTNOWRAD                     CYCLONE.NATNET.DU.EDU
RTPTSRC                      CYCLONE.NATNET.DU.EDU
RTWXTEXT                     CYCLONE.NATNET.DU.EDU
TOPO                         CYCLONE.NATNET.DU.EDU

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done

  I changed this to the remote server:

/export/home/mcidas/workdata% dataloc.k ADD RTNEXRAD cyclone.natnet.du.edu

Group Name                    Server IP Address
--------------------         ----------------------------------------
RTNEXRAD                     CYCLONE.NATNET.DU.EDU

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done
/export/home/mcidas/workdata% dataloc.k LIST

  And then tried to use the remote server for an IMGLIST:

/export/home/mcidas/workdata% imglist.k RTNEXRAD/N0R ID=LIST
Image file directory listing for:RTNEXRAD/N0R
imglist.k: TCP write failed
imglist.k: done

  So, something _was_ incorrectly setup for ADDE remote services.  I see
  that the TCP/IP daemon wrapper package is installed, so I am suspicious
  of it:

/export/home/mcidas/workdata% cat /etc/inetd.conf
#SEC           #
#SEC           #ident   "@(#)inetd.conf 1.33    98/06/02 SMI"   /* SVr4.0 1.5   
*/
#SEC           #
#SEC           #
#SEC           # Configuration file for inetd(1M).  See inetd.conf(4).
# WVtcpd 
# WVtcpd The 7.6 version of the TCP/IP daemon wrapper package is installed
# WVtcpd On this workstation. 
# WVtcpd 
# WVtcpd It was compilled with the Language extensions enable (See 
hosts_options(5))
# WVtcpd 
# WVtcpd Example of use:
# WVtcpd 
# WVtcpd finger  stream  tcp     nowait  nobody  /usr/bin/tcpd           
in.fingerd
# WVtcpd smtp    stream  tcp     nowait  root    /usr/bin/tcpd 
/usr/lib/sendmail -bs
# WVtcpd tftp  dgram  udp  wait  root  /usr/bin/tcpd  in.tftpd -s /tftpboot
# WVtcpd 

  BUT, I do _not_ see it being used for McIDAS ADDE:

cat /etc/inetd.conf
 ...
mcserv  stream  tcp     nowait  mcadde  /export/home/mcidas/bin/mcservsh        
mcservsh -H /export/home/mcidas
mccompress      stream  tcp     nowait  mcadde  
/export/home/mcidas/bin/mcservsh        mcservsh -H /export/home/mcidas

  While sniffing around, I did see one thing that I would change, but
  this is not the cause of the TCP write failed error above.  I would
  change the default shell for 'mcadde' to /bin/false (it is now
  /bin/csh).  This must be done as 'root'.  I made the change using
  sudo.

  After verifying that TCP wrappers are not being used for the McIDAS ADDE
  services, I started looking for a problem in ~mcidas/.mcenv.  What
  I found was that MCDATA was being defined in terms of MCDATA:

  MCDATA=$MCDATA/workdata

  This should be:

  MCDATA=$MCHOME/workdata

  I changed that line, and then retested an imglist.k for RTNEXRAD
  data from the 'mcidas' account:

/export/home/mcidas/workdata% imglist.k RTNEXRAD/N0R ID=LIST
Image file directory listing for:RTNEXRAD/N0R
 Pos Satellite/         Date       Time      Center   Band(s)
     sensor                                 Lat  Lon
 --- -------------  ------------  --------  ---- ---- ------------
   1  RADAR          5 FEB 01036  23:47:00     FTG    1
   2  RADAR          5 FEB 01036  23:50:00     GJX    1
   3  RADAR          5 FEB 01036  23:56:00     PUX    1
imglist.k: done


OK, so the remote ADDE server is now working for 'mcidas'.  Time to test it
out for you:

> imglist.k RTNEXRAD/N0R ID=LIST
Image file directory listing for:RTNEXRAD/N0R
 Pos Satellite/         Date       Time      Center   Band(s)
     sensor                                 Lat  Lon
 --- -------------  ------------  --------  ---- ---- ------------
   1  RADAR          5 FEB 01036  23:50:00     GJX    1
   2  RADAR          5 FEB 01036  23:56:00     PUX    1
   3  RADAR          5 FEB 01036  23:57:00     FTG    1
imglist.k: done

Looking good!

Let's recap the changes that were made:

-- in the 'mkeables' account --
1) corrected MCTABLE_READ definition in ~/.cshrc
2) uncommented the setenv MCCOMPRESS TRUE line in ~/.cshrc
3) removed FRAMENH.001 and TERMCHAR.001 in ~/mcidas/data
4) removed core in ~/mcidas/data
5) pointed McIDAS-X sessions at the ADDE remote server on cyclone for
   RTNEXRAD

-- in the 'mcidas' account --
1) removed FRAMENH.001 and TERMCHAR.001 in ~/workdata
2) removed Frame*.* in ~/help
3) pointed McIDAS-X sessions at the ADDE remote server on cyclone for
   RTNEXRAD
4) found and fixed the MCDATA definition in ~/.mcenv
5) changed the default shell for 'mcadde' to /bin/false in /etc/passwd


>On a totally unrelated matter, how can I set the font color on FRMLABEL?

FRMLABEL draws labels in the image plane, not the graphics:

   LEVel=text bg | brightness level for text and background (def=255 0)

If you want to annotate with text, then you need to use the ZA
command.  ZA allows you to select change the FONT and label colors.

>I've got the NEXRAD images up on our weather server
>(http://cyclone.natnet.du.edu/) and would rather diplay the font in some
>color other than magenta.

The color being used is one of the colors from the image enhancement.
You need to look at that enhancement and see which level is most
pleasing for the FRMLABEL label.  The image I see on your web page
shows that the enhancement has white at the upper range of brightness
levels.  A listing of the CREF enhancement used:

EU TABLE CREF
 Brightness  Blue      Green      Red
  min max   min max   min max   min max
  --- ---   --- ---   --- ---   --- ---
    0  15     0   0     0   0     0   0
   16  31   196 196   196 196   196 196
   32  47   118 118   118 118   118 118
   48  63   170 170   170 170   255 255
   64  79   140 140   140 140   238 238
   80  95   112 112   112 112   201 201
   96 111   144 144   251 251     0   0
  112 127     0   0   187 187     0   0
  128 143   112 112   255 255   255 255
  144 159    96  96   208 208   208 208
  160 175    96  96    96  96   255 255
  176 191     0   0     0   0   218 218
  192 207     0   0     0   0   174 174
  208 223   255 255     0   0     0   0
  224 239   255 255   255 255   255 255
  240 255   255 255     0   0   231 231
EU: Done

shows that you will have to specify LEV= in the range of 224 - 239.  This
would be something like:

FRMLABEL LEV=230 0 " DU CLIMATE LAB   DENVER RADAR  (ID) (DAY) (HHMM)Z

I displayed a Composite Reflectivity image from FTG in a McIDAS session
running in your account; drew a map; put up a data BAR; annotated it;
and saved it in frmlabel.gif as an example.  You will find this file in
~mkeables/mcidas/data/frmlabel.gif.  Please let me know if this is what
you had in mind.

>Thanks.

You are welcome.

One quick additional comment.  I have been working on the next addendum
for 7.7, 7.705.  In this addendum, I have updated the UWGRID program
and have fleshed out the GLOBAL.BAT file that can add grids to the
RTGRIDS/MRF-UW dataset files.  By configuring the uwgrid.sh shell
script and running it from cron at the appropriate times, one can
generate GRID files that are exactly the same (or are enhanced
versions) as the ones that used to be sent in the Unidata-Wisconsin
datastream.  This is useful since the Fkey menu was always geared for
using those files.

Additionally, I have updated the Fkey menu be about 100% ADDEized.  One
thing that I have added is access to the NOAAPORT GINI imagery as long
as one can contact remote ADDE servers that will provide the data.  I
setup GINI imagery access in your account for you to try:

DATALOC ADD GINIEAST PAPAGAYO.UNL.EDU
DATALOC ADD GINIWEST PAPAGAYO.UNL.EDU
DATALOC ADD GINICOMP STRATUS.AL.NOAA.GOV
DATALOC ADD RTGINI   ADDE.UCAR.EDU

The datasets GINIEAST, GINIWEST, and GINICOMP are subsets of RTGINI.
The reason for splitting up the dataset was so that sites can point
at different ADDE servers for imagery from different parts of the
country.

I encourage you to give these data sets a whirl and let me know
what you think.  Here are some sample IMGDISP invocations that should
be illustrative:

SF 1
EG B
IMGDISP GINIEAST/GE1KVIS STA=KBOS MAG=-3 REFRESH='EG;MAP H'
IMGDISP GINIEAST/GE1KVIS STA=KBOS MAG=1 REFRESH='EG;MAP VH'
IMGDISP GINIWEST/GW1KVIS STA=KASE MAG=-2 REFRESH='EG;MAP H'
IMGDISP GINICOMP/GSN8KIR STA=KTOP MAG=1 REFRESH='EG;MAP H'

DSINFO I GINIEAST
DSINFO I GINIWEST
DSINFO I GINICOMP
DSINFO I RTGINI

Tom

>From address@hidden Tue Feb  6 10:47:54 2001

Tom,

Thanks for everything that you did yesterday to straighten out the mess on
cyclone. I know where some of the problems came from (e.g. the : vs. ; was
my mistake), but I'm not sure about some of the others (I _KNOW_ I set UMASK
to 002 when I created the account, for example.) I appreciate you taking the
time to spell out exactly what was wrong and what you did to fix it.

Anyway, thanks for getting the ADDE back on track, and also for the advice
on FRMLABEL text colors.

--Mike

Michael J. Keables, Ph.D.                    email: address@hidden
Associate Professor                             voice: 303.871.2653
Director, Environmental Science               fax: 303.871.2201
Department of Geography                      GPS:  39 40 28.3 N
University of Denver                                       104 57 47.7 W
Denver, CO 80208                                 aural: "Hey Mike!!"