20020317: McIDAS-X/XCD installation and configuration on DEC OSF/1 5.1 (cont.)

>From:  "Dan A. Dansereau" <address@hidden>
>Organization:  Utah State University
>Keywords: 200203141936.g2EJa6a06867 McIDAS DEC OSF/1

Dan,

re:
>I have regenerate a new op system, ldm, and mcidas
>anyway before I started "fixing it" I thought you could
>look at it,

Things seem to be working correctly on tornado:

o the LDM is ingesting and decoding IMAGE, POINT, GRID and TEXT data
  from allegan

o XCD decoders are processing the data and writing the output into
  /var/data/ldm/xcd

o and McIDAS is setup to view the data being ingested by ADDE

Here is a list of things that I did:

LDM

1) logon as the user 'ldm'

2) stop the LDM:

   ldmadmin stop

3) edit the Bourne shell scripts batch.k, mcscour.sh, and xcd_run:

   cd decoders

   o make sure that all three scripts declare SHELL to be 'sh' and
     export that definition

   o make sure that all three scripts correctly set MCDATA, MCPATH, MCGUI,
     etc

   o modify PATH and LD_LIBRARY_PATH environment variables to remove
     SunOS specific settings

4) modify ldmd.conf to remove startup of pqexpire (not needed in LDM 5.1.x)

5) modify pqact.conf to remove unneeded decode actions and add 3 new
   actions that will be used if/when you start ingesting the FNEXRAD
   feed.  In LDM 5.1.2, FNEXRAD was not yet understood, so the feed
   type I used in pqact.conf was NMC3.  This will remain OK when
   you upgrade your LDM, but at some point you will want to change
   the NMC3 use to FNEXRAD as it is more clear as what feed is being
   acted on

McIDAS

1) logon as the user 'mcidas'

2) verify that the environment variable definitions in ~mcidas/.cshrc
   are correct.  The McIDAS definitions were OK, but the setting
   of the PATH was not:

   change:

set path = ( /sbin /usr/sbin /usr/bin /usr/bin/X11 . )

   to:

if ( ! ${?path} ) then
  set path = ( /sbin /usr/sbin /usr/bin /usr/bin/X11 . )
endif

   I believe that this was a BIG cause of the problems you originally
   reported.  What happens is that when a McIDAS session is started, a
   routine called 'mcenv' is actually run.  If the user runs under the
   C shell, .cshrc is sourced.  Given the 'path' setting that was in
   your .cshrc file, and given the construct in the McIDAS environment
   variable setting that guards against 'path' from getting continually
   reset, you would lose the /home/mcidas/bin directory in 'path', and
   then subsequent attempts to do McIDAS things (list, plot, etc.)
   would fail since the executables would not be found.

3) remove the copies of LWPATH.NAM, STRTABLE, and SKEDFILE in /home/mcidas

4) remove the copies of ALLOC.WWW, RESOLV.SRV, and STRTABLE in 
   /home/mcidas/mcidas7.8/data (created during test phase, but not needed
   anymore)

5) setup LOCAL.NAM, LSSERVE.BAT, and LOCDATA.BAT:

   cd ~mcidas/data

   edit LOCAL.NAM:
      change all occurrances of /var/data/xcd to /var/data/ldm/xcd (numerous)

   edit LSSERVE.BAT
      comment out all definitions for datasets that you will most likely never
        host on your machine (e.g., WNEXRAD, WNOWRAD).  Commenting out
        the definitions is done by adding a 'REM ' to each line that defines
        a group/descriptor pair

   edit LOCDATA.BAT
      change DATALOCs to point at external machines for all of those
        datasets that you will likely never have locally (e.g., GINICOMP,
        GINIEAST, GINIWEST, NEXRCOMP (you may eventually host NEXRCOMP))

      NOTE: I added a new dataset to your LOCDATA.BAT file.  The dataset
        is for composite NEXRAD reflectivity products.  This dataset
        will be announced either this week or the next, but I wanted
        to give you a jump on it.

6) make the REDIRECTions specified in LOCAL.NAM active:

   cd ~mcidas/workdata          <- NB: I CD to the directory first
   rm LWPATH.NAM
   redirect.k REST LOCAL.NAM

7) remove previous ADDE dataset definitions and define those setup in
   LSSERVE.BAT:

   cd ~mcidas/workdata
   rm RESOLV.SRV
   batch.k LSSERVE.BAT

8) remove previous DATALOC definitions and then define them as specified
   in LOCDATA.BAT:

   cd ~/mcidas/data
   rm ADDESITE.TXT

   cd ~mcidas/workdata          <- NB: I CD to the directory first
   batch.k LOCDATA.BAT

9) make sure that the McIDAS environment string XCDDATA is defined
   in the McIDAS string table:

   cd ~mcidas/workdata          <- NB: I CD to the directory first
   te.k XCDDATA \"/var/data/ldm/xcd

10) run the scripts that setup XCD decoding:

    batch.k XCD.BAT
    batch.k XCDDEC.BAT

11) I will be releasing a new addendum for McIDAS soon.  I uploaded some
    of the routines that are in the new addendum to your system so
    you could begin looking at the NEXRAD composite imagery now.

Back to LDM

6) I noticed  that you had copied SATANNOT and SATBAND from either the
   McIDAS distribution or from the ldm-mcidas distribution to the
   /var/data/ldm/xcd directory.  This is not needed, so I moved them
   to the /home/ldm/etc directory.  These files are used by the
   ldm-mcidas pnga2area decoder.  Their existence in ~ldm/etc are not
   needed for sites that are setup so that the HOME directory for
   'mcidas' is /home/mcidas (like yours is).  On other systems, I
   advise users to put the files in ~ldm/etc, and they would have
   to tell pnga2area where to find the files through '-a' and '-b'
   flags on the pqact.conf pnga2area invocation line.

7) verify that the mods I made to pqact.conf do not have errors and then
   restart the LDM:

   ldmadmin pqactcheck

   -- if no errors are found --

   ldmadmin start

OK, so that got things rolling!  I made, however, a couple more changes
on your system:

McIDAS

1) edit the McIDAS ADDE remote server environment file to setup logging
   of ADDE transactions:

   cd ~mcidas
   edit .mcenv
     add ADDE_LOGGING=YES and export that setting

2) add a REDIRECTION for the ADDE remote server log file so that it will
   be read/written in the ~ldm/logs directory:

   cd ~mcidas/workdata
   redirect.k ADD SERVER.LO\* \"/var/data/ldm/logs
   touch /var/data/ldm/logs/SERVER.LOG
   chmod 666 /var/data/ldm/logs/SERVER.LOG

LDM

1) setup rotation of the ADDE remote server logs to cron:

   setenv EDITOR vi
   crontab -e

   <add>
#
# Rotate ADDE access log files just past 12 midnight on first day of month
#
1 0 1 * * /home/ldm/bin/newlog logs/SERVER.LOG 3; chmod 666 logs/SERVER.LOG

After setting up ADDE server logging, you can easily see who is looking
at data through your remote server:

<login as 'mcidas'>
cd workdata
addeinfo.k

The default execution of addeinfo.k just lists machines that have used
your remote server, and how much data they transferred.  addeinfo.k has
other modes that allow you to see exactly what the external process
was getting.  Check out the online help for ADDEINFO for more information.

Wrap-up:

The changes I made were basically:

1) setup the McIDAS environment stuff in ~mcidas/.cshrc (and correct anything
   that was wrong there)

2) setup LOCAL.NAM, LSSERVE.BAT, and LOCDATA.BAT in ~mcidas/data

3) remove existing ADDE dataset definitions, file REDIRECTIONs, and
   DATALOCs so that the new setup in the files in 2) would be all
   that there is

4) define needed REDIRECTions, ADDE dataset definitions, and DATALOCs

5) setup ADDE server logging in McIDAS

6) tidy up the Bourne shell scipts in ~ldm/decoders

7) tidy up the entries in ~ldm/etc/ldmd.conf and ~ldm/etc/pqact.conf

8) stop and restart the LDM (at the appropriate times)

9) update your McIDAS distribution with some new source code and
   ancillary data files.  This step was not needed to get you going
   at all.  I did it so you could look at the NEXRCOMP NEXRAD composite
   images

Almost all of the steps above are what is presented in the online
McIDAS documentation, or, at least, it is what I am trying to present.

I think that your setup went "south" for two simple reasons:

o an errant 'path' definition in ~mcidas/.cshrc

o an editing mistake in ~mcidas/data/LOCAL.NAM (/data/ldm/xcd instead of
  /var/data/ldm/xcd)

Please let me know if you have any problems.  In particular, please
let me know what happens when you try to start the McIDAS MCGUI
interface:

<login as 'mcidas'>
<set your DISPLAY environmet variable>
mcidas

If you are setting up any user accounts, you may well have to take a hard
look at their ~/.cshrc file if it sets 'path' like what was done in the
'mcidas' .cshrc file.

I want to get this off to you so you can take a look...

Tom

Date:    Sun, 17 Mar 2002 16:40:28 MST
To:      Tom Yoksas <address@hidden>
From:    "Dan A. Dansereau" <address@hidden>
Subject: RE: 20020317: McIDAS-X/XCD installation and configuration on DEC OSF/1 
5.1

Tom

Thanks much - I took a quick glance at your notes - the only major 
error was that I had chaged the drive /var/data... paths just before
I turned it over to you - and had not updated the NAM file - sorry.

I will go thru everything - and give a wrap up later today/or in the AM

again - thanks much - you do GOOD WORK!!!!!

Dan