20021120: Corrupt ROUTE.SYS and ROUTE PostProcess BATCH files (cont.)

>From: Angel Li <address@hidden>
>Organization: U Miami/RSMAS
>Keywords: 200211150406.gAF46PL16731 McIDAS ROUTE.SYS

Angel,

Given:

re: would you be against combining the contents of the /ldm/data/mcidasd
and /ldm/mcidas/data directories

>No reason they are separate.

re: would you be against my changing the setup that runs the decoding and
BATCH files to use setups in the 'mcidas' account instead
of using setups in the 'ldm' account

>Nope

I made several changes to the LDM setup on sapodilla this morning:

<as 'ldm'>

o First thing, I stopped the LDM:

  cd ~ldm
  ldmadmin stop

o I edited ~ldm/.cshrc and commented out McIDAS environment variable definition
  lines.  This now looks like:

if ($?prompt) then
    setenv EXINIT "set ai aw ic sw=4 redraw nows terse|map g G|map v ~~~~"
    alias ls 'ls -FC'
    set prompt="ldm% "

#    setenv MCDATA /ldm/mcidas/data
#    setenv MCPATH "${MCDATA}:/mcidas/data:/mcidas/help:/ldm/data/mcidasd"
#    setenv MCGUI "/mcidas/bin"
#    set path = (. ~/decoders ~/bin $MCGUI /usr/local/bin /usr/bin /usr/bin/X11
/usr/sbin)
    set path = (. ~/decoders ~/bin /usr/local/bin /usr/bin /usr/bin/X11 /usr/sbi
n)
endif

  I would have removed the lines, but I didn't want to do so until
  you had a chance to take a look.

o I edited batch.k, xcd_run, and mcscour.sh in ~ldm/decoders to change
  the definitions of MCDATA, MCPATH, and MCLOG.  The are now:

# Define needed macros
MCHOME=/mcidas
MCDATA=$MCHOME/workdata
MCGUI=$MCHOME/bin
MCLOG=/ldm/logs/ROUTEPP.LOG

  This more-or-less matches the recommended setup in the McIDAS
  web pages at:  http://www.unidata.ucar.edu/packages/mcidas/2002.

  I changed these files to force log files to be written into ~ldm/logs.
  I also updated the REDIRECTion table in the 'mcidas' account so
  that ROUTEPP.LOG can be found so that it can be deleted by mcscour.sh
  when run from 'ldm's cron.

o I removed all of the XCD created data files, decoder status files, etc.
  from ~ldm/mcidas/data.  The directory is now almost empty:

ldm% cd ~/mcidas/data
ldm% ls
ALLOC.WWW      GOES8IR.WHR    LWPATH.NAM     RESOLV.SRV
GCMPVAR        GOES8VIS.WHR   MCTABLE.TXT

  The LWPATH.NAM, MCTABLE.TXT, and RESOLV.SRV files have definitions
  in them that pertain to the data that you are IMGCOPYing in the
  scripts that are being run from 'ldm's cron.  These definitions
  will need to be merged into corresponding ones in the 'mcidas' account.

  I don't know what GCMPVAR, GOES8IR.WHR, and GOES8VIS.WHR are.

  ALLOC.WWW is created each time a McIDAS process is run.  No need to
  worry about it.

o I noticed that the ldm-mcidas decoders had been copied to the ~ldm/bin
  directory.  Since this directory is a link to the latest LDM release
  bin directory, it is a good idea to put files that don't chante with
  an LDM distribution (like ldm-mcidas decoders) in a different directory.
  given this, I moved all of the ldm-mcidas decoders from ~ldm/bin to
  ~ldm/decoders.  Now, you can do LDM installations without having to
  worry about copying the ldm-mcidas decoders to the newly installed
  bin directory.

o I noticed that even though the LDM was setup to ingest NLDN data, it was
  not being decoded into McIDAS-compatible MD files.  I edited
  ~ldm/etc/pqact.conf and uncommented (and modified) the lines that
  run the ldm-mcidas NLDN decoder.  This will result in MD files 71-80
  being created in the /ldm/data/mcidasd directory.  They are already
  usable by McIDAS processes since the RTPTSRC/LIGHTNING dataset was setup
  to use those files.

o While cleaning up, I deleted the mcidasx780.tar.Z file in ~ldm.
  ROUTE.SYS.orig can be deleted as it is not needed.

o ** IMPORTANT **  I note that the /etc/syslog.conf entry for LDM logging
  is incorrect:

local0.err;local0.warning;local0.notice;local0.info     /s1/ldm/logs/ldmd.log

  The directory /s1/ldm/logs doesn't exist.  The entry should read:

local0.err;local0.warning;local0.notice;local0.info     /ldm/logs/ldmd.log

  Until this fix is made and a HUP is sent to syslogd, no LDM logging will
  occur:

ldm% cd /ldm/logs
ldm% ls -alt
 ...
-rw-rw-r--   1 ldm      users          0 Nov 14 19:27 ldmd.log
-rw-r--r--   1 ldm      users        564 Nov 14 19:23 ldmd.log.1
-rw-r--r--   1 ldm      users       6968 Nov 14 14:20 ldmd.log.2
-rw-r--r--   1 ldm      users       4458 Nov 14 10:45 ldmd.log.3
-rw-r--r--   1 ldm      users      28286 Nov 14 10:11 ldmd.log.4


<as 'mcidas'>

o finished the McIDAS-X, -XCD 2002 installation.  I noticed that even
  though both -X and -XCD had been built, only -X had been installed.
  I reinstalled everything just to make sure:

  cd ~mcidas/mcidas2002/src
  make install.all

o setup the 'mcidas' account for XCD decoding:

  cd ~mcidas/data
  <edit LOCAL.NAM and change directory references to /ldm/mcidas/data to
  /ldm/data/mcidasd>

  cd ~mcidas/workdata
  redirect.k REST LOCAL.NAM

  te.k XCDDATA \"/ldm/data/mcidasd
  batch.k XCD.BAT
  batch.k XCDDEC.BAT

  The last three steps created files in the /ldm/data/mcidasd directory
  that are needed by XCD decoders.


<back as 'ldm'>

o restart the LDM:

  ldmadmin start

Right now, LDM ingestion is proceeding, and XCD decoding is working.
what is not working is the decoding of NLDN data by the ldm-mcidas
decoder nldn2md into McIDAS MD files.  I couldn't troubleshoot this
problem since the LDM entry in /etc/syslog.conf is incorrect.

Remaining:

re: would you be opposed to moving the scripts being used to IMGCOPY
imagery from SSEC to the 'mcidas' account.  Is there any reason that
these scripts are being run from the 'ldm' account and not the 'mcidas'
account?  It would be cleaner to move them to the 'mcidas' account
totally.

>That's done easily. The scripts are ~ldm/gilmore/lister and 
>~ldm/gilmore/getOne.
>
>The same ones are in ~ldm/wallops. I can do this after you're done 
>rearranging.

OK.  I am done rearranging.  There might be a couple of mopping up
issues (like /etc/syslog.conf), but things seem to be working now.

re: have the IMGCOPY scripts run as 'mcidas'

>No problem there. The scripts can run as mcidas just as easily.

These can be moved to the 'mcidas' account now.  The things that will
need to be done are the merging of ADDE dataset locations currently in
/ldm/mcidas/data/MCTABLE.TXT into either a newly created
/mcidas/workdata/MCTABLE.TXT or, better yet, into the existing
/mcidas/data/ADDESITE.TXT.  Also, the ADDE dataset definitions in
/ldm/mcidas/data/RESOLV.SRV need to be merged into the existing dataset
definition file /mcidas/workdata/RESOLV.SRV.  Lastly, a resolution of
REDIRECTions in /ldm/mcidas/data/LWPATH.NAM must be made with the
corresponding file for 'mcidas', /mcidas/workdata/LWPATH.NAM.

>Thanks Tom.

No worries.

Tom