20001012: Post-enhancement PostProcessing Problems

>From: Joseph Smith <address@hidden>
>Organization: Oregon State University
>Keywords: 200010121833.e9CIXu404250 McIDAS-X LDM ROUTE PostProcess

Joe,

>As Wayne Gibson previously mentioned, we successfully compiled LDM with
>the new logging enhancements, and LDM is working fine.

Right, I saw that interchange.

>However, there has
>been an unforeseen change in the operation of the McIDAS PostProcessing
>facility.  Scripts that were being successfully executed from PP BATCH
>files via the OS command were no longer functioning correctly.
>Specifically, *.k commands inside mcenv blocks were no longer being found.

This would tell me that something in the user 'ldm's environment changed.

>I set up PP BATCH files to run NORTMAPR commands along with OS commands to
>run the shell scripts that were originally being run from the crontab.  I
>removed the PATH/MCPATH statements from these scripts as they were no
>longer necessary, this info being passed directly to the scripts from the
>ldm batch.k file.  Inside the scripts are shell commands to build
>filenames and move the image files, and two mcenv blocks to create images
>of the desired size:
>   mcenv -g 16 -i 238 -f 750x872 << 'EOF', & 
>   mcenv -g 16 -i 238 -f 375x436 << 'EOF'.

OK.

>This arrangement was working fine, creating the topo composite AREA files
>& satellite images, and doing the file manipulation.  However, after Wayne
>and I recompiled and installed ldm with the ulog enhancement, this setup
>quit working.

I have to believe that something changed in the environment.  The
functionality of the LDM to kick off processes from pqact.conf did not
change, and the issuing of ROUTE PostProcess is still controlled by the
decoder reading the ROUTE.SYS file in the output data directory.

>The NORTMAPR & OS commands were still being executed, as
>were the shell commands in the scripts.  However, the commands in the
>mcenv blocks were not being executed, hence no images are created.  
>Specifically, I was getting:
>   imgdisp.k: Command not found.
>   map.k: Command not found.
>   frmlabel.k: Command not found. (etc.)
>error messages in ROUTEPP.LOG.

I think that what is happening is that the PATH being set in batch.k
is not being passed into the mcenv block.  This is caused by .cshrc
being sourced when mcenv is run.

There are at least threee workarounds to this problem:

o remove the "'" from EOF in the mcenv invocation:

  mcenv -g 16 -i 238 -f 750x872 << EOF

o edit the .cshrc file for the user 'ldm' and put in a construct that
  accepts PATH definitions if the exist before setting additional paths

  Consider the following for a modification in .cshrc:

if ( ! ${?PATH} ) then

   set PATH=$PATH:...

else

   set PATH=...

endif

  (The '...' is the normal PATH set that you already have).

  The change is to not redefine PATH if it is already defined, but to
  define it if it is not already defined.

o pass the appropriate PATH settings into the mcenv block through
  a variable definition outside of the mcenv block

  Something like this might work in batch.k:

MYPATH=/home/mcidas/bin:/usr/local/bin:...

MYPATH=$MYPATH MCPATH=$MCPATH PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH mcenv 
<< 'EOF'

PATH=${MYPATH}:$PATH
imgdisp.k ...

EOF

>I printed out the environment from ~decoders/bin/batch.k, OCS_VIS.BAT (the
>PP BATCH file) and ocs_vis.sh (the shell script executed by OS in
>OCS_VIS.BAT).  All seem to have the correct PATHs and MCPATHs to run the
>*.k McIDAS files, but they apparently could not be found.  Example:
>[snip]
>HOME=/home/ocs/ldm
>HOST=ocs
>HOSTTYPE=sun4
>LANG=C
>LDMHOSTNAME=ocs.OCE.ORST.EDU
>LDMPQFNAME=/home/ocs/ldm/data/ldm.pq
>LD_LIBRARY_PATH=.:/usr/openwin/lib:/usr/lang/lib:/usr/lib/:/usr/local/lib:/usr
> /ucblib:/usr/lib/X11:/usr/lib/X
>11:/opt/SUNWspro/SC5.0/lib:/home/mcidas/lib
>LOGNAME=ldm
>MACHTYPE=sparc
>MAIL=/var/mail/ldm
>MANPATH=/usr/dt/man:/usr/man:/usr/openwin/share/man
>MCENV_NEGUC=4
>MCENV_POSUC=245000
>MCPATH=/home/ocs/mcidas/workdata:/home/ocs/mcidas/data:/home/ocs/mcidas/help:/
> home/ocs/ldm/.mctmp/245000
>OPENWINHOME=/usr/openwin
>OSTYPE=sunos4
>PATH=/home/ocs/mcidas/bin:/opt/bin:/home/ocs/ldm:/home/ocs/ldm/bin:/usr/local/
> bin.solaris2x:/usr/local/bin:/h
>ome/server/local/bin.scripts:/home/server/local/bin.solaris2x:/usr/openwin/bin
> :/usr/openwin/bin/xview:/usr/op
>enwin/demo:/usr/ucb:/usr/bin:/bin:/usr/new:/usr/sbin:/usr/ccs/bin:/etc:/usr/et
> c:/usr/local/etc:/home/server/l
>ocal/etc.solaris2x:/usr/games:/opt/SUNWspro/bin:.
>PWD=/home/ocs/ldm/runtime/bin
>SESSION_SVR=ocs
>SHELL=/usr/local/bin/tcsh
>ULOG_FACILITY_OVERRIDE=1
>[snip]
>from ocs_vis.sh.
>
>I noticed the MCENV_NEGUC & MCENV_POSUC variables and wondered if the
>mcenv blocks might be conflicting with them, so I commented out the mcenv
>lines and images began being created, albeit the wrong size.  I changed
>the OS command from
>OS "/home/ocs/mcidas/scripts/ocs_vis.sh
>to 
>OS "mcenv -g 16 -i 238 -f 1@750x872 /home/ocs/mcidas/scripts/ocs_vis.sh
>and the correct size images were created.
>
>I know I can split my scripts and pass the mcenv environment to each one
>seperately, but the underlying question remains:  Why did the
>functionality of the scripts run from the PP BATCH facility change after
>ldm was recompiled to use the new ulog override?

I don't know why the functionality changed.  Until assured otherwise, I
have to assume that a change was made to the PATH declaration in 'ldm's
.cshrc file.

>And for a new twist, I removed the link in ~ldm/bin to
>~ldm/decoders/bin/batch.k that I created to get PostProcessing to work in
>the first place, and the PostProcessing worked fine without it.  Looks
>like the recompilation affected this as well.

Since things continue to work without the link, doesn't it tell us that
PATH in .cshrc was changed?

>Any insights?

Let me know if the above does not make sense to you.  If it doesn't,
then inclusion of the set of 'batch.k', the PP BATCH file, and the
script run from the PP BATCH file would help in troubleshooting.

>Thanks for your help,
>Joseph

Tom Yoksas