20060109: No MD data being written to /data/mcidas

>From: John Weeks <address@hidden>
>Organization: USU
>Keywords: 200601100200.k0A20g7s021950 McIDAS-XCD LDM

Hi John,

re:
>We are not seeing any /data/mcidas/MD* files being generated
>on allegan.gis.usu.edu.   I have checked file permissions to be
>writeable by mcidas and ldm.   This problem started several
>weeks ago but I don't have a good estimate on the actual failure
>date.

After you enabled SSH from zero, I logged onto allegan and made several
changes to the LDM and McIDAS setup.  Here is a synopsis of what I
did:

<as 'ldm'>

- created a ~ldm/ldm-mcidas directory

- moved ldm-mcidas-2003 under ~ldm/ldm-mcidas

- downloaded ldm-mcidas-2004.tar.Z (binary distribution) to ~ldm/ldm-mcidas
  This was needed since the 2003 version of pnga2area had a problem that
  resulted in a damaged ROUTE.SYS file in /var/data/mcidas.  I corrected
  this by replacing ROUTE.SYS with the copy from the McIDAS distribution.
  More information below.

- unpacked ldm-mcidas-2004.tar.Z and copied appropriate decoders
  (pnga2area, nldn2md, proftomd) to ~ldm/decoders

- deleted archaic ldm-mcidas decoders from ~ldm/decoders

- edited mcidas scripts in ~ldm/decoders: xcd_run, batch.k, and mcscour.sh
  changes made include modifying LD_LIBRARY_PATH settings (this may have
  been your original problem) and forcing the scripts to run in 'sh'

- downloaded the latest LDM release, LDM-6.4.4.  I unpacked this release
  ran configure specifying the --disable-max-size flag so that large
  (> 2 GB) LDM support will not be built in.

  The LDM installation is not finished.  'root' should do the follwing:

  cd /home/ldm/ldm-6.4.4/src
  /usr/ccs/bin/make install_setuids

  After this step has been done, 'ldm' should do the following:

  cd ~ldm
  ldmadmin stop
  rm runtime
  ln -s ldm-6.4.4 runtime
  rehash
  ldmadmin start

- stopped running some programs from ~ldm/etc/ldmd.conf that are no
  longer needed (i.e., pqbinstats, and pqexpire)

- added reporting of real time stats in ~ldm/etc/ldmd.conf:

  exec  "rtstats -h rtstats.unidata.ucar.edu"

- changed the host from which you are requesting NEXRAD Level III
  images.  The host you were using (cirp.met.utah.edu) was not
  allowing your feed request.  You are now ingesting radar images
  and they are available in McIDAS.

- I notice that LDM logging is not working.  I think that this is due
  to the lack of needed configuration entries in /etc/syslog.conf.
  Since these changes need to be made by 'root', I was unable to
  implement the needed changes.  Given 'root' access (or sudo), I
  could make them; please let me know if you would like me to do this.


<as 'mcidas'>

- combined the /var/data/xcd directory with /var/data/mcidas.  I actually
  removed /var/data/xcd and its contents; I did not move the contents
  to /var/data/mcidas since I was going to recreate the various files
  anyway.

- modified ~mcidas/data/LOCAL.NAM to reflect decoding into /var/data/mcidas

- changed the STRTABLE entry for XCDDATA to reflect the fact that I
  changed McIDAS-XCD decoding to write in /var/data/mcidas:

  cd ~mcidas/workdata
  te.k XCDDATA \"/var/data/mcidas

- ran the XCD setup scripts:

  cd ~mcidas/workdata
  batch.k XCD.BAT
  batch.k XCDDEC.BAT

- copied SCHEMA, SYSKEY.TAB, and ROUTE.SYS from the McIDAS installation
  to /var/data/mcidas:

  cd ~mcidas/workdata
  cp ~mcidas/data/SCHEMA /var/data/mcidas
  cp ~mcidas/data/SYSKEY.TAB /var/data/mcidas
  cp ROUTE.SYS /var/data/mcidas

- RELeased ROUTE.SYS entries that allow for compositing of imagery:

  route.k REL C
  route.k REL N

- SUSpended ROUTE.SYS entries for images no longer broadcast in the
  UNIWISC (aka MCIDAS) IDD datastream

  route.k SUS UA UC UR

- cleaned up files that had been created in directories where they
  didn't belong:

  cd ~mcidas/workdata
  rm ~mcidas/help Frame*
  rm *.00*

  These files were probably created by scripts run to create raster
  products (e.g., GIFs, etc.), and the scripts probaby did not
  correctly and/or completely specify needed McIDAS environment
  variables (MCDATA, MCPATH, etc.)

After making the above changes and restarting the LDM, McIDAS-XCD
is decoding again.

NOTE:  The current release of McIDAS is v2005b.  If youwant me to
upgrade your McIDAS installation to this version, please do the following:

- register with the Unidata Website, http://www.unidata.ucar.edu

- after registering and logging in, proceed to the Downloads page
  and download the McIDAS v2005 distribution from the McIDAS section.
  Once the new McIDAS distribution is in the ~mcidas directory on
  allegan, I can build the executables; install the code; and update
  McIDAS-XCD decoding.

>I installed mcidas 2005 on webcat.gis.usu.edu and I am
>seeing similar results as well.

Hmm...  Do you want me to look at webcat as well?  If yes, it would
be better do do the work with the current McIDAS release.

>allegan# su - mcidas
>Sun Microsystems Inc.   SunOS 5.8       Generic February 2000
>You have mail.
>allegan% ptlist.k RTPTSRC/SFCHOURLY FORM=FILE ALL
>
>ptlist.k: No MD files found
>ptlist.k: Done

re:
>allegan% allegan% ls -ld /data /data/mcidas /data/mcidas/MD*
>drwxrwxrwx  10 ldm      ldm          512 Jan  9 10:44 /data
>drwxrwxr-x   2 ldm      ldm         6656 Jan  9 17:07 /data/mcidas
>-rw-rw-r--   1 ldm      ldm      10719232 Oct 16  2002 /data/mcidas/MD340456
>-rw-rw-rw-   1 ldm      ldm        29404 Jan  9 17:01 /data/mcidas/MDXX0079
>-rw-rw-rw-   1 ldm      ldm        31964 Jan  9 18:55 /data/mcidas/MDXX0080
>allegan% touch /data/mcidas/foo
>allegan% rm /data/mcidas/foo
>allegan%

I deleted all these MD files and some bad AREA files from /var/data/mcidas.

>Any pointers would be much appreciated.   I am taking over for the upgrades
>that Dan Dansereau used to perform and am fairly new to McIDAS but very
>familiar with Solaris.  

No worries.  Is Dan no longer at USU?  Also, I thought that he (Dan)
had setup a NOAAPORT ingest system.  If he did, you should be ingesting
much of the data you are getting from other systems from it.

>You are welcome to login as mcidas to allegan or webcat.

You sent your original message to the mcidas-x email list.  I believe
that you really meant to send it to Unidata User Support
<address@hidden>.  McIDAS-specific questions are best sent to
address@hidden.

Please let me know if you see anything else amiss and/or have questions.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.