19990915: LDM: Installation Problems

>From: Michael Keables <address@hidden>
>Organization: DU
>Keywords: 199909152035.OAA16060 LDM ldm-mcidas

Mike,

I saw your email and decided to answer since Robb was taking some
vacation time today.

>I'm trying to install the LDM on a Sun Ultra (Solaris 7) aka
>cyclone.natnet.du.edu. I've gone through the checklists on the web but am
>still unable to get the LDM to run properly.
>
>When I issue ldmadmin start & a process kicks off but nothing happens.

OK, when this happens, you should follow the recommendations at the
top of the file ~ldm/etc/ldmd.conf:

####
#
# This is the main configuration file for the LDM server. All lines that start
# with a "#" sign are comments.
#
# To debug an LDM that hangs on start up, run the following from LDM home:
# % bin/rpc.ldmd -vl - -q data/ldm.pq etc/ldmd.conf
#
# If the LDM still hangs, comment out all lines in this file, try again.

In doing so, you could have found out that things would work after you
commented out the following line from ldmd.conf:

#exec   "pqsurf"

The reason that this line needed to be commented out was that the
queue that pqsurf needs did not exist.  The great majority of sites
do not use pqsurf anymore, so I left this line commented out.

This was not all that was needed, however.  I also found that you had
two entries for NDLN data.  This was a typo from somewhere else since
the datasteam is NLDN (D and L transposed).  Also, the problem with doing:

request NLDN    ".*" cirrus.al.noaa.gov

is that the NLDN (lightning) data is not one of the ones that gets
relayed from upstream sites.  All lightning data is sent point-to-point
from SUNY Albany.  If you havn't already done so, you must request
(email) a feed of the NLDN data from:

                Name  David Knight (p)
         Institution  State Univ. of NY-Albany
          Department  Earth and Atmospheric Sciences
   Street Address #1  1400 Washington Ave.
   Street Address #2  Earth Sci. Bldg. ES 228
City, State, Zip, Co  Albany  NY 12222 
              Phone:  518 442-4204  Fax: 518 442-4494
       Email Address  address@hidden


Dave will enable your machine to receive the data.  Once he has done
this, you will need to change the request NLDN line from what it is in
~ldm/etc/ldmd.conf to:

request NLDN    ".*" striker.atmos.albany.edu

For convenience, I did this for you but left the line commented out:

#request NLDN    ".*" striker.atmos.albany.edu

When Dave has setup stuff up, all you need to do is uncomment the line
and stop and restart the LDM.

>After a few minutes I get the following emai: 
>
>Sep 15 19:54:40 UTC cyclone.natnet.du.edu : start_ldm: Server not started
>or registered.
>
>A ps -eaf | grep ldm yields: 
>
>cyclone% ps -ef | grep ldm
>     ldm 22126 22125  0 14:00:00 ?        0:00 /usr/local/bin/perl
>bin/ldmfail -p cirrus.al.noaa.gov -f cirrus.al.noaa.gov
>     ldm 22125   176  0 14:00:00 ?        0:00 sh -c bin/ldmfail -p
>cirrus.al.noaa.gov -f cirrus.al.noaa.gov
>     ldm 22135 22126  0 14:00:00 ?        0:00 /usr/local/bin/perl
>/usr/local/ldm/bin/ldmadmin start
>     ldm 22265 22259  0 14:07:16 pts/4    0:00 -csh
>
>Issuing notifyme shows that I have access to cirrus.al.noaa.gov (albeit I
>don't have a failover host yet so ldmd.primary and ldmd.failover are the
>same as ldmd.conf.)

It is great that you used notifyme to see if you have access to your
upstream feed site.  This was the exact right thing to try!  It is
also great that you used 'ldmadmin pqactcheck' to check the pqact.conf
file entries.  See below.

>I have deduced the following:
>
>1. there is a problem with the following statement in pqact.conf (which I
>downloaded from the web):
>
>cyclone% ldmadmin pqactcheck
>Sep 15 20:25:57 pqact[22391]: feedtype error at line 14: unknown feed name
>in feedtype expression: "MCIDAS  ^(LWTOA3 .*)"

Right.  I looked at your pqact.conf file and found that all of the
entries had spaces where tabs were called for.  The most likely cause
for this was someone cutting and pasting example pqact.conf entries
into the file; true?  I edited pqact.conf and changed all of the spaces
to tabs where needed.  I then ran 'ldmadmin pqactcheck' to discover
other lines that had problems. There were three lines that are very
long had line breaks in them.  They looked like:

#WSI    ^NEX/(...)/(BREF1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0
-6][0-9])

This entry was coming out as two lines instead of one.  This was also
probably due to cutting and pasting.

>2. ldm/decoders is empty ... I assumed that I needed to download the
>decoders from the web but I get a permissions violation when I try to
>download decoders.tar.Z

The other things that were missing from ~ldm/decoders were the ldm-mcidas
decoders.  Binary versions of these can be FTPed from ftp.unidata.ucar.edu
from the pub/binary/sunos_5.7-sparc directory.  I did this for you:

cd ~ldm
ftp ftp.unidata.ucar.edu
 <anonymous>
 <your email address>
 cd pub/binary/sunos_5.7-sparc
 bin
 get ldm-mcidas.tar.Z
 quit
zcat ldm-mcidas-tar.Z | tar xvf -

I then copied the ldm-mcidas decoders to the ~ldm/decoders directory:

cp ldm-mcidas-7.6.1/bin/* decoders

After that, I went into the decoders directory and setup the McIDAS
ROUTE PostProcessing script, batch.k, to match your setup.  This
script allows such things as composite images to be produced.  The
editing job consisted of nothing more than changing the definition
of MCHOME from /home/mcidas to /export/home/mcidas.  What remains
to be done is to enable the compositing by running ROUTE from
a McIDAS-X session running as the user 'mcidas'.  I didn't do this
for you since we need to talk about where you have setup data
file storage.  More on this below.

While in the decoders directory, I copied the file xcd_run from the
McIDAS distribution:

cp ~mcidas/mcidas7.6/src/xcd_run .

I then edited xcd_run (another Bourne shell script) and changed MCHOME
in the same way that was done for batch.k

Finally, since the FSL2 wind profiler decoder needs the McIDAS SCHEMA
file in the directory in which the decoder wants to create output
MD files, I copied it there.  I also copied ROUTE.SYS and SYSKEY.TAB
since they will be used/updated by the ldm-mcidas and XCD decoders:

cd /var/data/mcidas
cp ~mcidas/data/SCHEMA .
cp ~mcidas/data/SYSKEY.TAB .
cp ~mcidas/workdata/ROUTE.SYS .

After that, I was ready to start the LDM:

ldmadmin start
ldmadmin tail

The LDM started up and began receiving data from the upstream feed site.
Here is the contents of the /var/data/mcidas directory as I write this:

cyclone% cd /var/data/mcidas
cyclone% ls
AREA0060    AREA0140    AREA0170    AREA0205    ROUTE.SYS
AREA0120    AREA0150    AREA0191    AREA0210    SCHEMA
AREA0130    AREA0160    AREA0200    MDXX0099    SYSKEY.TAB

You can see that a number of images have already been received and decoded
and that one MD file has been created.  That MD file, 99, was created
from the FSL2 6-minute profiler data.

>Please advise on how to get out of the mess I'm in.

OK, now we need to talk about where the data are currently going:
/var/data/mcidas.  I did a quick check of disk space on your machine:

cyclone% df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/proc                      0       0       0     0%    /proc
/dev/dsk/c0t0d0s0      96455   40768   46042    47%    /
/dev/dsk/c0t0d0s6     877790  665487  150858    82%    /usr
fd                         0       0       0     0%    /dev/fd
/dev/dsk/c0t0d0s1     413639   77771  294505    21%    /var
/dev/dsk/c0t1d0s7    8509324  339070 8085161     5%    /export/home
/dev/dsk/c0t0d0s5    3007086  917816 2029129    32%    /opt
/dev/dsk/c0t0d0s7    4031022  140916 3849796     4%    /usr/local
swap                  657744     112  657632     1%    /tmp

and see that /var is very small (total of 400 MB to begin with).  This
will not be enough room to store McIDAS or GEMPAK data in.  The most
likely candidate for data storage is /export/home since it has
8 GB of disk available.  I ordinarily don't recommend that data files
be kept in /home (or /export/home in your case), but it would be painful
to go back and repartition your disk.  Is there more disk in the system
that hasn't been mounted?  If so, and if it is on the order of at
least 2-3 GB, then that is where the data should go.

In the meantime, in order to exercise the LDM and ldm-mcidas decoders,
I left things running.

On the McIDAS side of things, I see that you have not yet setup XCD.
Once you have done this (I would have done so for you, but there
is not enough disk space in /var/data), turning on XCD decoding
will be as simple as editing ~ldm/etc/pqact.conf and uncommenting
out the lines for XCD processing:

# Entries for XCD decoders
#DDPLUS|IDS     ^.*     PIPE
#       xcd_run DDS
#HRS     ^.*    PIPE
#       xcd_run HRS

(remove the '#' signs at the beginning of the lines while making sure
to NOT turn them into spaces).

Let's touch base on your setup tomorrow.

>Thanks in advance.

You are welcome.

Tom