[GEMPAK #VFW-525515]: A bit more about why the decoders might not be working....

I'm going to suggest the sym link $OS_BIN to ~ldm/decoders be removes, and copy 
(as user ldm) all dc programs to the directory ~ldm/decoders/ "cp $OS_BIN/dc* 
~ldm/decoders/.

Then rehash or re-login and check with "which dcgrib2" and test your scripts 
again.  Might simply be a permission issue with user gempak owning the programs.

And like you said, I'm also worried about the double ldm/data/data/ - send me 
back the current list of "env" from both gempak and ldm users, there's a 
conflict somewhere and I'm still not sure where.

-Michael

> Michael,
> 
> If I cd to ~ldm/decoders and run a script it seems to start
> 
> If I run the script as decoders/acprof,  I get errors....    See  >>>>>
> 
> [ldm@awipserver decoders]$ acprof
> [IP -2]
> [IP -10] ACPROF
> Creating process: gplt for queue 294914
> [LV -3]
> 
> 
> ^C out of there....
> 
> [ldm@awipserver decoders]$ cd ~
> [ldm@awipserver ~]$ decoders/acprof
> [IP -2]
> [IP -10] ACPROF
> [GEMPLT -101]
> [ACPROF -3]
> Error in message send = 22
> itype, ichan, nwords,0,294914,3
> [ldm@awipserver ~]$
> 
> Maybe it won't run through the symbolic link?
> 
> Ray
> On Wed, Dec 5, 2012 at 5:13 PM, Unidata GEMPAK Support
> <address@hidden> wrote:
> > As user ldm, confirm on login that the decoders are in your path: "which 
> > dcgrib2" should point to ~/decoders/dcgrib2 or $OS_BIN/dcgrib2
> >
> >
> >> Michael,
> >>
> >> Ok still did not get there......  I made a symbolic link to the
> >> directory pointed to by gempak $OS_BIN.  ldm tries to run them out of
> >> the gempak directory.
> >>
> >> decoders -> /home/gempak/GEMPAK6.7.0/os/linux64/bin/
> >>
> >> All the dc files are permission 755  rwxr-xr-x  so ldm should be able
> >> to run them.
> >>
> >> But if you look at the last ldmd.log  (attached) it says the decoders
> >> don't run.   Barring moving them directly to the ldm/decoders
> >> directory, see anything else?
> >>
> >> All the decoder calls are preceded by a "No such file or directory"
> >> but without going into the scripts for decoders/dcgrib2 for example, I
> >> don't know what directory it is looking for.
> >>
> >> Ray
> >>
> >>
> >> On Wed, Dec 5, 2012 at 3:01 PM, Unidata GEMPAK Support
> >> <address@hidden> wrote:
> >> > $LDMHOME for the gempak user must be able to read where the ldm is 
> >> > filing and decoding data.  This appears to be setup correctly on your 
> >> > system with the only change needed the directory ~ldm/decoders.
> >> >
> >> > However I don't see GEMDATA in 'env' for the gempak user, though from 
> >> > $HDS and other data environmental variables, it appears to be set.
> >> >
> >> > -Michael
> >> >
> >> >> Ok,
> >> >>
> >> >> I must have missed that instruction in the cookbook.  I'll make that 
> >> >> fix .
> >> >>
> >> >> The /usr/local/ldm/data   symbolic link then SHOULD (?) be to
> >> >> /mnt/sdb/ldm/var/data  ????
> >> >>
> >> >> Also ldm and gempak are separate users so why should ldm know about
> >> >> the gempak environment variables?  Should they be separate or in a
> >> >> group?  Could it be that gempak didn't write the decoders because it
> >> >> didn't have permissions in the ldm group?
> >> >>
> >> >> Thanks, I'll try these fixes later today and let you know the outcome .
> >> >>
> >> >> Ray
> >> >>
> >> >> On Wed, Dec 5, 2012 at 2:44 PM, Unidata GEMPAK Support
> >> >> <address@hidden> wrote:
> >> >> > I don't see a directory ~ldm/decoders to contain the dc* programs 
> >> >> > from $OS_BIN.  /usr/local/ldm/decoders is in the ldm user's $PATH, so 
> >> >> > it's looking there for decoders but not finding them.  This explains 
> >> >> > why some raw data were written - these pattern actions are using a 
> >> >> > FILE command rather than PIPE to a decoder.
> >> >> >
> >> >> > The solution is to copy all dc* programs from $OS_BIN to 
> >> >> > ~ldm/decoders, or to symlink the ~ldm/decoders directory to $OS_BIN.
> >> >> >
> >> >> > Michael
> >> >> >
> >> >> >> Michael,
> >> >> >>
> >> >> >> I took all the filters out of pqact.conf just so I could be sure that
> >> >> >> the only calls I was making was to a GEMPAK needed file.   I actually
> >> >> >> had one IDS call open yesterday..... bad form I guess, but that 
> >> >> >> didn't
> >> >> >> stop pqact.gempak from running.
> >> >> >>
> >> >> >> Here is the LDM environment:
> >> >> >>
> >> >> >> [ldm@awipserver ~]$ env
> >> >> >> MANPATH=/usr/local/ldm:man:/usr/man
> >> >> >> HOSTNAME=awipserver.physics.umbc.edu
> >> >> >> TERM=xterm-color
> >> >> >> SHELL=/bin/bash
> >> >> >> HISTSIZE=1000
> >> >> >> SSH_CLIENT=130.85.163.143 49658 22
> >> >> >> SSH_TTY=/dev/pts/2
> >> >> >> USER=ldm
> >> >> >> LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
> >> >> >> MAIL=/var/spool/mail/ldm
> >> >> >> PATH=/usr/local/ldm/decoders:/usr/local/ldm/util:/usr/local/ldm:bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/ldm/bin
> >> >> >> INPUTRC=/etc/inputrc
> >> >> >> PWD=/usr/local/ldm
> >> >> >> LANG=en_US.UTF-8
> >> >> >> SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
> >> >> >> SHLVL=1
> >> >> >> HOME=/usr/local/ldm
> >> >> >> LOGNAME=ldm
> >> >> >> LDMHOME=/usr/local/ldm
> >> >> >> CVS_RSH=ssh
> >> >> >> SSH_CONNECTION=130.85.163.143 49658 130.85.163.219 22
> >> >> >> LESSOPEN=|/usr/bin/lesspipe.sh %s
> >> >> >> G_BROKEN_FILENAMES=1
> >> >> >> _=/bin/env
> >> >> >> [ldm@awipserver ~]$
> >> >> >>
> >> >> >>
> >> >> >> And, no, "does /usr/local/ldm/data point to /mnt/sbd/var/ldm?"   No,
> >> >> >> it is  /usr/local/ldm/data > /usr/local/ldm/var/data ..... It is 
> >> >> >> using
> >> >> >> the var as a symbolic link and that should be changed so it 
> >> >> >> explicitly
> >> >> >> points to the right level on /mnt/sdb....
> >> >> >>
> >> >> >>
> >> >> >> So the symbolic link should be
> >> >> >>
> >> >> >> ln -s /mnt/sdb/ldm/var/data       data
> >> >> >>
> >> >> >> for the data symbolic link?
> >> >> >>
> >> >> >> the parallel structure on /mnt/sdb should be?????
> >> >> >>
> >> >> >> ldm
> >> >> >> var
> >> >> >> data    logs   queues?
> >> >> >>
> >> >> >>
> >> >> >> Here is the directory for /usr/local/ldm:
> >> >> >>
> >> >> >> lrwxrwxrwx 1 ldm  ldm    11 Nov 30 13:47 bin -> runtime/bin
> >> >> >> -rw-rw-r-- 1 ldm  ldm   214 Dec  3 14:22 crontab
> >> >> >> lrwxrwxrwx 1 ldm  ldm    23 Nov 30 13:47 data -> 
> >> >> >> /usr/local/ldm/var/data
> >> >> >> drwxr-xr-x 2 ldm  ldm  4096 Nov 30 13:26 Desktop
> >> >> >> drwxrwxr-x 2 ldm  ldm  4096 Dec  5 09:42 etc
> >> >> >> lrwxrwxrwx 1 ldm  ldm    15 Nov 30 13:47 include -> runtime/include
> >> >> >> drwxr-xr-x 7 ldm  ldm  4096 Nov 30 13:46 ldm-6.11.1
> >> >> >> lrwxrwxrwx 1 ldm  ldm    11 Nov 30 13:47 lib -> runtime/lib
> >> >> >> lrwxrwxrwx 1 ldm  ldm    23 Nov 30 13:47 logs -> 
> >> >> >> /usr/local/ldm/var/logs
> >> >> >> lrwxrwxrwx 1 ldm  ldm    10 Dec  3 13:33 runtime -> ldm-6.11.1
> >> >> >> lrwxrwxrwx 1 ldm  ldm    13 Nov 30 13:47 share -> runtime/share
> >> >> >> lrwxrwxrwx 1 ldm  ldm    11 Nov 30 13:47 src -> runtime/src
> >> >> >> lrwxrwxrwx 1 root root   12 Dec  3 13:45 var -> /mnt/sdb/var
> >> >> >> -rw-rw-r-- 1 ldm  ldm    25 Dec  5 09:37 watch.txt
> >> >> >>
> >> >> >>
> >> >> >> Ray
> >> >> >>
> >> >> >>
> >> >> >> On Wed, Dec 5, 2012 at 2:20 PM, Unidata GEMPAK Support
> >> >> >> <address@hidden> wrote:
> >> >> >> > Hi Raymond,
> >> >> >> >
> >> >> >> > Thanks for attaching the logs and environmental variable files.
> >> >> >> >
> >> >> >> > The first thing noticed in ldmd.log is
> >> >> >> >
> >> >> >> > Dec 05 14:35:36 awipserver pqact[19304] NOTE: Configuration-file 
> >> >> >> > "/usr/local/ldm/etc/pqact.conf" has no entries. You should 
> >> >> >> > probably not start this program instead.
> >> >> >> >
> >> >> >> > Which suggests the pqact.conf isn't being taken correctly.  Even 
> >> >> >> > so, there are decoder entries in the log file such as:
> >> >> >> >
> >> >> >> > Dec 05 14:35:36 awipserver pqact[19308] ERROR: [filel.c:1404] 
> >> >> >> > Couldn't execute decoder "decoders/dcnldn"
> >> >> >> >
> >> >> >> > So I'm not entirely sure what the problem is, but I can guess that 
> >> >> >> > it's due to difference between data directories in the gempak and 
> >> >> >> > ldm users' environment.
> >> >> >> >
> >> >> >> > What is 'env' os the user ldm, and does /usr/local/ldm/data point 
> >> >> >> > to /mnt/sbd/var/ldm?
> >> >> >> >
> >> >> >> >
> >> >> >> > -Michael
> >> >> >> >
> >> >> >> >> Michael,
> >> >> >> >>
> >> >> >> >> Ok, I have LDM and GEMPAK running but not much is getting decoded 
> >> >> >> >> by
> >> >> >> >> pqact.gempak.   I suspect it is due to an incorrect directory
> >> >> >> >> structure in the LDM directory.
> >> >> >> >>
> >> >> >> >> What I've done:
> >> >> >> >>
> >> >> >> >> 1) I have ldm and gempak as users on awipsserver.physics.umbc.edu.
> >> >> >> >> ldm's home directory is /usr/local/ldm  (as suggested in the ldm
> >> >> >> >> documentation).  gempak's home directory is /home/gempak  (with 
> >> >> >> >> other
> >> >> >> >> users)
> >> >> >> >>
> >> >> >> >> 2)  If I run pqact.conf that comes with ldm, I can pull feeds (see
> >> >> >> >> watch.txt attached) from idd.meteo.psu.edu and it looks like the
> >> >> >> >> UNIDATA feed (and the UNIWISC) feeds work.
> >> >> >> >>
> >> >> >> >> 3)  When I built ldm, I did not want the /var directory on my main
> >> >> >> >> hard drive on the server, but I put it on a raid  (/mnt/sdb/var) 
> >> >> >> >> and I
> >> >> >> >> symbolically linked /var and /data in ldm to point to 
> >> >> >> >> /mnt/sdb/var and
> >> >> >> >> /mnt/sdb/var/data).   That worked ok and the ldm data was pushed 
> >> >> >> >> to
> >> >> >> >> /mnt/sdb/var/data/surface/US.... for the demo pqact.conf feeds 
> >> >> >> >> from
> >> >> >> >> IDS.
> >> >> >> >>
> >> >> >> >> 4) In GEMPAK, all built out ok (except for the /gf bug that I sent
> >> >> >> >> previously) and GEMPAK runs fine.   I attach the env for gempak.  
> >> >> >> >>  I
> >> >> >> >> think the issue is here.   I left $GEMDATA as it was in 
> >> >> >> >> Gemenviron as
> >> >> >> >> it was built by the csh routine in GEMPAK.
> >> >> >> >>
> >> >> >> >> setenv GEMDATA       /data/ldm/gempak
> >> >> >> >>
> >> >> >> >> That seemed strange to me but it started putting the gempak data 
> >> >> >> >> in
> >> >> >> >> /usr/local/ldm/data/data/images/    etc....
> >> >> >> >>
> >> >> >> >> I only got /images   /jason  and  /nwx  built in that directory .
> >> >> >> >> /surface doesn't get decoded.    From the gempak_env, you can see 
> >> >> >> >> that
> >> >> >> >> it set up the /usr/local/ldm/data/data/gempak structure.
> >> >> >> >>
> >> >> >> >> I attach a zipped version of the ldmd.log file with the errors.  
> >> >> >> >> It
> >> >> >> >> looks like this is not feed related but the decoders are not all
> >> >> >> >> working.
> >> >> >> >>
> >> >> >> >> 5) To make GEMPAK read from the right files,  I had to change the
> >> >> >> >> above Gemenviron variable to be
> >> >> >> >>
> >> >> >> >> setenv GEMDATA      /usr/local/ldm/data/data/gempak
> >> >> >> >>
> >> >> >> >> and then GEMPAK could go and get the data that was available 
> >> >> >> >> (images
> >> >> >> >> mostly.... the GOES data is coming in ok.)
> >> >> >> >>
> >> >> >> >> Suggestions on how to fix the Gemenviron file (also attached)?
> >> >> >> >>
> >> >> >> >> Ray
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Raymond M. Hoff
> >> >> >> >> Professor of Physics
> >> >> >> >> Rm 426, Physics Dept.
> >> >> >> >> University of Maryland, Baltimore County
> >> >> >> >> 1000 Hilltop Circle
> >> >> >> >> Baltimore MD 21250
> >> >> >> >> p: 410-455-1943 f:410-455-1042
> >> >> >> >> e: address@hidden
> >> >> >> >> physics.umbc.edu/~hoff
> >> >> >> >>
> >> >> >> >>
> >> >> >> >
> >> >> >> > Ticket Details
> >> >> >> > ===================
> >> >> >> > Ticket ID: XDH-337278
> >> >> >> > Department: Support GEMPAK
> >> >> >> > Priority: Normal
> >> >> >> > Status: Open
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Raymond M. Hoff
> >> >> >> Professor of Physics
> >> >> >> Rm 426, Physics Dept.
> >> >> >> University of Maryland, Baltimore County
> >> >> >> 1000 Hilltop Circle
> >> >> >> Baltimore MD 21250
> >> >> >> p: 410-455-1943 f:410-455-1042
> >> >> >> e: address@hidden
> >> >> >> physics.umbc.edu/~hoff
> >> >> >>
> >> >> >>
> >> >> >
> >> >> >
> >> >> > Ticket Details
> >> >> > ===================
> >> >> > Ticket ID: XDH-337278
> >> >> > Department: Support GEMPAK
> >> >> > Priority: Normal
> >> >> > Status: Open
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Raymond M. Hoff
> >> >> Professor of Physics
> >> >> Rm 426, Physics Dept.
> >> >> University of Maryland, Baltimore County
> >> >> 1000 Hilltop Circle
> >> >> Baltimore MD 21250
> >> >> p: 410-455-1943 f:410-455-1042
> >> >> e: address@hidden
> >> >> physics.umbc.edu/~hoff
> >> >>
> >> >>
> >> >
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: XDH-337278
> >> > Department: Support GEMPAK
> >> > Priority: Normal
> >> > Status: Open
> >> >
> >>
> >>
> >>
> >> --
> >> Raymond M. Hoff
> >> Professor of Physics
> >> Rm 426, Physics Dept.
> >> University of Maryland, Baltimore County
> >> 1000 Hilltop Circle
> >> Baltimore MD 21250
> >> p: 410-455-1943 f:410-455-1042
> >> e: address@hidden
> >> physics.umbc.edu/~hoff
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: XDH-337278
> > Department: Support GEMPAK
> > Priority: Normal
> > Status: Open
> >
> 
> 
> 
> --
> Raymond M. Hoff
> Professor of Physics
> Rm 426, Physics Dept.
> University of Maryland, Baltimore County
> 1000 Hilltop Circle
> Baltimore MD 21250
> p: 410-455-1943 f:410-455-1042
> e: address@hidden
> physics.umbc.edu/~hoff
> 
> 


Ticket Details
===================
Ticket ID: VFW-525515
Department: Support GEMPAK
Priority: Normal
Status: Open