19990119: ADDE Config Problems (cont.)

>From: Bryan Rockwood <address@hidden>
>Organization: Creighton
>Keywords: 199901140557.WAA05876 McIDAS ADDE ROUTE PP BATCH

Bryan,

re: typo in .mcenv
>Darn typos.......

>Ok, now back to the ADDE stuff.  I now have it up and running on both
>cockeyed-bob and kona.  My next question is "What is the easy way to get
>to the data?"  I can do the command line stuff, but most of the students
>want simple user interface to the analysis tools.  I read that the GUI is
>slowly making its way to using the ADDE commands to access the data.  When
>will this become a reality for the rest of the community, or do I have to
>go and edit things by hand for a full ADDE implementation.  Or, is it
>better to do NFS right now knowing that the ADDE stuff is ready for when
>things make the full switch?

I am working on an ADDE version of the GUI now.  It is a top priority
to flesh out the functionality of the GUI in areas like GRID display
and analysis.  Given that it is the beginning of a semester and this
is not ready to go, it may be wise to go the NFS route for now.

>In a related question, I can do a PTLIST for surface obs, but that is
>about it.  I figure I am typing in something wrong, because when I do a
>RAOBRPT, I get a RC=2 error.  I believe as long as PTLIST is working, I
>should be fine, though.

RAOBRPT lists TEXT upper air obs (i.e. not decoded).  PTLIST lists decoded
obs.  If RAOBRPT is not working, it is likely that SFCRPT doesn't work
either.  This means one of two things:

o you are not using XCD to create the files needed by these routines
o you have an dataset configuration error

re: starting a session with whatever sized frames you want
>Found that, and it is pretty neat, thanks.

re: LDM logging
>Sorry, I do mean logging.  The FILE action does not log to the ldmd.log
>file.  I have not had a chance to try your recommendation, but will
>probably tomorrow.

Right, the FILE action does not produce verbose messages.

re: LDM logging
>I believe you were dead on.  This leads me to my next question.  In the
>ldmd.log file, there are some errors I would like to ask about.  First off
>is the following:
>
>Jan 20 02:33:22 whistler pqact[13300]: pbuf_flush (4) write: Broken pipe 
>Jan 20 02:33:22 whistler pqact[13300]: pipe_dbufput: -closelwtmd2-d/var/data/m
> cidas-v write error 
>Jan 20 02:33:22 whistler pqact[13300]: pipe_prodput: trying again
>
>At first, I thought it was a formatting issue, but when checking the log
>file, I found that it doesn't show up all the time.  Do you need to see
>the full log file?

The broken pipe message means that the process reading from the pipe (lwtmd2)
in this case goes away while there is still data in the pipe or data to
be written to the pipe.  If you are running XCD and have the ldm-mcidas
decoding of MD files in the Unidata-Wisconsin datastream turned off in
the McIDAS routing table (ROUTE.SYS), then the LDM will startup 'lwtmd2'
which will then read ROUTE.SYS (if it is in the output data directory and
is readable and writable).  'lwtmd2' will then see that it is not supposed
to decode the MD data and will exit.  The LDM doesn't know that this is
going to happen, so it is still setup to write to the pipe from which
'lwtmd2' was supposed to read.  As soon as a reader on the pipe goes away,
the LDM sees a problem and issues the Broken pipe message.  If this is
your situation, then the best way to get rid of the messages is to remove
the 'lwtmd2' action from your pqact.conf file.  This way, it never gets
started in the first place.

>Jan 20 00:27:03 whistler chinook[13301]: 4a3fc6bfb5722525c2039d17a6ec4dc0: nev
> er completed 
>and
>Jan 20 00:46:34 whistler sysu1[13886]: b6e412e949f38ca643baa72e381f88ba: never
>  completed 
>These errors I would imagine are related to the slow I'net during parts of
>the day, correct?

Yes, the messages are telling you that a product of the ID listed (the
long hex number) did not come in completely.  The upstream LDM should
know this and retransmit the product.

>My last question is about the post processing.  I see the log file being
>generated for the composite images in the /home/mcidas/workdata directory
>and it indicates the batches are being run.  When I check the routing
>table, route.k LIST, they don't show up as being processed.

First thing is that you should make sure that the copy of ROUTE.SYS
that you are looking at is the one that is in the output directory for
the ldm-mcidas decoders.  You can check this by running 'dmap.k
ROUTE.SYS'.  I just logged onto whistler and verified that this is so.
I also verified that the routing table does not show that the
composites have been created, just as you noted.

Next, I looked at the PP log file in /home/mcidas/workdata.  There I
see thta PP BATCH invocations are being kicked off, but I do not see
that they are actually being run.  For instance:

snippit from your /home/mcidas/workdata/ROUTEPP.LOG:

BATCH U3 203 99020 100524 1  TWIN=0 "MDR.BAT
BATCH UB 172 99020 101555 1  TWIN=0 "H2O9.BAT

snippit from our /home/mcidas/workdata/ROUTEPP.LOG:

BATCH U3 207 99020 040526 1  TWIN=0 "MDR.BAT
NORTMAPR 9018 207 9993 N5 NONE 0 9018 MAPAREA=9974 DEV=NNN
batch.k: BATCH done /home/mcidas/data/MDR.BAT
BATCH UB 170 99020 041606 1  TWIN=0 "H2O9.BAT
GOESCOMP 170 NONE 9988 C CW SCRAREA=9976 9977 DEV=NNN
batch.k: BATCH done /home/mcidas/data/H2O9.BAT

This tells me that the PP BATCH command is not actually being run for
some reason on your system.  The entry in ROUTEPP.LOG gets there by
virtue of the echoing of the BATCH command line to the log file in your
~ldm/bin/batch.k file.  If BATCH (actually batch.k) runs, you should
see output from the McIDAS commands run in ROUTEPP.LOG also.  The fact
that you are not seeing this means that the commands are not being
run.

I am poking around on whistler to see if I can figure out what is wrong
right now.

<later>
I added a line to ~ldm/bin/batch.k that CDs to the
/home/mcidas/workdata directory.  This is in the later versions of
batch.k (I'll check again to make sure).  It is needed since some of
the REDIRECTions that are defined in /home/mcidas/workdata/LWPATH.NAM
refer to the current directory (needed for things like STRTABLE,
etc.).  I think that this was the problem, but I will wait until more
products come in to make sure.

Looks like the BATCH PP stuff is working now.  Let me know if you see
any problems.

>As always, thanks.

You are welcome.

Tom