20010111: configuring McIDAS for ADDE access to NEXRAD Level III products

>From: "Stacy N. Wehmeyer" <address@hidden>
>Organization: University of Missouri-Columbia
>Keywords: 200101111805.f0BI5Xo04181 McIDAS-X NEXRAD ADDE LDM

Stacy,

>My name is Stacy Allen and I'm a graduate student at the University of
>Missouri-Columbia working on getting the new NEXRAD feed via the IDD
>working for local use in McIDAS.

Nice to meet you.

>After reading the recommendations for receiving radar, I decided to
>upgrade our system to McIDAS-X 7.7 from 7.6 that will support the
>zlib-compressed radar files.

Excellent.  I see that you grabbed the distribution on: Mon Jan  8 09:49:29.
Please be advised that there have been bugfix releases for 7.70 since you
FTPed the distribution.  For information on the McIDAS addendum process,
please review:

http://www.unidata.ucar.edu/packages/mcidas/770/mcx/addenda.html

>The installation went well, and the XCD
>installation went okay too, or so I thought.

OK.  One quick note here.  Since Unidata sites run the LDM, the handling
of the NEXRAD Level III data products will be totally outside of McIDAS-XCD.
More on this later if needed.

>When I went to the
>section on Running with the LDM, the step where I tried checking to see
>if startxcd was running, I didn't find the process with the command 'ps
>-eax | grep pqact'.

I would have tried:

ps -eax | grep startxcd

or

ps -eaf | grep DM

>I tried the troubleshooting step of seeing if
>there is a lockfile in the directory  /tmp,  and there was.  I removed
>the file startxcd.k, then did a 'ldmadmin stop' and 'ldmadmin start'.
>When I looked again in /tmp, there was another lockfile with the
>timestamp of the ldmadmin start process.  Several repetitions of the
>above have only shown that as soon as the LDM starts, a new lockfile is
>produced.

If your system is running McIDAS ROUTE PostProcess BATCH files, then
the lock file in /tmp will be created when they run.  I think that
worring about this at this juncture is unneeded.  If you really want
to make sure that the McIDAS-XCD stuff is not being run, then do the 
following:

<login as 'ldm'>
cd etc
<edit ldmd.conf and comment out the line:

exec "xcd_run MONITOR"

<also edit pqact.conf and comment out the lines:

DDPLUS|IDS      ".*"    PIPE
        xcd_run DDS
HDS     ".*"    PIPE
        xcd_run HDS

and then stop and restart the LDM.

>Here is what I have done that may have an effect on why xcd_run isn't
>fully operational and McIDAS images aren't viewable yet:

No images viewable in McIDAS are related to McIDAS-XCD.  The images in the
Unidata-Wisconsin datastream (LDM feed type MCIDAS) are decoded with the
ldm-mcidas decoder, pnga2area.  The NEXRAD images are ingested with and
FILEd by the LDM.  All that is needed to get the ADDE access to those
images is proper configuration of ADDE services.

>-I copied xcd_run from /home/mcidas/mcidas7.7/src to ~ldm/decoders, gave ldm 
>execute permission, and edited the library paths to include our Sun libraries 
>(SC4.2 changed to SC5.0).  

OK, good.

>-The LDM and McIDAS users are in the same group and both have umask of 2.

Excellent.

>-Then I edited our ldmd.conf to have the lines:
>
>exec    "pqexpire"
>exec    "xcd_run MONITOR"
>exec    "pqbinstats"
>exec    "pqact"
>#exec   "pqsurf" 

Good.  You are now going down the path of setting up XCD.  If your only
objective is access to the NEXRAD data, you do not need to do these steps.

>-Then I uncommented the lines in pqact.conf:
>
>#McIDAS-XCD section
>#
>DDPLUS|IDS      ^.*     PIPE
>        xcd_run DDS
>HRS     ^.*     PIPE
>        xcd_run HRS
>#

OK.

>-And I have pqact gathering the data as such:
>
>NNEXRAD         ^SDUS5. K... (..)(....) /p(N0R)(ARX|LSX|EAX)
>                FILE    -close  -overwrite      /asp/met/data/ldm/nexrad/\4/\3
> /\
>3_\1\2.\4

OK.  You are only going to FILE the N0R NEXRAD products (Base Reflectivity
at Tilt 1) for ARX, LSX, and EAX with this action.

>Here is a sample of our filing structure:
>
>/asp/met/data/ldm/nexrad/LSX/N0R/N0R_111637.LSX

Looks good.

>We're only requesting base reflectivity for the closest radars and the test 
>radar; we'll branch out later.  

OK.  Looks like you have done your homework.

>-I have searched the email archive and found two emails about configuring the 
>ADDE and have followed the instructions.  I made a copy of DSSERVE.BAT 
>(DSNEXRAD.BAT) and edited it to have only NEXRAD info with the following lines
> :
>
>DSSERVE ADD RTNEXRAD/N0R          NEXR    1 9999 TYPE=IMAGE INFO=MUNEXR.CFG   
>  " 
>      "Base Reflectivity Tilt 1
>and so on...

Super.

>-The copy of NNEXRAD.CFG (MUNEXR.CFG) in ~mcidas/workdata  has the following 
>lines:
>
>DIRMASK=/asp/met/data/ldm/nexrad\ID/\TYPE
>FILEMASK=\TYPE_*.\ID
>IPMASK=*

This looks good.

>-When I go to complete the instructions, 
>'batch.k DSNEXRAD.BAT'
>'cd ~mcidas/workdata'
>'dataloc.k ADD RTNEXRAD cires_adde_server_name'
>'dsinfo.k I RTNEXRAD'
>-all the above commands give great results, but then this:

a small comment here.  The order of these executions should have been:

cd ~mcidas/workdata
batch.k DSNEXRAD.BAT
dataloc.k ADD RTNEXRAD cires_adde_server_name
dsinfo.k I RTNEXRAD

and you should have replaced cires_adde_server_name with the actual name
of your server (like cires.snr.missouri.edu if that is the machine's name).
You should have seen an error when you ran this DATALOC command.

Another thing when doing a DATALOC (dataloc.k).  The machine that you specify
to go to when requesting the data must have already been setup for the
McIDAS remote ADDE server.  I will guess that this step has not been done
for cires.

Lets, however, plow on.  Do the following:

cd ~mcidas/workdata
dataloc.k ADD RTNEXRAD LOCAL-DATA

This says to not try to go out to a remote ADDE server for the data (remote
ADDE servers can be anywhere accessible by TCP/IP ethernet including the
same machine you are doing this all on).

>McIDAS =>imglist.k RTNEXRAD/N0R ID=LSX
>Image file directory listing for:RTNEXRAD/N0R
>imglist.k: SelectNexrImages:: error generating list of files
>imglist.k: done
>McIDAS =>

At this point, after changing the DATALOC to be LOCAL-DATA, the McIDAS
account should be able to list the LSX N0R products with the command
you used above.

imglist.k RTNEXRAD/N0R ID=LSX

>It seems I am close to the end,

Yes, very!

>but I am stumped as to what could be
>going wrong.  My guess is that the XCD problem is related to the fact
>that I cannot list the images, but I could be wrong.

Again, XCD has nothing to do with the NEXRAD data at all.

>Please keep in mind that my unix, McIDAS, LDM experience is limited to
>the last six months of immersion (that is probably why I have been so
>thorough in this narrative).

I appreciate your thoroughness and commend you on your efforts so far.
The only step you were missing was setting up the remote ADDE server
on your machine.  You can do most of the work there, but someone with
'root' privilege will have to finish the job (since one step must
be done by 'root').  Until you are ready to tackle the remote ADDE
server installation, you should work in the 'mcidas' account and
keep the DATALOCation for the NEXRAD data to LOCAL-DATA.

>I do appreciate all the help you've
>provided in the form of the email archive and basic instructions.
>Thanks!

You are welcome.  Again, you are very close indeed.

>Stacy Allen
>Graduate Research Assistant
>Mesoscale Dynamics Lab
>201 Gentry Hall
>Columbia, MO 65211

Tom Yoksas