[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20000106: McIDAS remote ADDE server (cont.)
- Subject: 20000106: McIDAS remote ADDE server (cont.)
- Date: Thu, 06 Jan 2000 18:34:45 -0700
>From: "Thomas L. Mote" <address@hidden>
>Organization: University of Georgia
>Keywords: 200001042125.OAA01383 McIDAS-XCD setup
Tom,
re: you using NIS+
>You are correct! I am running NIS+. I added the files entry
>under services in my /etc/nsswitch.conf file. After
>rebooting, I was able to do a a telnet cacimbo.ggy.uga.edu
>500 and get it to respond as it should.
Excellent! I just pointed my McIDAS session at your machine
and verified that I can talk to your ADDE remote server:
DATALOC ADD RTIMAGES cacimbo.ggy.uga.edu
DSINFO IMAGE RTIMAGES
Dataset Names of Type: IMAGE in Group: RTIMAGES
Name NumPos Content
------------ ------ --------------------------------------
ANTARCTIC 10 Antarctic IR Composite
EDFLOATER-I 10 Educational Floater
EDFLOATER-II 10 Educational Floater II
GE-IR 10 GOES-East North America IR
GE-IRTOPO 10 GOES-East IR/TOPO Composite
GE-VIS 10 GOES-East North America VIS
GE-VISTOPO 10 GOES-East VIS/TOPO Composite
GE-WV 10 GOES-East North America H2O
GEW-IR 10 GOES-East/West IR Composite
GEW-IRTOPO 10 GOES-East/West IR/TOPO Composite
GEW-VIS 10 GOES-East/West VIS Composite
GEW-VISTOPO 10 GOES-East/West VIS/TOPO Composite
GEW-WV 10 GOES-East/West H2O Composite
GW-IR 10 GOES-West Western US IR
GW-IRTOPO 10 GOES-West IR/TOPO Composite
GW-VIS 10 GOES-West Western US VIS
GW-VISTOPO 10 GOES-West VIS/TOPO Composite
GW-WV 10 GOES-West Western US H2O
MDR 10 Manually Digitized Radar
MDRTOPO 10 MDR/TOPO Composite
MOLL-IR 10 Mollweide Composite IR
MOLL-IRTOPO 10 Mollweide IR/TOPO Composite
MOLL-WV 10 Mollweide Composite H2O
RESFLOATER 10 Research Floater
DSINFO -- done
I also verified that I can see images on your system:
IMGLIST RTIMAGES/GE-IR.ALL
Image file directory listing for:RTIMAGES/GE-IR
Pos Satellite/ Date Time Center Band(s)
sensor Lat Lon
--- ------------- ------------ -------- ---- ---- ------------
1 G-8 IMG 6 JAN 00006 12:15:00 23 71 4
2 G-8 IMG 6 JAN 00006 13:15:00 23 71 4
3 G-8 IMG 6 JAN 00006 14:15:00 23 71 4
4 G-8 IMG 6 JAN 00006 15:15:00 23 71 4
5 G-8 IMG 6 JAN 00006 16:15:00 23 71 4
6 G-8 IMG 6 JAN 00006 19:15:00 23 71 4
7 G-8 IMG 6 JAN 00006 20:15:00 23 71 4
8 G-8 IMG 6 JAN 00006 21:15:00 23 71 4
9 G-8 IMG 6 JAN 00006 22:15:00 23 71 4
10 G-8 IMG 6 JAN 00006 11:15:00 23 71 4
IMGLIST: done
Next, I tried to see if your point source data files were accessible:
DATALOC ADD RTPTSRC cacimbo.ggy.uga.edu
Group Name Server IP Address
-------------------- ----------------------------------------
RTPTSRC CACIMBO.GGY.UGA.EDU
<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done
PTLIST RTPTSRC/PTSRCS.ALL FORM=FILE
PTLIST: No MD files found
PTLIST: Done
This tells me that either no MD files are being created from XCD processes
or that the REDIRECTions in the 'mcidas' account are not valid.
re: setup the 'mcadde' account to have the same HOME directory as 'mcidas'
>OK. This I didn't realize. I also (before rebooting)
>changed the shell on user mcadde to /bin/false and changed
>the home directory for mcadde to /unidata/home/mcidas. I
>also changed my inetd.conf file. I have attached these
>files.
Very good.
re: testing point source data access
>OK. As before, I had already gone through all of these
>steps before I sent the question. (Not trying to be
>smart... just want you to know I'm making an effort.)
I understand completely. I included the information just in case you
hadn't done the steps.
>I went through the steps again after getting the ADDE server
>responding. I have attached the LOCDATA.BAT file. I also
>checked the redirections with a REDIRECT LIST. Everything
>is set to /data/mcidasd.
>
>I then tried a SFCPLOT T USA. No dice.
This is not good. The failure on your side matches my test above.
>I tried a DATALOC ADD RTPTSRC LOCAL-DATA before the SFCPLOT
>command. Nada.
This must mean that the REDIRECTions are wrong or that MD files are
not being created by the XCD decoders.
>So, I went to the SFCPLT command. Still not finding data.
>BTW, I see no MD files in my /data/mcidasd directory.
Bingo!
>I will assume this is significant! I have all sorts of files,
>but no MD files. I have attached a listing of my
>/data/mcidasd directory.
The other thing I discovered is that I can't access GRID files from
your system.
re: turning on GRID decoding
>I had set up the GRIB decoding before writing. (Again, not
>trying to be smart...) DECINFO does show the GRIB decoder
>to be active.
OK. I am baffled right now.
>I still wonder about the MD files. I should be getting
>them.
You should if the following was done:
o added the xcd_run entries to pqact.conf and restarted your LDM
o copied SCHEMA from the ~mcidas/data directory to /data/mcidasd
(the point source decoders need the information in the schema
file in order to know how to write the data)
>I still have all of the "old" decoders running in
>addition to the XCD.
Old decoders? Do you mean ldm-mcidas decoders? If so, I need to
remind you that the point source and grid files that used to be
in the Unidata-Wisconsin channel back on July 1 of 1999. The only
products coming in the Unidata-Wisconsin datastream anymore are
images.
>I have attached my pqact.conf file.
Great.
>Maybe the problem is there. (A pqactcheck shows it to
>be OK.)
I am glad that you tested the file contents integrity!
>Some of this stuff is probably out of date since I
>was gone for over a year. I haven't had a chance to update
>the pqact.
OK.
>I'm not sure what to try next, but we do seem to be
>chipping away at problems.
Yes, we are making progress. The fact that remote access works to
your ADDE server is good. The whole ADDE thing is what is so cool
about McIDAS! To prove this to yourself, try the following from
a McIDAS-X session started as the user 'mcidas':
DATALOC ADD RTPTSRC ADDE.UNIDATA.UCAR.EDU
SFCPLOT T USA
DATALOC ADD RTPTSRC cacimbo.ggy.uga.edu
Pretty cool, huh!?
Comments on files you attached:
services::
...
# UNIDATA stuff
ldm 388/udp unidata # UCAR Unidata LDM
ldm 388/tcp unidata # UCAR Unidata LDM
mcserv 500/tcp # McIDAS ADDE port
mccompress 503/tcp # McIDAS ADDE compression port
Looks good.
inetd.conf::
mcserv stream tcp nowait mcadde /unidata/home/mcidas/bin/mcservsh
mcservsh -H /unidata/home/mcidas
mccompress stream tcp nowait mcadde
/unidata/home/mcidas/bin/mcservsh mcservsh -H /unidata/home/mcidas
This also looks correct. The routine /unidata/home/mcidas/bin/mcservsh is
a Bourne shell script that will read in information from the file
/unidata/home/mcidas/.mcenv. '.mcenv', contains McIDAS envirionment
information (e.g. MCDATA, MCPATH, MCGUI, PATH, etc.) that is needed
to run McIDAS applications. Since I can list out (and display) imagery
from your machine, I figure that '.mcenv' is setup correctly.
LOCDATA.BAT::
This file was zero lenght in your message.
pqact.conf::
My comments are sorrounded by '******************************' below.
#####
#Entries for XCD decoders
#
DDPLUS|IDS ^.* PIPE
xcd_run DDS
HRS ^.* PIPE
xcd_run HRS
******************************
Looks good. The other things that are needed are (you probably already
did these, but others may read these transactions to see what could
be wrong):
o copy xcd_run to ~ldm/decoders directory
o edit ~ldm/decoders/xcd_run and adjust MCDATA, MCPATH, PATH, LD_LIBRARY_PATH
to match your McIDAS installation
o make sure that xcd_run has execute permission set
******************************
...
#### MCIDAS
# Pattern/actions for the Unidata/Wisconsin (i.e. McIDAS) data stream.
# See the man page documentation for gunrv2, lwfile, lwtmd2, and lwtoa3
# for more info on how to customize these.
#
# GRID files:
#
MCIDAS ^(GUNRV2 .*)
PIPE
-close /unidata/home/mcidas/ldm-mcidas/bin/gunrv2 -d /data/mcidasd -v
# Opaque files:
#
MCIDAS ^(LWFILE .*)
PIPE
-close /unidata/home/mcidas/ldm-mcidas/bin/lwfile -d /data/mcidasd -v
# MD files:
#
MCIDAS ^(LWTMD2 .*)
PIPE
-close /unidata/home/mcidas/ldm-mcidas/bin/lwtmd2 -d /data/mcidasd -v
# AREA files:
#
MCIDAS ^(LWTOA3 .*)
PIPE
-close /unidata/home/mcidas/ldm-mcidas/bin/lwtoa3 -d /data/mcidasd -v
# Profiler files:
#
MCIDAS ^(LWFILE PROFILER.CDF)
PIPE
-close /unidata/home/mcidas/ldm-mcidas/bin/lwfile -d /data/mcidasd -v
MCIDAS ^(LWFILE) (PROFILER.CDF) .* DIALPROD=(..)
EXEC
cdftomd -d /data/mcidasd \3 \2 81
# Fronts for McIDAS
WMO ^ASUS01 .... ([0-3][0-9])([0-2][0-9])
FILE /data/surface/front/ASUS1%j
******************************
You can comment out the entries for GUNRV2, LWFILE, and LWTMD2 above.
Again, the only data in the Unidata-Wisconsin datastream since July 1
is imagery.
If you are going to want FSL wind profiler data decoded into McIDAS
MD files, you need to add the following entries to pqact.conf:
FSL2 ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(.*)\..*
PIPE -close
/unidata/home/mcidas/ldm-mcidas/bin/proftomd -v
-l /usr/local/ldm/logs/ldm-mcidas.log -d /data/mcidasd U2 WPRO 81
FSL2 ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min\.(.*)\..*
PIPE
/unidata/home/mcidas/ldm-mcidas/bin/proftomd -v
-l /usr/local/ldm/logs/ldm-mcidas.log -d /data/mcidasd U6 WPR6 91
This decoder requires that the McIDAS SCHEMA file be in the ouput data
directory, /data/mcidasd.
******************************
#### WSI
# Nexrad data
#
******************************
The current release of McIDAS-X has an ADDE server for both NIDS and
WSI NOWrad (tm) data. It can use the raw files the same way that
GEMPAK/GARP can, so there is no need to turn those products into AREA
files.
******************************
####
# NLDN decode actions
#
#
# nldn format: YYJJJHHMbMe -- YY year, JJJ julian day, HH hour
# Mb begin minute bin, Me end minute bin
******************************
The format of the NLDN broadcast product header as of January 1, 2000 is:
# nldn format: CCYYJJJHHMbMe -- CCYY year, JJJ julian day, HH hour
This means that the regular expression for the following NLDN actions
will no longer be valid (they would match 9YJJJHHMbMe. If you
want to process NLDN data, you should change the regular expressions
from:
^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
to:
^([0-9][0-9][0-9]|[0-9][0-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
This will match both YYJJJHHMbMe and CCYYJJJHHMbMe formats for the NLDN
data products.
Also, you will probably what to check the pathname for the McIDAS NLDN
decoder as the one in the following lines does not follow what you
have for lwtoa3 above.
******************************
#
# Store 6 min bins with group parts
#
NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
FILE
-close /data/nldn/\1\2\3\4\5.ingest
#
# Store 6 min bins verbatim
NLDN (.*)
FILE
-close /data/nldn/\1.ingest
#
# Store transmitted, binary form as YYJJHH file
NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
FILE
-close /data/nldn/\1\2\3.ingest
#
# Decode transmitted binary form as GEMPAK ship file (named YYMMDDHH.gem)
NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
PIPE /usr/local/ldm/decoders/bin/dcnldn -m 10000 -s minute10
-d /data/gempak/logs/dcnldn.log
-p /unidata/home/gempak/gempak5.4/tables/pack/nldn.pack
/data/gempak/nldn/YYMMDDHH.gem
#
# Decode transmitted, binary form as McIDAS MD file
NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
PIPE
-close /usr/local/ldm/util/bin/nldn2md -v -d /data/mcidasd 70 NLDN
DIALPROD=LD \1\2 \3\400 DEV=CNN
#
...
******************************
The next two NLDN actions appear to be repeats of ones above:
******************************
# Store transmitted, binary form as YYJJHH file
NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
FILE -close /data/mcidasd/\1\2\3.ingest
# Decode transmitted, binary form as McIDAS MD file
NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
PIPE
-close /usr/local/ldm/util/bin/nldn2md -v
-d /data/mcidasd 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CCN
******************************
OK, the XCD stuff in pqact.conf looks good. There some mods needed
as I indicate above, but these would not keep the XCD point source
decoders from working.
******************************
data.mcidasd::
numerous .IDX files
-rw-r--r-- 1 ldm apps 128 Jan 5 18:44 AVM00005.IDX
...
numerous AREA files, but no composites. More on this later.
-rw-rw-rw- 1 ldm apps 607776 Jan 6 07:34 AREA0060
...
.XCD files (these contain ALL of the DDPLUS|IDS data for a single
day):
-rw-rw-r-- 1 ldm apps 80072960 Jan 5 18:59 DD000050.XCD
-rw-r--r-- 1 ldm apps 177427840 Jan 6 18:09 DD000060.XCD
XCD rapid access files:
-rw-rw-r-- 1 mcidas apps 142596 Jan 5 09:39 FOUS14.RAP
-rw-rw-r-- 1 mcidas apps 6617092 Jan 5 09:39 RAOB.RAP
-rw-rw-r-- 1 mcidas apps 26088580 Jan 5 09:39 SAOMETAR.RAP
-rw-rw-r-- 1 mcidas apps 6364676 Jan 5 09:39 SYNOPTIC.RAP
-rw-rw-r-- 1 mcidas apps 4896388 Jan 6 18:09 TERMFCST.RAP
-rw-rw-r-- 1 ldm apps 6346420 Jan 6 18:09 TERMFCST.RAT
HRS spool file:
-rw-rw-r-- 1 ldm apps 16781313 Jan 6 18:09 HRS.SPL
ROUTE.SYS, SYSKEY.TAB, SCHEMA:
-rw-rw-r-- 1 mcidas apps 14080 Jan 6 18:06 ROUTE.SYS
-rw-rw-r-- 1 mcidas apps 432640 Jan 5 09:39 SCHEMA
-rw-r--r-- 1 mcidas apps 24000 Jan 6 18:34 SYSKEY.TAB
The *.IDX and *.XCD files look normal. The *.RAP files were created
when you ran 'BATCH XCDDEC.BAT'. Each one of these should have a
corresponding *.RAT file. The only one I see is the one for terminal
forecasts. This goes along with you not decoding any point source
data.
The HRS.SPL file shows that model data is being written to the spool,
but the lack of GRID files shows that the GRIB data monitor, DMGRID,
is not decoding any of them into McIDAS data files.
Where do we stand? It seems that there is a problem running the
point source and grid XCD decoders (duh! :-). The question is what
is the problem? To learn more, we both need to look at the contents
of:
o ~mcidas/workdata/XCD_START.LOG
o ~mcidas/.mcenv
o the output from 'DMAP LWPATH.NAM'
(if this file is not in ~mcidas/workdata, we have a problem)
o the contents of ~mcidas/workdata/LWPATH.NAM
o the output from 'DMAP MD'
o a directory listing of ~mcidas/workdata
Tom