[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20050121: Unidata McIDAS-XCD Trouble (cont.)
- Subject: 20050121: Unidata McIDAS-XCD Trouble (cont.)
- Date: Fri, 21 Jan 2005 09:58:24 -0700
>From: "James T. Brown" <address@hidden>
>Organization: Michigan State University
>Keywords: 200501201933.j0KJXOv2020815 McIDAS-XCD ADDE
Hi Jim,
>Thanks for the reply. I checked the notes that you sent and
>for the most part, the steps of the installation seem to match
>the steps I have taken.
OK. I thought this was the case given your excellent reporting of
your situation, but I had to list out the steps just in case...
>Some of my installation folders are
>different, but all of the proper environment variables should
>be set.
Sounds good.
>Only main difference is the "SCHEMA", "SYSKEY.TAB",
>and "ROUTE.SYS" files that reside in my /data/mcidas folder
>came from the "ldm-mcidas" decoders and not from the McIDAS
>installation - any problem with that?
Depending on which ldm-mcidas version you installed, there should
be no problem. The McIDAS v2004a distribution has the latest
versions of these files, so if you installed v2004a it would be
best to use the copies of the files from it.
>If you would like to connect to my ADDE services for the
>RTPTSRC group, my server name is:
>
> pileus.geo.msu.edu
OK, thanks. I am able to contact your ADDE server so the installation
looks OK. I note, however, that you do not have an surface MD file
for today; it would be MDXX0001.
>Here is a new listing of the MDxx files in my "/data/mcidas"
>folder:
>
> % ls -lm MD*
> -rw-rw-r-- 1 9002 wx 49999236 Jan 21 09:32 MDXX0009
> -rw-rw-r-- 1 9002 wx 338536 Jan 20 18:48 MDXX0010
> -rw-rw-r-- 1 9002 wx 44560 Jan 19 18:49 MDXX0019
> -rw-rw-r-- 1 9002 wx 151312 Jan 20 02:53 MDXX0020
> -rw-rw-r-- 1 9002 wx 16640 Jan 19 18:49 MDXX0029
> -rw-rw-r-- 1 9002 wx 112752 Jan 20 13:38 MDXX0030
> -rw-rw-r-- 1 9002 wx 8071800 Jan 21 09:32 MDXX0059
> -rw-rw-r-- 1 9002 wx 887448 Jan 20 06:51 MDXX0060
> -rw-rw-r-- 1 9002 wx 5951016 Jan 20 01:45 MDXX0069
> -rw-rw-r-- 1 9002 wx 17360 Jan 20 13:37 MDXX0070
>
>Other than some changes in some of the modification times, most of
>the file sizes are the same as the listing that was producded yesterday.
The surface MD file for yesterday, MDXX0010, is too small to hold
much data. A full MD file will be on the order of 50 MB per day.
>Here is the latest output from the "PTLIST" file listing:
>
> % ./ptlist.k RTPTSRC/PTSRCS FORM=FILE ALL
> Pos Description Schema NRows NCols Proj#
>Created DataDate
> ------ -------------------------------- ------ ----- ----- -----
>------- --------
> 9 SAO/METAR data for 19 JAN 2005 ISFC 72 7000 0
>2005019 2005019
> 10 SAO/METAR data for 20 JAN 2005 ISFC 72 7000 0
>2005020 2005020
> 19 Mand. Level RAOB for 19 JAN 2005 IRAB 8 1500 0
>2005019 2005019
> 20 Mand. Level RAOB for 20 JAN 2005 IRAB 8 1500 0
>2005020 2005020
> 29 Sig. Level RAOB for 19 JAN 2005 IRSG 16 6000 0
>2005019 2005019
> 30 Sig. Level RAOB for 20 JAN 2005 IRSG 16 6000 0
>2005020 2005020
> 59 SYNOPTIC data for 19 JAN 2005 SYN 8 6500 0
>2005019 2005019
> 60 SYNOPTIC data for 20 JAN 2005 SYN 8 6500 0
>2005020 2005020
> 69 PIREP/AIREP data for 19 JAN 2005 PIRP 24 1500 0
>2005019 2005019
> 70 PIREP/AIREP data for 20 JAN 2005 PIRP 24 1500 0
>2005020 2005020
> ./ptlist.k: Done
I can get the same listing from here.
>I am no expect in McIDAS, but shouldn't there be MDxx files ending
>with "1" for today's date (Jan 21)?
Yes, absolutely.
>You asked for output of the "PTLIST RTPTSRC/SFCHOURLY" command:
>
> % ./ptlist.k RTPTSRC/SFCHOURLY
> Row : 1 Col : 885
> TYPE = 0 | DAY = 2005020
>CYD |
> TIME = 0 HMS | NREC =
>3 |
> ID = KPIH | LAT = 42.9202
>DEG |
> LON = 112.5711 DEG | ZS = 1356
>M |
> ST = ID | CO =
>US |
> MOD = 0 | HMS = 234600
>HMS |
> CIGC = _missing_ | CC1 =
>_missing_ |
> CC2 = _missing_ | CIGH =
>_missing_ |
> ZCL1 = _missing_ | ZCL2 =
>_missing_ |
> VIS = 1.5 MI | WX1 =
>_missing_ |
> WX2 = _missing_ | T = 277.16
>K |
> TD = _missing_ | DIR = 0
>DEG |
> SPD = 0.0 MPS | GUS =
>_missing_ |
> PSL = _missing_ | PCP =
>_missing_ |
> SNO = 4 IN | PRE =
>_missing_ |
> P24 = _missing_ | WXC1 =
>_missing_ |
> WXC2 = _missing_ | WXC3 =
>_missing_ |
> WXC4 = _missing_ |
>
>---------------------------------------------------------------------------
> Number of matches found = 1
> ./ptlist.k: Done
The reason I asked for the listing was to see if data was being decoded.
It is, but there should be lots more data than is represented by the
size of yesterday's surface MD file.
>I've used McGUI to attempt to plot Meteorograms and most stations produce
>a blank screen (station=KPIH from the record listed above only plots data
>from the entry listed above).
I tried as well and see almost no data in your MD files.
>If you would like to logon to take a look around, feel free - I would be
>very interested if you are able to determine the source of the problem.
>
>I have set a temporary password on the "mcidas" account - you should be
>able to SSH into the machine where McIDAS and McIDAS-XCD have been
>built:
OK.
>My LDM is running on another machine which is behind a firewall, but
>is temporarily accessible via SSH from the "pileus" machine (using
>the same McIDAS username and password listed above):
>
> LDM HOST:
>
> hostname: accas.geo.msu.edu
> LDM home dir: /soft/ldm
>
>Let me know if you would like temporary access to the "ldm" user
>account.
I logged on and see one major problem right off. 'mcidas' does not
have write permission for the /data/mcidas directory:
> hostname
accas
> touch /data/mcidas/xxx
touch: /data/mcidas/xxx cannot create
I also see that files that should have been created by the XCD setup
BATCH scripts do not exist in /data/mcidas. I feel confident that this
is one (and perhaps the only) reason your decoding is not working
correctly.
The first part of the fix is to add group write permissions to
/data/mcidas. The easiest thing to do next is to redo the XCD
configuration from scratch (this will take only a couple of minutes):
- as 'ldm':
ldmadmin stop
cd /data
chmod 775 mcidas
cd /data/mcidas
rm *
- as 'mcidas' do the following:
cp ~mcidas/data/SCHEMA /data/mcidas
cp ~mcidas/data/SYSKEY.TAB /data/mcidas
cp ~mcidas/workdata/ROUTE.SYS /data/mcidas
cd /data/mcidas
chmod 664 SCHEMA SYSKEY.TAB ROUTE.SYS
mkdir bufr grib
cd ~mcidas/workdata
rm DECOSTAT.DAT
rm DCLSTIDX.PTR
batch.k XCD.BAT
batch.k XCDDEC.BAT
At this point, you should have a set of files in /data/mcidas that
were created by the BATCH scripts. If you only have SCHEMA, SYSKEY.TAB,
ROUTE.SYS it means that you still have some sort of permission problem.
- if the BATCH scripts were successful in producing additional files
in /data/mcidas, you should restart your LDM:
<as 'ldm'>
ldmadmin start
>I can certainly try to rebuild things again later in the day
>and can combine the ADDE services on the same machine as the
>LDM if that would make any difference.
That should not make any difference. The problem is that some of the
files needed in XCD decoding do not exist in /data/mcidas. The reason
they don't exist is that 'mcidas' does not have write permission for
the directory (and it needs to!).
>However, if you are
>able to login and take a quick look, I appreciate any suggestions
>that you may have before I do anything too drastic.
The fix should be quick and easy, and you should see proper decoding
almost immediately.
>Let me know if you have any trouble accessing the machines and
>accounts listed above.
No problems, thanks for the access. It was very useful to be able to
see your setup. I would not have thought about /data/mcidas not being
open for writing to 'mcidas' until all other things have been
exhausted.
One last thing. The set of DATALOCs that are setup on in the 'mcidas'
account are not quite correct given the data you are ingesting.
> dataloc.k
Group Name Server IP Address
-------------------- ----------------------------------------
AMRC UWAMRC.SSEC.WISC.EDU
BLIZZARD ADDE.UCAR.EDU
CIMSS PILEUS.GEO.MSU.EDU
GINICOMP PILEUS.GEO.MSU.EDU
GINIEAST PILEUS.GEO.MSU.EDU
GINIWEST PILEUS.GEO.MSU.EDU
GOESEAST PILEUS.GEO.MSU.EDU
ME7 IO.SCA.UQAM.CA
MYDATA <LOCAL-DATA>
NEXRCOMP PILEUS.GEO.MSU.EDU
RTGRIDS PILEUS.GEO.MSU.EDU
RTIMAGES PILEUS.GEO.MSU.EDU
RTNEXRAD PILEUS.GEO.MSU.EDU
RTPTSRC PILEUS.GEO.MSU.EDU
RTWXTEXT PILEUS.GEO.MSU.EDU
TOPO <LOCAL-DATA>
<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done
In looking at the request lines in your ~ldm/etc/ldmd.conf file, I see
that you are not ingesting the data that would allow you to have the
data files necessary for:
GINICOMP, GINIEAST, and GINIWEST - NOAAPORT GOES imagery available in the
NIMAGE IDD datastream
GOESEAST - GOES satellite imagery available
only from the ADDE server on
unidata2.ssec.wisc.edu
NEXRCOMP - NEXRAD Level III composite imagery
available in the FNEXRAD IDD datastream
RTGRIDS - GRID decoding is turned off
RTNEXRAD - NEXRAD Level III products available
in the IDD NNEXRAD datastream
Here is what I see for data request lines in ~ldm/etc/ldmd.conf:
> grep ^request ~ldm/etc/ldmd.conf
request UNIDATA|FSL ".*" f5.aos.wisc.edu PRIMARY
request NLDN ".*" striker2.atmos.albany.edu PRIMARY
request FSL3 "MSR_mpe" crgateway.ncrfc.nws.gov
request CONDUIT "/MT.ruc_CY" f5.aos.wisc.edu
So, if you have no intentions of ingesting data needed for the above
datasets, I suggest that you do the following:
<as 'mcidas'>
cd ~mcidas/data
edit LOCDATA.BAT and set it up to look like the following:
DATALOC ADD CIMSS pileus.geo.msu.edu
DATALOC ADD GINICOMP pscwx.plymouth.edu
DATALOC ADD GINIEAST pscwx.plymouth.edu
DATALOC ADD GINIWEST papagayo.unl.edu
DATALOC ADD GOESEAST unidata2.ssec.wisc.edu
DATALOC ADD NEXRCOMP adde.ucar.edu
DATALOC ADD RTGRIDS adde.ucar.edu
DATALOC ADD RTIMAGES pileus.geo.msu.edu
DATALOC ADD RTNEXRAD atm.geo.nsf.gov
DATALOC ADD RTPTSRC pileus.geo.msu.edu
DATALOC ADD RTWXTEXT pileus.geo.msu.edu
DATALOC ADD TOPO LOCAL-DATA
Leave the other DATALOC settings in LOCDATA.BAT the same and then do:
cd ~mcidas/workdata
batch.k LOCDATA.BAT
After doing this, the 'mcidas' account will be setup to access data in
the datasets you are not ingesting data for from remote ADDE servers.
If you do have intentions for ingesting data necessary for the datasets,
there is still some work to be done to define the datasets. We can
talk about this in detail if/when you decide to change what you are
ingesting.
>Thanks.
No worries.
Cheers,
Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web. If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.