20011116: McIDAS vis-a-via Solaris 7/8 (cont.)

>From: "James R. Frysinger" <address@hidden>
>Organization: College of Charleston
>Keywords: 200111061842.fA6Igt112242 McIDAS platfomrs

Jim,

re: use /export/home/mcdata

>       OK. Did that.

>> The ports that need to be opened through the firewall are:
>>
>> Port  Use
>> -----+---------------------------
>> 388   LDM
>> 500   ADDE uncompressed transfers
>> 503   ADDE compressed transfers

>       I'll try to get through to the right person tomorrow (Friday) to have 
>them check and make sure they are open. But if we were ingesting data 
>by LDM 10 months ago, didn't those ports have to be open then?

Port 388 would have had to been opened before, not 500 or 503.

>Could 
>you try our ADDE from there without too much trouble, just to see what 
>kind of response you get?

I tried yesterday, and could not get through to 500 or 503 (I just tried
again a couple of seconds ago).

 re: we get two copies of each email you send

>       I'll mention that to them, too. I've cc'd address@hidden on 
>only two messages so far so if all of them are doubling, that's not the 
>problem. I'm quite willing to believe it's a campus mail server problem.

OK, thanks.

re: only the brave modify LSSERVE.BAT

>       Fine by me! Didn't touch them.

OK.

re: realtime MD files MUST get scoured before they get to be 10 days old

>       Last time we had data coming in, 3 days was the longest I let things 
>hang around before they were scoured.

Sounds good.

re: te.k XCDDATA \"/export/home/mcdata

>       Did that.

or

>> TE XCDDATA "/export/home/mcdata
>
>       Did that (again) once I got to the point of having mcidas back up. 
>Belt + suspenders.

Fair enough.

>       Went to /export/home/mcdata and did mkdir XCD against that future need 
>and before it slipped my mind.

OK.

re: LOCDATA entries

>       Great! Makes sense laid out like that. So I made mine look that way. 

Good.

>So, if I've got the big picture, this is sort of a master file that 
>tells users where various kinds of data are available.

Yes, exactly!

>The ones that we 
>flag with /export/home/mcdata are the types that we will ingest via LDM 
>or other means (DODS?).

Ingest via LDM and decode with XCD and ldm-mcidas decoders.

>For other types of data files, we provide a 
>"referal address".

Right.  Sort of like the machine name portion of a URL.

>And LOCAL DATA is going to send them to the XCD 
>directory I just created, not to the big swamp of ingested data.

LOCAL data means that one's on McIDAS session will be responsible for
knowning the mapping of dataset name (group/descriptor) and actual
data files, AND be able to find those data files.  McIDAS "finds"
files through two facilities: file REDIRECTions and MCPATH.  File
REDIRECTions are mappings between file name masks (regular expressions)
and directories.  A REDIRECTion like:

AREA012* /export/home/mcdata

tells McIDAS applications to look in the dierctory /export/home/mcdata
for any/all files whose names match the regular expression AREA012*.
To complicate life, more than one regular expression can match a file
name.  For instance, all of the following match the name AREA0128:

AREA012* /export/home/mcdata
AREA0*   /jim/data/mcidas
AREA01*  /export/home/imagres
AREA*    /mnt/data/mcidas

The one that "wins" (matches most exactly) is AREA012* since it is the
most specific.

MCPATH, on the other hand, is a colon-separated list of directories
that will be searched by McIDAS routines IF they can not locate a
file by a REDIRECTion.  The order that the directories are searched
is the same for other Unix directory lists (like PATH): left to
right.

>For 
>the referal addresses, was I supposed to have known those addresses 
>from somewhere in the instructions?

Here is where McIDAS falls down somewhat.  There is no "data discovery"
in ADDE (yet).  You have to know a machine name/IP and what dataset(s)
it has in order to "point" to that machine and access data.  We are
working on extending data discovery to ADDE through our THREDDS
initiative.  The original design of ADDE considered no data discovery
to be a security "feature".  Sort of security through obscurity.

>Did I overlook something? Or is 
>that just something one picks up (as sysadmin) by reading the various 
>unidata mail lists (which I'm signed up for and receiving OK)?

You get told who has data and what data (dataset names) they have.
This will bet more flexible as time goes on.

re: MCTABLE_READ and MCTABLE_WRITE

>       OK. Here's where I started running into some strange stuff. Most of it 
>I think is just innocuous (non-fatal) warnings.
>
>       Under "Making File Redirections, ....":
>
>DMAP AREA -- got the following output above the command line:
>WARNING:  cannot examine directory /home/mcidas/data: no such
>   file or directory. [line was repeated]

This says that your LOCAL.NAM file had one or more REDIRECTion definitions
that used /home/mcidas/data.  Since your McIDAS installation is in
/export/home/mcidas, you need to reedit ~mcidas/data/LOCAL.NAM and
change all occurrances of /home/mcidas to /export/home/mcidas.  After
making the change, rerun:

REDIRECT REST LOCAL.NAM

from your McIDAS session, and then rerun the DMAP command.  Repeat
this procedure until you get no warning messages AND you see all of
the AREA files in ~mcidas/data (AREA9000, AREA9010, etc).

>Perm/Size/.....
>   got three lines for AREA 9982, AREA 9983, and AREA 9985 that 
>referred to /export/home/mcidas/data and seemed "normal".
>       Was that normal for DMAP to try the default location first prior to 
>reading our global environment variable values?

No, not if the REDIRECTions are correct.

>LA 9000 9019 -- worked OK

It did?  Did it have a listing for 9011?

>DF 9011 1 AU 0 0 -3 EU=TOPO SF=YES -- got the following complaint:
>Specified image is not a valid type.

Does

DMAP AREA9011

show that the file is in /export/home/mcidas/data?

>       Is that because we don't have anything in that category in memory yet? 

No.

>Or is this a problem?

I suspect your REDIRECTions as I indicated above.

>TE XCDDATA "/export/home/mcdata
>BATCH "LSSERVE.BAT
>       worked fine, watched a lot of impressive comment lines go by that 
>looked "normal" and expected. Again, got the XCDDATA := [as above]

Right, there will be a lot of output.

>DSINFO IMAGE TOPO -- listed 10-12 categories; looked "normal"

Good.

>IMGLIST TOPO/CONF -- complaint:
>No images satisfy the selection criteria.

This must be related to the REDIRECTion issue.

>IMGDISP TOPO/CONF 1 EU=TOPO MAG=-2 LATLON42 100 SF=YES -- complaint:
>Invalid element magnification factor.
>2ND MAG= argument has invalid integer form --> LATLON42
>for more help key in: ARGHELP [disregarded, didn't do this]
>Cannot use magnification factor of zero.

Two things:

o if IMGLIST doesn't work, IMGDISP shouldn't either
o your IMGDISP invocation has a typo.  It should read:

IMGDISP TOPO/CONF 1 EU=TOPO MAG=-2 LATLON=42 100 SF=YES

>BATCH "LOCDATA.BAT -- got lots of nice looking lines ending with
>DATALOC ADD RTWXTEXT weather.cofc.edu
>DATALOC -- done

OK.

>DATALOC ADD TOPO
>DATALOC: Could not resolve TOPO. Entry not filed.
>DATALOC failed, RC=2
>BATCH: BATCH job abandon /export/home/mcidas/data/LOCDATA.BAT
>BATCH: BATCH done /export/home/mcidas/data/LOCDATA.BAT
>       Does this mean that everything went OK except for TOPO stuff? And is 
>that a moot complaint?

We have to worry about the REDIRECTion before we get to this.

>I checked and indeed there is a /export/home/mcidas/data/ADDESITE.TXT 
>and there is no /export/home/mcidas/workdata/MCTABLE.TXT. Looks good, 
>eh?

Yes, looks good!

>DId EXIT without breaking anything except a cold sweat.  ;-)

:-)

>Went on to next section, "Configuring User Accounts". Went to my 
>~mcidas/mcidas7.8/src and found no user_env.csh or user_env.sh. Went to 
>McIDAS Download page and then selected unix and then 780. No luck 
>there, either.

These can be found in:

http://www.unidata.ucar.edu/packages/mcidas/780/mcx/config_users.html

Links to them are near the top of the page.

>Did my make and install wipe those files out?

No.  They are not part of the distribution (but maybe they should be?).
They are accessible on the web page listed above.

>I remember 
>a lot of rm's going by then. Oh, well, I'll just use a standard shell 
>template and type in the MCIDAS stuff in the directions. OK with that? 

Sure.  The information on the web page is the McIDAS stuff that is to
be added to one's shell definition file, not one's entire shell
definition file.

>BTW, here's where your new code for path first shows up.

Yup.

>I see that I've got a little editing to do in the ~mcidas/.Xdefaults 
>and ---/dtwmrc files.

Only if you are going to be using the Fkey menu.  Since I am moving away
from the Fkey menu towards my GUI, MCGUI, I think you can skip this
step.

>However, doing ls -a doesn't list those under 
>~mcidas. Oh....yes, this account is using CDE, not Open Windows.

The dtwmrc dile is foe CDE, not Openwindows.

>Is 
>that a Solaris 8 thing? I didn't find any Edit Dtwmrc icon on the 
>Desktop_Tools area of the pop up desktop menu, either. What do I do for 
>Solaris 7? BTW, I just bought a copy of the Unix System Administrator's 
>Handbook by Nemeth, et al. I'm figuring on doing a little light reading.

Let's skip the CDE setup studd for right now.  Again, the mods are to
be able to use all function keys in the McIDAS Fkey menu.  If you
don't use it, then there is no need to mess with the CDE setup.

>Under, "Copying Needed FIles", I see the files to copy from. I take it 
>that when a user account is set up, it should include a mcidas 
>directory and a mcidas/data directory.

Yes.

>Just do a mkdir for those?

Yes.

>Any others?

No, not for non-'mcidas' uses.

>I figure on making my own frysingj account a user account for 
>practice.

Good idea.

>And that account is in the same group as mcidas, not the 
>normal user group.

Not what I recommend, but have at it.  Just remember that being in the
same group as mcidas (and mcadde and ldm) will allow you to modify
files that non-'mcidas' uses should never modify.

>Off-the-wall: just to check something, the owner uid for mcadde is 
>mcadde and not mcidas, correct?

I guess I don't understand the question (my thickheadedness).  The
user 'mcadde' is separate from 'mcidas', BUT it is in the same group
and has the same HOME directory.  One can think of it as an alias
for 'mcidas', but one with not all of the permissions as 'mcidas'
(it has group privilege, but not owner privilege).

>       Hangin' tight! End in sight?

Yes, you are close to being finished for account stuff.  The next
step is in setting up XCD stuff in the 'mcidas' account, and ldmd.conf
and pqact.conf entries in the 'ldm' account.  These only have to be
done once, so they are simply a one-time hurdle.

>(Of course, then there's LDM, LDM-MCIDAS, 
>DODS, Gempak, scripting the decoders, establishing a feed, and a big 
>bottle of Tylenol ... not necessarily in that order.)    ;-)

Right ;-)  We'll be here to help if/when needed.  The funny thing is
that once a dull install has been done and understood, a new full
install should only take on the order of a half hour start-to-finish.

Tom

>From address@hidden Fri Nov 16 09:28:44 2001
>Subject: Firewall holes for data access
>Sender: address@hidden
>To: address@hidden, address@hidden
>Cc: Unidata Support <address@hidden>,
>   address@hidden, address@hidden,
>   "Dr. Jon Hakkila" <address@hidden>

Dear Tony and Charlie,

I'm not sure which of you (or who else) is in charge of providing access
through our firewall for data access, so please pass this to the proper
person if my aim is off the mark.

I am close to finishing the process of restoring mcidas and mcadde to
our weather.cofc.edu which means that again we will be wanting data port
access on/off campus with Unidata/UCAR providers.

The ports I need to have open are:
   388 LDM [the data ingester here at weather.cofc.edu]
   500 ADDE uncompressed transfers
   503 ADDE compressed transfers
ADDE is the local server into which data comes and out of which it is
served, both locally and remotely, as controlled by LDM, our local data
manager. I believe that they were open last January when we were
successfully ingesting data, but this bears checking!

PLease let me know ASAP what the status of these ports is for access
through the firewall to and from weather.cofc.edu.

As an aside, please pass to the postmaster of ashley.cofc.edu that my
Unidata colleagues tell me that my messages to them are doubling.

regards,
Jim Frysinger
also at:
   address@hidden
   (Sysadmin, weather.cofc.edu)
 
-- 
James R. Frysinger                  University/College of Charleston
10 Captiva Row                      Dept. of Physics and Astronomy
Charleston, SC 29407                66 George Street
843.225.0805                        Charleston, SC 29424
http://www.cofc.edu/~frysingj       address@hidden
Cert. Adv. Metrication Specialist   843.953.7644