19990625: McIDAS Comments and Problems

>From: Bryan Rockwood <address@hidden>
>Organization: Creighton
>Keywords: 199906240351.VAA10949 McIDAS NLDN DAY ADDE NT

Bryan,

re: Linux memory mapping does not play nicely with LDM
>I would have to say that I have noticed the same thing, so add another
>comment to that list.

OK, we have pretty good feedback that there is a real problem.  Our
problem is that our LDM chief designer, Glenn Davis, was killed in
an airplane accident in early May.  This has left a big hole in our
development group that will take us some time to recover from.

re: impending change in Unidata-Wisconsin datastream
>Yes, I have been working on that over the last month.

Excellent!

>I have moved most
>of the affected data over to the XCD with the only thing left being the
>model data.  I am trying to decided how I am going to do this as we have
>very little HD space left for data (currently, I can only keep about two
>days worth and not go above 75% capacity of the /var/data partition which
>leaves 400 Meg).  I am in the process of trying (or begging as the case
>may be) for a much larger HD.  The system has a 3 Gig now, and I have seen
>8 for as low as $99 bucks at a local Best Buy.  Some of the faculty have
>expressed a need for seven days worth of XCD data, so I am actively
>pursuing this.

Isn't it incredible that it is so hard to come up with a few hundred
dollars that could be used to add much needed capabilities to existing
systems.  I went through this all the time when I was the system
administrator numerous years ago at UW (Wyoming, that is).

re: what kind of disk(s)
>Sadly, I was not involved in the purchase of the system, so it is a
>P200MMX with an EIDE HD.  If I had my way, I would have forced SCSI on
>them :-).

We also believe that SCSI disks stand up to hard use better than IDE
drives.  I got the totally different opinion from a person who had
worked at CSU/CIRA, however.  He said that their experience was that
SCSI disks did NOT last as long as EIDE drives.  I suspect that the
difference was that they were using NT and we were using Unix.  NT
conforms to the track/sector/cylinder disk access in xIDE drives;
Unix does the block addressing that is integral to SCSI disks.

re: lack of colors when using CDE on x86
>I have come across this problem.  I am tossing around the idea of going
>with OpenWindows.  I had some experience with that on the old Sparc we
>used to have so it would not be new territory.  It is not as easy for the
>new user then CDE, but it is an option.

I would be interested in your experience if you did start using OpenWindows
for your installations.

re: memory use on x86 vs Linux
>Agreed, the swap file activity really can slow the system down.

re: Linux crashes
>Actually, the recent crash that give me the opportunity to play with
>Solaris was, I believe, the result of a cracker doing the usual trash job
>to the system.  I had a chance to use a RedHat rescue disk and when I
>booted up, the partitions were just gone.  Before that, it would crash
>(from exhaustion, I suppose) about every two months.

Of the five sites that have let us know that they were hacked into,
all have been running Linux!

re: going to x86 was a good move
>Agreed.

>I got your second letter, as well, about what was done to the ldm-mcidas
>NLDN decoder and the NIDS ADDE stuff.  It is refreshing to find it was
>something that wasn't in the documentation.

To you, but not to me.

>Unfortunately, I have not had
>a chance to go down to campus yet.  I will do so this afternoon.  I am
>terribly excited about this ADDE stuff as opposed to the old NFS way of
>doing things.

It is extremely useful.  I crank up McIDAS-X at home and load data through
a 28.8 modem.  Not terribly fast or anything, but it is a LOT faster than
trying to logon and run a remote X session.

>I was testing this at home and doing some speed comparisons
>between NFS and the ADDE.  My home connection is a cablemodem, but to
>campus is terribly slow (this will change this weekend when I switch to a
>different cablemodem provider on the same backbone as Creighton so this is
>kind of a moot issue).  The ADDE system displayed the data two to three
>times faster then the NFS.

That has been our and SSEC's experience as well: ADDE can be significantly
faster than reading data on NFS mounted file systems.

>I can only imagine the impact on performance
>this will have on campus.  From what I have been able to gather, the full
>ADDE implementation will be in McIDAS 8, correct?

McIDAS-X 7.6, the one I am currently working on, is not yet complete.
The outstanding things that are missing ADDE-wise are:

o speed ups in ADDE GRID data access
o cross sections
o servers for GINI imagery in NOAAPORT
o servers for TeraScan (tm) imagery
o several other smaller items

I have been in contact with SSEC about code for GINI conversion to AREA
and got the code yesterday.  I will be creating ADDE AGET and ADIR servers
for GINI sometime in the next several weeks.  Since there will apparently
be some work on a TeraScan conversion utility, I will wait to do any
development there.  

>Also, the way things
>have been going, it looks like you are shooting for the 6-8 month time
>frame to release this.

I am looking to release 7.6 by the end of July.

>I am just trying to get a time frame to let the
>profs know what we can expect.

The current release schedule for McIDAS from SSEC is once-per-year.
I have been pushing for this for a number of years.  We have been getting
so many new releases that it has been next to impossible to do any
significant development work on the areas that we feel are lacking.
With only one release per year (bug fix updates in between), I will
be able to get to a number of the things that just had to be shelved
in the past.

>Since I have ADDE on the mind, I have one more question.  With the
>licensing restrictions on the NIDS data, what kind of security can be
>setup to keep outside connections from accessing the ADDE serer, or am I
>jumping the gun a little and this will be discussed in the next major
>release?

I have hooks in the new ADDE server code that, when flushed out, will
allow a site to define an IP mask of machines that will be allowed to
get the data.  My intention is to allow remote listing of the data,
but only allow loading of the data to selected sites.  The formalism
for this will be the IPMASK= keyword in the NIDS/NOWrad configuration
files.

>The NT stuff:
>
>I am terribly interested.  It would be nice to have a unified platform in
>the lab, specially since I will be leaving in about eight months to bigger
>and better things (i.e. graduating).

I hope that there will be some effort to spin up a new contact person
at Creighton before you leave!

>I have already notified the local
>teaching staff about the possibility and will try and talk them into
>getting the appropriate software.

Sounds good.

re: are you interested?
>I am.  I will not have any more time this week, but next week looks good.

OK.

>I am trying to talk the locals here into letting me use one of the Solaris
>machines and put NT/Interix (a Pentium 133 with 64 meg of RAM), we really
>need new machines...) on it to give it a rip.  I have this setup at home up
>and running on my PII 400, so I will give it a try there, as well.  Again,
>next week looks like a good time frame.

OK.

>What kind of feedback are you looking for?

I want the end user's perspective on the viability of using an NT platform
to run McIDAS.  What is clunky, what works nicely, what fails miserably,
etc.

I have to quickly remind you that the version of McIDAS-X for NT that
is out on the FTP site does not have any of the Unidata
additions/modifications/bugfixes.  This means that NIDS data access is
not there, nor is profiler plots/contours/cross sections, nor is NLDN
lightning, etc.  If the word back from you and Kean University (the
other site that will be poking around with the NT stuff) is that this
is very useful, then I will attempt to roll our mods into an NT
distribution.  Whether or not we can/will officially support this
platform has still not been decided, however.  I would imagine that
enthusiastic users would help sway some opinions.

Tom