Re: 950913: Query on behalf of a number of people re: CRI machines & beta NetCDF

>Organization: National Energy Research Supercomputer Center
>Keywords: 199509132004.AA06993

Hi Tim,

> This is an inquiry about the status of the 2.4 beta release of NetCDF.
> The URL about "The NetCDF Project," in discussing changes in release 2.4,
> mentions Cray-specific optimizations from Jeff Kuehn.  We and a number of
> other people at various institutions I know about have been after improved
> NetCDF performance on the C90 and other CRI machines for years now; we are
> anxious to see the sources and try the beta version including the
> Cray-specific optimizations.  What is the status and (future?)
> availability of the beta release?

The Cray-specific optimizations have been integrated into the the alpha-test
version of release 2.4.  The remaining tasks that are delaying the
beta-release of version 2.4 are

 1.  Getting some library modifications for Windows NT that a user has
     developed, so the release would support that platform.  (If we don't
     get these in another week, we won't hold up the release for them, but
     would just wait for a later release).

 2.  Making a final pass through the documentation, updating everything
     that's out-of-date and fixing the Fortran examples.  This includes
     using the latest texinfo so we can embed HTML in the documentation for
     use by the texi2html filter.

 3.  Add ncrec functions to the C++ interface.  Figure out what to do about
     the C++ interface documentation.

 4.  Fix four remaining ncgen bugs.

I'm working on all of these now.  Our work has been delayed by an unusually
high load of netCDF support questions, by a need to prepare and give a
presentation on netCDF status and plans, and by vacations.  My current
estimate is that we'll have a beta test release by the end of next week,
barring any unanticipated problems.  Since unanticipated problems always
occur, it might be better to bet on 29 September for the announcement.

> In a related matter: A Cray analyst here at NERSC has built a parallel I/O
> layer for the T3D machine and built a couple of higher-level I/O libraries,
> including NetCDF, on top of it.  I have found this to work well in my physics
> simulation code, which also runs on C90 vector machines.  Having efficient
> NetCDF implementations on both platforms would be a *major* boost to my work.
>  The CRI analyst here, in addition to creating the parallel I/O underlayer,
> also made some simple changes to the NetCDF sources that improved scalar
> performance on each node of the T3D trememdously, by avoiding the
> one-word-at-a-time data transfers done by the unmodified NetCDF sources.  We
> presume that these changes are along the lines of Jeff Kuehn's optimizations
> for the Cray vector machines, and would like very much to compare with those
> and adopt Jeff's version, since it will part of the standard NetCDF sources
> beginning with 2.4.  For this purpose, then, access to the 2.4 beta release
> would be extremely useful to us.

The Cray optimizations were accomplished by implementing XDR on the CRI FFIO
file interface.  Users can tune the buffering by changing an environment
variable.  This implementation was all accomplished in a single file,
xdrffio.c, that replaces the xdrposix.c.  If it would help, I can send a
copy of xdrffio.c (it's only about 6K bytes).

______________________________________________________________________________

Russ Rew                                           UCAR Unidata Program
address@hidden                              http://www.unidata.ucar.edu