970230: netCDF on Cray T3E

Tony,

> From: Tony Drummond <address@hidden>
> Subject: NETCDF for CRAY T3E
> Organization: University of california Los Angeles
> Keywords: 199703210022.RAA17785 netCDF Cray T3E

In the above message, you wrote:

> I was wondering if there is a version of NETCDF
> available for the T3E?

Yes and no.  At this point in time, it depends on what -- exactly -- you
want: netCDF version 2 or 3?  FORTRAN support or not?  If you tell me 
your circumstances and environment, I might be able to suggest a course
of action.

I've enclosed some relevant email from our searchable netCDF support
email system, which you can reach at the following URL:

    <http://www.unidata.ucar.edu/packages/netcdf/index.html>

--------
Steve Emmerson   <address@hidden>

From address@hidden  Thu Mar 20 14:37:32 1997
        by unidata.ucar.edu (8.8.5/8.8.5) with SMTP id OAA14906;
        Thu, 20 Mar 1997 14:37:32 -0700 (MST)
Message-Id: <address@hidden>
To: "Al Bourgeois" <address@hidden>
Subject: Re: 970320: netCDF on Cray T3D 
In-reply-to: Your message of "Thu, 20 Mar 1997 13:38:07 MST."
Organization: UCAR Unidata Program
Date: Thu, 20 Mar 1997 14:37:31 -0700
From: Russ Rew <address@hidden>

>To: address@hidden
>From: "Al Bourgeois" <address@hidden>
>Subject: netCDF on Cray T3D
>Organization: Lockheed Martin Technical Services
>Keywords: 199703202035.NAA13072

Hi Al,

> I have an old version of netCDF that was "hacked" to work on the T3D.
> My (limited!) understanding is that this was necessary to account for the
> different representation of C floats and FORTRAN reals on the T3D.  I don't
> know the version number of the netCDF library I'm using (I don't have the
> source code), but I obtained it around October of 1995.  Is there a newer
> version you support that works on the T3D?

Yes, you're in luck.  We have a netCDF-3 pre-release version that works
very well on the T3D, at least according to one of our test users, whose
message about it I've appended.

You can get it from the directory ftp://ftp.unidata.ucar.edu/pub/netcdf/
as file netcdf-3.3a.tar.Z.  There are some release notes about netCDF-3
in http://www.unidata.ucar.edu/packages/netcdf/prerelease.html

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



--- Forwarded mail from address@hidden (John Sheldon)

From: address@hidden (John Sheldon)
Subject: NetCDF-3.3a performance on T90
To: address@hidden
Date: Fri, 7 Mar 1997 13:35:13 -0500 (EST)

Hi Glenn-

Sorry it took a while to get some performance numbers for you.  Cray
has a problem with their new IEEE conversion library that I was unlucky
enough to stumble into.  While they work on that, I had to work around
it...

Here are my notes regarding a pair of test codes which write/read a
netCDF file and a companion IEEE file.  As a separate option, the
"read" program can compare the data in the two files.  You'll be happy
(if unsurprised) to know that the result of that check was fine.

Notes:
---------
   Test code(s) write/read a NetCDF file and am IEEE file containing
     the same data, a mixture of 2- and 3-D data arrays with multiple
     time levels.

   File sizes = 190,516,800  IEEE
                190,521,716  NetCDF

   FFIO: cachea:512:4 in both cases (plus f77 layer and -N for IEEE case)

                         Elapsed    User CPU   Sys CPU
                         Seconds     Seconds   Seconds
                        ========== ========== ==========
    Write:
         IEEE             23.5227     2.5561     0.4676
         netCDF-2.4.2     15.1192     4.1711     0.3661
         netCDF-3.3a      11.6088     2.3618     0.3125
          (- w/ prefill)  38.2105     4.0388     0.8282

    Read:
         IEEE              9.3383     0.7726     0.3961
         netCDF-2.4.2     11.0291     1.7275     0.3222
         netCDF-3.3a       8.6275     1.4250     0.3192
---------

Note that the order of hyperslab writing/reading is simple, in order to
gauge the "best" speed capable by each method.  That means that an
entire time-slice of a variable is written/read at a time.  I haven't
yet tested writing vertically slabbed (i-k-j-t) data to a conventional
(x-y-z-t) file - that's probably more an issue of setting up the FFIO
layer correctly.

Note that there is no "pre-fill" timing for 2.4.2.  Recall that
pre-fill for that version on the T90 was a prohibitive option,
requiring approximately 100x as much time as actually writing the
data.  That seems to have been fixed in 3.3.

The IEEE-conversion write/read times above represent Cray latest and
greatest libraries (the ones that work, anyhow).

In summary, it looks like netCDF-3 writes are at least as fast as IEEE
writes, and about 40% faster than 2.4.2.  Netcdf-3 reads are within
about a factor of two of IEEE writes, and about 20% faster than 2.4.2.

Now, I must say, however, that we never did very much in the way of
using aggressive optimization when we built 2.4.2, as you did with
3.3a.  I don't know how much that may have to do with some of the
timing differences, but this is all good news in any case.

Great work!  As soon as netCDF-3 comes off beta testing, I'll be sure
to make it default here.  In the meantime, I'll make it available to
anyone who's interested.

Thanks again for all your efforts!

John


---End of forwarded mail from address@hidden (John Sheldon)


From address@hidden Wed Jun 12 03:28:50 1996
Message-Id: <address@hidden>
Full-Name: Russ Rew
To: Rick Light <address@hidden>
Subject: Re: NetCDF, shadow files, parallel I/O on T3D 
In-Reply-To: Your message of "Tue, 11 Jun 1996 17:16:02 MDT."              
<address@hidden>
Organization: UCAR Unidata Program
Date: Wed, 12 Jun 1996 09:28:50 -0600
From: Russ Rew <address@hidden>

Hi Rick,

> I'm about to begin working with some folks who are
> using a special version of NetCDF in which a parallel I/O
> library has been integrated into it for use on the T3D as
> a nodal I/O library.  Tim Williams is the one who
> filled me in on all of these details, says in response
> to my question as to which version of NetCDF he's using:
> 
> "... 2.4b4, I believe. The modifications to the source to make it work
> with par_io are going to be adopted by the Unidata people in future versions,
> if they haven't already.  Steve Luzmoor of CRI (sits, or at least used to sit,
> at NERSC) developed par_io and made the modifications to the netcdf source
> (very minor) to make it work."
> 
> 
> So my questions to you:
> 
>       1. do you have a "very latest" version of NetCDF that
>          has these mods in it for a Cray T3D?

The current version, 2.4.2, available as

    ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf.tar.Z

contains the mods for the T3D that we got from Steve Luzmoor.  These
reference functions from a prototype Cray par_io library that we can't
distribute with netCDF, so you may have to obtain the current version of
that library from Cray.  While we were integrating these changes, it seemed
that the par_io interfaces were changing as well and more changes were
planned, so it's possible that the current netCDF release may need some
tweaking to work with the current par_io release.  If you have problems
building the current release for T3D with the par_io library that CRI can't
solve, please let us know.

>       2. did you ever include in your master release the
>          concept we discussed of "shadow files" in which the
>          metadata is copied to a tiny local file so when
>          the completed terabyte file is written to archival
>          storage the metadata is still easily accessable?

No, although it seemed like a good idea and it's come up again recently, it
didn't make it to the top of our rather long list of what should be added
to the library, especially since we couldn't convince ourselves that it
couldn't be done above the level of the library by someone interested in
that functionality.  Currently, our top priority is adding type-conversion
interfaces that eliminate the need for void* pointers, and incorporating
packing into the external types, so users can store 11-bit data
efficiently, for example.

>          And, if so, do you have the "very latest" version
>          mentioned in 1 above with this shadowing capability built in?
> 
> If the answer to these is Yes, which I do hope is the case,
> please fill me in on how I can obtain this library.  Thanks
> very much.  I hope all is well with you.

Thanks.  Details about the current version and what's changed are in the
"what's New with netCDF" link at

    http://www.unidata.ucar.edu/packages/netcdf/whatsnew.html

______________________________________________________________________________

Russ Rew                                           UCAR Unidata Program
address@hidden                              http://www.unidata.ucar.edu