Re: 970320: netCDF on Cray T3D

>To: address@hidden
>From: "Al Bourgeois" <address@hidden>
>Subject: netCDF on Cray T3D
>Organization: Lockheed Martin Technical Services
>Keywords: 199703202035.NAA13072

Hi Al,

> I have an old version of netCDF that was "hacked" to work on the T3D.
> My (limited!) understanding is that this was necessary to account for the
> different representation of C floats and FORTRAN reals on the T3D.  I don't
> know the version number of the netCDF library I'm using (I don't have the
> source code), but I obtained it around October of 1995.  Is there a newer
> version you support that works on the T3D?

Yes, you're in luck.  We have a netCDF-3 pre-release version that works
very well on the T3D, at least according to one of our test users, whose
message about it I've appended.

You can get it from the directory ftp://ftp.unidata.ucar.edu/pub/netcdf/
as file netcdf-3.3a.tar.Z.  There are some release notes about netCDF-3
in http://www.unidata.ucar.edu/packages/netcdf/prerelease.html

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



--- Forwarded mail from address@hidden (John Sheldon)

From: address@hidden (John Sheldon)
Subject: NetCDF-3.3a performance on T90
To: address@hidden
Date: Fri, 7 Mar 1997 13:35:13 -0500 (EST)

Hi Glenn-

Sorry it took a while to get some performance numbers for you.  Cray
has a problem with their new IEEE conversion library that I was unlucky
enough to stumble into.  While they work on that, I had to work around
it...

Here are my notes regarding a pair of test codes which write/read a
netCDF file and a companion IEEE file.  As a separate option, the
"read" program can compare the data in the two files.  You'll be happy
(if unsurprised) to know that the result of that check was fine.

Notes:
---------
   Test code(s) write/read a NetCDF file and am IEEE file containing
     the same data, a mixture of 2- and 3-D data arrays with multiple
     time levels.

   File sizes = 190,516,800  IEEE
                190,521,716  NetCDF

   FFIO: cachea:512:4 in both cases (plus f77 layer and -N for IEEE case)

                         Elapsed    User CPU   Sys CPU
                         Seconds     Seconds   Seconds
                        ========== ========== ==========
    Write:
         IEEE             23.5227     2.5561     0.4676
         netCDF-2.4.2     15.1192     4.1711     0.3661
         netCDF-3.3a      11.6088     2.3618     0.3125
          (- w/ prefill)  38.2105     4.0388     0.8282

    Read:
         IEEE              9.3383     0.7726     0.3961
         netCDF-2.4.2     11.0291     1.7275     0.3222
         netCDF-3.3a       8.6275     1.4250     0.3192
---------

Note that the order of hyperslab writing/reading is simple, in order to
gauge the "best" speed capable by each method.  That means that an
entire time-slice of a variable is written/read at a time.  I haven't
yet tested writing vertically slabbed (i-k-j-t) data to a conventional
(x-y-z-t) file - that's probably more an issue of setting up the FFIO
layer correctly.

Note that there is no "pre-fill" timing for 2.4.2.  Recall that
pre-fill for that version on the T90 was a prohibitive option,
requiring approximately 100x as much time as actually writing the
data.  That seems to have been fixed in 3.3.

The IEEE-conversion write/read times above represent Cray latest and
greatest libraries (the ones that work, anyhow).

In summary, it looks like netCDF-3 writes are at least as fast as IEEE
writes, and about 40% faster than 2.4.2.  Netcdf-3 reads are within
about a factor of two of IEEE writes, and about 20% faster than 2.4.2.

Now, I must say, however, that we never did very much in the way of
using aggressive optimization when we built 2.4.2, as you did with
3.3a.  I don't know how much that may have to do with some of the
timing differences, but this is all good news in any case.

Great work!  As soon as netCDF-3 comes off beta testing, I'll be sure
to make it default here.  In the meantime, I'll make it available to
anyone who's interested.

Thanks again for all your efforts!

John


---End of forwarded mail from address@hidden (John Sheldon)