Re: 970422: NetCDF on CRAY-T3D

>To: address@hidden
>From: Martin Marc <address@hidden>
>Subject: NetCDF on CRAY-T3D
>Organization: Parallel Computing Group, University of Geneva CUI
>Keywords: 199704220804.CAA22557

Hi Marc,

>       I'm currently working on a CRAY-T3D mpp system and
>     I would like to use NetCDF (v2.4.3) on it. But the 
>     'configure' utilitly seems to bo not agree with 
>     me (see below). Maybe because the front-end computer
>     is a CRAY YMP?
> 
> 
>       I know that your don't support this system, but
>     I have read that someone have compiled netCDf for it.
>     So can you give me some adresses where I can get
>     binaries for T3D?
> 
> 
>     Marc.
> - - - - - - - - - - - - - - - - - - - - - - - - 
> It doesn't works:
> 
> 40)pascal-/t1/mmartin/netcdf-2.4.3/src$ configure
> loading cache ./config.cache
> checking for catman... no
> checking for fill-value usage... new usage
> checking the installation prefix... /t1/mmartin/netcdf-2.4.3
> checking the installation exec-prefix... /t1/mmartin/netcdf-2.4.3
> checking for c89... c89
> checking type of operating system... unicos-mpp
> checking for ranlib... ranlib
> checking how to run the C preprocessor... c89 -E
> checking the C preprocessor... works
> checking for f77... no
> checking for cf77... cf77
> checking for CC... CC
> checking for yacc... yacc
> checking for neqn... neqn
> checking for tbl... tbl
> checking for ar... ar
> checking for makeinfo... no
> checking for tar flags... -chof
> checking for dependency generation mechanism... c89 -M
> checking for C position-independent-code compile-option... 
> checking whether cross-compiling... yes
> checking size of int... configure: error: can not run test program while
> cross compiling

The last two lines indicate the configure script couldn't compile a
simple C program on this platform using the c89 compiler.  The reason
for this failure should be in the file "config.log".  You should verify
that it is possible to preprocess and compile a simple program using the
c89 compiler, otherwise set your CC environment variable to a different
compiler before invoking configure.

Here's information about netCDF on the T3D from a previous support reply:

    Steve Luzmoor of CRI (sits, or at least used to sit, at NERSC) developed
    the first version of a "par_io" library for parallel I/O and made the
    modifications to the netCDF source to make it work on a T3D.

    The current version, 2.4.3, contains the mods for the T3D that we got
    from Steve Luzmoor.  These reference functions from the Cray par_io
    library prototype that we can't distribute with netCDF, so you may have
    to obtain the current version of that library from Cray.  While we were
    integrating these changes, it seemed that the par_io interfaces were
    changing as well and more changes were planned, so it's possible that
    the current netCDF release may need some tweaking to work with the
    current par_io release.  If there are problems building the current
    release for T3D with the par_io library that CRI can't solve, we would
    want to know about them.

Alternatively, you can try the pre-release version of netCDF-3,
available as

   ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-3.3a.tar.Z

It works very well on the T90, at least according to one of our test
users, whose message about it I've appended.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu

--- Forwarded mail from address@hidden (John Sheldon)

From: address@hidden (John Sheldon)
Subject: NetCDF-3.3a performance on T90
To: address@hidden
Date: Fri, 7 Mar 1997 13:35:13 -0500 (EST)

Hi Glenn-

Sorry it took a while to get some performance numbers for you.  Cray
has a problem with their new IEEE conversion library that I was unlucky
enough to stumble into.  While they work on that, I had to work around
it...

Here are my notes regarding a pair of test codes which write/read a
netCDF file and a companion IEEE file.  As a separate option, the
"read" program can compare the data in the two files.  You'll be happy
(if unsurprised) to know that the result of that check was fine.

Notes:
---------
   Test code(s) write/read a NetCDF file and am IEEE file containing
     the same data, a mixture of 2- and 3-D data arrays with multiple
     time levels.

   File sizes = 190,516,800  IEEE
                190,521,716  NetCDF

   FFIO: cachea:512:4 in both cases (plus f77 layer and -N for IEEE case)

                         Elapsed    User CPU   Sys CPU
                         Seconds     Seconds   Seconds
                        ========== ========== ==========
    Write:
         IEEE             23.5227     2.5561     0.4676
         netCDF-2.4.2     15.1192     4.1711     0.3661
         netCDF-3.3a      11.6088     2.3618     0.3125
          (- w/ prefill)  38.2105     4.0388     0.8282

    Read:
         IEEE              9.3383     0.7726     0.3961
         netCDF-2.4.2     11.0291     1.7275     0.3222
         netCDF-3.3a       8.6275     1.4250     0.3192
---------

Note that the order of hyperslab writing/reading is simple, in order to
gauge the "best" speed capable by each method.  That means that an
entire time-slice of a variable is written/read at a time.  I haven't
yet tested writing vertically slabbed (i-k-j-t) data to a conventional
(x-y-z-t) file - that's probably more an issue of setting up the FFIO
layer correctly.

Note that there is no "pre-fill" timing for 2.4.2.  Recall that
pre-fill for that version on the T90 was a prohibitive option,
requiring approximately 100x as much time as actually writing the
data.  That seems to have been fixed in 3.3.

The IEEE-conversion write/read times above represent Cray latest and
greatest libraries (the ones that work, anyhow).

In summary, it looks like netCDF-3 writes are at least as fast as IEEE
writes, and about 40% faster than 2.4.2.  Netcdf-3 reads are within
about a factor of two of IEEE writes, and about 20% faster than 2.4.2.

Now, I must say, however, that we never did very much in the way of
using aggressive optimization when we built 2.4.2, as you did with
3.3a.  I don't know how much that may have to do with some of the
timing differences, but this is all good news in any case.

Great work!  As soon as netCDF-3 comes off beta testing, I'll be sure
to make it default here.  In the meantime, I'll make it available to
anyone who's interested.

Thanks again for all your efforts!

John


---End of forwarded mail from address@hidden (John Sheldon)