Russ Rew: Re: HDF and netCDF

Forgot to CC: support-netcdf on this reply ...

------- Forwarded Message

Date:    Tue, 19 Nov 2002 13:50:56 -0700
From:    Russ Rew <address@hidden>
To:      "Ted Habermann" <address@hidden>
cc:      address@hidden, address@hidden,
         Brand Fortner <address@hidden>,
         address@hidden
Subject: Re: HDF and netCDF 

Hi Ted,

Here is a recent (February 2002) distributed visualization paper that
includes a section on data representation, which discusses CDF, HDF,
netCDF from the viewpoint of use in visualization applications:

   http://cg.its.tudelft.nl/~jan/Paper/paper.html#sect-distvispipe-data

As pointed out in the above, you have to distinguish between HDF4 and
HDF5.  HDF5 has the following strengths:
  
 - ability to directly represent multidimensional arrays of structures
 - ability to represent nested structures
 - parallel I/O using the MPI-2 standard
 - support for very large files
 - support for "chunking", also known as tiling, which permits
   efficient access along more than one dimension
 - support for packing low-precision data very efficiently

NetCDF has the following complementary strengths:

 - a simpler API than HDF5
 - support by more analysis and visualization tools than HDF5
 - in addition to C, C++, Java, and Fortran-90 interfaces (which HDF5
   has), additional interfaces supporting access from Fortran-77,
   Python, Perl, Ruby, Ada-95

Currently HDF5 is supported by a group of about 10, whereas netCDF
support has dwindled to about 1/4, so HDF5 wins on level of support.
But Unidata is requesting another developer for netCDF support in our
upcoming 5-year proposal.

Recently Mike Folk and I have submitted a proposal to NASA to provide
a netCDF-4 interface over HDF5 which would build on the strengths of
both packages.  It would also fund another developer for netCDF
work and reduce by one the number of formats you would have to
consider in determining what format to use for your scientific data
archives.  (Here's hoping you get to review it!)

--Russ



------- End of Forwarded Message