[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

netCDF performance

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: netCDF performance
  • Date: Wed, 20 Mar 2002 13:39:38 -0700
Hi Ann,

You wrote:
 ...
>     The UCAR Software product netCDF is used for archiving  model
>     output products and we are currently investigating ways to improve
> the I/O throughput from netCDF to disk, where there are filesystems
> with fixed yet differing cluster sizes. I would like to find out about
> any similar work that may have already occured with the development
> team at UCAR.

Some work was done in the past on tuning netCDF I/O for FFIO on Crays,
and the results are described in a section on Unicos optimization:

  
http://www.unidata.ucar.edu/packages/netcdf/f90/Documentation/f90-html-docs/guide9.html#1005235

Some work has also been done on parallelizing netCDF for Cray T3E
platforms:

  http://hpcf.nersc.gov/software/libs/io/netcdf/pusage.html

We also have a plan for parallelizing netCDF I/O using MPI-I/O by
layering the next version of netCDF on top of HDF:

  http://www.unidata.ucar.edu/proposals/NetCDFHDF/description.html

but haven't yet identified resources to carry out this work.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu