[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #ZFB-587742]: Reading variable with strides very slow

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #ZFB-587742]: Reading variable with strides very slow
  • Date: Fri, 09 Aug 2013 10:49:42 -0600
Patrick-

Vars in netcdf is inherently slow
(when stride > 1) because it cannot
easily make use of bulk read operations.
So the library must read element by element
from the underlying disk storage. This has
a noticeable effect on performance. This is not
easy to fix because it must do the read using only
the memory that is passed to it by the client. 

For netcdf versions before 4.3.0 (including 4.1.3)
there was an additional factor. For historical
reasons, vars was implemented in terms of varm
so there was some additional overhead.

If you upgrade to 4.3.0, you will see some performance
improvement but not, probably, enough to solve your problem.

Sorry I do not have better news.
=Dennis Heimbigner
 Unidata
=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: ZFB-587742
Department: Support netCDF
Priority: Normal
Status: Closed