[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #GCR-931481]: Help regarding slow reading from and writing to netCDF3 files

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDFJava #GCR-931481]: Help regarding slow reading from and writing to netCDF3 files
  • Date: Thu, 23 Feb 2017 15:19:46 -0700
Hi Daniel,

In NetCDF, multi-dimensional arrays are laid out in row-major order. For good 
performance, it's crucial to operate on those arrays in a row-oriented manner. 
I suspect you're accessing them in a column-oriented manner, which is the cause 
of your trouble. This article [1] gives an excellent explanation of 
column-major vs row-major ordering, and the associated performance 
implications. I highly suggest you read the whole thing.

So, operate on the arrays in a row-oriented manner if possible. Sometimes, a 
particular problem will require you do column-oriented access instead. In those 
situations, there's not much you can do to improve performance, except to read 
the entire array into memory, as I mentioned before.

Good luck,
Christian

[1] http://eli.thegreenplace.net/2015/memory-layout-of-multi-dimensional-arrays

Ticket Details
===================
Ticket ID: GCR-931481
Department: Support netCDF Java
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.