[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDFJava #GCR-931481]: Help regarding slow reading from and writing to netCDF3 files
- Subject: [netCDFJava #GCR-931481]: Help regarding slow reading from and writing to netCDF3 files
- Date: Wed, 22 Feb 2017 16:51:15 -0700
Hi Daniel,
I'm not aware of any such performance issues in the library –– we read a LOT of
large files every day and we'd have encountered that problem by now if it
existed.
For I/O performance in NetCDF, the two dominant factors are number of
operations and access pattern.
Are you minimizing the number of reads and writes? Try to make big reads and
cache the data in memory for later use.
Are you reading and writing the data sequentially, or are you jumping around
within the file, resulting in expensive seek operations? [1] It's important to
understand how data is laid out in a NetCDF file. But in short, sequential
access will always be fastest. [2]
Cheers,
Christian
[1]
http://docs.oracle.com/javase/7/docs/api/java/io/RandomAccessFile.html#seek(long)
[2]
http://www.unidata.ucar.edu/software/netcdf/docs/file_structure_and_performance.html#netcdf_3_io
Ticket Details
===================
Ticket ID: GCR-931481
Department: Support netCDF Java
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.