[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #NNS-961385]: nf90_put_var slows down erratically

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #NNS-961385]: nf90_put_var slows down erratically
  • Date: Tue, 08 Jan 2008 22:58:48 -0700
Hi David,

I was able to duplicate your results with the test program.  At first
I thought the problem was not explicitly specifying the "count"
parameter to the nf90_put_var() call, but it looks like the default is
actually OK for that.  However, you might want to specify the "count"
parameter explicitly for future maintainers who have to figure out
what's happening.

The inconsistency in timing has nothing to do with the f90 interface,
as I was able to translate your example into C and get the same sort
of results just using the C interface.  The anomalies you are seeing
seem to be due to a pattern of disk writes that defeats common caching
strategies, combined with running on the edge of using all of the
available cache memory, so that if anything else happens on your
system that uses memory, you get a cache miss for each of the 20
variables, rather than usually getting a cache hit for each of them.

I'm still studying the problem to see if there's anything you can do.
I just verified that turning off fill mode doesn't seem to help.

I'd also like to see if using netCDF-4's chunking parameters would
speed up the I/O.

I've also noticed that our Solaris ZFS file system runs your example
more than 20 times faster than just writing to local disk on the same
machine (1m43s versus 38m44s), which is a much better speedup than we
have seen on other programs.

--Russ


Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: NNS-961385
Department: Support netCDF
Priority: Normal
Status: Closed