[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDF #VRU-236841]: Re: Known problem URL for large block size, silent file corruption
- Subject: [netCDF #VRU-236841]: Re: Known problem URL for large block size, silent file corruption
- Date: Thu, 28 Apr 2011 10:19:52 -0600
Hi Charlie,
> Sounds like you are close to a fix for this nasty bug.
> I'll test your fix on cisl bluefire and mirage, if you want.
> And I'll wait awhile until releasing 4.0.8.
Just to keep you appised of progress, I've checked in a fix to our svn trunk,
consisting of a 20-line addition to the libsrc/posixio.c code. The conditions
for the bug appear to be pretty rare, but are more likely with larger disk
block sizes. Examples of the bug with small disk block sizes require
relatively small files and involve:
- writing data to a file in nofill mode
- writing more than one disk-block beyond the end of the file, as might
happen in writing the last slice of a multidimensional variable before
writing other slices
- crossing disk-block boundaries with the region to be written
- having the in-memory buffer in a state in which the region to be written
corresponds to the upper half of the buffer and recently written data in
the lower half of the buffer hasn't been flushed to disk yet.
The last condition makes it difficult to give users an easy way to determine
whether they have been a victim of this problem. I'm still struggling with
a better description of the conditions under which it might occur, and I still
need to understand why we can duplicate the problem for 4K disk blocks if we
use the double-underbar function nc__create(), but not if we use the more
common nc_create().
When I have that mystery solved, I should be able to send out a netcdfgroup
posting, and maybe create an FAQ or blog entry about the bug with more
information than people are likely to want to read in an email posting.
--Russ
would
--Russ
Russ Rew UCAR Unidata Program
address@hidden http://www.unidata.ucar.edu
Ticket Details
===================
Ticket ID: VRU-236841
Department: Support netCDF
Priority: Normal
Status: Closed