[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #VKC-963616]: [netcdf-hdf] unexpected chunking behavior

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #VKC-963616]: [netcdf-hdf] unexpected chunking behavior
  • Date: Wed, 17 Jun 2009 07:40:23 -0600
> Hi All,
> 
> Here are a simple command from a non-simple program, and the
> messages/errors that it produces. The command tries to create
> and output file with a single variable (time), copied from the
> input file. The input file is completely unchunked, and the
> command tries to explicitly turn off chunking on each variable
> before it calls nc_enddef() on the output file.
> 
> ncks -O -4 -D 4 -v time --cnk_plc=uck ~/nco/data/in.nc ~/foo.nc
> 
> So, at its core, the command calls
> 
> nc_def_var_chunking(nc_id,var_idx,srg_typ,cnk_sz);
> 
> with srg_typ=NC_CONTIGUOUS and cnk_sz=NULL once for "time".
> Then it calls nc_enddef(). Then it fails with the messages below.
> Messages from other unexpected failures (whether chunking
> or unchunking) are quite similar.
> 
> Any insight appreciated!
> 
> Charlie
> 
> zender@givre:~$ ncks -O -4 -D 4 -v time --cnk_plc=uck ~/nco/data/in.nc
> ~/foo.nc
> ncks: CONVENTION File "Conventions" attribute is "CF-1.0"
> 

Howdy Charlie!

I just got back from vacation... ;-)

Your understanding of how chunking should work is correct. Any variable may be 
chunked or contiguous without reference to the settings of other variables.

I have written and run a quick test which shows that I can create a file with 
lots of variables, and turn on chunking for every other variable. This works 
for me, but I did detect one bug...

When vars are created, they are contiguous by default, if they have no 
compression, checksum, or unlimited dimensions.

The bug is that nc_inq_chunking fails to report this until after the first 
enddef.

I am fixing this - I'll let you know when the fix appears in the daily snapshot.

But could this be the cause of your problem? It seems unlikely. I will continue 
to investigate this.

Thanks,

Ed  


Ticket Details
===================
Ticket ID: VKC-963616
Department: Support netCDF
Priority: Normal
Status: Open