[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #VKC-963616]: [netcdf-hdf] unexpected chunking behavior

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #VKC-963616]: [netcdf-hdf] unexpected chunking behavior
  • Date: Fri, 12 Jun 2009 13:08:47 -0600
Hi Charlie,

(I meant to send this support response rather than a private email.)

Ed is away for a couple of weeks, and he's the one best able to answer 
your question.  It might be helpful if you can provide us with a simple 
example of a program that creates a chunked netCDF-4 file fails with a 
code -101 HDF error.  

If that's not convenient, then it would be helpful to get the actual HDF
error, which you can do by configuring like this:

  ./configure --enable-logging --enable-netcdf-4 --disable-f77 --disable-cxx 
--with-hdf5=...
  make check

Then if your test fails, insert the following line:

  nc_set_log_level(5);

in your test case, before the point where the failure occurs. This will
produce some log output telling us what is going on.

Thanks!

--Russ

> I have implemented access to the netCDF4 chunking features in NCO.
> The "all or nothing" approach---chunking all the variables or none---
> usually works as expected. However, "moderately chunked" approaches---
> where, say, only the 3D variables are chunked, or only the variables
> containing a certain dimension---often fail in the HDF library
> (error code -101 = "HDF error"). Also, unchunking a completely chunked
> dataset fails.
> 
> As I understand it, the chunking of one variable is completely
> independent of the chunking of the rest of the variables, i.e.,
> netCDF4 imposes no special restrictions on chunking (or not)
> coordinate variables, record variables, etc.
> To me, this indenpendence means that if I chunk one variable that
> contains a certain dimension, "longitude" say, then I do not have
> to chunk all the other variables that contain "longitude".
> My understanding is that I can chunk (or not) any variables I like
> as long as each dimension's chunk size is <= the dimension size.
> Are there any chunking constraints/limitations present in the HDF
> layer that might cause the symptoms I encounter?
> 
> As important, I do (sometimes) call nc_def_var_chunking() multiple
> times for the same variable before an nc_enddef(). I assume this is OK
> and that each variable will be written with the parameters set for it
> in the last nc_def_var_chunking() call before nc_enddef().
> 
> Thanks,
> Charlie
> --
> Charlie Zender, Department of Earth System Science
> University of California, Irvine (949) 824-2987 :)
> 
> _______________________________________________
> netcdf-hdf mailing list
> address@hidden
> For list information or to unsubscribe, visit: 
> http://www.unidata.ucar.edu/mailing_lists/
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: VKC-963616
Department: Support netCDF
Priority: Normal
Status: Closed