[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #WNT-807985]: deflation question/clarification

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #WNT-807985]: deflation question/clarification
  • Date: Thu, 05 Dec 2013 14:36:39 -0700
Hi Charlie,

> A user wants to uncompress their files with NCO and setting the
> deflation level to zero works yet the variables are still listed
> as having the deflation filter set. Now I make NCO manually turn-off
> the deflate flag when the deflation level is zero yet I'm wondering
> why the netCDF library does not do this automatically when I call
> var_deflate() with level 0. This makes me think I don't fully
> understand what deflation level equals zero means. And I'm wary
> of introducing bugs through naïvete.

I'm CC:ing hdf5 support, in case they have a correction for my
understanding of why deflation level 0 is distinguished from
uncompressed.

I think netCDF-4 is following HDF5's meaning for deflate level 0.  The
HDF5 reference documentation for H5P_set_deflate() says:

    Note that setting the compression level to 0 (zero) does not turn 
    off use of the gzip filter; it simply sets the filter to perform no
    compression as it processes the data.

> Is there any distinction between uncompressed variables and variables
> with compression set to level zero?

I think this means that the variable must still use chunked rather than 
contiguous layout, whereas an uncompressed variable could use contiguous 
layout.

> Are such variables fully uncompressed?

Yes.

> Perhaps this could be indicated in the manual.

Right, if hdf5 support confirms my understanding, I'll make this 
difference explicit in the documentation.

> Is there a reason not to unset the deflation flag when deflation
> level is zero?

I'll pass the buck to HDF5 support, in case they know a good reason 
why the deflation property is set for data "compressed" with deflation 
level 0.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: WNT-807985
Department: Support netCDF
Priority: Normal
Status: Closed