[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #ISA-863293]: Behaviour of nc_def_var_deflate

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #ISA-863293]: Behaviour of nc_def_var_deflate
  • Date: Wed, 08 Nov 2017 13:52:22 -0700
Also, as an experiment, you might try applying the zip command
to the whole file and see what kind of size reduction you get.


> Hello Dinesh,
> 
> Currently, compression is passed to the 'zlib' library, which handles 
> compression.  What sort of compression ratio are you getting?  It's entirely 
> possible that the data you're working with does not lend itself to 
> compression.  You might try writing a dataset of the same size using the same 
> value for the variable across the grid.  This should lend itself very well to 
> compression, and I would expect to see a better compression ratio.  It will 
> certainly establish a 'best possible case' for what can be expected via 
> compression.
> 
> I hope this helps, have a great day!
> 
> -Ward
> 
> > Hi everyone,
> > I am developing an application using NetCDF library to read/write NetCDF
> > files. I am using version 4.3.3.1 of the library.
> >
> > I am trying to understand how the NetCDF variables are being compressed.
> >
> > I am creating a NetCDF4 file with a variable having dimensions 10000x20000
> > and datatype uint16. This is about ~381 MB.
> >
> > When I create the variable, I set the Deflate level to 5. So, when I write
> > data to the entire variable, I expect the resulting file size to be much
> > smaller than 381MB. However, this does not appear to be the case.
> >
> > Am I missing something or is this because the data I am is not very
> > correlated for the compression to do a good job?
> >
> > Dinesh
> >
> >
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: ISA-863293
Department: Support netCDF
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.