[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #AMR-714212]: netcdf file size for limited vs unlimited

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #AMR-714212]: netcdf file size for limited vs unlimited
  • Date: Wed, 24 Mar 2010 13:50:32 -0600
Oscar,

> I found a poor man's solution that seems to work:
> 
> I don't know the exact dimension of the nc-file beforehand, but I can
> make a reasonable estimate of the maximum possible length of the file
> and I add some 10% to that just to be sure. Using now a dimension that
> is always a bit larger than the maximum record-length, I get nc-files
> with the same file-size as with exact-dimension information. The trade
> off, is that the processing time is a bit longer, but I much prefer that
> my files are half the size!
> 
> Any objections to this procedure?

That seems like a fine and practical solution.

> regards,.............................Oscar
> 
> 
> PS I just received your mail comparing file-sizes for different
> netcdf-versions. Netcdf-4 is not very efficient in terms of disk usage!

It can be very efficient when using compression, which is not available
with the netCDF-3 classic format.  For example, some of the CMIP5 3D ocean
model fields are reduced by more than 50% in size when stored in netCDF-4
with compression.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: AMR-714212
Department: Support netCDF
Priority: Normal
Status: Closed