[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20030408: netCDF packing

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: 20030408: netCDF packing
  • Date: Tue, 08 Apr 2003 12:47:40 -0600
>To: address@hidden
>From: Sudharshan Sathiyamoorthy <address@hidden>
>Subject: netCDF packing
>Organization: Woods Hole
>Keywords: 200304081807.h38I7V7U008801

Hi Sudharshan, 

> Hi, I read that compression and packing are not available for the current 
> release of netCDF.  However, I notice that netCDF files I obtained from 
> CDC (Climate Data Center) are much smaller than files I create using the 
> following simple script:
> 
>   ncquiet
>   nc=netcdf(filename, 'clobber');
>   nc('lat')=length(lat);
>   nc('lon')=length(lon);
>   nc('time')=t;
>   nc{'lat'}='lat';
>   nc{'lon'}='lon';
>   nc{'time'}='time';
>   nc{'nswrs'}={'time','lat','lon'};
>   nc{'lat'}.units='degrees';
>   nc{'lon'}.units='degrees';
>   nc{'time'}.units='month';
>   nc{'nswrs'}.units='W/m^2';
> 
>   nc{'lat'}(:)=lat;
>   nc{'lon'}(:)=lon;
>   nc{'nswrs'}(:)=nswrs;
>   nc=close(nc);
> 
> The size differences are quite significant: the ones I create are 4 times 
> larger than the ones from CDC (same amount of data, etc. in both cases).  
> Is there some other way I can create the netcdf file in order to have 
> smaller file sizes?

The CDC is probably packing the floating-point data into 8- or 16-bit
integers, by using the documented convention for packing with the
attributes "add_offset" and "scale_factor".  For an explanation of
their use of these attributes, see

  http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml

If you aren't seeing the packed values, you must be accessing the data
through an interface that transparently handles the unpacking whenever
the "add_offset" and "scale_factor" attributes are present.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://my.unidata.ucar.edu