Re: 20030408: netCDF packing

>To: address@hidden
>From: Sudharshan Sathiyamoorthy <address@hidden>
>Subject: netCDF packing
>Organization: Woods Hole
>Keywords: 200304081807.h38I7V7U008801

Hi Sudharshan, 

> Hi, I read that compression and packing are not available for the current 
> release of netCDF.  However, I notice that netCDF files I obtained from 
> CDC (Climate Data Center) are much smaller than files I create using the 
> following simple script:
> 
>   ncquiet
>   nc=netcdf(filename, 'clobber');
>   nc('lat')=length(lat);
>   nc('lon')=length(lon);
>   nc('time')=t;
>   nc{'lat'}='lat';
>   nc{'lon'}='lon';
>   nc{'time'}='time';
>   nc{'nswrs'}={'time','lat','lon'};
>   nc{'lat'}.units='degrees';
>   nc{'lon'}.units='degrees';
>   nc{'time'}.units='month';
>   nc{'nswrs'}.units='W/m^2';
> 
>   nc{'lat'}(:)=lat;
>   nc{'lon'}(:)=lon;
>   nc{'nswrs'}(:)=nswrs;
>   nc=close(nc);
> 
> The size differences are quite significant: the ones I create are 4 times 
> larger than the ones from CDC (same amount of data, etc. in both cases).  
> Is there some other way I can create the netcdf file in order to have 
> smaller file sizes?

The CDC is probably packing the floating-point data into 8- or 16-bit
integers, by using the documented convention for packing with the
attributes "add_offset" and "scale_factor".  For an explanation of
their use of these attributes, see

  http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml

If you aren't seeing the packed values, you must be accessing the data
through an interface that transparently handles the unpacking whenever
the "add_offset" and "scale_factor" attributes are present.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://my.unidata.ucar.edu