Re: 20020805: Modifying netCDF headers

>To: address@hidden
>From: address@hidden
>Subject: Modifying netCDF headers
>Organization: UCAR/Unidata
>Keywords: 200208051025.g75APX925711

Hi Kek,

> What's the easiest (least resource consuming) way of editing the header
> of a netCDF 3.5 file?  i.e. the stuff you get from 'ncdump -h'.
> 
> Currently I "ncdump" the whole file, edit the cdl file, then 'ncgen'.
> This is fine for a relatively small file, but not so convenient for a
> 300MB file.
> 
> Thanks for any pointers.

The netCDF file format allows some changes to the header that can be
done in place, such as changing the names of dimensions, variables,
and attributes, but I assume you want to know how to handle more
significant changes to the structure of the netCDF dataset, such as
adding new dimensions and variables.

You didn't say which language interface you're using, so I'll assume
the C interface.  If you call nc_redef after opening a netCDF
dataset, you can add new dimensions, variables, and attributes, and
when you later call nc_enddef, everything gets written to the
dataset.  The catch is that if the new header information takes more
space than was available in the original netCDF file, all the data is
copied from the old dataset to a new one, which can be time consuming.

If you can anticipate that you will be adding to the header
information, you can make sure this is done efficiently by reserving
extra space in the header when you first create the file, or after you
add to the header.  You do this by calling the "underbar underbar"
functions nc__create or nc__enddef, which are documented in the man
page reference documentation. but not yet in the User Guides (except
for the Fortran 90 User Guide, which does document them).  Here is the
C man page:

  http://www.unidata.ucar.edu/cgi-bin/man-cgi?netcdf+-s3

By using these functions, you can sometimes just change the header in
place with no data copying.  However, copying data will still be
required if you add a new record variable or change the shape of an
existing variable, an operation not supported by the netCDF interface.

If your header change requires something like changing the shape of a
variable, I don't know of any simpler or more efficient way of
doing this for large files than creating the new file with changed
headers using ncgen, then writing a program that reads and copies the
data from the old file to the new file.  There are some library
functions such as the documented nc_copy_att() to copy all the values
of an attribute and the undocumented nc_copy_var() to copy all the
values of a variable that might be of some help.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu