Re: 20030903:NCL and NC_MAX_VARS

>To: address@hidden
>cc: address@hidden
>From: Mary Haley <address@hidden>
>Subject: Re: NCL and NC_MAX_VARS
>Organization: UCAR/Unidata
>Keywords: 200309031508.h83F8SLd024439

Hi Bernd and Mary,

> installing and scripting NCL (ncl-4.2.0.a030.Solaris9_sun4.tar.gz)
> for our netcdf-files was easy, but since NCL seems to use the netcdf
> default of NC_MAX_VARS == 2048, I cannot reach all the variables in
> our files:
> 
> 'fatal:The file (xxx.nc) contains (3714) variable which  exceeds the number 
> of allowable variables (2048)'
> 
> Are there chances, that for a later NCL build NC_MAX_VARS could be >
> 2048, i.e. a netcdf.h like
> 
> #define NC_MAX_VARS   9999     /* max variables per file */ ?
> 
> May be not, because of compatibility issues to software using NCL,
> but I do not want to pass up the opportunity to ask.

It is generally OK to bump this limit up to as large as you need.  We
have increased it to 4096 in the beta release of netCDF-3.5.1.  The
only reason for not making it huge in the library (or unlimited) is
that some Fortran application programs and utilities that are designed
to handle any netCDF data may allocate static arrays of this size.  As
it says in the comments in netcdf.h:

  /*
   * These maximums are enforced by the interface, to facilitate writing
   * applications and utilities.  However, nothing is statically allocated to
   * these sizes internally.
   */
  #define NC_MAX_DIMS   512      /* max dimensions per file */
  #define NC_MAX_ATTRS  4096     /* max global or per variable attributes */
  #define NC_MAX_VARS   4096     /* max variables per file */
  #define NC_MAX_NAME   128      /* max length of a name */
  #define NC_MAX_VAR_DIMS       NC_MAX_DIMS /* max per variable dimensions */

The only negative implication of having a larger limit than is in the
released library version in netcdf.h (or the Fortran version
netcdf.inc), is that this makes it possible to create netCDF datasets
that are not completely portable, because exceeding the advisory
limits makes files that can only be accessed by versions of the
library compiled with the larger limits.  Also third party software,
such as IDL, has the smaller limits compiled in, so your datasets that
exceed the advisory limits cannot be accessed completely in such
applications.  But note that this is already a problem for earlier
versions of the netCDF library, which had smaller limits, so the
portability issue may be minor, depending on the applications you want
to use.

I would say go ahead and bump the limit up to at least 4096 (which is
in the current beta release) and to a larger number if you need it.
Note that although NC_MAX_ATTRS has the same value as NC_MAX_VARS,
there is no reason these have to be the same, so you could leave
NC_MAX_ATTRS at 4096.

Our current design for mapping netCDF-3 datasets to HDF5 datasets does
depend on no more than 9999 variables per netCDF dataset, so we may
increase the advisory limit to 9999 before releasing netCDF 3.5.1 ...

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://my.unidata.ucar.edu