20010327: netCDF and NC_MAX_VARS=2000 limitation

Harsh,

> To: address@hidden
> cc: address@hidden,
> cc: address@hidden
> From: Harsh Anand <address@hidden>
> Subject: netCDF and NC_MAX_VARS=2000 limitation
> Organization: Lawrence berkeley National Laboratory
> Keywords: 200103270213.f2R2DwL14596

The above message contained the following:

> Some of the netCDF library users at NERSC are finding it hard to 
> work within the maximum number of variables limit (NC_MAX_VARS=2000) 
> imposed in current version of NetCDF. Is it possible to raise
> the NC_MAX_VARS limit in the future releases of NetCDF library?   
> 
> > I realize that we could locally build a version of NetCDF with a raised
> > NC_MAX_VARS, here at PPPL, but we would risk losing the binary portability
> > of our files, which is the whole motivation for using NetCDF in the first
> > place.  (We pass these files, which contain close to 2000 variables and
> > are of order 50Mbytes large, amongst collaborators, many of whom have
> > their own NetCDF installations not controlled by us).  Or, we could break
> > up our files or consolidate multiple variables into a singly named multi-
> > dimensional glob of numbers, but these types of solutions would impose 
> > lasting inconvenience on our users.
> > 
> > We think the cleanest solution (for us, at any rate) would be if NetCDF
> > would allow a higher limit on the total number of variables per file.
> >
> > At the level we are using it (aside from this NC_MAX_VARS limit which
> > has tripped us only once, recently), we find the robustness, portability,
> > and performance aspects of NetCDF all to be excellent.  But we expect,
> > with the development of our code, for NC_MAX_VARS=2000 to become an
> > intolerable limitation at some point in the not too distant future.
> 
> Thanks.
> 
> Harsh Anand
> 
>  Harsh Anand ----------------------------------   address@hidden  -|
>  NERSC User Services                      phone:  (510) 486-5575   |
>  Lawrence Berkeley Lab                      Fax:  (510) 486-7202   |
>  Univ. of California, Berkeley ------------------------------------

We'll have to discuss it.

I think we could do it tomorrow if not for the Fortran-77 interface.
I believe the problem is that that interface allocates static storage
based on NF_MAX_VARS (the Fortran equivalent to NC_MAX_VARS).  Raising
it might put an unreasonable and unnecessary burder on some programs.

I'll discuss it with Russ when he returns from vacation.

Regards,
Steve Emmerson   <http://www.unidata.ucar.edu>