This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
> Hi Steve, > > Thanks for the binary you sent me. The good news is it works. The bad news is > the new version of netcdf uses lots of memory compare to the old version we have > here. The high consumption of memory causes the application crashed due to run > out of memory. Do you have any ideas why the new version uses so much memory? > > Thanks. > David Wong David: The only significant memory allocation done directly by the netcdf is for an i/o buffer. For CRAY systems (using ffio), we get a default size for this from the system using a call to fffcntl(). We use the value returned in the st_oblksize field of struct ffc_stat_s. It could well be that this is an absurd number. Talk it over with your Cray experts. See libsrc/ffio.c:blksize(). The good news is that you can override the default using the "underbar underbar" versions of the create or open function. See NF__OPEN in the fortran man page. The 'chunksize' parameter sets the size of this allocation. One can tune this to be the size appropriate. It is also probable that large allocations are made on behalf of the netcdf library by the ffio layer. I think you have control of this via the NETCDF_FFIOSPEC environment variable. (On the T3E using the latest NERSC modifications, this can also be controlled by an assign statement.) Hope this helps. -glenn >From address@hidden Fri Jan 22 10:37:38 1999 From: Al Bourgeois <address@hidden> Date: Fri, 22 Jan 1999 12:37:33 -0500 Organization: Lockheed Martin To: "Glenn P. Davis" <address@hidden> CC: "David C. Wong" <address@hidden>, address@hidden, Steve Emmerson <address@hidden>, address@hidden Subject: Re: T3D / T3E netcdf memory usage References: <address@hidden> <address@hidden> It appears that our problem is solved on both the T3D and T3E by setting the NETCDF_FFIOSPEC variable, as you suggested below. For example, with a little experimentation, we found that a setting of "cachea:1:50" speeded up NCVGT (for a specific data set) by a factor of 200 on both the T3E and T3D. Thanks for the help. -Al