This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
>From: "Reggie Chang" <address@hidden> >Subject: file size >Organization: ZONA Technology >Keywords: 200107310123.f6V1NH117343 netCDF file size Reggie, > Yesterday I also figured out the cause of the larger > size file written in NetCDF format. As you mentioned > in the email, all the items with unlimited declared will be > extended to the maximum 'time'. > > I've been wondering if there is a way to get around with > that. Since there is only one unlimited dimension in a > NetCDF file, it would not be possible to introduce another > unlimited dimension. However, can one change the 'fixed' > dimension of an array as the program execution continues? > (Say change 100x100xN to 100x100x2N later) Or is it > possible to make the second array bigger to contain earlier > array and then remove the first written array. (I checked > the User's Guide but only saw nf_del_att with the delete > feature.) No, sorry, netCDF avoids any kind of garbage collection in order to keep the format and access code simple and fast. If data were moved around in the file to fill up holes created by arrays dynamically growing and shrinking, netCDF would have to be more like a database system than a small interface for array-oriented data access. Issues of free space management, fragmentation, and algorithms for efficient coalescing of adjacent free blocks would all be involved. HDF5 from NCSA comes closer to permitting that kind of flexibility, at the cost of a more complex interface and implementation. I think it permits multiple resizable dimensions and supports more sophisticated data structures. If you really need to be able to grow data long any dimension efficiently, you probably should look at HDF5. If we had the resources, we would like to eventually implement netCDF 4 using something like the HDF 5 format to support multiple resizable dimensions, parallelism, larger files, tile/chunked access, and packed data. --Russ _____________________________________________________________________ Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu