This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
>Organization: MIT Lincoln Laboratory >Keywords: 199509261903.AA27182 Hi Jennifer , > I've been searching the archives and haven't found a definitive answer > to the following question. My apologies if it's been asked before. > > Is it possible to create and write a netCDF file directly to a mag tape? In general, no, because netCDF is a direct-access I/O library, which means it must be able to seek to fairly arbitrary locations in a file to write or read data. In particular, netCDF permits you to write an array in a different order from which it is read. For example, you can write a matrix by columns and read it back by rows. The direct-access (sometimes called "random-access") features of the interface permit you to efficiently access small subsets of data from very large datasets, and in particular to read the n-th record of a record variable without first reading through the first n-1 records. If you have a tape device that permits using the "lseek()" call to position to an arbitrary byte boundary (subject to alignment constraints, which really means seeks are restricted to 4-byte boundaries), then netCDF can be implemented to read and write it directly. Most tape drives cannot do anything like this, they can only seek to some sort of tape record boundaries, which are typically much larger. > If so, is it generally a bad idea? The general concensus around here is > that the header isn't necessarily written before the data begins, so it'd > be difficult, if not impossible, but we'd like to hear the answer from > the horse's mouth, as it were... That's another consideration that makes it a bad idea. NetCDF files are appendable, which means you can open an existing file and add some more data to variables that use the unlimited dimension without recopying the entire file. However when you do this, it not only lengthens the file, but also updates the current size of the unlimited dimension, which is located in the header. Since the header is at the front of the file, this would involve rewinding the tape to update this number in the header after each write of a record at the end, which would make for fairly inefficient tape access. It's better to read and write netCDF files on a device capable of direct access, like a disk, and then stage them from and to magnetic tape before or after you access the data. --Russ "The Horse's Mouth" Rew ______________________________________________________________________________ Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu