This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
The latest release of netcdf java (http://www.unidata.ucar.edu/packages/netcdf-java/) has a contributed class InMemRandomAccessFile.java which does what you want. We havent finished integrating it by adding a NetcdfFile constuctor for it, but that wouldnt be hard (see public NetcdfFile(URL url)). I could add that if you are interested, and willing to test it. ----- Original Message ----- From: "Russ Rew" <address@hidden> To: "ahoward" <address@hidden> Sent: Monday, April 01, 2002 4:47 PM Subject: Re: in memory netcdf > >To: address@hidden > >From: ahoward <address@hidden> > >Subject: Re: 20020327: in memory netcdf > >Organization: NOAA/FSL > > Hi Ara, > > > don't know if you recall my name, but i was one of the early users of your > > netcdf java library a few years back. in any case, i've switched > > departments here at fsl and am now in the ITS group responsible for the > > realtime data ingest. we're currently in the process of revamping our > > systems design and implementation language (c -->> c++). i'm working on > > generic handling of point data and need a good general data structure in > > which to store all point data for eventual munging into output files of > > either netcdf or grib format. netcdf itself would be an ideal data > > structure as the metadata contained in the file is complete enough for my > > purposes (with some creative uses of attributes) and the typed accessor > > methods (c++ interface) are exactly the type of thing needed. > > > > my problem is that any use of netcdf requires a disk hit to read in an > > existing cdf (output of ncgen) and each subsequent put/get also hits disk. > > i'm looking into replacing the IO layer (posixio.c) with a layer that > > operates entirely in memory. that would leave only the hit to read in an > > existing template (i'm also planning on looking at ncgen to see if it > > could be modified to place a netcdf file structure in memory instead of on > > disk). eg. i want to eliminate most, if not all, of the disk accesses in > > the c/c++ netcdf api. > > > > alternatively, applying the same set of requirements, and availibility of > > a constructor to the NcVar class might prove sufficient for our purposes. > > > > my questions are : > > > > * do you know of any work along these lines, 3rd party or > > otherwise? > > Andrew Janke <address@hidden> proposed implementing something > similar in the Java interface, which you can read about in this reply > from John Caron to his question: > > http://www.unidata.ucar.edu/cgi-bin/mfs/70/4439 > > Glenn Davis also once implemented an experimental memory-mapped > version of the i/o layer for netCDF 3.4 built on top of Unix mmap(2), > which is described here: > > http://www.unidata.ucar.edu/cgi-bin/mfs/70/2756 > > and available in this file: > > ftp://ftp.unidata.ucar.edu/pub/netcdf/contrib/mmapio.c > > that saves some disk I/O. > > > * am i completely off base in thinking the IO layer could be > > replaced with an in memory IO layer > > > > any input appreciated, i was planning on cc'ing john caron on this as well > > but have lost his email. > > The I/O layer could be replaced with an in-memory layer. But if you > just want to access a small slice of data from a huge file, using > random-access to read from the disk is cheaper than reading the whole > file into memory first before you try to access the data slice from > the in-memory copy. But it sounds like that's not a concern for your > application. > > By the way, Charlie Zender has developed a new C++ interface to > netCDF, known as libnco_c++, which he describes in this netcdfgroup > posting: > > http://www.unidata.ucar.edu/glimpse/netcdfgroup-list/1484 > > which emphasizes easy migration from the C interface ... > > --Russ >