This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Roy, > Okay guys, here is the "example" we are working on. A long time ago > we put the World Ocean Database and the GTSPP data, all subsurface > obs that we have or get, into the old HDF-EOS point structure. The > way we had it structured was that each file was a 10-degree square > with a name that contained the box number as in COADS. The HDF-EOS > point structure had levels that made what was a series of flat tables > look like a hierarchy by having a link field. So level 0 data would > be all the box 2's in the box 10, and the level 1 data would be the > data with the box2 number as the link field. This allowed you to > search for all the data in a box2 efficiently > > A downside was that the number of rows in the field had to be equal > to the sum total of depths at which there were observations. This > meant that there is a huge amount of empty space. I understand the COADS box numbers for 10 degree or two degree boxes, but I don't understand what you're saying about levels and link fields. Is there a simpler or more detailed description of this data structure somewhere that I could study? I'd like to understand the requirements and whether netCDF-4 could eliminate the wasted space while preserving fast access. > ... So how might we > improve on this - why netcdf4! We use vlen's to deal with the > varying number of depths and a compound data type to combine together > the info. Groups and (sub)-Groups provide the same structure that > was "emulated" in the point structure. Or at least that was what we > thought would be a good design given netcdf4's features, but that is > what prompted my email about examples and design - there may well be > better ways of doing this. > > However, unless I miraculously find both money and a good C > programmer, or get Fortran wrappers to work, it will be sometime > before it happens. > > BTW, at one point I used the hdf4tohdf5 program to convert these > files to HDF5, so they can be read using that library. I don't know > that the conversion worked that cleanly to use with the HDF-EOS5 > library, but it should be possible to both read and write using the > libs that are a part of Netcdf4. I actually had a few days to begin > to work on this, hence the series of emails. > > -Roy > > > > > ********************** > "The contents of this message do not reflect any position of the U.S. > Government or NOAA." > ********************** > Roy Mendelssohn > Supervisory Operations Research Analyst > NOAA/NMFS > Environmental Research Division > Southwest Fisheries Science Center > 1352 Lighthouse Avenue > Pacific Grove, CA 93950-2097 > > e-mail: address@hidden (Note new e-mail address) > voice: (831)-648-9029 > fax: (831)-648-8440 > www: http://www.pfeg.noaa.gov/ > > "Old age and treachery will overcome youth and skill." > >