[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #NLS-992754]: gbx[8] index files don't speed up subsetting

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDFJava #NLS-992754]: gbx[8] index files don't speed up subsetting
  • Date: Mon, 17 May 2010 15:44:28 -0600
Tom,

The gbx8 index files contain the metadata for the Grib files including the 
pointers to the data offsets in the files. 

Here's the detailed definition of what's in a index file:
http://www.unidata.ucar.edu/software/decoders/grib/javadoc/IndexFormat.txt
It's a little out of date but it's the basic idea.  I'll fix the document soon. 

When a Grib file is opened as a NetcdfFile() only the index gbx8 file is read. 
For a data request, the data location is obtained from the index, seek to data 
offset in the Grib file, read/return the data. The index gbx8 can be used again 
and again without reverting back to the Grib file. Reading a large gigabyte 
Grib file can be expensive and time consuming especially if your only looking 
for one parameter and the process is repeated. If you delete the index file, it 
will be recreated the next time you open the Grib file. 

It might be a benefit to create the index files before there are any data 
requests. On our motherlode TDS data server there's an external process to make 
the index files so there is no delay caused by creating the indexes. The 
external program is called ucar.nc2.iosp.grib.GribBinaryIndexer(). If you want, 
I'll send you the details about setup and configurations.

Robb...
 

Ticket Details
===================
Ticket ID: NLS-992754
Department: Support netCDF Java
Priority: High
Status: Open