[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

netcdf-Java error reading HDF5 with large attributes

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: netcdf-Java error reading HDF5 with large attributes
  • Date: Tue, 23 Sep 2014 21:30:21 +0000
John,

A group at NASA JPL is gearing up to begin serving a new collection of Earth 
observations data next year and plan to provide HDF5 datasets. They have run 
into a problem accessing test datasets using my Panoply app, and the exception 
traces back to code in netCDF-Java’s reading of the HDF5 header.

One thing that the JPL group is doing that is out of the ordinary is that they 
are stuffing a very large XML string into a single global attribute. In some 
cases, the string is longer than 64 kB. When NJ tries to read the file header 
for those cases, a "Negative seek offset” exception gets thrown. The stack 
trace that results looks like the following...

java.io.IOException: java.io.IOException: Negative seek offset
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:425)
        at 
ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:699)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:421)
        at ucar.nc2.dataset.NetcdfDataset.acquireDataset(NetcdfDataset.java:516)
        at ucar.nc2.dataset.NetcdfDataset.acquireDataset(NetcdfDataset.java:493)
        at gov.nasa.giss.data.nc.NcDataset.initMe(NcDataset.java:107)

Caused by: java.io.IOException: Negative seek offset
        at ucar.unidata.io.RandomAccessFile.seek(RandomAccessFile.java:402)
        at ucar.nc2.iosp.hdf5.FractalHeap.readDirectBlock(FractalHeap.java:463)
        at ucar.nc2.iosp.hdf5.FractalHeap.<init>(FractalHeap.java:180)
        at 
ucar.nc2.iosp.hdf5.H5header$DataObject.processAttributeInfoMessage(H5header.java:2344)
        at ucar.nc2.iosp.hdf5.H5header$DataObject.<init>(H5header.java:2332)
        at ucar.nc2.iosp.hdf5.H5header$DataObject.<init>(H5header.java:2180)
        at ucar.nc2.iosp.hdf5.H5header.getDataObject(H5header.java:2027)
        at ucar.nc2.iosp.hdf5.H5header.access$600(H5header.java:70)
        at 
ucar.nc2.iosp.hdf5.H5header$DataObjectFacade.<init>(H5header.java:2073)
        at ucar.nc2.iosp.hdf5.H5header.readGroupNew(H5header.java:3911)
        at ucar.nc2.iosp.hdf5.H5header.access$900(H5header.java:70)
        at ucar.nc2.iosp.hdf5.H5header$H5Group.<init>(H5header.java:2153)
        at ucar.nc2.iosp.hdf5.H5header$H5Group.<init>(H5header.java:2118)
        at ucar.nc2.iosp.hdf5.H5header.readSuperBlock2(H5header.java:354)
        at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:206)
        at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:128)
        at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1521)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:813)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:422)
        ... 15 more

Note: The above comes from NJ 4.3.22. I have not yet switched up to NJ 4.5.X, 
although I plan to do so after the next Panoply release.

rbs


--
Robert B. Schmunk
Webmaster / Senior Systems Programmer
NASA Goddard Institute for Space Studies
2880 Broadway, New York, NY 10025