[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #FJB-731534]: ncks produces (possibly) invalid HDF-5 file.

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #FJB-731534]: ncks produces (possibly) invalid HDF-5 file.
  • Date: Wed, 05 Nov 2014 08:38:54 -0700
John,

> Antonio has reported this problem to netcdf-java email list, referencing
> NCO conversation, reproduced below, also at
> 
> http://sourceforge.net/p/nco/discussion/9829/thread/a99790ed/
> 
> netcdf-java barfs on opening, but h5dump apparently does not, nor does
> ncdump, using HDF 1.8.9. Message below references 1.8.13. Could there be a
> dependency on the HDF-5 library version?

No, our current netCDF C library works with any HDF5 bersion later than 1.8.8.
Also, nccopy works fine on the file, and can produce a bit-for-bit copy using 
only
netCDF C calls, so we see nothing funny about the file.  Charlie also said it
actually works fine through nco operators.  So I'm closing this ticket.

--Russ

> The fact that ncwa and nccopy reportedly barfs also makes me think maybe
> its not a netcdf java problem ??
> 
> original (netcdf3) and rechunked (netcdf4) are in
> /share/testdata/AntonioRodriques/
> 
> ----------------
> 
> Dear Antonio,
> 
> Forgive the delayed response. We prefer that NCO issues be posted on
> Sourceforge forum as described on nco.sf.net. Thank you for doing so
> later and for providing the test file. The issue you first raised on
> the netCDF group list occurs not only with NCO on Windows, but also
> with NCO and with nccopy on all OSs attempted. The output file
> produced by the chunking allows simple reads (e.g., with ncks), but
> produces an exotic menagerie of error when read by more sophisticated
> applications (e.g., Java or ncwa). Since NCO and nccopy are completely
> independent implementations of chunking that rely on the same library,
> it's a good bet that there is a problem with the library. The
> information below will help Unidata investigate this and hopefully
> produce a fix that works with both NCO and nccopy.
> 
> Best,
> Charlie
> 
> These two commands produce files that crash when ingested by ncwa
> (all OSs) and, according to the OP, lib Java on Windows.
> 
> ncks -4 --cnk_dmn latitude,4 --cnk_dmn longitude,4 \ --cnk_dmn time,512 ~/
> 2014.nc ~/2014_ch.nc
> nccopy -c time/512,latitude/4,longitude/4 ~/2014.nc \ ~/2014_ch_nccopy.nc
> 
> zender@roulee:~$ ncwa -O ~/2014_ch.nc ~/foo.nc
> Bus error
> 
> zender@roulee:~$ ncwa -O ~/2014_ch_nccopy.nc ~/foo.nc
> *** Error in `HDF5-DIAG: Error detected in HDF5 (1.8.13) thread 0:
> #000: H5Dio.c line 149 in H5Sselect_hyperslab(): not a data space
> HDF5-DIAG: Error detected in HDF5 (1.8.13) Aborted
> 
> Input file is available at
> 
> http://glace.ess.uci.edu/tmp/2014.nc
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: FJB-731534
Department: Support netCDF
Priority: Normal
Status: Closed