This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi John & others,A lot of us are creating large netcdf files these days, and I was wondering if anyone is having the same kind of problems I am:
When I open a large (~2Gb) netcdf file using the latest stable release, I've been getting an error. The code was working when the file size was smaller (fewer time steps, smaller lat/lon resolution).
My data is simply dimensions: LON = 1276 ; LAT = 950 ; TIME = UNLIMITED ; // (601 currently) variables: double LON(LON) ; LON:units = "degrees_east" ; LON:point_spacing = "even" ; double LAT(LAT) ; LAT:units = "degrees_north" ; LAT:point_spacing = "uneven" ; double TIME(TIME) ; TIME:units = "SECONDS" ; float HA(TIME, LAT, LON) ;I start by opening the file, getting the time dimension & variable. It croaks when reading the variable into an Array (note that I'm only reading the TIME axis array, fairly small):
... Variable testVar = timeDim.getCoordinateVariable(); System.out.println("testVar.getName: " + testVar.getName()); System.out.println("testVar.getDataType: " + testVar.getDataType()); System.out.println("testVar.getRank: " + testVar.getRank()); int[] testShape = testVar.getShape(); for (int q=0; q<testShape.length; q++) System.out.println("testShape["+q+"]: " + testShape[q]); int[] testOrigin = new int[testVar.getRank()]; Array testArr2 = testVar.read(testOrigin,testShape); error: java.io.IOException: Negative seek offset java.io.IOException: Negative seek offset at java.io.RandomAccessFile.seek(Native Method) at ucar.netcdf.RandomAccessFile.read_(RandomAccessFile.java:508) at ucar.netcdf.RandomAccessFile.seek(RandomAccessFile.java:350) at ucar.netcdf.NetcdfFile$V1DoubleIo.readArray(NetcdfFile.java:1520) at ucar.netcdf.NetcdfFile$V1Io.copyout(NetcdfFile.java:896) at ucar.netcdf.Variable.copyout(Variable.java:276) at ucar.nc2.Variable.read(Variable.java:237)I immediately thought it was a Large File Support problem, but a little investigation turned up the fact that I don't (yet) need 64-bit addressing because my record sizes are small. In fact, I can read the file fine with applications that use the C and FORTRAN netcdf libraries (version 3.6 or higher). And ncdump works.
But I went ahead and created the large file with 64-bit addressing anyway by simply changing one line in my FORTRAN model from
iret = nf_create(ncfn, NF_CLOBBER, ncid) to iret = nf_create(ncfn, OR(NF_CLOBBER, NF_64BIT_OFFSET), ncid) It creates the file just fine, checking with: od -An -c -N4 foo.nc gives the expected C D F 002 and ncdump gives netcdf s_2903-563_ha64 { // format variant: 64bit dimensions: ...but my java code still croaks (in the same way Ferret does if not compiled against netcdf lib > 3.6):
Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Not a netcdf file
at ucar.netcdf.NetcdfFile.readV1(NetcdfFile.java:1745) at ucar.netcdf.NetcdfFile.<init>(NetcdfFile.java:130) at ucar.netcdf.NetcdfFile.<init>(NetcdfFile.java:148) at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:61) Anybody else out there creating java applications that read large files?I'm running RHEL4 on an EMT64 x86_64 machine, with java 1.5.0_06 and java netcdf version 2.2.16
Yours, Chris __________________________________________________________________ Christopher W. Moore email: address@hidden Research Scientist, Oceanography tel: 206.526.6779 University of Washington/JISAO/NOAA-PMEL fax: 206.526.6744 NOAA Center for Tsunami Research Seattle, WA ------------------------------------------------------------------ =============================================================================== To unsubscribe netcdf-java, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ===============================================================================