[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #QSJ-228724]: large variable issue

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [Support #QSJ-228724]: large variable issue
  • Date: Thu, 13 May 2010 09:17:40 -0600
Hi Stefano,

Sorry to have taken so long to respond to your question ...

> I am running a Fortran90 program in order to build a netcdf file
> containing a large size variable (360x180x60x600). I am using netcdf-3.6.1
> on an IBM Fortran compiler (XL Fortran V11.1). What I get is a
> Segmentation Fault on running time, which disappears if I set the last
> variable to a smaller number. After some calculation, together with
> CISL-NCAR customer support I realized that the program crashes when the
> variable is larger than 4GB.
> 
> At this page
> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/NetCDF-Classic-Format-Limitations.html#NetCDF-Classic-Format-Limitations
> 
> I found the documentation to overcome issues related to variables larger
> than 4GB. The variable is actually the last and the only big one in the
> dataset, but that does not seem to help. Also, I tried to create a record
> dimension, which should allow to write large variables, but the problem
> persists. The dataset is created in cmode=nf90_64bit_offset.
> I also tried to run the program with netcdf-4.0.1 and I got the same
> Segmantation Fault.

Could you please send the output of "ncdump -c" on your file, so we can see
the type and structure of the file?  From what you have described, this should
work fine, and tests with such files are run  as part of "make check" when
installing.

There was a bug in netcdf-3.6.2 and earlier versions with variables larger than 
4 GB, as described here:

  http://www.unidata.ucar.edu/netcdf/docs/known_problems.html#large_vars_362

but I can't tell whether that applies in your case without seeing the types and
shapes of your variable.  That bug was fixed in 4.0.1, so it doesn't sound like
that's what you encountered.

We have also recently discovered and fixed a bug with writing multidimensional 
variables that have more than 4294967296 values, on 32-bit platforms, but that
also doesn not fit your problem description.  Are you running on a 64-bit 
platform
(sizeof(size_t) == 8 ?).

It would also help if you have a relatively small program that demonstrates the
problem, so we could reproduce it here and see if it's platform dependent or a
symptom of another bug in large variables ...

--Russ




Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: QSJ-228724
Department: Support netCDF
Priority: Emergency
Status: Closed