[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #SOA-856908]: NECDF file size

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #SOA-856908]: NECDF file size
  • Date: Thu, 21 Jun 2012 10:02:02 -0600
Hi Lin,

> I have a problem associated with netcdf file size. I already set supporting 
> large file in environment variables,"setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1".  
> But, it still has problem when writing a file as large as 30G. So, is the 
> netcdf do not permit to write a file to 30G?

NetCDF can support writing files larger than 30GB.  For details, see this set of
questions and answers:

  
http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#Large%20File%20Support

There have been a few bug fixes related to large files in previous versions of 
netCDF software, described in the RELEASE_NOTES file that is included with each 
new version, for example:

  release notes for version 4.2.1-rc1:
  http://www.unidata.ucar.edu/software/netcdf/release-notes-4.2.1-rc1.html

has this:

    Fixed bug in 64-bit offset files with large records,
    when last record variable requires more than 2**32
    bytes per record.  
    [NCF-164]

with a link to more information about the fix in the "[NCF-164]" link:

    https://www.unidata.ucar.edu/jira/browse/NCF-164

Other similar fixes in recent versions are described briefly and then in 
more detail in these links:

  Version 4.2.0:
  http://www.unidata.ucar.edu/software/netcdf/release-notes-4.2.html

    Fixed problem with 64-bit offset format where writing more
    than 2**31 records resulted in erroneous NC_EINVALCOORDS error.
    [NCF-101]
    https://www.unidata.ucar.edu/jira/browse/NCF-101

    Dimension size in classic model netCDF-4 files now allows
    larger sizes than allowed for 64-bit offset classic files.
    [NCF-117]
    https://www.unidata.ucar.edu/jira/browse/NCF-117

  Version 4.1.2:

    Fixed two large-file bugs with using classic format or
    64-bit offset format and accessing multidimensional
    variables with more than 2**32 values.
    (No detailed description available ...)

From the information you provided, I can't tell if any of the above bug fixes
are relevant to the problem you are seeing, but if you could provide a small
program that can be used to reproduce the problem, we can test it against the
current software release to see if it still occurs.

It may also be a problem with WRF configuration, but for that, you would have to
contact WRF support ...

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: SOA-856908
Department: Support netCDF
Priority: Normal
Status: Closed