[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #HQN-423106]: netcdf4 with NC_NETCDF4 and NC_64BIT_OFFSET

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #HQN-423106]: netcdf4 with NC_NETCDF4 and NC_64BIT_OFFSET
  • Date: Thu, 01 Sep 2011 14:28:01 -0600
> On 9/1/11 12:47 PM, Unidata netCDF Support wrote:
> > Hi Greg,
> >
> >> I tried to create a netcdf file using NC_NETCDF4&  NC_64BIT_OFFSET&
> >> NC_CLASSIC_MODEL and it fails.  Looking at nc4file.c, there is an
> >> explicit check that disallows for both NC_NETCDF4 and NC_64BIT_OFFSET
> >> both being set.
> >>
> >> Is this intentional (or, obviously this is intentional, what is
> >> the background on this decision).   I have a model that needs to use
> >> NC_NETCDF4 and the NC_CLASSIC_MODEL, but has a dimension that exceeds 2^31
> >> (but is<  2^32).
> > Yes, it's intentional.  NC_64BIT_OFFSET is only intended to specify use of
> > the variant of the classic format that uses 64-bit offsets instead of 32-bit
> > offsets.  It corresponds to format 2 of the four netCDF format variants.  
> > All
> > netCDF-4/HDF5 files (formats 3 and 4) uses 64-bit offsets as far as I know.
> I do know that there is this check in libsrc4/nc4dim.c:
> 
> /* For classic model, stick with the classic format restriction:
> * dim length has to fit in a 32-bit signed int. For 64-bit offset,
> * it has to fit in a 32-bit unsigned int. */
> if (h5->cmode&  NC_CLASSIC_MODEL)
> if((unsigned long) len>  X_INT_MAX) /* Backward compat */
> return NC_EDIMSIZE;
> 
> Compared to this check in libsrc/dim.c:
> 
> if ((ncp->flags&  NC_64BIT_OFFSET)&&  sizeof(off_t)>  4) {
> /* CDF2 format and LFS */
> if(size>  X_UINT_MAX - 3) /* "- 3" handles rounded-up size */
> return NC_EDIMSIZE;
> } else {
> /* CDF1 format */
> if(size>  X_INT_MAX - 3)
> return NC_EDIMSIZE;
> }
> 
> 
> So, it looks like the NETCDF4 model isn't permitting the same size
> dimensions as the 64BIT_OFFSET model...
> Maybe this check needs to be changed to compare against X_UINT_MAX
> instead of X_INT_MAX?

Yes, I think you're right that this is a bug, and I think you've also
identified the right fix.  I'll try testing the fix to make sure there
aren't unintended consequences.  We're tracking this in Jira as

  https://www.unidata.ucar.edu/jira/browse/NCF-117

if you want to follow its progress or add comments.

Thanks for reporting the problem!

--Russ


Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: HQN-423106
Department: Support netCDF
Priority: Normal
Status: Closed