[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #TFT-400775]: [netcdfgroup] Writing files > 4GB with 16-bit variales

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #TFT-400775]: [netcdfgroup] Writing files > 4GB with 16-bit variales
  • Date: Wed, 04 Nov 2009 09:53:24 -0700
Hi,

> I am trying to write a large file by combining multiple files. When I
> try to write data (type short) greater than 4GB to a file, I get the
> following error  *"ncx.c:1810: ncx_put_size_t: Assertion `*ulp <=
> 4294967295U' failed."
> *I was wondering if this is an issue with my code or if it is a known and
> resolved issue or even if there is a workaround? Appreciate any help!

One of the limitations of the netCDF-3 classic or 64-bit format is that a 
variable cannot only exceed 4GiB under certain conditions:

See the three FAQs here:

  
http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#Large%20File%20Support10

If you use a record dimension or the netCDF-4 format, you will be able to write 
larger variables.  Alternatively, the new parallel netCDF package has introduced
a new variant of the netCDF format (not yet readable by other netCDF software)
that also permits variables larger than 4 GiB:

  http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/ReleaseNotes-1.1.0

--Russ

> Cheers.
> Code :
> ...........
> short data[1490][1636][1720];
> int NX=1720, NY=1636, NZ=1490;
> for (k=1; k<=2; k++) {
> sprintf(name,"test%d.nc",k);
> nc_open(name, NC_NOWRITE, &ncid);
> 
> if (nc_inq_ndims(ncid,&ndims) != NC_NOERR)
> printf("Another error");
> size_t dim_siz[ndims];
> printf("The file has %d dimensions\n",ndims);
> for (dim =0; dim < ndims; dim++)  {
> nc_inq_dim(ncid, dim, dim_name, &dim_siz[dim]);
> printf("Name of dimension %d is %s\n",dim,
> dim_name);
> printf("Size of dimension %d is %d\n",dim,
> dim_siz[dim]);
> }
> int varid;
> if (nc_inq_varid(ncid, "segmented", &varid) != NC_NOERR)
> printf("Error 1");
> if (nc_get_var_short(ncid, varid, &data[745*(k-1)) !=
> NC_NOERR)
> printf("Error 2");
> nc_close(ncid);
> }
> printf("Done reading data\n");
> if(retval=nc_create(FILE_NAME, NC_CLOBBER | NC_64BIT_OFFSET,
> &ncid2))
> ERR(retval);
> if ((retval = nc_def_dim(ncid, "x", NX, &x_dimid)))
> ERR(retval);
> if ((retval = nc_def_dim(ncid, "y", NY, &y_dimid)))
> ERR(retval);
> if ((retval = nc_def_dim(ncid, "z", NZ, &z_dimid)))
> ERR(retval);
> 
> 
> dimids[0] = x_dimid;
> dimids[1] = y_dimid;
> dimids[2] = z_dimid;
> 
> if ((retval = nc_def_var(ncid, "segmented", NC_SHORT, 3, dimids,
> &varid)))
> ERR(retval);
> 
> if ((retval = nc_enddef(ncid)))
> ERR(retval);
> 
> if ((retval = nc_put_var_short(ncid, varid, &data[0][0][0])))
> ERR(retval);
> 
> if ((retval = nc_close(ncid)))
> ERR(retval);
> printf("*** SUCCESS writing example file simple_xy.nc!\n");
> 
> 
> return 0;
> }
> 
> _______________________________________________
> netcdfgroup mailing list
> address@hidden
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: TFT-400775
Department: Support netCDF
Priority: Normal
Status: Closed