This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Unidata Support wrote:
no problem with streaming data - its just like any other. the main thing is to make sure your outer dimension is the unlimited dimension, so that you can continue to expand the file.------- Forwarded MessageTo: address@hidden From: "Peyush Jain" <address@hidden> Subject: netCDF Java - Newbie Question: Is it possible to archive streaming data using NetCDF? Organization: UCAR/Unidata Keywords: 200503291623.j2TGNUiv002216Institution: NASA Package Version: Version 2.1 Operating System: Windows XP Pro Hardware Information: P4, 3.0GHz, 2GB RAM Inquiry: Hello, I was wondering if it is possible to archive streaming data using netCDF. If it is, can you please point me to an example? A
example below
ssuming the data coming in has 5 one dimensional arrays (of different types), I was able to create 5 variables and save them to a netCDF file. Everytime I receive this data, it overwrites the previous data in the file. I am sure that it is not appending the data because the file size never increases. I am using the following function to write to netCDF file: ncfile.write(name, ArrayAbstract.factory(doubleArray));
problem is you are using this call, where origin assumed to be 0/** Write data to the named variable, origin assumed to be 0. Must not be in define mode. * @param varName name of variable. IllegalArgumentException if variable name does not exist.
* @param values write this array; must be same type and rank as Variable * @throws IOException */public void write(String varName, Array values) throws java.io.IOException, InvalidRangeException;
you should use this one, and create an origin array; then increment the outer dimension each time you write (leave the others zero):
/** Write data to the named variable. Must not be in define mode.* @param varName name of variable. IllegalArgumentException if variable name does not exist.
* @param origin offset within the variable to start writing. * @param values write this array; must be same type and rank as Variable * @throws IOException */public void write(String varName, int [] origin, Array values) throws java.io.IOException, InvalidRangeException {
public void testNC3WriteWithRecord() throws IOException {
NetcdfFileWriteable ncfile = new
NetcdfFileWriteable("C:/temp/writeRecordExample.nc", false);
// define dimensions, including unlimited
Dimension latDim = ncfile.addDimension("lat", 64);
Dimension lonDim = ncfile.addDimension("lon", 128);
Dimension timeDim = ncfile.addDimension("time", -1);
// define Variables
Dimension[] dim3 = new Dimension[3];
dim3[0] = timeDim;
dim3[1] = latDim;
dim3[2] = lonDim;
// double T(time, lat, lon) ;
// T:long_name="surface temperature" ;
// T:units = "degC" ;
ncfile.addVariable("T", DataType.DOUBLE, dim3);
ncfile.addVariableAttribute("T", "long_name", "surface temperature");
ncfile.addVariableAttribute("T", "units", "degC");
// float lat(lat) ;
// lat:units = "degrees_north" ;
ncfile.addVariable("lat", DataType.FLOAT, new Dimension[] {latDim});
ncfile.addVariableAttribute("lat", "units", "degrees_north");
// float lon(lon) ;
// lon:units = "degrees_east" ;
ncfile.addVariable("lon", DataType.FLOAT, new Dimension[] {lonDim});
ncfile.addVariableAttribute("lon", "units", "degrees_east");
// int time(time) ;
// time:units = "hours" ;
ncfile.addVariable("time", DataType.INT, new Dimension[] {timeDim});
ncfile.addVariableAttribute("time", "units", "hours");
// :title = "Example Data" ;
ncfile.addGlobalAttribute("title", "Example Data");
// create the file
try {
ncfile.create();
} catch (IOException e) {
System.err.println("ERROR creating file");
e.printStackTrace();
return;
}
System.out.println( "ncfile = "+ ncfile);
// now write one record at a time
Variable v = ncfile.findTopVariable("T");
ArrayDouble data = new ArrayDouble.D3(1, latDim.getLength(),
lonDim.getLength());
ArrayInt timeData = new ArrayInt.D1(1);
int[] origin = new int[v.getRank()];
int[] timeOrigin = new int[1];
for (int time=0; time<100; time++) {
// fill the data array
Index ima = data.getIndex();
for (int j=0; j<latDim.getLength(); j++)
for (int k=0; k<lonDim.getLength(); k++)
data.setDouble(ima.set(0,j,k), (double) time*j*k);
timeData.setInt( timeData.getIndex(), time);
// write to file
origin[0] = time;
timeOrigin[0] = time;
try {
ncfile.write("T", origin, data);
ncfile.write("time", timeOrigin, timeData);
} catch (IOException e) {
System.err.println("ERROR writing file");
} catch (InvalidRangeException e) {
e.printStackTrace();
}
}
// all done
try {
ncfile.close();
} catch (IOException e) {
System.err.println("ERROR writing file");
}
System.out.println( "**** TestWriteRecord done");
}