This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Unidata Support wrote:
no problem with streaming data - its just like any other. the main thing is to make sure your outer dimension is the unlimited dimension, so that you can continue to expand the file.------- Forwarded MessageTo: address@hidden From: "Peyush Jain" <address@hidden> Subject: netCDF Java - Newbie Question: Is it possible to archive streaming data using NetCDF? Organization: UCAR/Unidata Keywords: 200503291623.j2TGNUiv002216Institution: NASA Package Version: Version 2.1 Operating System: Windows XP Pro Hardware Information: P4, 3.0GHz, 2GB RAM Inquiry: Hello, I was wondering if it is possible to archive streaming data using netCDF. If it is, can you please point me to an example? A
example below
ssuming the data coming in has 5 one dimensional arrays (of different types), I was able to create 5 variables and save them to a netCDF file. Everytime I receive this data, it overwrites the previous data in the file. I am sure that it is not appending the data because the file size never increases. I am using the following function to write to netCDF file: ncfile.write(name, ArrayAbstract.factory(doubleArray));
problem is you are using this call, where origin assumed to be 0/** Write data to the named variable, origin assumed to be 0. Must not be in define mode. * @param varName name of variable. IllegalArgumentException if variable name does not exist.
* @param values write this array; must be same type and rank as Variable * @throws IOException */public void write(String varName, Array values) throws java.io.IOException, InvalidRangeException;
you should use this one, and create an origin array; then increment the outer dimension each time you write (leave the others zero):
/** Write data to the named variable. Must not be in define mode.* @param varName name of variable. IllegalArgumentException if variable name does not exist.
* @param origin offset within the variable to start writing. * @param values write this array; must be same type and rank as Variable * @throws IOException */public void write(String varName, int [] origin, Array values) throws java.io.IOException, InvalidRangeException {
public void testNC3WriteWithRecord() throws IOException {NetcdfFileWriteable ncfile = new NetcdfFileWriteable("C:/temp/writeRecordExample.nc", false);
// define dimensions, including unlimited Dimension latDim = ncfile.addDimension("lat", 64); Dimension lonDim = ncfile.addDimension("lon", 128); Dimension timeDim = ncfile.addDimension("time", -1); // define Variables Dimension[] dim3 = new Dimension[3]; dim3[0] = timeDim; dim3[1] = latDim; dim3[2] = lonDim; // double T(time, lat, lon) ; // T:long_name="surface temperature" ; // T:units = "degC" ; ncfile.addVariable("T", DataType.DOUBLE, dim3); ncfile.addVariableAttribute("T", "long_name", "surface temperature"); ncfile.addVariableAttribute("T", "units", "degC"); // float lat(lat) ; // lat:units = "degrees_north" ; ncfile.addVariable("lat", DataType.FLOAT, new Dimension[] {latDim}); ncfile.addVariableAttribute("lat", "units", "degrees_north"); // float lon(lon) ; // lon:units = "degrees_east" ; ncfile.addVariable("lon", DataType.FLOAT, new Dimension[] {lonDim}); ncfile.addVariableAttribute("lon", "units", "degrees_east"); // int time(time) ; // time:units = "hours" ; ncfile.addVariable("time", DataType.INT, new Dimension[] {timeDim}); ncfile.addVariableAttribute("time", "units", "hours"); // :title = "Example Data" ; ncfile.addGlobalAttribute("title", "Example Data"); // create the file try { ncfile.create(); } catch (IOException e) { System.err.println("ERROR creating file"); e.printStackTrace(); return; } System.out.println( "ncfile = "+ ncfile); // now write one record at a time Variable v = ncfile.findTopVariable("T");ArrayDouble data = new ArrayDouble.D3(1, latDim.getLength(), lonDim.getLength());
ArrayInt timeData = new ArrayInt.D1(1); int[] origin = new int[v.getRank()]; int[] timeOrigin = new int[1]; for (int time=0; time<100; time++) { // fill the data array Index ima = data.getIndex(); for (int j=0; j<latDim.getLength(); j++) for (int k=0; k<lonDim.getLength(); k++) data.setDouble(ima.set(0,j,k), (double) time*j*k); timeData.setInt( timeData.getIndex(), time); // write to file origin[0] = time; timeOrigin[0] = time; try { ncfile.write("T", origin, data); ncfile.write("time", timeOrigin, timeData); } catch (IOException e) { System.err.println("ERROR writing file"); } catch (InvalidRangeException e) { e.printStackTrace(); } } // all done try { ncfile.close(); } catch (IOException e) { System.err.println("ERROR writing file"); } System.out.println( "**** TestWriteRecord done"); }