[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDF #OPQ-508225]: netCDF General - Record data with two unlimited records
- Subject: [netCDF #OPQ-508225]: netCDF General - Record data with two unlimited records
- Date: Fri, 23 Mar 2007 14:40:37 -0600
Hi,
Here are a few suggestions:
- Use indirection, by having "index" and "length" variables for each record,
providing the location in a single large variable array for where the data
starts for each frame and how many values there are for that frame:
dimensions:
frame=unlimited;
numvals=1000000; // total number of values for all frames
variables:
int index(frame); // where the values for each frame start
int length(frame); // how many values for each frame
float values(numvalues);
...
- Use two passes, storing the data in a different organization as it comes in,
and later making a separate pass over the raw data to store it as a better
organized netCDF file. That way, you will know exactly how many values are in
each frame and you would not have to use an "unlimited" dimension, because you
would also know how many frames there were. For example, instead of one
variable, you could use three: a small, medium, and large variable, each
containing all the values for small-sized, medium-sized, and large-sized
frames, with a separate variable telling for each frame, whether it is small,
medium, or large and where its values are.
- Try netCDF-4, which is currently available for testing. It supports both
multiple unlimited dimensions and a "variable length" user-defined type for
structures like ragged arrays. However, you would have to wait for the
released version of netCDF-4 (expected in a few months) to make sure the format
was supported and future netCDF programs could read it.
- use HDF5 instead of netCDF, because HDF5 already supports variable-length
types and multiple unlimited dimensions. However, HDF5 is somewhat more
complex than netCDF-3 or netCDF-4 and is not supported by as much third-party
software.
You might also look at the solutions and conventions for multi-rate data used
for aircraft data, since it solves a similar problem with netCDF-3:
http://www.eol.ucar.edu/raf/Software/netCDF.html
--Russ
Russ Rew UCAR Unidata Program
address@hidden http://www.unidata.ucar.edu
Ticket Details
===================
Ticket ID: OPQ-508225
Department: Support netCDF
Priority: Normal
Status: Closed