[netCDF #TLB-677315]: FW: Net CDF-4 Issues

Coy,

I'm attaching two CDL files for you to compare.  The first,
nc3time.cdl, is just a copy of an earlier CDL file you sent showing
the current way you organize the data using netCDF-3.

The second, nc4time.cdl, is supposed to represent exactly the same
data, except it uses a variable-length type.  The differences are 

 1.  the addition of the type definition "heights_t" representing a
     variable length sequence of float data at different heights
     
        types:
          float (*) heights_t ;

 2.  the omission of the "Heights" dimension, which is no longer used

 3.  the declarations of the five variables 

        heights_t MeanDopplerVelocity(Time) ;
        heights_t SNR(Time) ;
        heights_t Power(Time) ;
        heights_t SpectralWidth(Time) ;
        heights_t NoiseLevel(Time) ;

 4.  the CDL notation for the data values for those variables, which
     now use "{val1, val2, ..., valn}" for a variable length sequence of
     n values at various heights.

Using ncgen and the experimental utility ncgen4, available in the
current snapshot distribution, the corresponding netCDF files can be
generated for these two CDL files:

  $ ncgen -b nc3time.cdl
  $ ncgen4 -b nc4time.cdl

Comparing sizes shows nc3time.nc is still smaller (24076 bytes) than 
nc4time.nc (93399 bytes), but I think that's because of the larger 
constant overhead for netCDF-4/HDF5 files, and that space would be 
saved if using more realistic amounts of data.  Note also that the
use of variable length types means you really aren't limited to the
use of a set of fixed heights such as 10, 20, ..., 100, but could
instead have any number of heights for any variable/time combination.

Disadvantages of use of this variable length type include:

  - You have to access each set of heights for a particular variable
    at a particular time all at once, rather than getting a single
    value.  In you case, this means writing anywhere from 10 to 100
    values of MeanDopplerVelocity (for example) in one call for a
    particular time, and similarly all heights for a particular
    variable and time would have to be read with one call.

  - You can't access a cross section of values at the same height but
    different times for a variable, as you could if the values were
    stored in a rectangular array.

  - The variable-length type does not support compression, so
    variables that use it, such as the five in your example, can't use
    the compression features available for other types.

  - At this time only a limited number of applications, such as
    ncdump, support the variable-length type available in netCDF-4.

I hope this helps make clearer how the variable-length type could be
used in your application.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: TLB-677315
Department: Support netCDF
Priority: Normal
Status: Closed

Attachment: nc3time.cdl
Description: Binary data

Attachment: n4ctime.cdl
Description: Binary data