[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #DVB-160103]: NetCDF and Image Sequences with varying image sizes

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [Support #DVB-160103]: NetCDF and Image Sequences with varying image sizes
  • Date: Mon, 28 Aug 2006 11:09:24 -0600
Hi Carsten,

You are correct that netCDF-3 only allows appending along the record dimension, 
and each record must be of the same size.  So the images would all be of the 
same size, and there is no way to directly represent tensors of varying 
dimension as a single variable.

I can suggest three ways to get around this limitation:

 1.  Use HDF5:
http://hdf.ncsa.uiuc.edu/HDF5/
which already supports multiple unlimited dimensions and a variable-length data 
type.

 2.  Try the netCDF-4 alpha-release, which also supports both multiple 
unlimited dimensions and a variable-length data type, since it uses HDF5 as its 
storage layer: 
       ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/netcdf-4.0-alpha16.tar.gz
However, since this is an alpha (internal testing) release, the format will 
likely change when HDF5 version 1.8 is released, so this would only be useful 
for testing in preparation for a real release of netCDF-4 in early 2007 
(barring unforeseen difficulties).

 3.  Write a thin software layer above netCDF that uses the record dimension to 
store fixed-size blocks and that stores variable sized images and variable 
dimension tensors using those fixed-size record blocks.  This would probably 
require another two variables for index and length for the nth image into the 
fixed-size blocks.  Of course, if you have to write a layer like this, you 
might argue that you might as well write an application-specific binary layer 
or use a database system that supports BLOBs (binary large objects).

A fourth approach is to just accept wasted space and make the record size 
correspond to the largest image you will write, with something similar for 
tensors of varying dimensions.

Sorry I can't suggest something better, but your problem is a year or two ahead 
of our development ...

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: DVB-160103
Department: Support netCDF
Priority: Normal
Status: Closed