[netCDF #ORR-618715]: Question about formatting data with NETCDF-4

Hi Paul,

> My name is Isidore Paul AKAM BITA, I am working for LuxSpace, a company
> specialized on Satellite Telecommunication Applications located in
> Luxembourg.
> 
> For a project purpose, we have to used NETCDF-4 to disseminate the
> images to user's. The images in fact are first compressed using an
> external compression algorithm  (not the one embedded in the NETCDF-4
> libraries). In fact, as input, we have a satellite multi-component image
> with N = 12 spectral bands. After the compression of the original image
> (we used a kind of JPEG codec (coding-decoding), and the compression is
> done on each file independently), we obtain 12 files (file1,file2, ...
> file 12), each file representing one spectral band. The 12 files have
> different sizes. I would then like to know (I am new to NETCDF-4 and
> still try to understand how it works) how to proceed, in order to send
> to the user, the 12 files (with different sizes) using NETCDF-4. Since
> the 12 files represent one multi-component image, it would be better to
> have all the 12 files as input and to generate one output that is the
> output of the NETCDF-4 libraries. How should I proceed to reach my goal?

Interesting question, but we need more information.  In what format are
file1, file2, ..., file12?  That is, when uncompressed are they just values
of the image, or are they netCDF files with associated metadata describing
the variable, band, units, etc.?

> In the other hand, I would also need that the user's, from the NETCDF-4
> file, is able to decode the NETCDF-4 file and retrieve the different
> files (file1,file2, ... file12).
> 
> Do you think that I could achieve it using NETCDF-4 libraries ?

It sounds like you're trying to store a different format in netCDF files, which
are just being used as a container and to attach metadata.  The resulting files
will not be completely self-describing, because the a user could only extract
the compressed bytes, stored as byte arrays, and then would have to apply your
decompression to get at the data.  That means you couldn't use the netCDF 
facilities
for chunking (multidimensional tiling), which are pretty important for efficient
access to netCDF-4 data.

For example, if the user wants a small subset of the data in file1, the entire 
file
a byte array would have to be read, uncompressed, and the subset extracted.  
With
chunking and compression supported by netCDF-4, only the tiles that include the
desired data subset would have to be read and uncompressed, which can be 
significantly
faster.

> I would also appreciate if you could  provide your advice, explaining me
> what is more suitable in that case, binary file or txt file, or another
> kind of files.
> 
> By looking forward to your help and feedback in order to help me to
> understand how to use the NETCDF-4 in an efficient way,

Sorry to say we won't be able to answer your question in any more detail until 
next week.
There are currently only 2 developers working on netCDF here, and we get lots 
of support
questions.  We're currently trying to prepare for a two-day workshop we're 
giving on
Thursday and Friday.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: ORR-618715
Department: Support netCDF
Priority: Normal
Status: Closed