[netCDF #SJD-391075]: compression and parallel Netcdf4

Hello Reto,

Our compression/parallel I/O functionality for netCDF4 is provided via the hdf5 
library.  Currently, I don't believe they have any plans for adding that 
functionality to their library, and we (unfortunately) do not have the 
resources to re-implement compression/MPI in a way that they work together.  I 
agree with you, however, that it would be a great feature to offer, and if it 
is something we are able to provide in the future, we most assuredly will.  Or, 
if somebody were to implement it and provide it as a patch via a 'Pull Request' 
on our GitHub page, that would also be graciously accepted :).

I'm sorry I can't provide a more positive response,

-Ward

> Dear NetCDF Support
> 
> On your online documentation
> 
> http://www.unidata.ucar.edu/software/netcdf/workshops/2011/nc4chunking/CompressionAdvice.html
> 
> it is written that compression cannot be used with parallel I/O.
> 
> I've tested the feature and it really does not work :-)
> So, my question goes:
> 
> Is it planned to include per-variable compression also with MPI parallel
> I/O support? It would possibly speed up the define and write of big data
> quite extensively depending on data structure, and it would decrease
> the file size of parallel NetCDF 4 files very much.
> 
> Reto
> 

Ticket Details
===================
Ticket ID: SJD-391075
Department: Support netCDF
Priority: Normal
Status: Closed