[netCDF #IEP-993725]: netCDF4 update

Hi Cathy,

> We finally got all our netCDF application code working with netCDF4
> files. Our biggest holdup was Perl as we greatly rely on it, but after a
> lot of work and reinstallation of various pieces, that now works.

Great, glad to hear it.

> NCL still doesn't produce files with the correct chunking but the next
> release will have the flexibility to change the default chunking that it
> comes with (end of January). We would redo any files that were affected
> at that point.

The version of the nccopy utility that's in the current snapshot (and
will be in 4.1.2) supports chunking as well as compression of classic
files to netCDF-4 classic model files.  It rechunks variables by
dimension, rather than having to chunk each variable individually,
which is easier than using the C API on files with lots of variables
that use shared dimensions that you'd like to chunk the same way.  But
if you want to chunk variables that use the same dimensions
differently, you'll still have to use that API rather than nccopy (and
similarly for compression levels).

This is  a new feature of nccopy, so if you end up making use of it,
I'd appreciate feedback about things that you don' like.

> One question I had was about rounding. I gather with correct rounding it
> is possible to store the data that is more tightly packed, comparible to
> the incoming grib. Jeff Whitaker gave me some python code which I will
> try but I was wondering if that can be built into the API in some way. I
> know it's not as simple as rounding the data and then storing.

We're have "bit shaving" on our list as an additional compression
option or filter.  I could build it into nccopy as an option, but I'm
not sure I want to do that, as currently all the transformations
nccopy does on the data are lossless, and bit shaving would actually
change the data by making the lower n bits of the floating-point
mantissa 0.

> I am looking at our CFSR files now and making sure we have all the
> monthly mean files and accompanying doc and then we will put them out
> for the public.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: IEP-993725
Department: Support netCDF
Priority: Normal
Status: Closed