Re: 20020411: NUWG conventions

>To: <address@hidden>
>From: "Tony Simmers" <address@hidden>
>Subject: NUWG conventions 
>Organization: UCAR/Unidata
>Keywords: 200204110120.g3B1KLa25329

Hi Tony,

> I am working on software (a colleague, Tim Hume, has done most of
> the hard work) to store various types of nwp data (mostly from grib)
> in netCDF files.  We are pretty much following the NUWG conventions
> with a few additions to handle identifying ensemble members etc.  I
> have based my initial ideas on the NUWG documentation and the CDLs
> distributed with your decoders.  ...

We established the NUWG conventions in about 1992 and haven't done
much to maintain them since then.  We have occasionally updated the
CDLs associated with our "gribtonc" decoder to handle new parameters,
levels, or grids, but gribtonc is also getting somewhat creaky from
lack of maintenance and refactoring.  There are undoubtedly GRIB
products that it doesn't handle well.  I've been waiting until GRIB2
forces us into a rewrite.

The most complete netCDF conventions for gridded data may be the new CF
Conventions at 

  http://www.cgd.ucar.edu/cms/eaton/cf-metadata/index.html

which generalize and extend the COARDS conventions.

The Java applications we are working on handle NUWG, COARDS, and CF
conventions where possible.  I'm CC:ing John Caron on this in case he
has more to add about that.

>                             ...  As much as possible we want to keep
> in line with what others do, which raises a couple of questions:
> 
> 1. In the nav variables there seems to be some variation between
> using i_dim and x_dim. I can see why the use of Ni and Nx varies
> (the grib tables use these names for different projection types) but
> I thought the NUWG conventions specified attribute x_dim to identify
> the dimension. ruc2.cdl is one of the few (only) ones to use x_dim.
> Can you advise the best one to use (a last resort would be to define
> both i_dim and x_dim...)

From the correspondence on the NUWG conventions available at

  http://www.unidata.ucar.edu/packages/netcdf/NUWG/caron.email

I found this:

  We use "x" and "y" when the GRIB GDS specification uses "x" and "y"
  (e.g. for polar stereographic projections, parameterized in terms of
  Nx, Ny, Dx, Dy, ...) and "i" and "j" when the GRIB GDS specification
  uses "i" and "j" (e.g. latitude/longitude grids, parameterized in
  terms of Ni, Nj, Di, ...).  For a different georeferencing model, we
  would presumably name the parameters using names specific to a
  standard description of that model.  The use of parameter names from
  a standard reference avoids specifying their order by a
  NUWG-specific convention.

It looks like "x_dim" and "y_dim" are used in ruc.cdl, avn-q.cdl,
ngm-q.cdl, eta.cdl, all projected grids such as Lambert conformal that
use (x,y) dimensions instead of (lat,lon) dimensions.

But "i_dim" and "j_dim" are used in ecmf.cdl, avn-x.cdl, sst-a.cdl,
mrf-e.cdl, mrf-a.cdl, ocean.cdl, which are defined on strictly
(lat,lon) grids.

But I really can't defend this difference as being a good convention.
I think it would be better to just use "x_dim" and "y_dim"
consistently, when practical.  There are probably skewed grids where
it's somewhat arbitrary which dimensions are identified with "x" and
"y".

> 2. Any reason why the grib1 scanning mode (GDS octet 28) does not
> appear in any of the CDL's (perhaps related to the previous point?)

That's because the scanning mode was already accounted for in
converting the data to netCDF, so would convey no information about
the netCDF representation of the data.  If we kept the data in the
same order as in the GRIB product, it would have to be in a
1-dimensional array rather than a 2-dimensional array (or m-1
dimensional instead of m dimensional).  The reason it's sometimes
ordered differently in GRIB is this can improve the compression, but
since the netCDF form is not compressed, it's better if applications
can access the data in a natural array form.

> 3. Have you had any thoughts on grib2?  My feeling is that there
> will be little real impact, but I would be especially interested if
> you had any ideas on how you might handle ensemble data.

I haven't been following GRIB2 developments closely enough to know
much about it except that there are additional complexities to provide
even better compression.  Is it actually being used now as an approved
standard?  If so, we may have to start rewriting decoders and
rethinking conventions ...

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu