20000308: NetCDF-GEMPAK-Re: info

Darien,

I provided a NetCDF grid interface to our release of Gempak 5.4. 
That was our last release. I'm working on Gempak 6.0 now, but
haven't done any porting of that old work to the new release.
NCEP in the mean time has had lots of internal releases that we never
received. 

Basically, we assumed grid files in the NUWG NetCDF convention.
Unfortunatly all the other data sets like the reanalysis data
didn't follow NUWG...in particular, without projection information
in the CDL, plotting is a little hard. The reanalysis data just gives
lat/lon for every point. As a result, for data sets like that, it is just
easier to convert the data from NetCDF to a gempak file once rather than
trying to have Gempak use the NetCDF file directly. 

In order to allow GEMPAK to read either NetCDF or GEMPAK format files,
4 gemlib files had to be modified:
dginit.f  dgofil.f  dgrdat.f  dgrgrd.f grtlst.f

The basic changes are to look at the first few words of the file and determine
if the file is CDF or GEM, and then, if its a NetCDF, use the separate library
routines we created to get the projection, parameter, time, coordinate
and level information. Our goal was to limit the changes to the gemlib
routines since we don't have control of that development.

I had to maintain a lookup table for converting "netcdf" variable names to
GEMPAK style coordinate names. For instance, GEMPAK uses TMPC for temperature
regardless of whether its on a TROP, MAXW, PRES, HGHT etc level. The NUWG has
different temperature names, like T_sfc(lon,lat,time), T(lon,lat,level,time),
T_trop(lon,lat,time). It would be much easier if NetCDF files were implemented
using the standard GEMPAK idea of a grid with PARM, LEV1, LEV2, TIME1, TIME2
and GVCORD so that every grid in NetCDF would look like:
tmpc(lat,lon,lev1,lev2,coord,time2,time1). 
Unfortunately that was would create a very sparse NetCDF file which would lead
to a large file for ncgen.

Most plotting program work just as well with the NetCDF once the initial startup
hit of mapping the NetCDF file takes place. Accessing individual grids to plot
is no problem The biggest perfomance hit is with programs like gdinfo which
for a gempak format file only have to list out the available headers.
For NetCDF, there was no way of telling whether a grid had been received or not.
Without a flag as to whether the grid had been received yet, the only way
to see is to read in the grid which is very slow.

The worst performing program is gddiag. Gddiag lets the user create
a new parameter from a computed function and store the grid back to
a file for later use. To store the new parameter to a NetCDF file forces a 
copy of the entire file to take place so that the new pararameter can be
created. On large grids, that is an unacceptable hit for NetCDF.

Steve Chiswell
Unidata User Support


On Wed, 8 Mar 2000, Linda Miller wrote:

> Hi Darien,
> 
> --On Wed, Mar 8, 2000 1:42 AM +0000 Darien Davis <address@hidden> wrote:
> 
> > Hi Linda,
> > 
> > Hope all is going well. Let's get together soon...
> 
> I'd like that.  Give me a call.
> > 
> > all the fun stuff aside, I'm trying to find out the
> > status of gempak/netcdf (i.e., is there a version
> > of gempak calls that read from netcdf files?) I thought
> > that Steve Chiswell was working on this. Do you
> > know who I might contact along these lines?
> > 
> I know that Chiz has created decoders for GEMPAK that deal with the ACARS and
> the 6-minute profiler data.  I've copied him so that he can respond with the
> status.
> 
> Take care,
> Linda
> 
> > Any info would be appreciated!
> > Thanks
> > Darien
> > 
> 
> 
> 
> Linda Miller - address@hidden
> External Liaison, Unidata
> University Corporation for Atmospheric Research
> P.O. Box 3000
> Boulder, CO 80307-3000
> 303 497-8646 fax: 303-497-8690
> URL:  http://www.unidata.ucar.edu/staff/lmiller/un.act.html
> 
>