Re: a few NetCDF questions

Hi Steve and Nawajish,

> We had a design review meeting for some of our prototypes yesterday
> discussing NetCDF support in ArcGIS, and came up with a few questions you
> might be able to address.
> 
> * Is it possible, and/or common to have NetCDF point data that might be
> fixed locations, such as weather stations that are sparsely distributed and
> not on a regular grid; and also to have NetCDF files where the points move
> as through space and time, such as a radiosonde? ...

Yes to both questions.  A good example of the first sort of data is
one of the examples linked from the "Example netCDF files" web page
at: 

  http://www.unidata.ucar.edu/packages/netcdf/examples/files.html

namely, the file named "2004091003_metar.nc" which actually links to

  http://www.unidata.ucar.edu/packages/netcdf/examples/metar.nc

with structure shown in the CDL file "2004091003_metar.cdl" which
links to

  http://www.unidata.ucar.edu/packages/netcdf/examples/metar.cdl

I've just gotten access to a lot more of these sorts of files today,
as I finally got a username/password login to the NOAA MADIS ftp site.
These are netCDF point data from lots of local mesonets all over the
U.S., such as are listed here:

  http://www-sdd.fsl.noaa.gov/MADIS/network_info.html

Unfortunately, I just noticed the password was not included in the
email, so I'll have to call for that tomorrow, then I'll put one or
more of those examples on the web page.  They should be better for GIS
use, as most are local to a relatively small geographical area.

As for the second kind of file (moving point data), that's common for
aircraft data and dragged buoys.  An example of aircraft instrument
data is the file on the examples page described by
"GOTEX.C130_N130AR.LRT.RF06.PNI.cdl", which has variables CALT(time),
CLAT(time), and CLON(time), for "CMIGITS-III" altitude, latitude, and
longitude at a given time and GPS height, latitude, annd longitude at
a specified time given by variables GGALT(time), GGLAT(time), and
GGLON(time), and "inertial" latitude and longitude, given by variables
LAT(time) and LON(time).  And then there are lots of other variables
for sensor measurements at the specified time, such as temperature,
relative humidity, etc.  For the actual data, see the entry in the
table for the corresponding ".nc" file corresponding to the above
".cdl" file.

There are lots more oceanographic data that are ship measurements, and
the ARGOS free drifting buoys are another source of such data.  Both
of these are available in netCDF form, and I'll see if I can locate
them and include them on the examples page.

>                                               ... If the answer to each of
> these is yes, any chance you know someone with an example of such data that
> you could post on the website or send to us?  And is there any good general
> guidelines we might follow in determining if the appropriate default
> interpretation of a dataset should be as points instead of as a raster?

If the data includes 1-dimensional latitude, longitude, and possibly
height variables that are not just coordinate variables (for example
they are indexed by "time" or "station id" dimensions), then they
represent either a trajectory (for time) or a list of fixed
observation points or stations (for "station id").  If instead the
latitude, longitude, and height variables are coordinate variables
(using identically-named dimensions to index them), then they
represent axes for grids (rasters?) instead.

> * Can you verify that it is either not possible or else very rare that
> someone would store lines or polygons in a NetCDF file and not something we
> need to worry about for a initial release of NetCDF support?

It is rare but possible, so you probably don't need to worry about
it.  I believe the widely-used GMT (Generic Mapping Tools) system from 

  http://gmt.soest.hawaii.edu/

uses a netCDF scheme to store its high resolution map boundary data,
but the representation is not straightforward.  I know there is an
"arcGMT" package for converting between GMT and Arc/Info, but I'm not
sure if the conversion is for the GMT netCDF data or for an image
format it can output.

> * Is it possible, or likely that someone might create a netcdf file that has
> several spatial coordinate systems in the same file?  Might someone create a
> file that has some variables in lat/long and some other variables in
> stereographic or some other projection?

Yes, it's possible, but it's not done much.  The NUWG conventions
explicitly allow for that possibility by having a dimension "nav" for
multiple different grids within a single file, but I don't think
anyone makes use of that part of the convention.  On the other hand,
some models such as WRF use multiple grids offset from each other by
half a grid cell to form "staggered" grids, in which some variables
are defined on the grid cell corners, others defined on the grid cell
centers, and still others on the midpoints of the sides of grid
cells.  For an example of such staggered grids, see the
"wrfout_v2_Lambert.cdl" CDL file and corresponding netCDF file on the
examples page.  There are as yet no well-established conventions for
representing the relationship between staggered grids using netCDF, so
we're still struggling with how to handle these well.

> * If we add the ability to write netcdf  data, will the API be changing with
> future versions of netcdf, such as the HDF work, or will the netcdf API
> remain stable?

The API will remain stable.  We have stringent backward compatibility
requirements, and still support every netCDF function call used since
1989.  When we introduce a new version of the interface, the previous
version is typically supported as a thin layer over the new interface,
both to test its adequacy to deliver at least equivalent functionality
annd to provide complete backward compatibility.

> I will be out of town Thursday thru Monday, so I have cced Nawajish Noman on
> this email since he is working closely with the programmers who are building
> this.

Great, I hope this is helpful.

--Russ