Re: Satellite Data in netCDF

> Organization: NCAR/MMM
> Keywords: 199411092242.AA00943 TeraScan TDF terascan SeaSpace Fahle

Hi David,

> A couple of days ago I was talking with Ben Domenico about the 
> possibility that NCAR might be able to acquire GOES-8 satellite
> data from a receiving station of our own.  BACKGROUND:  We have 
> recently acquired a Terascan analysis workstation from Sea Space 
> (San Diego) which has all the necessary software and hardware 
> hooks to acquire the direct transmission satellite data broadcast 
> from geostationary and polar-orbiting satellites (all we would have 
> to do is add a suitable antenna system).
> 
> One of the things that came up in my discussions with Ben was the 
> desirability in maintaining satellite data in netCDF format.  I believe 
> that the Joe Fahle's Terascan Data Format (TDF) used by Sea Space is
> "essentially" netCDF, but with some extentions.  I don't know how
> critical the extentions are that they have introduced.  I sent 
> Joe an e-mail asking his view on the current compatibility of his
> TDF with "standard" netCDF.  I haven't heard anything back from
> him yet, but I just "discovered" an interesting discussion of TDF
> data sets in the on-line "man" pages that came with their software.
> 
> In case you're interested, I'll append the man pages to the end
> of this e-mail.  NOTE the reference to Rew (1988) for netCDF.  

Thanks for appending the man pages.  Your version is more recent than what
was in the TDF library sources I got from Fahle last year.

While TDF and netCDF originally used a similar model for data access, the
implementations were always quite different.  As far as I know, there is no
software currently available for converting files in one format to the
other.

Fahle has introduced an additional "relation" abstraction into the TDF
data model and added support for "links" that reference data from other
files and allow the import of non-TDF data.  The "relation" objects are
ordered lists of variables, so can be mapped into a netCDF character array
variable using suitable conventions.

Support for something equivalent to TDF links has been on our development
list for netCDF, but we haven't had the resources to do add any extensions
to netCDF in the last year or so.  With some programming effort, someone
could probably add a TDF software layer over netCDF that would provide the
TDF interfaces to netCDF files, enforcing appropriate conventions for
representing relations and links using netCDF objects, but if the TeraScan
software makes heavy use of TDF links, there might be a significant
performance cost.

There are other operations supported by TDF that are expensive to implement
on top of the netCDF interface, for example deleting a variable.  With
netCDF, this requires work equivalent to copying the dataset.

In general, I think the TDF data model and interface are in some ways more
advanced than netCDF's.  However no Fortran or C++ interface is available
for TDF, as far as I know.  One other consideration that may be important:
TDF files don't use XDR (a machine-independent eXternal Data
Representation), so they are not portable across machine architectures that
use different floating-point representations or byte-ordering for integers,
as far as I know.

--
Russ Rew                                                UCAR Unidata Program
address@hidden                                          P.O. Box 3000
http://www.unidata.ucar.edu/                          Boulder, CO 80307-3000