[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NetCDF Java IOSPs



Wright, Bruce wrote:

Hi Ethan,

Thanks for the clarification.

The point you make on implementing convention mapping at a different
level is a very good one...and actually sits well with what Phil has
been looking at, which is the convention mapping, rather than the
'format conversion' aspect. 
I think it would be useful to handle the conversion mapping through the
IOSP in a pluggable way, as you suggest. This is probably something we
can consider with the work that Sheila should (hopefully) be doing with
the THREDDS Data Server - we want to look at how easy it would be to
build a IOSP for our bespoke data formats, and we could consider how the
'convention mapping' work that Phil is doing could be brought into this
in clean way.
The CoordSysBuilder interface is one way to plug in convention parsing. More info is at:
http://www.unidata.ucar.edu/software/netcdf-java/tutorial/CoordSysBuilder.html
Ethan is right that the IOSP and Conventions operate on different levels, but still, when writing your own IOSP, it is often convenient to add the metadata to use an existing convention (CF or our own internal _Coordinate). this obviates the need for a seperate CoordSysBuilder. to undersstand the details, you probably have to implement one.
Note that the main point of Coordinate Conventions in the netcdf-java library is to identify the coordinate systems, which allows you to serve the dataset with WCS etc. You certainly can also use it as the place to add standard names if you wish. 
Regards,
Bruce
--
Bruce Wright  IT Architect
Met Office  FitzRoy Road Exeter EX1 3PB United Kingdom
Tel: +44 (0)1392 886481 Fax: 0870 9005050 E-mail: address@hidden http://www.metoffice.gov.uk 

This e-mail is intended for the addressee only.  Its contents are
provided 'in confidence' and may be covered by contractual, legal or
other privilege.  If you are not the addressee, you may not use or copy
it to any other person.


-----Original Message-----
From: Ethan Davis [mailto:address@hidden] Sent: 28 November 2007 17:32 
To: Wright, Bruce
Cc: John Caron; Bentley, Philip; Needham, Sheila
Subject: Re: NetCDF Java IOSPs

Hi Bruce,

Wright, Bruce wrote:
I hadn't realised that the IOSPs did not map to standard names; is this due to the difficulty in obtaining unique mapping for all variables? I'd naively assumed that there would be a strong focus on standard names for interoperability, but having started to look into this (the work that Phil's been doing), I can begin to see the difficulty in this, but I do think it's very useful to try to provide 
this mapping.


The IOSP was designed simply to allow datasets to be read and mapped
into the netCDF data model and read through the netCDF API. It was not
designed to deal with any of the netCDF conventions, like CF. We have
always tried to keep the conventions at a different level than the data
model and API.

Since conventions are so important, I wonder if we shouldn't consider a
more pluggable way to add convention attributes and such at the IOSP
level.
However, my interest was about the use of configuration files to handle the mappings within the IOSPs; do I understand correctly form your comment about "100% Java for portability", that all mappings are hard coded into the Java, with no separate configuration files (e.g. CSV, XML, etc)? 

By "100% Java", we mean that we haven't linked into any existing
FORTRAN, C, etc code for reading data files. Instead we use/write Java
code to read the data files.

That Java code could certainly use separate configuration files for
adding information to the netCDF view of the data that isn't in the
dataset being read. For instance, I'm pretty sure our GRIB code has
external tables it references. Others are more hard coded.

Ethan