Re: Standard for observational data

On Sat, Feb 18, 2012 at 5:34 PM, John Caron <address@hidden> wrote:

Hi Arlindo:

This proposal is now part of CF, see chapter 9 and Appendix H in:

http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html

Ill look at your specifics when i can get to it, it looks like a collection of profiles at first glance.

Meanwhile have a look at that doc.

I just read it, it is very close to what is in the draft document. My main grip with the current Profile related feature types is that they prescribe the rank of the coordinates (third column is mandatory), preventing people from exploring regularity in their data:

profile

an ordered set of data points along a vertical line at a fixed horizontal position and fixed time

data(i,o)

x(i) y(i) z(i,o) t(i)

timeSeriesProfile

a series of profile features at the same horizontal position with monotonically increasing times

data(i,p,o)

x(i) y(i) z(i,p,o) t(i,p)

trajectoryProfile

a series of profile features located at points ordered along a trajectory

data(i,p,o)

x(i,p) y(i,p) z(i,p,o) t(i,p)

A more general approach would let people drop dimensions in their coordinates (or data) as the regularity in their data permits. (It makes no sense for me to have to define z(i,p,o) when z(p) will suffice.) For example, something along these lines would accomodate not only the 3 types of profiles above but also many other permutations:

genericProfile

a generic collection of profile features.

data(i,p,o)|data(i,p)

x(i,p)|x(i) y(i,p)|y(i) z(i,p,o)|z(i,p)|z(p) t(i,p)|t(i)

(Notice that for a given rank there is only possibility for the coordinates). 

   Am my missing something more fundamental in CF-1.6?

        Arlindo

 

John

On 2/18/2012 10:53 AM, Arlindo da Silva wrote:

On Tue, Dec 30, 2008 at 1:24 PM, John Caron <address@hidden> wrote:

I have proposed a point obs standard to CF here:

 http://cf-pcmdi.llnl.gov/trac/ticket/37
 http://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions

Feedback would be appreciated, as we'd like to start swimming if we dont sink ;^)

This proposal (when accepted) will supercede the "Unidata Observation Dataset Convention". We are looking at being able to encode some of the NWS obs data stream, as it comes in on the IDD. If there are specific requirements you have, send them along and we will include them in our use cases.

I am not sure where is the best place to post this question/coment, so I am starting with you. I am trying to produce CF compliant NetCDF files for NASA's DISCOVER-AQ airborne campaign. Specifically, I have a collection of profiles from "aircraft spirals" around fixed locations which have been binned at a fixed vertical grid. The natural way to index the data is:

    data(profile,altitude)

with coordinates:

   longitude(profile)

   latitude(profile)

   time(profile)

but

   altitude(altitude)  // vertical grid is the same for *all* profiles.

None of the CF feature types described in this word document 

      https://cf-pcmdi.llnl.gov/trac/attachment/ticket/37/CFch9-may4.docx

seems to cover this case. The closest is "trajectory" but it has no provision for vertical coordinates and my time coordinate is certainly *not* monotonic.  To describe the data I'd need something like:

griddedProfile	a generic collection of profile features on a fixed vertical grid; time is not necessarily monotonic.
	data(i,p)	x(p) y(p) z(i) t(p)

BTW, this same data structure would also be needed for the airborne HSRL LIDAR data.  

As for my feedback on the current CF draft. I find this notion of "featureType" a bit awkward, given that you may need a large number of permutations to cover all use cases. In particular, the mandatory space-time coordinates is problematic. While requiring (x,y,z,t) coordinates is OK, requiring that they are dimensioned a certain way is too restrictive, specially because this information is easily discoverable. For example, if this restriction would be lifted, the featureType "profile" could be used to describe my data.

  Thank you!

      Arlindo