[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

orthogonality (was Re: New attempt)

Benno, John,
As I am currently immersed in Web-service-think (particularly WSDL, which seems to basically be a more generalized version of what THREDDS catalog is attempting for scientific data services) I would propose that the principle of orthogonality might be a useful tool for deciding on these issues.
For an XML design, orthogonality means whether or not a given tag or attribute represents a distinct concept that is in no case expressible using existing tags and attributes.
While a completely orthogonal tag set results in somewhat less succinct documents than an approach which defines a number of "special case" tags, the advantage is that it yields the maximum ratio of expressiveness to schema complexity.
WSDL is an extreme example of this approach. Unlike them, we may want to define special-case tags for very common cases to make the actual documents less unwieldly.
The purpose of this message is not to suggest exactly which cases these might be; rather, I am just suggesting we take a look the current and proposed DTD down from this perspective. 

-------------------
A summary of the completely orthogonal concepts I believe we have introduced thus far (not necessarily named the same as the tags that currently express the concept): 

service type - a named mechanism for accessing scientific data

access - a (named?) binding of a URI to a specific service type

metadata type - a named convention for description of scientific data

metadata - a (named?) binding of a text fragment to a specific metadata type

metadata reference - a (named?) binding of a URI to a specific metadata type

dataset - a named collection of access objects and metadata

collection - a named collection of datasets
collection reference - the URI of a THREDDS XML document containing a collection 

----------------

In contrast, the following concepts are not clearly orthogonal to me:
service path, server path, collection path, dataset path, suffix - since they are only used in the context of the access object, they don't really add meaning - any catalog using these attributes is equivalent to one without them, which uses absolute uri's for all of its access objects
compound service / service list - this also doesn't strictly speaking add meaning since services are only used in the context of access objects - thus, one access with a compound service type is functionally equivalent to n access objects with simple service types.
service subtype - unless the values for this attribute are given standard meanings, this is equivalent to a named access object. even if the values do have standard meanings, there still seems to be some overlap with metadata type.
catalog, server - if you factor out the path attribute, these are equivalent to collections 

documentation - equivalent to metadata with a human-readable metadata type
document - same as documentation, except with the connotation that it is not critical to interpreting the dataset 

--------------------
One that I am not sure about is the "attribute" tag, since I am not clear on how this is intended to be used. Is it for the THREDDS parser, or passed directly to the user? Will there be standardized names and values for attributes?
A reminder, I am not trying to say specifically whether any of these tags should be kept or dropped. I am merely suggesting that we might want to focus on tags and attributes that represent orthogonal concepts, and be a bit more choosy about the rest.
Also, I would suggest that any proposed extensions that *are* genuinely orthogonal to the original tag set (although I'm not sure we've had any thus far) be given special consideration, since by definition, there is no workaround if they are not included. 

John, hope this is useful input.

- Joe



Benno Blumenthal wrote:


John Caron wrote:
Im trying to think what is the meaning of serviceType="Catalog" in 
    general?
    What should the client assume? It seems that if you want the client
    to be
    able to get the collection as a dataset, then you add a dataset
    element. If
    you want the client to "drill down" further, then a collection or
    catalogRef
    element can do that. What I have removed is the clear association
    between
    the dataset element and the collection, eg that these are the same
    thing. I
    have also made it more cumbersome(need two elements). I agree these are
    weaknesses.
The client does not know that these are two different ways of looking at the same thing -- the key piece of information that was trying to be conveyed. The client does not have to present both -- maybe the client only presents THREDDS choices because it has no DODS capabilities, another client does not present the drill-down because it does have DODS capabilities.
The fact that you can produce a dataset as COARDS vs DIF, etc is 
    also for me
    not so great of an example. Rather than modifying the underlying
    data acess
    (eg DODS), it seems simpler to add a metadata element. I admit that
    this is
    just an idea which has not been done yet. And you already have a
    server that
    does in fact modify the data access. But think of it from a client POV.
    Should she search through the services looking for a service of type
    DODS,
    subtype COARDS? Or search through the metadata looking for COARDS
    metadata,
    independent of service type?
My point was your metadata tag was services for metadata. clients that can only handle COARDS metadata would ask for COARDS metadata services. 


    A more compelling example would be where the dataset is served up
    through
    FTP and DODS, and ADDE, etc. But then I wonder/doubt whether one URL is
    likely to be able to be used for all these services.
DODS already has multiple services -- ascii is not necessarily present, some of the selection interfaces are optional, metadata is optional.
> > I am also concerned about XYZT clients (4D world view) -- how can I 
    protect
     > them against higher (and other) dimensional data (ensemble member
    count,
     > spectral, different kinds of time (forecast start, lead, target
    time)?   I
     > could convert to multiple datasets, or spatial grids, but it
    would be nice
    to
     > advertise the service.  As well as supporting various binary and
    ascii
    data
     > formats.  Or the THREDDS dataset (as opposed to collection/catalog)
     > description...

    I am not clear of "protect", did you mean "project" ?
Protect -- 4D world clients simply fail when given something else, I would like them to have an alternative.
> 2) The access for the dataset LEVITUS94 is again via THREDDS 
    (the present
     > collection) or via DODS (the access statement).  Adding another
    dataset
    inside
     > the collection called "Daily" is not the same meaning at all.

    Sorry, I should have had:

    <dataset name="LEVITUS94 dataset" urlPath="SOURCES/.LEVITUS94/dods"/>
    <catalogRef xlink:title="Drill down into dataset"
    xlink:href="http://iridl.ldeo.columbia.edu/SOURCES/LEVITUS94/thredds.xml";
    />

    In this case the dataset is presented to the user for immediate
    selection
    AND a link is presented for drilling further down.
This is the wrong example: LEVITUS94 was a collection that was also available through DODS -- now you have lost it completely. It is an example for the subdatasets ANNUAL, etc, with the flaw that clients no longer can tell that these are two different ways of getting the same thing as mentioned above.
Besides, multiple services also will show up in aggregations of THREDDS catalogs -- multiple servers serving the same dataset could be represented as a single entry with multiple services -- in this case, services with identical attributes except for path information.
The main thing service does is to let you specify a type and factor 
    out the
    common URL base. then this is passed to "protocol aware" code.
    Because the
    das,dds,dods,info,ascii subservice URLs are always regular in how
    they are
    formed, it seems unnecessary to actually specify them. In principle
    subservices are probably useful but some concrete examples are needed.
As I mentioned earlier, not all DODS servers have all the services. Even if they did, it would not hurt to be able to list them. 

     While I have not given examples, different datasets will have
    different
> services, which is why I kept specifying using the access tag. Some 
    datasets
     > will be incompatible with certain representations, so the service
    lists
    will
     > vary.   One could argue that the THREDDS standard collection is a
    dataset
    not
     > available via DODS -- certainly that is the case for me.

    I understand you want to compactly specify what services are
    available for
    datasets. Im not sure we have enough examples to make sure we are
    doing it
    right. I am also oriented towards incremental design, doing what we
    can get
    right and iterating.
OK with me, but we have lost the structure I was trying to express -- alternate ways of accessing the same object, with emphasis on the same object. 

Benno

--
Dr. M. Benno Blumenthal          address@hidden
International Research Institute for climate prediction
Lamont-Doherty Earth Observatory of Columbia University
Palisades NY 10964-8000                  (845) 680-4450




--
Joe Wielgosz
address@hidden / (707)826-2631
---------------------------------------------------
Center for Ocean-Land-Atmosphere Studies (COLA)
Institute for Global Environment and Society (IGES)
http://www.iges.org