Re: latest Catalog XML

Daniel Holloway wrote:
> 
> John Caron wrote:
> 
> > heres another strawman, incorporating our latest discussions:
> >
> > http://www.unidata.ucar.edu/projects/THREDDS/xml/InvCatalog.0.6d.dtd
> >
> > Recent changes:
> >
> 
> ....
> 
>     I've used the DTD listed above to create two examples of what
> I believe I could return in an XML-based dods-dir response.  The
> catalogs represent simple directory representations of dods-accessible
> data; level1.xml is a higher-level directory containing only subdirs
> and level2.xml is one of the subdirs containing the dods-accessible
> datafiles.
> 
>     Both of these files validate, I get some warnings but mostly
> due to style issues, there are no errors.   These examples represent
> the minimum information that I could provide, there are ways I
> could augment this service to provide some of the additional fields
> represented in the DTD but this example is meant to illustrate a
> minimum set.
> 
>     Points to note:
> 
> 1: I'd need to provide 2 service elements, one for
> type = DODS, and one for type = Other, where Other represents
> the dods-dir service.   In many cases there will be both datafiles
> and subdirectories in a file system.   Do I need to have two
> service elements?   If so, in the 'Level2.xml' example how do
> I indicate which service to use without adding an explicit
> access element for each dataset element?

In each dataset element you can indicate the service it uses with the
serviceName attribute; its value would be the same as the value of the service
elements name attribute.

> 2:  I use 'collection' as a simple filesystem collection, there
> may not be any meaningful relation between the files contained
> in the directory other than they're accessible via the DODS DAP.
> 
> 3:  Am I using 'catalogRef' correctly?  The intention is to
> indicate another collection level that should only be traversed
> at the user's request.

catalogRef should reference a THREDDS catalog. Yours is referencing a dods-dir
page. I've reworked and attached your examples (with fewer years in level1.xml
and fewer example datasets in level2.1985.xml).

1) I changed the catalogRefs in level1.xml to point to the level2.*.xml files.
2) I added datasets in level1.xml that give access to the dods-dir pages.
3) I added a serviceName attribute in the dataset elements in the level2.*.xml
files.

A few alternatives:

1) You could have just one level. Each year could be a collection that contains
the individual datasets.

2) Again, just one level. Each year is a dataset with the dods-dir access and
contains sub-datasets for each individual dataset.



I'm working on the THREDDS catalog generator tool. Currently I'm working on
expanding DODS file servers into THREDDS catalogs. After I get that working I'd
like to work on crawling dods-dir pages. So, I'll be wanting to pick your brains
on this stuff.

Ethan

>    Dan
> 
> >
> > 1) Collections as datasets. Ethan and I think the cleanest way to allow
> > "collections as datasets" is to allow nested datasets. Collections remain
> > just groups of datasets. A Collection as a dataset is done as a dataset with
> > nested datasets. Nested dataset elements (Joe's meaning #2) should imply (in
> > some way we need to clarify better) nested datasets (Joe's meaning #1 and
> > #3).
> >
> > 2) allow compound services again. I have talked myself into that these will
> > be often useful.
> >
> > 3) services are now contained within any collection, rather than having to
> > be all in the top catalog element. They are scoped by the collection they
> > are in (so we no longer use ID, since those are global). This makes a
> > catalogRef have (almost) the same semantics as a collection.
> >
> > 4) a catalog now only has exactly one collection element. (i considered
> > eliminating catalog but i think its better this way).
> >
> > 5) "attribute" changed to "property" (tired of saying "the attribute
> > attribute")
> >
> > 6) access element can specify an absolute URL with a serverType -or- a
> > reletive URL with a serverID.
> >
> > Unless I get a barrage of objections, I will document this in more detail
> > ASAP. I will be gone for a week starting next Tuesdy, so Ethan will continue
> > the conversation as needed. Hopefully, we can converge soon and take a
> > break! I think I have incorporated all the good ideas i have heard in this
> > discussion. Have I left out something, or do you think any feature is not
> > worth the complexity?

-- 
Ethan R. Davis                       Telephone: (303) 497-8155
Software Engineer                    Fax:       (303) 497-8690
UCAR Unidata Program Center          E-mail:    address@hidden
P.O. Box 3000
Boulder, CO  80307-3000              http://www.unidata.ucar.edu/

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE catalog SYSTEM 
"http://www.unidata.ucar.edu/projects/THREDDS/xml/InvCatalog.0.6d.dtd";>
<catalog name="htn_sst_decloud" version="0.6d">
   <collection name="htn_sst_decloud/">
      <service name="URI-DODS" serviceType="DODS"
               base="http://dods.gso.uri.edu/dods-3.2/nph-dods/"/>
      <service name="DODSdirectory" serviceType="Other"
                     base="http://dods.gso.uri.edu/dods-3.2/nph-dods/"/>

      <dataset name="DODS_dir of htn_sst_decloud/1985/"
               serviceName="DODSdirectory"/>
      <catalogRef xlink:title="htn_sst_decloud/1985/"
                  xlink:href="./level2.1985.xml"/>

      <dataset name="DODS_dir of htn_sst_decloud/1986/"
               serviceName="DODSdirectory"/>
      <catalogRef xlink:title="htn_sst_decloud/1986/"
                  xlink:href="./level2.1986.xml"/>

   </collection>
</catalog>

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE catalog SYSTEM 
"http://www.unidata.ucar.edu/projects/THREDDS/xml/InvCatalog.0.6d.dtd";>
<catalog name="htn_sst_decloud/1985" version="0.6d">
  <collection name="htn_sst_decloud/1985">
     <service name="URI-DODS" serviceType="DODS"
              base="http://dods.gso.uri.edu/dods-3.2/nph-dods/"/>
     <service name="DODSDirectory" serviceType="Other"
              base="http://dods.gso.uri.edu/dods-3.2/nph-dods/"/>
     <dataset name="k85001182828.htn_d" serviceName="URI-DODS"
              urlPath="htn_sst_decloud/1985/k85001182828.htn_d.Z"/>
     <dataset name="k85001182828.htn_d" serviceName="URI-DODS"
              urlPath="htn_sst_decloud/1985/k85002181750.htn_d.Z"/>
  </collection>
</catalog>