[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: metadata question

Hi Bruce,

Bruce Flynn wrote:

Hi Ethan - I hope your prepared for this!
Specifically, the metadata element that does not work with the TDS HTML is <metadata xlink:href="http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/"; xlink:title="Parent Dataset" />. 
It fails with the error output found at 
http://www.ssec.wisc.edu/~brucef when navigating to the dataset 
containing the above metadata.  The above XML works if you change the 
tag to documentation and place it in a metadata element.  ???

One problem is that your catalog is not valid according to the 
InvCatalog schema. The TDS is pretty lenient when reading catalogs so I 
don't think any of this is causing the problem you are seeing. But here 
are the details. Just a few changes fix it. At the catalog level, 
service elements must come before dataset elements. The datasetRoot 
element needs a location attribute instead of a dirLocation attribute 
(either is allowed in datasetScan but not in datasetRoot). Inside of 
datasets, access elements come after metadata and dataSize elements. 
Also, metadata elements have to come before dataset elements. I've 
attached a new version of your catalog that fixes these problems.

I think the main problem is that the resource you are linking to (looks 
like your TDS is down currently, so I haven't actually been able to look 
at it) appears to not be well formed. Here's the part of the error 
message that leads me to this conclusion:

   **Error: InvCatalogFactory.readOtherXML failed on
   
http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/

   Exception= org.xml.sax.SAXParseException The element type "meta"
   must be terminated by the matching end-tag "</meta>".

   ** failed to read thredds metadata at =
   
http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/
   for datasetmichelle.tar



One other thing that might cause problems. It looks like the TDS HTML 
view only displays a link for the metadata xlink:href if that is the 
only metadata element. Sounds like a TDS bug, so we'll look into that.

Can you try the attached catalog and check for well-formed XML in the 
linked resource and see how well that works?

What I am trying to accomplish is kinda hard to explain, but I'll give 
it a shot...

I have 2 parallel TDS representations for the same data. There is a 
"Shelves" tree generated using datasetScan that is an exact recreation 
of our datastore back-end on our XSAN.  The data-store is in a format 
that is not user navigable, so for users to get to their data we 
provide soft-links to something more "friendly," which we call the 
"user links."  The XSAN back-end will never change, but the user links 
might.  I have python scripts that generate TDS catalogs from the user 
links that replicate the soft-linked user navigable access to the XSAN 
back-end.

So...  If a link to a particular dataset(ie. netcdf file) is 
distributed from one user to another, that user may also be interested 
in the related files in the parent dataset.  Therefore, there was a 
requirement to have a link to the parent dataset containing related 
files. Since the user links may change it made sense to provide that 
link to the shelves tree, which is static.

For example, the user links URL for the above metadata tag is 
http://localhost:8080/thredds/dodsC/User_Links/Users/johnm/catalog.html, 
which actually represents the same data.

So, it seems unnatural to classify the link to a parent dataset as 
documentation.

Well, I'm not sure I really understand but it sounds like you want to 
explicitly capture a parent/child relationship that the THREDDS catalogs 
being produced don't capture. The only way to capture this information 
would be to build a single THREDDS catalog with the parent/child 
relationships showing up as the hierarchical structure of the catalog. 
The reason we don't have a place for this kind of information outside of 
the hierarchical structure is because a catalog can be pointed to from 
multiple locations so there isn't necessarily a single parent to any 
given catalog.

Since there isn't an explicitly defined way of doing this in THREDDS 
catalogs and it sounds like you simply want this information displayed 
to the user rather than making this a standard, machine readable bit of 
information, I would suggest using the documentation element. (Here's my 
thinking on this ... Both documentation and metadata elements are simply 
providing additional information about the dataset. The difference is 
the intended audience: a human user, in which case the free-text and/or 
XHTML of the documentation element should be used; or a machine, in 
which case the structured information of the metadata element should be 
used.)

You could just stick some text and an HTML link into a documentation 
element:

<documentation>Other related datasets are available in this datasets <a 
href="???">parent collection</a>.</documentation>

or as we've been discussing, an xlink to another document.

Hope this all helps.

Ethan

--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    address@hidden
P.O. Box 3000
Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
---------------------------------------------------------------------------


<?xml version="1.0" encoding="UTF-8"?>


<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
         xmlns:xlink="http://www.w3.org/1999/xlink";>

  <service name="group" base="" serviceType="Compound">

    <service name="dods" base="/thredds/dodsC/" serviceType="OpenDAP"/>

    <service name="http" base="/thredds/fileServer/" serviceType="HTTPServer"/>

    <datasetRoot path="Shelves" location="/Volumes/gulf1/.bruce_dev/"/>

  </service>

  <service name="other" base="/thredds/fileServer/" serviceType="HTTPServer"/>

  <dataset name="7a754fd85b39c6b6795936bbcd26955d" 
ID="7a754fd85b39c6b6795936bbcd26955d">

    <metadata inherited="true">

      <documentation type="summary">

        MM5 code for michelle dataset setup by John M

      </documentation>

      <keyword>MM5</keyword>

      <keyword>michelle</keyword>

      <keyword>Jan2003</keyword>

      <property name="link" value="Users/johnm"/>

      <creator>

        <name>eriko</name>

        <contact email="address@hidden"/>

      </creator>

      <publisher>

        <name>Administrator</name>

        <contact email="address@hidden"/>

      </publisher>

    </metadata>

    <dataset name="michelle.tar" 
ID="7a754fd85b39c6b6795936bbcd26955d-michelle.tar">

      <metadata>

        <documentation>

          The dataset in which this file is contained is available at:

        </documentation>

        <property name="MD5 Checksum" value="a41252c9243d6ee8b66ff353736a3fc4"/>

        <dataFormat>text/plain</dataFormat>

      </metadata>

      <metadata
              
xlink:href="http://sands01.ssec.wisc.edu:8080/thredds/Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/";
              xlink:title="Parent Dataset"/>

      <dataSize units="bytes">1984233675</dataSize>

      <access 
urlPath="Shelves/7a75/4fd8/7a754fd85b39c6b6795936bbcd26955d/michelle.tar" 
serviceName="other"/>

    </dataset>

  </dataset>

</catalog>