[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #HIP-310486]: issue about "latest" proxy dataset

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [THREDDS #HIP-310486]: issue about "latest" proxy dataset
  • Date: Thu, 10 Jul 2008 11:13:45 -0600
Hi Zhu Jiang,

> > I configured the thd following the instruction of reference(
> > http://www.unidata.ucar.edu/Projects/THREDDS/tech/reference/DatasetScan.html).
> > I got problem with the <addProxies> element. It appeared the error
> > message as the attached picture showed. The catalog.xml file is also
> > attached. Could you point out what I had done wrong?
> >
> > Another question is that the data we organized by day catalog in local
> > file system like:
> >   NAFP_T213_FOR_FTM
> >             -- 01
> >             -- 02
> >             -- 03
> >             -- 04
> >             -- 05
> >             -- 06
> >                --01
> >                --02
> >                  -- NAFP_T213_FOR_FTM-advtmp-2008060400.nc
> >                  -- NAFP_T213_FOR_FTM-covr-2008060400.nc
> >                  -- ...
> >            ...*
> > *I used filter element to seperate the data. I got the lastest.xml in
> > each NAFP_T213_FOR_FTM/06/* directory. However,
> > NAFP_T213_FOR_FTM/lastest.xml pointing to
> > NAFP_T213_FOR_FTM/06/lastday/*.nc is what I hope to get. Should I
> > reorganize the data in the tree of
> > NAFP_T213_FOR_FTM/NAFP_T213_FOR_FTM-###/*.nc? or other suggestions?

Not sure I understand your directory structure. Is it 
NAFP_T213_FOR_FTM/<month>/<day>? And the current month also contains a lastday/ 
directory?

From your catalog datasetScan, filter, and proxy:

  <datasetScan name="NAFP_T213_FOR_FTM Data" path="t213" 
location="/space/MetData/NAFP_T213_FOR_FTM/" >
    ...
    <filter>
      <include wildcard="*.nc"/>
    </filter>
    ...
    <addProxies>
      <simpleLatest serviceName="tdsserver" />
      <latestComplete name="latestComplete.xml" top="true" 
serviceName="tdsserver" lastModifiedLimit="60" />
    </addProxies>
    ...
  </datasetScan>

you should be getting a catalog for each directory under the NAFP_T213_FOR_FTM/ 
directory. Each directory that contains *.nc files should list those files and 
a "latest.xml" and "latestComplete.xml".

Is that what you are seeing? Are you getting any latest datasets in the 
NAFP_T213_FOR_FTM/ directory? I would guess you were not seeing the latest 
datasets there.

There isn't a way to tell the latest proxy datasets to scan a directory tree or 
look in a particular directory. It currently only deals with scanning one 
directory at a time.

Sorry, I don't have any other suggestions. Reorganizing your directory 
structure might help but I think you would have to collapse the entire 
<month>/<day> structure before the latest proxy dataset would give you what you 
want. And obviously that has plenty of drawbacks.

Ethan

Ticket Details
===================
Ticket ID: HIP-310486
Department: Support THREDDS
Priority: Normal
Status: Open