[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Subject: [THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2
Date: Thu, 08 Jul 2010 07:07:36 -0600
[ your THREDDS server ]

is running 4.1. Did you revert?

> On Wed, Jul 7, 2010 at 10:30 AM, Unidata THREDDS Support
> <address@hidden> wrote:
> > Hi rich: can you send a sample file?
> >
> > also, what about getting "remote debugging" working so i can poke around? 
> > is that possible?
> >
> > john
> >
> >> Hi Rich,
> >>
> >> After a quick look, nothing is jumping out at me. I'll try to take a 
> >> deeper look tomorrow or Tuesday. If not, John gets back Wednesday and may 
> >> see something right away.
> >>
> >> Ethan
> >>
> >> On 6/29/2010 3:49 PM, Rich Signell wrote:
> >> > New Ticket: Struggling with new FMRC specification in TDS 4.2
> >> >
> >> > Guys,
> >> >
> >> > I'm struggling with new FMRC specification in TDS 4.2.
> >> >
> >> > Here's my old dataset spec, which works:
> >> >
> >> >         <datasetFmrc name="coawst_2_2" path="coawst_2_2/fmrc">
> >> >                 <metadata inherited="true">
> >> >                     <documentation type="summary">ROMS Output from
> >> > Coawst_2_2 starting 1/10/06</documentation>
> >> >                 </metadata>
> >> >                 <netcdf
> >> > xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> >> >                     <aggregation dimName="runtime"
> >> > type="forecastModelRunCollection"
> >> >                         recheckEvery="5min" timeUnitsChange="true">
> >> >                         <variable name="temp">
> >> >                             <attribute name="_FillValue" type="float"
> >> > value="0.0"/>
> >> >                         </variable>
> >> >                         <variable name="salt">
> >> >                             <attribute name="_FillValue" type="float"
> >> > value="0.0"/>
> >> >                         </variable>
> >> >                         <variable name="Hwave">
> >> >                             <attribute name="_FillValue" type="float"
> >> > value="0.0"/>
> >> >                         </variable>
> >> >                         <variable name="zeta">
> >> >                             <attribute name="_FillValue" type="float"
> >> > value="0.0"/>
> >> >                         </variable>
> >> >                       <variable name="Nbed" shape="Nbed" type="double">
> >> >                         <attribute name="long_name"
> >> > value="S-coordinate at sedbed-points" />
> >> >                         <attribute name="standard_name"
> >> > value="ocean_s_coordinate" />
> >> >                         <attribute name="formula_terms" value="s: Nbed
> >> > eta: zeta depth: h a: theta_s b: theta_b depth_c: hc" />
> >> >                         <values start="-1.0" increment="-0.1"/>
> >> >                       </variable>
> >> >                         <scan
> >> > location="/usgs/data0/barmstrong/coawst/coawst_2_2/Output"
> >> >                             regExp=".*coawst.*\.nc$"
> >> > dateFormatMark="coawst_us_#yyyyMMdd_HH"
> >> >                             subdirs="false" olderThan="5 min"/>
> >> >                     </aggregation>
> >> >                 </netcdf>
> >> >             </datasetFmrc>
> >> >
> >> >
> >> > Here's my attempt to convert this to the new FMRC <collection> for TDS
> >> > 4.2, which doesn't work:
> >> >
> >> >     <featureCollection name="coawst_2_2" featureType="FMRC"
> >> > harvest="true" path="coawst_2_2/fmrc">
> >> >         <metadata inherited="true">
> >> >           <documentation type="summary">ROMS Output from Coawst_2_2
> >> > starting 1/10/06</documentation>
> >> >         </metadata>
> >> >         <collection
> >> >           spec="
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$"
> >> >           recheckAfter="15 min" olderThan="5 min"/>
> >> >         <update startup="true" rescan="0 5 3 * * ? *"/>
> >> >         <protoDataset choice="First" />
> >> >         <fmrcConfig regularize="true" datasetTypes="Best"/>
> >> >         <netcdf 
> >> > xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> >> >           <variable name="temp">
> >> >             <attribute name="_FillValue" type="float" value="0.0"/>
> >> >           </variable>
> >> >           <variable name="salt">
> >> >             <attribute name="_FillValue" type="float" value="0.0"/>
> >> >           </variable>
> >> >           <variable name="Hwave">
> >> >             <attribute name="_FillValue" type="float" value="0.0"/>
> >> >           </variable>
> >> >           <variable name="zeta">
> >> >             <attribute name="_FillValue" type="float" value="0.0"/>
> >> >           </variable>
> >> >           <variable name="Nbed" shape="Nbed" type="double">
> >> >             <attribute name="long_name" value="S-coordinate at 
> >> > sedbed-points"/>
> >> >             <attribute name="standard_name" value="ocean_s_coordinate"/>
> >> >             <attribute name="formula_terms"
> >> >               value="s: Nbed eta: zeta depth: h a: theta_s b: theta_b
> >> > depth_c: hc"/>
> >> >             <values start="-1.0" increment="-0.1"/>
> >> >           </variable>
> >> >         </netcdf>
> >> >       </featureCollection>
> >> >
> >> > I get this in the FeatureCollectionScan.log:
> >> > [29/Jun/2010:07:16:23] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> >> > initial scan found n datasets = 342
> >> > [29/Jun/2010:07:16:23] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> >> > initial scan found n datasets = 342
> >> > [29/Jun/2010:07:16:32] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> >> > ugrid variable dep_net in collection
> >> > fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> >> > [29/Jun/2010:07:16:32] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> >> > ugrid variable ero_flux in collection
> >> > fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> >> > [29/Jun/2010:09:12:54] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> >> > at Tue Jun 29 09:12:54 EDT 2010
> >> > [29/Jun/2010:11:18:35] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> >> > at Tue Jun 29 11:18:35 EDT 2010
> >> > [29/Jun/2010:11:23:57] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> >> > at Tue Jun 29 11:23:57 EDT 2010
> >> > [29/Jun/2010:11:37:22] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> >> > at Tue Jun 29 11:37:22 EDT 2010
> >> > [29/Jun/2010:11:52:21] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> >> > at Tue Jun 29 11:52:21 EDT 2010
> >> > [29/Jun/2010:15:29:55] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> >> > initial scan found n datasets = 342
> >> > [29/Jun/2010:15:29:55] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> >> > initial scan found n datasets = 342
> >> > [29/Jun/2010:15:30:04] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> >> > ugrid variable dep_net in collection
> >> > fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> >> > [29/Jun/2010:15:30:04] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> >> > ugrid variable ero_flux in collection
> >> > fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> >> > [29/Jun/2010:15:31:00] INFO  thredds.servlet.DataRootHandler.fcScan:
> >> > Schedule startup scan for
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$
> >> > at Tue Jun 29 15:31:30 EDT 2010
> >> > [29/Jun/2010:15:31:00] INFO  thredds.servlet.DataRootHandler.fcScan:
> >> > Schedule recurring scan for
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$
> >> > cronExpr=0 5 3 * * ? *
> >> > [29/Jun/2010:15:31:29] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_1/Output.*coawst.*\.nc$:
> >> > initial scan found n datasets = 431
> >> > [29/Jun/2010:15:31:29] INFO
> >> > thredds.inventory.DatasetCollectionManager:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_1/Output.*coawst.*\.nc$:
> >> > initial scan found n datasets = 431
> >> > [29/Jun/2010:15:31:30] INFO  thredds.servlet.DataRootHandler.fcScan:
> >> > Trigger scan for coawst_2_2
> >> > [29/Jun/2010:15:31:30] ERROR
> >> > thredds.inventory.DatasetCollectionManager: coawst_2_2:
> >> > DatasetCollectionManager Invalid collection= MCollection{name='
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output', dirName='
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output', wantSubdirs=false,
> >> > ff=thredds.inventory.filter.Composite@36c02df}
> >> > [29/Jun/2010:15:31:30] INFO
> >> > thredds.inventory.DatasetCollectionManager: coawst_2_2: initial scan
> >> > found n datasets = 0
> >> > [29/Jun/2010:15:31:30] ERROR ucar.nc2.ft.fmrc.FmrcDataset: Fmrc
> >> > collection is empty =fmrc:coawst_2_2
> >> > [29/Jun/2010:15:31:30] ERROR ucar.nc2.ft.fmrc.Fmrc:
> >> > /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$:
> >> > initial fmrcDataset creation failed
> >> > java.lang.IllegalStateException: Fmrc collection is empty 
> >> > =fmrc:coawst_2_2
> >> >     at ucar.nc2.ft.fmrc.FmrcDataset.buildProto(FmrcDataset.java:214)
> >> >     at ucar.nc2.ft.fmrc.FmrcDataset.setInventory(FmrcDataset.java:189)
> >> >     at ucar.nc2.ft.fmrc.Fmrc.checkNeeded(Fmrc.java:241)
> >> >     at ucar.nc2.ft.fmrc.Fmrc.triggerRescan(Fmrc.java:166)
> >> >     at 
> >> > thredds.catalog.InvDatasetFeatureCollection.triggerRescan(InvDatasetFeatureCollection.java:211)
> >> >     at 
> >> > thredds.servlet.DataRootHandler$ScanFmrcJob.execute(DataRootHandler.java:251)
> >> >     at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
> >> >     at 
> >> > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
> >> > [29/Jun/2010:15:31:30] ERROR thredds.servlet.DataRootHandler.fcScan:
> >> > InitFmrcJob failed
> >> > java.lang.RuntimeException: java.lang.IllegalStateException: Fmrc
> >> > collection is empty =fmrc:coawst_2_2
> >> >     at ucar.nc2.ft.fmrc.Fmrc.checkNeeded(Fmrc.java:248)
> >> >     at ucar.nc2.ft.fmrc.Fmrc.triggerRescan(Fmrc.java:166)
> >> >     at 
> >> > thredds.catalog.InvDatasetFeatureCollection.triggerRescan(InvDatasetFeatureCollection.java:211)
> >> >     at 
> >> > thredds.servlet.DataRootHandler$ScanFmrcJob.execute(DataRootHandler.java:251)
> >> >     at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
> >> >     at 
> >> > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
> >> > Caused by: java.lang.IllegalStateException: Fmrc collection is empty
> >> > =fmrc:coawst_2_2
> >> >     at ucar.nc2.ft.fmrc.FmrcDataset.buildProto(FmrcDataset.java:214)
> >> >     at ucar.nc2.ft.fmrc.FmrcDataset.setInventory(FmrcDataset.java:189)
> >> >     at ucar.nc2.ft.fmrc.Fmrc.checkNeeded(Fmrc.java:241)
> >> >     ... 5 more
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: PEF-311987
> > Department: Support THREDDS
> > Priority: Normal
> > Status: Open
> >
> >
> 
> 
> 
> --
> Dr. Richard P. Signell   (508) 457-2229
> USGS, 384 Woods Hole Rd.
> Woods Hole, MA 02543-1598
> 
> 


Ticket Details
===================
Ticket ID: PEF-311987
Department: Support THREDDS
Priority: Normal
Status: Open
Prev by Date: [THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2
Next by Date: [THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2
Previous by thread: [THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2
Next by thread: [THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2
Index(es):
- Date
- Thread