[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [THREDDS #PEF-311987]: Struggling with new FMRC specification in TDS 4.2
  • Date: Thu, 01 Jul 2010 18:11:33 -0600
Hi Rich,

After a quick look, nothing is jumping out at me. I'll try to take a deeper 
look tomorrow or Tuesday. If not, John gets back Wednesday and may see 
something right away.

Ethan

On 6/29/2010 3:49 PM, Rich Signell wrote:
> New Ticket: Struggling with new FMRC specification in TDS 4.2
> 
> Guys,
> 
> I'm struggling with new FMRC specification in TDS 4.2.
> 
> Here's my old dataset spec, which works:
> 
>         <datasetFmrc name="coawst_2_2" path="coawst_2_2/fmrc">
>                 <metadata inherited="true">
>                     <documentation type="summary">ROMS Output from
> Coawst_2_2 starting 1/10/06</documentation>
>                 </metadata>
>                 <netcdf
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>                     <aggregation dimName="runtime"
> type="forecastModelRunCollection"
>                         recheckEvery="5min" timeUnitsChange="true">
>                         <variable name="temp">
>                             <attribute name="_FillValue" type="float"
> value="0.0"/>
>                         </variable>
>                         <variable name="salt">
>                             <attribute name="_FillValue" type="float"
> value="0.0"/>
>                         </variable>
>                         <variable name="Hwave">
>                             <attribute name="_FillValue" type="float"
> value="0.0"/>
>                         </variable>
>                         <variable name="zeta">
>                             <attribute name="_FillValue" type="float"
> value="0.0"/>
>                         </variable>
>                       <variable name="Nbed" shape="Nbed" type="double">
>                         <attribute name="long_name"
> value="S-coordinate at sedbed-points" />
>                         <attribute name="standard_name"
> value="ocean_s_coordinate" />
>                         <attribute name="formula_terms" value="s: Nbed
> eta: zeta depth: h a: theta_s b: theta_b depth_c: hc" />
>                         <values start="-1.0" increment="-0.1"/>
>                       </variable>
>                         <scan
> location="/usgs/data0/barmstrong/coawst/coawst_2_2/Output"
>                             regExp=".*coawst.*\.nc$"
> dateFormatMark="coawst_us_#yyyyMMdd_HH"
>                             subdirs="false" olderThan="5 min"/>
>                     </aggregation>
>                 </netcdf>
>             </datasetFmrc>
> 
> 
> Here's my attempt to convert this to the new FMRC <collection> for TDS
> 4.2, which doesn't work:
> 
>     <featureCollection name="coawst_2_2" featureType="FMRC"
> harvest="true" path="coawst_2_2/fmrc">
>         <metadata inherited="true">
>           <documentation type="summary">ROMS Output from Coawst_2_2
> starting 1/10/06</documentation>
>         </metadata>
>         <collection
>           spec="
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$"
>           recheckAfter="15 min" olderThan="5 min"/>
>         <update startup="true" rescan="0 5 3 * * ? *"/>
>         <protoDataset choice="First" />
>         <fmrcConfig regularize="true" datasetTypes="Best"/>
>         <netcdf 
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>           <variable name="temp">
>             <attribute name="_FillValue" type="float" value="0.0"/>
>           </variable>
>           <variable name="salt">
>             <attribute name="_FillValue" type="float" value="0.0"/>
>           </variable>
>           <variable name="Hwave">
>             <attribute name="_FillValue" type="float" value="0.0"/>
>           </variable>
>           <variable name="zeta">
>             <attribute name="_FillValue" type="float" value="0.0"/>
>           </variable>
>           <variable name="Nbed" shape="Nbed" type="double">
>             <attribute name="long_name" value="S-coordinate at 
> sedbed-points"/>
>             <attribute name="standard_name" value="ocean_s_coordinate"/>
>             <attribute name="formula_terms"
>               value="s: Nbed eta: zeta depth: h a: theta_s b: theta_b
> depth_c: hc"/>
>             <values start="-1.0" increment="-0.1"/>
>           </variable>
>         </netcdf>
>       </featureCollection>
> 
> I get this in the FeatureCollectionScan.log:
> [29/Jun/2010:07:16:23] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> initial scan found n datasets = 342
> [29/Jun/2010:07:16:23] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> initial scan found n datasets = 342
> [29/Jun/2010:07:16:32] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> ugrid variable dep_net in collection
> fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> [29/Jun/2010:07:16:32] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> ugrid variable ero_flux in collection
> fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> [29/Jun/2010:09:12:54] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> at Tue Jun 29 09:12:54 EDT 2010
> [29/Jun/2010:11:18:35] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> at Tue Jun 29 11:18:35 EDT 2010
> [29/Jun/2010:11:23:57] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> at Tue Jun 29 11:23:57 EDT 2010
> [29/Jun/2010:11:37:22] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> at Tue Jun 29 11:37:22 EDT 2010
> [29/Jun/2010:11:52:21] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$: rescan
> at Tue Jun 29 11:52:21 EDT 2010
> [29/Jun/2010:15:29:55] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> initial scan found n datasets = 342
> [29/Jun/2010:15:29:55] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$:
> initial scan found n datasets = 342
> [29/Jun/2010:15:30:04] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> ugrid variable dep_net in collection
> fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> [29/Jun/2010:15:30:04] ERROR ucar.nc2.ft.fmrc.FmrcDataset: cant find
> ugrid variable ero_flux in collection
> fmrc:/usgs/data0/barmstrong/coawst/coawst_2_2/Output.*coawst.*\.nc$
> [29/Jun/2010:15:31:00] INFO  thredds.servlet.DataRootHandler.fcScan:
> Schedule startup scan for
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$
> at Tue Jun 29 15:31:30 EDT 2010
> [29/Jun/2010:15:31:00] INFO  thredds.servlet.DataRootHandler.fcScan:
> Schedule recurring scan for
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$
> cronExpr=0 5 3 * * ? *
> [29/Jun/2010:15:31:29] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_1/Output.*coawst.*\.nc$:
> initial scan found n datasets = 431
> [29/Jun/2010:15:31:29] INFO
> thredds.inventory.DatasetCollectionManager:
> /usgs/data0/barmstrong/coawst/coawst_2_1/Output.*coawst.*\.nc$:
> initial scan found n datasets = 431
> [29/Jun/2010:15:31:30] INFO  thredds.servlet.DataRootHandler.fcScan:
> Trigger scan for coawst_2_2
> [29/Jun/2010:15:31:30] ERROR
> thredds.inventory.DatasetCollectionManager: coawst_2_2:
> DatasetCollectionManager Invalid collection= MCollection{name='
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output', dirName='
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output', wantSubdirs=false,
> ff=thredds.inventory.filter.Composite@36c02df}
> [29/Jun/2010:15:31:30] INFO
> thredds.inventory.DatasetCollectionManager: coawst_2_2: initial scan
> found n datasets = 0
> [29/Jun/2010:15:31:30] ERROR ucar.nc2.ft.fmrc.FmrcDataset: Fmrc
> collection is empty =fmrc:coawst_2_2
> [29/Jun/2010:15:31:30] ERROR ucar.nc2.ft.fmrc.Fmrc:
> /usgs/data0/barmstrong/coawst/coawst_2_2/Output/coawst_us_#yyyyMMdd_HH#.nc$:
> initial fmrcDataset creation failed
> java.lang.IllegalStateException: Fmrc collection is empty =fmrc:coawst_2_2
>       at ucar.nc2.ft.fmrc.FmrcDataset.buildProto(FmrcDataset.java:214)
>       at ucar.nc2.ft.fmrc.FmrcDataset.setInventory(FmrcDataset.java:189)
>       at ucar.nc2.ft.fmrc.Fmrc.checkNeeded(Fmrc.java:241)
>       at ucar.nc2.ft.fmrc.Fmrc.triggerRescan(Fmrc.java:166)
>       at 
> thredds.catalog.InvDatasetFeatureCollection.triggerRescan(InvDatasetFeatureCollection.java:211)
>       at 
> thredds.servlet.DataRootHandler$ScanFmrcJob.execute(DataRootHandler.java:251)
>       at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
>       at 
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
> [29/Jun/2010:15:31:30] ERROR thredds.servlet.DataRootHandler.fcScan:
> InitFmrcJob failed
> java.lang.RuntimeException: java.lang.IllegalStateException: Fmrc
> collection is empty =fmrc:coawst_2_2
>       at ucar.nc2.ft.fmrc.Fmrc.checkNeeded(Fmrc.java:248)
>       at ucar.nc2.ft.fmrc.Fmrc.triggerRescan(Fmrc.java:166)
>       at 
> thredds.catalog.InvDatasetFeatureCollection.triggerRescan(InvDatasetFeatureCollection.java:211)
>       at 
> thredds.servlet.DataRootHandler$ScanFmrcJob.execute(DataRootHandler.java:251)
>       at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
>       at 
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
> Caused by: java.lang.IllegalStateException: Fmrc collection is empty
> =fmrc:coawst_2_2
>       at ucar.nc2.ft.fmrc.FmrcDataset.buildProto(FmrcDataset.java:214)
>       at ucar.nc2.ft.fmrc.FmrcDataset.setInventory(FmrcDataset.java:189)
>       at ucar.nc2.ft.fmrc.Fmrc.checkNeeded(Fmrc.java:241)
>       ... 5 more



Ticket Details
===================
Ticket ID: PEF-311987
Department: Support THREDDS
Priority: Normal
Status: Open