Re: DODS AS

Joe McLean wrote:

> Hi John,
>
> I have tried to use an NcML file in the latest Agg Server 
> configuration I downloaded last Friday.  I repeatedly receive a java 
> error from the server at http://stout.pmel.noaa.gov:8180/dodsC/
>
> This is the pertinent snippet of the error:
>
> java.lang.NullPointerException
>         at 
> dods.servers.agg.CatalogServlet.addAccess(CatalogServlet.java:611)
>         at 
> dods.servers.agg.CatalogServlet.readCatalog(CatalogServlet.java:257)
>         at dods.servers.agg.CatalogServlet.init(CatalogServlet.java:178)
>         at javax.servlet.GenericServlet.init(GenericServlet.java:258)
are you still getting this error? i cant reproduce it. I am seeing this 
catalog:

<catalog name="Test NCML Aggregation" version="0.6">
 <dataset name="Top-Level Dataset" dataType="Grid" serviceName="this">
   <service name="this" serviceType="DODS" base=""/>
   <service name="stdmet" serviceType="NetCDF" 
base="file://WEB-INF/dodsAS/"/>

   <dataset name="NDBC stdmet buoy data">
      <property name="internalService" value="stdmet"/>
      <dataset name="stdmetNcML Agg test" urlPath="TestNcML.xml"/>
   </dataset>
 </dataset>
</catalog>


with this external URL:  http://stout.pmel.noaa.gov:8180/dodsC/TestNcML.xml

and this internal URL: file://WEB-INF/dodsAS/TestNcML.xml

this is a relative URL to the current path, so the absolute path is 
apparently:

file:/usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/TestNcML.xml

and i get the error message:

message = "CatalogServlet ERROR opening NcDataset TestNcML.xml
   access = 
file:/usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/TestNcML.xml

Is this the correct path? is not, try using the absolute path in the "stdmet" service element. 
absolute paths i think are system dependent; on linux try using 3 slashes or 1 slash:

   <service name="stdmet" serviceType="NetCDF" 
base="file:///path/something/WEB-INF/dodsAS/"/>
   <service name="stdmet" serviceType="NetCDF" 
base="file:/path/something/WEB-INF/dodsAS/"/>






> I used your Catalog and NcML aggregation as templates (following this 
> message) and ctreated the following
> Both are in $tomcat_root/dodsC/WEB-INF/dodsAS/  the web.xml file 
> points to the correct Catalog.xml file.
> Any ideas?  Is it that the NcML netcdf element can't handle a DODS 
> URL?  AmI confused about where the aggregation should take place?
>
> --------------------------------------
> AggServTestCatalog.xml
> ---------------------------------------
> <!DOCTYPE catalog SYSTEM
> "http://www.unidata.ucar.edu/projects/THREDDS/xml/AggServerCatalog.dtd";>
> <catalog name="Test NCML Aggregation" version="0.6"
> xmlns="http://www.unidata.ucar.edu/thredds";
> xmlns:xlink="http://www.w3.org/1999/xlink";>
>         <service name="this" serviceType="DODS" base=""/>
>         <service name="stdmet" serviceType="NetCDF"
>            base="file://WEB-INF/dodsAS/"/ <file://WEB-INF/dodsAS/>>
>         <dataset name="NDBC stdmet buoy data">
>             <property name="internalService" value="stdmet"/>
>             <dataset name="stdmet NcML Agg test" 
> urlPath="TestNcML.xml" />
>         </dataset>
> </catalog>
-
<catalog name="Test NCML Aggregation" version="0.6">
-
<dataset name="Top-Level Dataset" dataType="Grid" serviceName="this">
<service name="this" serviceType="DODS" base=""/>
<service name="stdmet" serviceType="NetCDF" base="file://WEB-INF/dodsAS/"/>
-
<dataset name="NDBC stdmet buoy data">
<property name="internalService" value="stdmet"/>
<dataset name="stdmetNcML Agg test" urlPath="TestNcML.xml"/>
</dataset>
</dataset>
</catalog>


> Also tried full path for service:
> <service name="stdmet" serviceType="DODS"
>            
> base="file:///usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/"/ 
> <file:///usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/>>
>
> Also tried serviceType="NetCDF"
> <service name="stdmet" serviceType="NetCDF"
>
> ---------------------------------------
> TestNcML.xml
> ---------------------------------------
> <?xml version='1.0' encoding='UTF-8'?>
> <netcdf xmlns='http://www.ucar.edu/schemas/netcdf'
>     xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
>     xsi:schemaLocation='http://www.ucar.edu/schemas/netcdf
> http://www.unidata.ucar.edu/schemas/netcdfDataset2.xsd' >
>
>   <variable name='recnum' type='int' shape='recnum' >
>     <attribute name='long_name' type='string' value='time coordinate' />
>     <attribute name='units' type='string' value='hours since 
> 2001-09-31 00:00:00 UTC' />
>     <values start='0' increment='1' />
>   </variable>
>
>   <aggregation dimName='recnum' type='joinExisting' />
>
>   <netcdf 
> uri='http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_09_stdmet.nc'/ 
> <http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_09_stdmet.nc%27/>>
>   <netcdf 
> uri='http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_10_stdmet.nc' 
> />
>   <netcdf 
> uri='http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_11_stdmet.nc'/ 
> <http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_11_stdmet.nc%27/>>
>  
> </netcdf>
> ---------------------------------------
>  
>
> Joe
> ------------------------------------------------
> John Caron wrote:
>
> > ok, you need to download a new dodsC.war file, follow instructions from
> > this page:
> >
> > http://www.unidata.ucar.edu/projects/THREDDS/tech/aggServer/AggServerInstall.html 
> >
> >
> > here is an example NcML  aggregation that i tested on 3 of your files;
> > it assumes a uniform 10 minute increment to add a coordinate variable.
> > it does a join on the recnum dimension on the listed 3 files. it should
> > work for you, except that obviously you replace with your files, in time
> > order, and get the starting time right.
> >
> > <?xml version='1.0' encoding='UTF-8'?>
> > <netcdf xmlns='http://www.ucar.edu/schemas/netcdf'
> >     xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
> >     xsi:schemaLocation='http://www.ucar.edu/schemas/netcdf
> > http://www.unidata.ucar.edu/schemas/netcdfDataset2.xsd' >
> >
> >   <variable name='recnum' type='int' shape='recnum' >
> >     <attribute name='long_name' type='string' value='time coordinate' />
> >     <attribute name='units' type='string' value='minutes since 2001-8-31
> > 23:00:00 UTC' />
> >     <values start='0' increment='10' />
> >   </variable>
> >
> >   <aggregation dimName='recnum' type='joinExisting' />
> >
> >   <netcdf uri='C:/data/buoy/cwind1.nc'/>
> >   <netcdf uri='C:/data/buoy/cwind2.nc' />
> >   <netcdf uri='C:/data/buoy/cwind3.nc'/>
> >
> > </netcdf>
> >
> > the agg server configuration looks like this:
> >
> > <!DOCTYPE catalog SYSTEM
> > "http://www.unidata.ucar.edu/projects/THREDDS/xml/AggServerCatalog.dtd";>
> > <catalog name="Test NCML Aggregation" version="0.6"
> > xmlns="http://www.unidata.ucar.edu/thredds";
> > xmlns:xlink="http://www.w3.org/1999/xlink";>
> >         <service name="cwind" serviceType="NetCDF"
> > base="file:///c:/data/buoy/"/ <file:///c:/data/buoy/>>
> >         <dataset name="cwind buoy data">
> >             <property name="internalService" value="cwind"/>
> >             <dataset name="cwind NcML Agg test" 
> > urlPath="cwindAgg.xml" />
> >         </dataset>
> > </catalog>
> >
> > where the NcML file (as above) lives at 
> > file:///c:/data/buoy/cwindAgg.xml
> >
> > hopefully this will get you started, let me know how it goes.
> >
> > BTW, this wont scale too huge numbers of netcdf files, but a few dozen
> > should be ok. i will need to add some file caching to speed it up, but
> > the above shoul be ok for small sets of files.