Re: Error with re-hosted FMRC best time series

Hi Kevin:

Heres a summary of the situation so far:

1. I still have the missing values attribute issue on my TODO list for 
4.2 Fmrc.
2. The "inconsistent array length read" on the client always indicates 
an error on the server. If you own that server, can you look in the 
threddsServlet.log files and find the corresponding entry and send to 
me? alternately, send me the entire log file for the time it happened.
3. it appears that you are wrapping an opendap dataset with the ncml. Is 
that the same TDS or a different server?
4. weve delayed the TDS release to get some WMS fixes in, ill check that 
you have the latest bug fixes.
5. I may have to get some files from you and reproduce the problem. Are 
those files available on the server?

Im out of town till Weds, so i probably cant do much more till Im back 
in the office. Ill check in then after ive had time to check stuff.

Kevin O'Brien wrote:
> Hi John -
>
> Sure -here's what it looks like.  Sorry, the cut and past will likely 
> wrap some of the lines...
>
> Is the belw what you want?
>
> thanks -
> Kevin
>
>
>
>
> <?xml version="1.0" encoding="UTF-8"?>
> <catalog name="kob test server"
>        
> xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
>        xmlns:xlink="http://www.w3.org/1999/xlink";>
>
>  <service name="all" base="" serviceType="compound">
>    <service name="odap" serviceType="OpenDAP" base="/thredds/dodsC/" />
>    <service name="http" serviceType="HTTPServer" 
> base="/thredds/fileServer/" />
>    <!--service name="wcs" serviceType="WCS" base="/thredds/wcs/" /-->
>    <service name="wms" serviceType="WMS" base="/thredds/wms/" />
>    <!--service name="ncss" serviceType="NetcdfSubset" 
> base="/thredds/ncss/" /-->
>  </service>
>
>              <dataset name="NCOM Region 2 Aggregation">
>                <dataset name="NCOM Region 2 Best Time Series" 
> ID="NCOM_reg_2_best_time_series" urlPath="NCOM_Region_2_Aggregation">
>                  <serviceName>all</serviceName>
>                  <netcdf 
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
> location="http://edac-dap.northerngulfinstitute.org/thredds/dodsC/ncom_reg2_agg/NCOM_Region_ 
>
> 2_Aggregation_best.ncd">
>                   <variable name="water_temp">
>                     <attribute name="_FillValue" type="float" 
> value="NaN" />
>                     <attribute name="missing_value" type="float" 
> value="NaN" />
>                   </variable>
>                   <variable name="salinity">
>                     <attribute name="_FillValue" type="float" 
> value="NaN" />
>                     <attribute name="missing_value" type="float" 
> value="NaN" />
>                   </variable>
>                   <variable name="water_u">
>                     <attribute name="_FillValue" type="float" 
> value="NaN" />
>                     <attribute name="missing_value" type="float" 
> value="NaN" />
>                   </variable>
>                   <variable name="water_v">
>                     <attribute name="_FillValue" type="float" 
> value="NaN" />
>                     <attribute name="missing_value" type="float" 
> value="NaN" />
>                   </variable>
>                   <variable name="surf_el">
>                     <attribute name="missing_value" type="float" 
> value="NaN" />
>                     <attribute name="_FillValue" type="float" 
> value="NaN" />
>                   </variable>
>                  </netcdf>
>                </dataset>
>              </dataset>
>
>  <service name="remoteOPeNDAP_00A" serviceType="OPENDAP" 
> base="http://edac-dap.northerngulfinstitute.org/thredds/dodsC/"; />
>
>              <dataset name="NCOM Region 2 Aggregation remote">
>                <dataset name="NCOM Region 2 Best Time Series remote" 
> ID="NCOM_reg_2_best_time_series_remote" 
> urlPath="ncom_reg2_agg/NCOM_Region_2_Aggregation_best.ncd">
>                  <serviceName>remoteOPeNDAP_00A</serviceName>
>                  <dataType>GRID</dataType>
>                </dataset>
>              </dataset>
>
>
>
> john caron wrote:
> > Kevin O'Brien wrote:
> > > Hi John -
> > >
> > > The TDS version at
> > >    http://ferret.pmel.noaa.gov/geoide/geoIDECleanCatalog.xml
> > >
> > > is at 4.1.0.
> > >
> > > I just upgraded my development version that is serving the same 
> > > dataset to 4.1.20100428.1437, which is the most recent download 
> > > version, and am seeing the same thing.  Would it be easier for you 
> > > if I upgraded the public version to the latest?
> > >
> > > thanks
> > > Kevin
> > >
> > >
> > >
> > > john caron wrote:
> > > > Hi Kevin:
> > > >
> > > > What version of the TDS/
> > > >
> > > > Kevin O'Brien wrote:
> > > > > Hi John -
> > > > >
> > > > > I'm seeing another interesting problem in terms of looking at 
> > > > > forecast model collections.  One of the problems we had with the 
> > > > > FMRC putting in NAN's without changing the attributes to indicate 
> > > > > such was that certain clients (notably ferret), kinda freaked out 
> > > > > when dealing with the NaN's it found.  One way I can fix this 
> > > > > without a change in the FMRC  (that you've been discussing with 
> > > > > Steve) is to of course wrap some NcML around the data and serve it 
> > > > > myself.  Something like:
> > > > >
> > > > >               <dataset name="NCOM Region 2 Aggregation">
> > > > >                 <dataset name="NCOM Region 2 Best Time Series" 
> > > > > ID="NCOM_reg_2_best_time_series" urlPath="NCOM_Region_2_Aggregation">
> > > > >                   <serviceName>all</serviceName>
> > > > >                   <netcdf 
> > > > > xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
> > > > > location="http://edac-dap.northerngulfinstitute.org/thredds/dodsC/ncom_reg2_agg/NCOM_Region_2_Aggregation_best.ncd";> 
> > > > >
> > > > >                    <variable name="water_temp">
> > > > >                      <attribute name="_FillValue" type="float" 
> > > > > value="NaN" />
> > > > >                      <attribute name="missing_value" type="float" 
> > > > > value="NaN" />
> > > > >                    </variable>
> > > > >                    <variable name="salinity">
> > > > >                      <attribute name="_FillValue" type="float" 
> > > > > value="NaN" />
> > > > >                      <attribute name="missing_value" type="float" 
> > > > > value="NaN" />
> > > > >                    </variable>
> > > > >                    <variable name="water_u">
> > > > >                      <attribute name="_FillValue" type="float" 
> > > > > value="NaN" />
> > > > >                      <attribute name="missing_value" type="float" 
> > > > > value="NaN" />
> > > > >                    </variable>
> > > > >                    <variable name="water_v">
> > > > >                      <attribute name="_FillValue" type="float" 
> > > > > value="NaN" />
> > > > >                      <attribute name="missing_value" type="float" 
> > > > > value="NaN" />
> > > > >                    </variable>
> > > > >                    <variable name="surf_el">
> > > > >                      <attribute name="missing_value" type="float" 
> > > > > value="NaN" />
> > > > >                      <attribute name="_FillValue" type="float" 
> > > > > value="NaN" />
> > > > >                    </variable>
> > > > >                   </netcdf>
> > > > >                 </dataset>
> > > > >               </dataset>
> > > > >
> > > > >
> > > > > So, I've done that, and in doing so have come across an issue 
> > > > > where, for some reason, Java netcdf is not able to read that 
> > > > > particular file/aggregation.  I'll use IDV as an example of what I 
> > > > > see.
> > > > >
> > > > > If I tell IDV to add the following catalog:
> > > > >
> > > > >     http://ferret.pmel.noaa.gov/geoide/geoIDECleanCatalog.xml
> > > > >
> > > > > and pick the appropriate dataset, like so:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > And then ask IDV to load the data, here's the error message that I 
> > > > > get:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I actually see the same error when trying to get LAS configuration 
> > > > > data for the same dataset using our addXML Java utility, but 
> > > > > thought showing it in IDV would probably make more sense to you.
> > > > > If I just point right directly to the best time series in the 
> > > > > FMRC, I don't have this problem...
> > > > >
> > > > > Any ideas?  Is there any other information I can supply that might 
> > > > > help...
> > > > >
> > > > > Thanks
> > > > > Kevin
> > > > >
> > > > > p.s  the full error dump from above is:
> > > > >
> > > > > java.io.IOException: Inconsistent array length read: 1165128303 != 
> > > > > 1914731274
> > > > >     at 
> > > > > ucar.nc2.dods.DODSNetcdfFile.readArrays(DODSNetcdfFile.java:1216)
> > > > >     at ucar.nc2.dods.DODSNetcdfFile.<init>(DODSNetcdfFile.java:318)
> > > > >     at 
> > > > > ucar.nc2.dataset.NetcdfDataset.acquireDODS(NetcdfDataset.java:783)
> > > > >     at 
> > > > > ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:657) 
> > > > >
> > > > >     at 
> > > > > ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:434)
> > > > >     at 
> > > > > ucar.nc2.dataset.NetcdfDataset.acquireDataset(NetcdfDataset.java:523)
> > > > >     at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:103)
> > > > >     at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:89)
> > > > >     at 
> > > > > ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:1084) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:1106) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:471) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.data.DataManager.createDataSource(DataManager.java:1565)
> > > > >     at 
> > > > > ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1992) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1909) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1843) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1816) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:943) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.chooser.XmlChooser.makeDataSource(XmlChooser.java:280) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.chooser.ThreddsHandler.processNodes(ThreddsHandler.java:819) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.chooser.ThreddsHandler.doLoad(ThreddsHandler.java:677) 
> > > > >
> > > > >     at 
> > > > > ucar.unidata.idv.chooser.XmlChooser.doLoadInThread(XmlChooser.java:715) 
> > > > >
> > > > >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > >     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > > >     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown 
> > > > > Source)
> > > > >     at java.lang.reflect.Method.invoke(Unknown Source)
> > > > >     at ucar.unidata.util.Misc$2.run(Misc.java:1068)
> > > > >     at ucar.unidata.util.Misc$3.run(Misc.java:1096)
> > > > >
> > > > >
> > > > > --
> > > > > Kevin O'Brien                   UW/JISAO   Research 
> > > > > Scientist              NOAA/PMEL/TMAP
> > > > > 206-526-6751                    http://www.pmel.noaa.gov
> > > > >
> > > > > "The contents of this message are mine personally and do
> > > > > not necessarily reflect any position of the Government
> > > > > or the  National Oceanic and Atmospheric Administration."
> >
> > Hey Kevin , what Id like to try is the latest 4.2 with some new 
> > configuration. Can you send me the TDS config catalog for that 
> > particular dataset? thanks