[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[THREDDS #QCX-921252]: Wrong dates reported by thredds 4.3
- Subject: [THREDDS #QCX-921252]: Wrong dates reported by thredds 4.3
- Date: Mon, 29 Apr 2013 12:39:34 -0600
Rostislav,
I've been thinking about your question, and I'm afraid I could not come up with
a mechanism to add the variables you mention to a netcdf subset.
Unfortunately, for reasons to do with CF imlementation requirements, this is
not a feature we support (though we may implement this in a future release).
Essentially, if you were serving up individual files, it would be no problem to
wrap the files in a bit of NcML to add the dimension and variables. However,
as an aggregation, the best solution I can offer is to add the information as a
global attribute, like this:
<dataset name="Rostislav">
<featureCollection name="Rostislavnc4data" featureType="FMRC"
harvest="true" path="rostislav/fmrc">
<metadata inherited="true">
<serviceName>all</serviceName>
<dataFormat>netCDF</dataFormat>
</metadata>
<collection spec="C:/Users/madry/work/RostislavKouznetsov/.*nc4" />
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
<dimension name="dayhours" value="24"/>
<attribute name="dayhours" value="24"/>
</netcdf>
</featureCollection>
</dataset>
It may be worth noting that the information you would like to add is more akin
to metadata than actual data.
Regards,
Lansing Madry
Unidata
Boulder, Colorado
> Hello Lansing,
>
> Thank you for ther reply.
> On the second point.
> If you download a file from
> http://silam.fmi.fi/thredds/fileServer/fires/files/reanal_v1_3_global_2000.nc4
> you will find there a dimension
> dayhours = 24 ;
> and two variables
> float dayhours(dayhours) ;
> float hourlyadj(dayhours) ;
>
> They do not appear in files that come from netcdfsubset
> http://silam.fmi.fi/thredds/ncss/grid/fires/files/reanal_v1_3_global_2000.nc4?var=emis_PM_FRP&north=70&west=0&east=20&south=50&horizStride=1&time_start=2000-01-01T00%3A00%3A00Z&time_end=2000-12-31T00%3A00%3A00Z&timeStride=1&addLatLon=true&accept=netcdf
>
> Is it possible to make the dimension "dayhours" and correspondinf
> variables to appear in netcdf subsets?
>
> Thank you once more.
>
> Best regards,
>
> Rostislav
>
>
>
> On Sat, Apr 20, 2013 at 01:09:30AM +0300, Unidata THREDDS Support wrote:
> >Rostislav,
> >
> >I was able to reproduce the failure you saw when asking for NetCDF4 versus
> >NetCDF files. I wasn't sure why that was happening, so I asked the
> >developer who worked on the ncss service. He thought it was probably a bug
> >of some sort, so I opened a ticket to address the issue. I don't know if it
> >is an easy fix or not yet, but it is in the queue. Thank you for bringing
> >this to our attention!
> >
> >https://bugtracking.unidata.ucar.edu/browse/TDS-408
> >
> >On your second point, I'm not sure what you mean by the small, non-grid
> >variable. When I do an ncdump on the file, I see:
> >
> > dimensions:
> > lon = 3600;
> > lat = 1800;
> > time = UNLIMITED; // (1 currently
> > variables:
> > float lon(lon=3600);
> > :long_name = "longitude";
> > :units = "degrees_east";
> > :standard_name = "longitude";
> >
> > float lat(lat=1800);
> > :units = "degrees_north";
> > :standard_name = "latitude";
> > :long_name = "latitude";
> >
> > double time(time=1);
> > :units = "seconds since 1970-01-01 00:00:00";
> > :calendar = "standard";
> > :_ChunkSize = 1; // int
> >
> > float emis_PM_FRP(time=1, lat=1800, lon=3600);
> > :units = "kg/s";
> > :_ChunkSize = 1, 1800, 3600; // int
> >
> > // global attributes:
> > :CDI = "Climate Data Interface version 1.4.1";
> > :Conventions = "CF-1.0";
> > :netcdf_generated = "2013-04-15 09:01 UTC";
> > :producer = "Finnish Meteorological Institute";
> > :data_produced = "2013-04-13";
> > :dataset = "IS4FIRES wildland-fire emission reanalysis";
> > :contact = "Mikhail Sofiev: address@hidden";
> > :PM2P5_10fraction = 0.25999999046325684; // double
> > :PM2P5fraction = 0.7400000095367432; // double
> > :quantity = "PM emission rate";
> > :CDO = "Climate Data Operators version 1.4.1
> > (http://www.mpimet.mpg.de/cdo)";
> >
> >Which is the non-grid variable you mean?
> >
> >On the third point, I will have to see if there is a way to get the
> >information you want served up in fewer steps, but I don't know if it is
> >possible off-hand.
> >
> >Regards,
> > Lansing
> >> Hello Lansing,
> >>
> >> Thank you for the detailed reply.
> >> I have set up the service according to your instructions.
> >> http://silam.fmi.fi/thredds/firesEmission.html
> >> Few more questions.
> >>
> >> 1. The subsetting of files does not work with nc4 output
> >> when accessing by file:
> >> http://silam.fmi.fi/thredds/ncss/grid/fires/files/reanal_v1_3_global_2000.nc4?var=emis_PM_FRP&time_start=2000-01-01T00%3A00%3A00Z&time_end=2000-01-02T00%3A00%3A00Z&accept=netcdf
> >> works, but
> >> http://silam.fmi.fi/thredds/ncss/grid/fires/files/reanal_v1_3_global_2000.nc4?var=emis_PM_FRP&time_start=2000-01-01T00%3A00%3A00Z&time_end=2000-01-02T00%3A00%3A00Z&accept=netcdf4
> >> (difference in the last argument)
> >> causes "I/O Exception handled : NetCDF: Invalid argument
> >> nc_def_var_chunking on variable time"
> >> Is it a bug or a feature?
> >>
> >> 2. I have a small non-grid variable in my file. It disappears when
> >> accessing the files via thredds (except for http).
> >> Can i make thredds to include it into any netcdf produced
> >> from the datasets?
> >>
> >> 3. The data are obtained from reanalisys, so they have a single
> >> time dimension. To get to the metadata page for "Fire emission this year"
> >> i have to click "Fire emission this year" and then "Best timeseries".
> >> Is it possible to avoid the second step?
> >>
> >>
> >> Thank you once more!
> >>
> >> Best regards,
> >> Rostislav
> >>
> >>
> >> On Wed, Apr 17, 2013 at 11:49:39PM +0300, Unidata THREDDS Support wrote:
> >> >Hello Rostislav,
> >> >
> >> >I downloaded a few days worth of files from your TDS and experimented
> >> >with setting up a local TDS to help with your questions. To answer your
> >> >first question, what you are observing is a feature, but I think you are
> >> >misinterpreting the data. With a datasetScan such as you have, the time
> >> >reported is the last time the file was modified (i.e., touched on your
> >> >server), not the time or date information contained in the file. To
> >> >answer your second question, I believe what you really want to do is to
> >> >set up a featureCollection, instead of a datasetScan. Here is a simple
> >> >example that I used locally:
> >> >
> >> ><?xml version="1.0" encoding="UTF-8"?>
> >> ><catalog name="THREDDS Server Default Catalog : You must change this to
> >> >fit your server!"
> >> >
> >> > xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0"
> >> > xmlns:xlink="http://www.w3.org/1999/xlink">
> >> >
> >> > <service name="all" base="" serviceType="compound">
> >> > <service name="odap" serviceType="OpenDAP" base="/thredds/dodsC/" />
> >> > <service name="http" serviceType="HTTPServer"
> >> > base="/thredds/fileServer/" />
> >> > <!--service name="wcs" serviceType="WCS" base="/thredds/wcs/" /-->
> >> > <service name="wms" serviceType="WMS" base="/thredds/wms/" />
> >> > <service name="ncss" serviceType="NetcdfSubset"
> >> > base="/thredds/ncss/grid/" />
> >> > </service>
> >> >
> >> > <dataset name="Rostislav">
> >> > <featureCollection name="Rostislavnc4data" featureType="FMRC"
> >> > harvest="true" path="rostislav/fmrc">
> >> > <metadata inherited="true">
> >> > <serviceName>all</serviceName>
> >> > <dataFormat>netCDF</dataFormat>
> >> > </metadata>
> >> > <collection spec="C:/Users/madry/work/RostislavKouznetsov/.*nc4" />
> >> > </featureCollection>
> >> > </dataset>
> >> ></catalog>
> >> >
> >> >This sets your data up to be served as a single logical unit via the
> >> >NetCDF Subset Service. After you have changed the names and paths to
> >> >suit your system, you will also need to edit the threddsConfig.xml file
> >> >to turn the NetcdfSubsetService on. You do this by uncommenting the
> >> >relevant block of xml and setting <allow> to true:
> >> >
> >> > <!--
> >> > The Netcdf Subset Service is off by default.
> >> > -->
> >> > <NetcdfSubsetService>
> >> > <allow>true</allow>
> >> > <scour>10 min</scour>
> >> > <maxAge>-1 min</maxAge>
> >> > </NetcdfSubsetService>
> >> >
> >> >Let me know if you need any further assistance.
> >> >
> >> >Regards,
> >> > Lansing Madry
> >> > Unidata
> >> > Boulder, Colorado
> >> >
> >> >> Hello,
> >> >>
> >> >> I have put a collection of daily files into thredds, one field per day.
> >> >> http://silam.fmi.fi/thredds/catalog/fires3/catalog.html
> >> >>
> >> >> Unfortunately it reports wrong times for them. e.g.
> >> >>
> >> >> http://silam.fmi.fi/thredds/catalog/fires3/catalog.html?dataset=fires3/reanal_v1_3_global_20130410.nc4
> >> >>
> >> >> shows "date 2013-04-15T09:01:19Z (modified)" instead of something like
> >> >> 2013-04-10T00:00:00Z.
> >> >>
> >> >> The dataset is described as:
> >> >> <datasetScan name="Fires3" path="fires3"
> >> >> location="/data/thredds/Fires/" suffix=".nc4">
> >> >> <serviceName>all</serviceName>
> >> >> </datasetScan>
> >> >>
> >> >> Is it a bug or a feature?
> >> >>
> >> >> Can I somehow configure thredds to report these files as a single
> >> >> dataset?
> >> >>
> >> >> Thank you!
> >> >>
> >> >> Best regards,
> >> >> Rostislav
> >> >>
> >> >> --
> >> >> Rostislav Kouznetsov
> >> >>
> >> >> **************************************
> >> >> Air quality
> >> >> Finnish Meteorological Institute
> >> >> PL-503 FI-00101 Helsinki Finland
> >> >> Phone: +358 29 539 4630
> >> >> **************************************
> >> >>
> >> >>
> >> >
> >> >
> >> >Ticket Details
> >> >===================
> >> >Ticket ID: QCX-921252
> >> >Department: Support THREDDS
> >> >Priority: Normal
> >> >Status: Open
> >> >
> >>
> >> --
> >> Rostislav Kouznetsov
> >>
> >> **************************************
> >> Air quality
> >> Finnish Meteorological Institute
> >> PL-503 FI-00101 Helsinki Finland
> >> Phone: +358 29 539 4630
> >> **************************************
> >>
> >>
> >
> >
> >Ticket Details
> >===================
> >Ticket ID: QCX-921252
> >Department: Support THREDDS
> >Priority: Normal
> >Status: Open
> >
>
> --
> Rostislav Kouznetsov
>
> **************************************
> Air quality
> Finnish Meteorological Institute
> PL-503 FI-00101 Helsinki Finland
> Phone: +358 29 539 4630
> **************************************
>
>
Ticket Details
===================
Ticket ID: QCX-921252
Department: Support THREDDS
Priority: High
Status: Open