[THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2

Fan,

I reproduced the behavior locally, and it is a bug.  Thanks for finding and 
reporting it!  I've set up bug-ticket in jira, our issue tracking system, which 
is here:

https://bugtracking.unidata.ucar.edu/browse/TDS-435

The developer who really knows grib is out for a long weekend, but I'll make 
sure he sees it next week.

-Lansing

> Hi Lansing,
> 
> You can get as many as you want from here:
> 
> ftp://hydro1.sci.gsfc.nasa.gov/data/s4pa/GLDAS_V1/GLDAS_CLM10SUBP_3H/1979/002/
> 
> My sample was the first file (at top) of that day.  The rest .grb files 
> represent the hourly data for that day.  For more than one-day worth of data, 
> go to 1979/003, 1979/004, etc. for year 1979.
> 
> -Fan
> ________________________________________
> From: Unidata THREDDS Support [address@hidden]
> Sent: Thursday, May 23, 2013 1:37 PM
> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
> address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
> 
> Fan,
> 
> Can you put a few more of the grb files up?
> 
> -Lansing
> 
> > Hello Lansing,
> >
> > Thanks for the tip.  You can get the sample data and grib table from:
> >
> > ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/GLDAS_CLM10SUBP_3H.A1979002.0000.001.2008086151108.grb
> > ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/gribtab_clm.tab
> >
> > My TDS configuration for the dataset is:
> >
> > <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H"
> > path="GLDAS_CLM10SUBP_3H"
> > location="/ftp/data/s4pa_TS2/TEST_OPENDAP/GLDAS_CLM10SUBP_3H/">
> > <netcdf
> > xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
> > iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
> > </netcdf>
> > <metadata inherited="true">
> > <serviceName>odapncss</serviceName>
> > <dataType>Grid</dataType>
> > </metadata>
> > <filter>
> > <include wildcard="*.grb"/>
> > </filter>
> > </datasetScan>
> >
> > -Fan
> >
> > On 05/22/2013 03:55 PM, Unidata THREDDS Support wrote:
> > > Fan,
> > >
> > > Two things - first, don't serve the ncx files.  It doesn't matter where 
> > > they are located (in the data directory or elsewhere), but it's not meant 
> > > to be served as data.
> > >
> > > Second, it sounds like you are describing a bug in how user tables are 
> > > applied.  I'd like to set up a tds locally in a debugger with your files, 
> > > collection spec, and local grib table.  Please bundle them up and send 
> > > them to me, or point me to where I can get them.
> > >
> > > Thanks,
> > >   Lansing
> > >
> > >> Hello Lansing,
> > >>
> > >> Any comments/findings about my questions?
> > >>
> > >> With or without featureCollection we need to serve grib data in their 
> > >> granule form, so how to make grib table work for individual granules is 
> > >> remaining question number one.  So far our grib table works with 
> > >> aggregated featureCollection dataset, but not for granules under 'files' 
> > >> folder.  I also tried the 'iospParam' attribute of the 'netcdf' element 
> > >> outside featureCollection, like
> > >>
> > >> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H" 
> > >> path="GLDAS_CLM10SUBP_3H" 
> > >> location="/ftp/data/s4pa_TS2/GLDAS_CLM10SUBP_3H/">
> > >> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
> > >> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
> > >> </netcdf>
> > >> ...
> > >>
> > >> and TDS didn't seem to read the grib table.
> > >>
> > >> The other main question is about the collection .ncx file usage.  I 
> > >> found accessing that file very speedy in TDS and wonder if it should be 
> > >> used to serve our extremely long time series datasets (~30 years of 
> > >> hourly data), i.e. shall we let TDS make the file at the data location 
> > >> instead of under cache/cdm, and treat it as the timeseries dataset?
> > >>
> > >> -Fan
> > >>
> > >> ________________________________________
> > >> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> > >> Sent: Friday, May 17, 2013 2:50 PM
> > >> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; address@hidden
> > >> Cc: address@hidden
> > >> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
> > >> TDS 4.2
> > >>
> > >> I take it back - the .tab extension apparently works.  We previously 
> > >> made an error renaming the grib table.
> > >>
> > >> I also changed '/s' to '.s' but both came out the same.  For example, 
> > >> kg/m^2/s or kg/m^2.s would show up as kgm^2s, which is wrong.  I suppose 
> > >> we have to change it to kgm^-2s^-1 in the table?  I hope it is 
> > >> consistent with software like 'wgrib'.
> > >>
> > >> It seems the variable names are essentially the long names and units in 
> > >> grib table plus something like '_surface', with space chars replaced by 
> > >> underscores.  I guess we can survive with that.
> > >>
> > >> These show up in the collection 'Best Timeseries' (is there a way to 
> > >> re-configure this collection name?), but not for individual files under 
> > >> 'files' folder in TDS (again, is there a way to configure for the folder 
> > >> name?).  Is this expected?
> > >>
> > >> One final question: if I drop the collection index file .ncx at the data 
> > >> location, TDS seems to be able to serve it like a dataset, and for time 
> > >> series it is surprisingly fast to serve the .ncx file instead.  Is this 
> > >> intended - in other words we shall direct user to use .ncx, especially 
> > >> for long time series?  I know in default it's under cache/cdm and not 
> > >> exposed to users.  What's the catch here?
> > >>
> > >> Thanks.
> > >>
> > >> -Fan
> > >> ________________________________________
> > >> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> > >> Sent: Thursday, May 16, 2013 6:13 PM
> > >> To: address@hidden for
> > >> Cc: address@hidden
> > >> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
> > >> TDS 4.2
> > >>
> > >> I can confirm changing the grib table name to have .tab as extension 
> > >> still does not work.
> > >>
> > >> I'll try the modifications in the units, but wonder why it fails for all 
> > >> since some of them have legit units, like 'K' for temperature.
> > >>
> > >> -Fan
> > >> ________________________________________
> > >> From: Unidata THREDDS Support [address@hidden]
> > >> Sent: Thursday, May 16, 2013 2:37 PM
> > >> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> > >> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
> > >> address@hidden
> > >> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
> > >>
> > >> Fan,
> > >>
> > >> I did a little more digging, and I don't think the file name extension 
> > >> will matter - that was my misinterpretation of how the table was getting 
> > >> picked up.  However, you will probably have the best result directly 
> > >> embedding the table information in your featureCollection element using 
> > >> the xml notation.  It looks like you tried this already, so I am not 
> > >> sure why it did not work for you.  It is possible that it broke because 
> > >> you did not follow udunits standard -- for instance, square meters is 
> > >> notated as m^2, rather than m(superscript)2.  The code you are asking 
> > >> about is just the number that identifies the grib parameter.
> > >>
> > >> Inside your <featureCollection> element:
> > >>
> > >> <gribConfig>
> > >> <parameterMap>
> > >> <parameter code="2">  <<-- This will override the standard table values 
> > >> for parameter 2 (code < 128 are standard, > 128 are local)
> > >> <description>Pressure reduced to MSL</description>
> > >> <units>Pa</units>
> > >> <name>PRMSL</name>
> > >> </parameter>
> > >> <parameter code="131">
> > >> <description>Snowfall rate</description>
> > >> <units>kg/m^2.s</units> <<--Compound units are notated with a dot, so 
> > >> /m^2/s is just /m^2.s
> > >> <name>Snof</name>
> > >> </parameter>
> > >> ...
> > >>
> > >> </parameterMap>
> > >>
> > >> </gribConfig>
> > >> -Lansing Madry
> > >> Unidata
> > >> Boulder, Colorado
> > >>
> > >> Ticket Details
> > >> ===================
> > >> Ticket ID: GJF-200148
> > >> Department: Support THREDDS
> > >> Priority: Low
> > >> Status: Closed
> > >>
> > >>
> > > -Lansing Madry
> > >   Unidata
> > >   Boulder, Colorado
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: GJF-200148
> > > Department: Support THREDDS
> > > Priority: Low
> > > Status: Open
> > >
> >
> >
> 
> -Lansing Madry
> Unidata
> Boulder, Colorado
> 
> Ticket Details
> ===================
> Ticket ID: GJF-200148
> Department: Support THREDDS
> Priority: Low
> Status: Open
> 
> 

-Lansing Madry
  Unidata
  Boulder, Colorado

Ticket Details
===================
Ticket ID: GJF-200148
Department: Support THREDDS
Priority: Low
Status: Open