[THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2

Fan,

You can download a 4.3.19 snapshot here:

https://docs.google.com/file/d/0B7jP7ooGP2ZzdFh5NUprdEowMkE/edit?usp=sharing

Enjoy!

-Lansing

On 9/3/2013 11:24 AM, Fan Fang wrote:
> New Client Reply: TDS 4.3 GRIB names different from TDS 4.2
>
> Hi Lansing,
>
> I notice that the bug is fixed in version 4.3.18.  Is it downloadable
> from somewhere?
>
> Thanks.
>
> -Fan
>
> On 05/23/2013 04:37 PM, Unidata THREDDS Support wrote:
>> Fan,
>>
>> I reproduced the behavior locally, and it is a bug.  Thanks for finding and 
>> reporting it!  I've set up bug-ticket in jira, our issue tracking system, 
>> which is here:
>>
>> https://bugtracking.unidata.ucar.edu/browse/TDS-435
>>
>> The developer who really knows grib is out for a long weekend, but I'll make 
>> sure he sees it next week.
>>
>> -Lansing
>>
>>> Hi Lansing,
>>>
>>> You can get as many as you want from here:
>>>
>>> ftp://hydro1.sci.gsfc.nasa.gov/data/s4pa/GLDAS_V1/GLDAS_CLM10SUBP_3H/1979/002/
>>>
>>> My sample was the first file (at top) of that day.  The rest .grb files 
>>> represent the hourly data for that day.  For more than one-day worth of 
>>> data, go to 1979/003, 1979/004, etc. for year 1979.
>>>
>>> -Fan
>>> ________________________________________
>>> From: Unidata THREDDS Support [address@hidden]
>>> Sent: Thursday, May 23, 2013 1:37 PM
>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
>>> address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
>>>
>>> Fan,
>>>
>>> Can you put a few more of the grb files up?
>>>
>>> -Lansing
>>>
>>>> Hello Lansing,
>>>>
>>>> Thanks for the tip.  You can get the sample data and grib table from:
>>>>
>>>> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/GLDAS_CLM10SUBP_3H.A1979002.0000.001.2008086151108.grb
>>>> ftp://hydro1.sci.gsfc.nasa.gov/private/ffang/gribtab_clm.tab
>>>>
>>>> My TDS configuration for the dataset is:
>>>>
>>>> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H"
>>>> path="GLDAS_CLM10SUBP_3H"
>>>> location="/ftp/data/s4pa_TS2/TEST_OPENDAP/GLDAS_CLM10SUBP_3H/">
>>>> <netcdf
>>>> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
>>>> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
>>>> </netcdf>
>>>> <metadata inherited="true">
>>>> <serviceName>odapncss</serviceName>
>>>> <dataType>Grid</dataType>
>>>> </metadata>
>>>> <filter>
>>>> <include wildcard="*.grb"/>
>>>> </filter>
>>>> </datasetScan>
>>>>
>>>> -Fan
>>>>
>>>> On 05/22/2013 03:55 PM, Unidata THREDDS Support wrote:
>>>>> Fan,
>>>>>
>>>>> Two things - first, don't serve the ncx files.  It doesn't matter where 
>>>>> they are located (in the data directory or elsewhere), but it's not meant 
>>>>> to be served as data.
>>>>>
>>>>> Second, it sounds like you are describing a bug in how user tables are 
>>>>> applied.  I'd like to set up a tds locally in a debugger with your files, 
>>>>> collection spec, and local grib table.  Please bundle them up and send 
>>>>> them to me, or point me to where I can get them.
>>>>>
>>>>> Thanks,
>>>>>    Lansing
>>>>>
>>>>>> Hello Lansing,
>>>>>>
>>>>>> Any comments/findings about my questions?
>>>>>>
>>>>>> With or without featureCollection we need to serve grib data in their 
>>>>>> granule form, so how to make grib table work for individual granules is 
>>>>>> remaining question number one.  So far our grib table works with 
>>>>>> aggregated featureCollection dataset, but not for granules under 'files' 
>>>>>> folder.  I also tried the 'iospParam' attribute of the 'netcdf' element 
>>>>>> outside featureCollection, like
>>>>>>
>>>>>> <datasetScan name="GLDAS_CLM10SUBP_3H" ID="GLDAS_CLM10SUBP_3H" 
>>>>>> path="GLDAS_CLM10SUBP_3H" 
>>>>>> location="/ftp/data/s4pa_TS2/GLDAS_CLM10SUBP_3H/">
>>>>>> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
>>>>>> iospParam="gribParameterTable=/home/ffang/gribtab_clm.tab">
>>>>>> </netcdf>
>>>>>> ...
>>>>>>
>>>>>> and TDS didn't seem to read the grib table.
>>>>>>
>>>>>> The other main question is about the collection .ncx file usage.  I 
>>>>>> found accessing that file very speedy in TDS and wonder if it should be 
>>>>>> used to serve our extremely long time series datasets (~30 years of 
>>>>>> hourly data), i.e. shall we let TDS make the file at the data location 
>>>>>> instead of under cache/cdm, and treat it as the timeseries dataset?
>>>>>>
>>>>>> -Fan
>>>>>>
>>>>>> ________________________________________
>>>>>> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>>>>> Sent: Friday, May 17, 2013 2:50 PM
>>>>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; address@hidden
>>>>>> Cc: address@hidden
>>>>>> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
>>>>>> TDS 4.2
>>>>>>
>>>>>> I take it back - the .tab extension apparently works.  We previously 
>>>>>> made an error renaming the grib table.
>>>>>>
>>>>>> I also changed '/s' to '.s' but both came out the same.  For example, 
>>>>>> kg/m^2/s or kg/m^2.s would show up as kgm^2s, which is wrong.  I suppose 
>>>>>> we have to change it to kgm^-2s^-1 in the table?  I hope it is 
>>>>>> consistent with software like 'wgrib'.
>>>>>>
>>>>>> It seems the variable names are essentially the long names and units in 
>>>>>> grib table plus something like '_surface', with space chars replaced by 
>>>>>> underscores.  I guess we can survive with that.
>>>>>>
>>>>>> These show up in the collection 'Best Timeseries' (is there a way to 
>>>>>> re-configure this collection name?), but not for individual files under 
>>>>>> 'files' folder in TDS (again, is there a way to configure for the folder 
>>>>>> name?).  Is this expected?
>>>>>>
>>>>>> One final question: if I drop the collection index file .ncx at the data 
>>>>>> location, TDS seems to be able to serve it like a dataset, and for time 
>>>>>> series it is surprisingly fast to serve the .ncx file instead.  Is this 
>>>>>> intended - in other words we shall direct user to use .ncx, especially 
>>>>>> for long time series?  I know in default it's under cache/cdm and not 
>>>>>> exposed to users.  What's the catch here?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> -Fan
>>>>>> ________________________________________
>>>>>> From: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>>>>> Sent: Thursday, May 16, 2013 6:13 PM
>>>>>> To: address@hidden for
>>>>>> Cc: address@hidden
>>>>>> Subject: RE: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from 
>>>>>> TDS 4.2
>>>>>>
>>>>>> I can confirm changing the grib table name to have .tab as extension 
>>>>>> still does not work.
>>>>>>
>>>>>> I'll try the modifications in the units, but wonder why it fails for all 
>>>>>> since some of them have legit units, like 'K' for temperature.
>>>>>>
>>>>>> -Fan
>>>>>> ________________________________________
>>>>>> From: Unidata THREDDS Support [address@hidden]
>>>>>> Sent: Thursday, May 16, 2013 2:37 PM
>>>>>> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
>>>>>> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]; 
>>>>>> address@hidden
>>>>>> Subject: [THREDDS #GJF-200148]: TDS 4.3 GRIB names different from TDS 4.2
>>>>>>
>>>>>> Fan,
>>>>>>
>>>>>> I did a little more digging, and I don't think the file name extension 
>>>>>> will matter - that was my misinterpretation of how the table was getting 
>>>>>> picked up.  However, you will probably have the best result directly 
>>>>>> embedding the table information in your featureCollection element using 
>>>>>> the xml notation.  It looks like you tried this already, so I am not 
>>>>>> sure why it did not work for you.  It is possible that it broke because 
>>>>>> you did not follow udunits standard -- for instance, square meters is 
>>>>>> notated as m^2, rather than m(superscript)2.  The code you are asking 
>>>>>> about is just the number that identifies the grib parameter.
>>>>>>
>>>>>> Inside your <featureCollection> element:
>>>>>>
>>>>>> <gribConfig>
>>>>>> <parameterMap>
>>>>>> <parameter code="2">  <<-- This will override the standard table values 
>>>>>> for parameter 2 (code < 128 are standard, > 128 are local)
>>>>>> <description>Pressure reduced to MSL</description>
>>>>>> <units>Pa</units>
>>>>>> <name>PRMSL</name>
>>>>>> </parameter>
>>>>>> <parameter code="131">
>>>>>> <description>Snowfall rate</description>
>>>>>> <units>kg/m^2.s</units> <<--Compound units are notated with a dot, so 
>>>>>> /m^2/s is just /m^2.s
>>>>>> <name>Snof</name>
>>>>>> </parameter>
>>>>>> ...
>>>>>>
>>>>>> </parameterMap>
>>>>>>
>>>>>> </gribConfig>
>>>>>> -Lansing Madry
>>>>>> Unidata
>>>>>> Boulder, Colorado
>>>>>>
>>>>>> Ticket Details
>>>>>> ===================
>>>>>> Ticket ID: GJF-200148
>>>>>> Department: Support THREDDS
>>>>>> Priority: Low
>>>>>> Status: Closed
>>>>>>
>>>>>>
>>>>> -Lansing Madry
>>>>>    Unidata
>>>>>    Boulder, Colorado
>>>>>
>>>>> Ticket Details
>>>>> ===================
>>>>> Ticket ID: GJF-200148
>>>>> Department: Support THREDDS
>>>>> Priority: Low
>>>>> Status: Open
>>>>>
>>> -Lansing Madry
>>> Unidata
>>> Boulder, Colorado
>>>
>>> Ticket Details
>>> ===================
>>> Ticket ID: GJF-200148
>>> Department: Support THREDDS
>>> Priority: Low
>>> Status: Open
>>>
>>>
>> -Lansing Madry
>>    Unidata
>>    Boulder, Colorado
>>
>> Ticket Details
>> ===================
>> Ticket ID: GJF-200148
>> Department: Support THREDDS
>> Priority: Low
>> Status: Open
>>
>
>
> Ticket Details
> ===================
> Ticket ID: GJF-200148
> Department: Support THREDDS
> Priority: Low
> Status: Open
> Link:  
> https://www.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=22083



Ticket Details
===================
Ticket ID: GJF-200148
Department: Support THREDDS
Priority: Low
Status: Open