Re: [IDV #GKC-227821]: CF Compliant Observation Data

Hi Bruce: I see two problems with the file:

1. the time coordinate has

  :standard_name ="forecast_reference_time";

should be

  :standard_name ="time";

in order for the time coordinate to be correctly identified.

2. all of the coordinates attributes should be

  :coordinates = "time lat lon alt";

instead of

  :coordinates = "time lat lon";

in order for the height coordinate to be correctly identified.

I might also propose

  :standard_name ="station_description";

that you might use for your station_info variable.

Anyway, thanks for your example, very helpful for me to test and debug. If you
generate any more
variants, please send. Im not sure when IDV will be able to visualize these,
but it shouldnt be too
long.


Bruce Flynn wrote:
> ftp://ftp.ssec.wisc.edu/pub/ssec/brucef/rig_tower.2009-02-01.nc
>
> In this file I had to replace the ':' in CF:featureType with a '-'
> because my libs are pre 3.6.2 on my generation machine and I haven't
> been able to upgrade yet.  If that causes issues for you I can generate
> and make available a new file with the correct attribute name when I get
> in tomorrow.
>
> Bruce
>
> On Feb 10, 2009, at 6:26 PM, Unidata netCDF Java Support wrote:
>
>> Now im sorry:
>>
>> this seems to have been timed off, can you restore?
>>
>>> Sorry for the delay, I've been wrapped up on other projects.  Here is
>>> a link to a file that I believe conforms to the contiguous ragged
>>> array time series station data.
>>>
>>> ftp://ftp.ssec.wisc.edu/pub/ssec/brucef/rig_tower.2009-01-13.nc
>>>
>>> Hope it helps,
>>> Bruce
>>>
>>> On Dec 18, 2008, at 8:44 PM, Unidata netCDF Java Support wrote:
>>>
>>>> Hi Bruce:
>>>>
>>>> If you get a test file, send it to me, as I am testing our software.
>>>> thanks.
>>>>
>>>>> Well, it doesn't appear that I'm able to leave my thoughts on the
>>>>> ticket page so I'll just send them to you.
>>>>>
>>>>> I currently have a 2 stations at the center, a mobile one that goes
>>>>> out on experiments, and a stationary one on the rooftop.  In addition
>>>>> we have a stationary buoy on the lake and some folks that would like
>>>>> me to work with data from their Antarctic stations.  Eventually I
>>>>> would like to archive our radiosonde launches as well.  I would
>>>>> foresee only using 1 station per file, except in the Antarctic case
>>>>> where there may be up to 60, but to keep the format as consistent as
>>>>> possible I'm leaning towards using a station dimension in all cases.
>>>>>
>>>>> With that being said I believe the ragged array representations fit
>>>>> all my needs.  Since space utilization is not really an issue, I
>>>>> would
>>>>> lean towards the non-contiguous case to maintain flexibility and
>>>>> generality in the case I ever had to write non-contiguous data.  I'm
>>>>> not sure I would be concerned about the performance unless it was
>>>>> shown to be more than a theoretical problem, and since I'll be
>>>>> providing most my data via OPeNDAP I'm not sure it would be the
>>>>> bottleneck.
>>>>>
>>>>>
>>>>> Thanks for your hard work,
>>>>> Bruce
>>>>>
>>>>>
>>>>> On Oct 21, 2008, at 6:46 PM, Unidata netCDF Java Support wrote:
>>>>>
>>>>>> Hi Bruce:
>>>>>>
>>>>>> Officially it will take a minimum of three weeks to be accepted, but
>>>>>> if you follow the conversation, you will probably see that any real
>>>>>> issues will be resolved quickly. As you'll see in my last post, it
>>>>>> would be helpful if you sent in your opinion on which
>>>>>> representation(s) you will want to use.
>>>>>>
>>>>>> http://cf-pcmdi.llnl.gov/trac/ticket/37
>>>>>>
>>>>>> You can comment by joinging the list, or send to me and I will
>>>>>> forward if you prefer.
>>>>>>
>>>>>>> Hi Ethan, Mr. Caron,
>>>>>>>
>>>>>>> There's some interesting information that was forwarded my way.  I
>>>>>>> seems there is a ticket in with the CF folks proposing a point
>>>>>>> observation standard submitted by Mr. Caron.  After reading through
>>>>>>> this it's clear that this meets all my requirements and would work
>>>>>>> very well for the various stations I'm working with.
>>>>>>>
>>>>>>> While I believe the proposed standard solves some issues it also
>>>>>>> presents some.  Because it's only "proposed" I'm left with the
>>>>>>> decision on whether to use this format or not.  I have to start
>>>>>>> publishing files for general access, but at the same time don't
>>>>>>> want
>>>>>>> to be in the situation where I've published files in the proposed
>>>>>>> standard and then I have to regenerate and re-publish everything
>>>>>>> when
>>>>>>> the standard is rejected or modified.
>>>>>>>
>>>>>>> Can you or Mr. Caron provide any insight as to whether this will be
>>>>>>> adopted by the CF or not?  I imagine there's not a competing
>>>>>>> proposal,
>>>>>>> so I would think chances are good.  I'm aware it's not really a
>>>>>>> fair
>>>>>>> question, asking you to predict the future and all, but I'm sure
>>>>>>> you
>>>>>>> all may have a better understanding of the process that I do.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Bruce
>>>>>>>
>>>>>>>
>>>>>>> On Oct 16, 2008, at 5:30 PM, Unidata netCDF Java Support wrote:
>>>>>>>
>>>>>>>> Hi Bruce,
>>>>>>>>
>>>>>>>> Currently, the main focus of CF is gridded data. The other types
>>>>>>>> of
>>>>>>>> data (e.g., station and trajectory) are not as widely used or
>>>>>>>> tested.
>>>>>>>>
>>>>>>>> Currently, the CF convention is really only fully formed for
>>>>>>>> gridded
>>>>>>>> data. The station timeseries section you refer to is not very
>>>>>>>> complete nor is it widely (if at all) used. We have done some work
>>>>>>>> on conventions for this and other types of data but they are still
>>>>>>>> evolving. Some discussion of them has taken place on the CF
>>>>>>>> discussion list but they haven't moved much beyond that.
>>>>>>>>
>>>>>>>> The netCDF conventions page
>>>>>>>> (http://www.unidata.ucar.edu/software/netcdf/conventions.html
>>>>>>>> ) lists a number of existing conventions. Two that I'm aware of
>>>>>>>> address station data to some degree:
>>>>>>>>
>>>>>>>> 1) The "Unidata Observation Dataset Convention" is the one we have
>>>>>>>> developed. Though it  probably needs some updating. And I'm not
>>>>>>>> sure
>>>>>>>> how fully we support it in the netCDF-Java/CDM code.
>>>>>>>>
>>>>>>>> 2) The PMEL-EPIC convention also deals with timeseries data.
>>>>>>>> However, I'm not very familiar with it so can't really comment on
>>>>>>>> its capabilities.
>>>>>>>>
>>>>>>>>
>>>>>>>> This is definitely an area that needs work. If you have any
>>>>>>>> insights, suggestions, or time to work on convention development
>>>>>>>> that would be great.
>>>>>>>>
>>>>>>>> Hope that helps,
>>>>>>>>
>>>>>>>> Ethan
>>>>>>>>
>>>>>>>>
>>>>>>>>> I have met station observations from our tower on the roof of the
>>>>>>>>> building going back to 2001.  Currently the data is really only
>>>>>>>>> used
>>>>>>>>> by a couple of widgets for displaying the current or recent
>>>>>>>>> conditions.  I would like to make the data more widely
>>>>>>>>> available in
>>>>>>>>> NetCDF files that are both CF compliant and displayable in IDV.
>>>>>>>>> I've
>>>>>>>>> been through the CD-1.3 documents and sections 5.4 seems to
>>>>>>>>> prefer a
>>>>>>>>> format similar to the CDL below.
>>>>>>>>>
>>>>>>>>> Before I generate files for all my data I would like to know if
>>>>>>>>> NetCDF
>>>>>>>>> files with this format display in IDV in a meaningful manner?  If
>>>>>>>>> not
>>>>>>>>> what format should I use?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Bruce
>>>>>>>>>
>>>>>>>>> netcdf test {
>>>>>>>>> dimensions:
>>>>>>>>> time = UNLIMITED ; // (0 currently)
>>>>>>>>> station = 1 ;
>>>>>>>>> id_len = 4 ;
>>>>>>>>> variables:
>>>>>>>>> double time(time, station) ;
>>>>>>>>> time:long_name = "Time offset from midnight" ;
>>>>>>>>> time:units = "seconds since 1970-1-1 0:00:00" ;
>>>>>>>>> char station_id(station, id_len) ;
>>>>>>>>> char station_description(station, id_len) ;
>>>>>>>>> double temp(time, station) ;
>>>>>>>>> temp:long_name = "Air Temperature" ;
>>>>>>>>> temp:standard_name = "air_temperature" ;
>>>>>>>>> temp:units = "degC" ;
>>>>>>>>> temp:valid_min = -50. ;
>>>>>>>>> temp:valid_max = 50. ;
>>>>>>>>> temp:precision = 0.1 ;
>>>>>>>>> temp:missing_value = -9999. ;
>>>>>>>>> temp:_FillValue = -9999. ;
>>>>>>>>> int qc_temp(time, station) ;
>>>>>>>>> qc_temp:long_name = "Quality check results on field: Relative
>>>>>>>>> Humidity" ;
>>>>>>>>> qc_temp:missing_value = -9999 ;
>>>>>>>>> qc_temp:_FillValue = -9999 ;
>>>>>>>>> double lat(station) ;
>>>>>>>>> lat:long_name = "north latitude" ;
>>>>>>>>> lat:units = "degrees_north" ;
>>>>>>>>> lat:valid_min = -90. ;
>>>>>>>>> lat:valid_max = 90. ;
>>>>>>>>> double lon(station) ;
>>>>>>>>> lon:long_name = "east longitude" ;
>>>>>>>>> lon:units = "degrees_east" ;
>>>>>>>>> lon:valid_min = -180. ;
>>>>>>>>> lon:valid_max = 180. ;
>>>>>>>>> double alt(station) ;
>>>>>>>>> alt:long_name = "height above mean sea level" ;
>>>>>>>>> alt:units = "meters" ;
>>>>>>>>>
>>>>>>>>> // global attributes:
>>>>>>>>> :title = "Rooftop Instrument Group(RIG) Tower obervations" ;
>>>>>>>>> :institution = "University of Wisconsin Space Science and
>>>>>>>>> Engineering Center 1225 W. Dayton St. Madison, WI 53706" ;
>>>>>>>>> :source = "surface observation" ;
>>>>>>>>> :history = "" ;
>>>>>>>>> :references = "" ;
>>>>>>>>> :comment = "" ;
>>>>>>>>> :Conventions = "CF-1.3" ;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Ticket Details
>>>>>>>> ===================
>>>>>>>> Ticket ID: GKC-227821
>>>>>>>> Department: Support netCDF Java
>>>>>>>> Priority: Normal
>>>>>>>> Status: Closed
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Ticket Details
>>>>>> ===================
>>>>>> Ticket ID: GKC-227821
>>>>>> Department: Support netCDF Java
>>>>>> Priority: Normal
>>>>>> Status: Open
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> Ticket Details
>>>> ===================
>>>> Ticket ID: GKC-227821
>>>> Department: Support netCDF Java
>>>> Priority: Critical
>>>> Status: Closed
>>>>
>>>
>>>
>>
>>
>> Ticket Details
>> ===================
>> Ticket ID: GKC-227821
>> Department: Support netCDF Java
>> Priority: Critical
>> Status: Open
>>