Re: Point data CDF format

Hi Bruce:

For station data, use the "contiguous list" option if you can. Its quite 
efficient and allows variable number of obs per station. If you only 
have one station, you can just call if a "point observation collection" 
and you dont need any linked list at all.

Bruce Flynn wrote:
> Hi John,
>
> I did look at the Unidata Obs Convention and also what was available 
> for the CF convention.  Looking at the 2 different conventions was 
> confusing me, but now I think I have a better understanding of what I 
> was reading.
>
> It looks like the Unidata standard says that you have to use a linked 
> list of some sort.  Because my main goal is to make the data easy to 
> use for scientists.  That being said, it would be really nice to be 
> able to use the data in IDV, but I would rather not add the extra 
> confusion of the linked list.  Is there a way to make the data useful 
> in IDV without using a linked list data structure?  The Unidata 
> standard seems to suggest that there isn't.
>
> Thanks
> Bruce
>
>
> On Dec 6, 2007, at 1:59 PM, John Caron wrote:
>
> > Hi Bruce:
> >
> > CF "point obs" conventions are incomplete, and we dont have an 
> > adapter for them. as Don probably mentionaed, we are working on an 
> > improved version that we will submit to CF soon. in the meantime, if 
> > you need to get working with the IDV now, you should use the 
> > conventions at
> >
> > http://www.unidata.ucar.edu/software/netcdf-java/formats/UnidataObsConvention.html 
> >
> >
> >
> > Don Murray wrote:
> > > Hi Bruce-
> > >
> > > I'm going to pass this off to John Caron who knows the convention
> > > better than I do. I don't think you want CF 1.0 as the convention
> > > and I don't see that you have the  relation between the station
> > > and the record.
> > >
> > > John?
> > >
> > > Don
> > >
> > >
> > > Bruce Flynn wrote:
> > > > Don,
> > > >
> > > > I now have a file that my CF validator says is valid and has the 
> > > > coordinate variable time.  I tried to open it in IDV with the 
> > > > datatype of netCDF Point Data and it does not seem to work.  If I 
> > > > leave the file chooser at "I'm feeling lucky" and select "netCDF 
> > > > Point Data" it fails silently.  If I select "netCDF Point Data" 
> > > > from the start I get the following exception:
> > > >
> > > > ucar.unidata.data.BadDataException: Unable to make a 
> > > > PointObsDataset from 
> > > > /Users/brucef/dev/cvs/TOOLS/dev/rig/python/db2nc/test.nc
> > > > Error = **Failed to find Datatype Factory for= 
> > > > /Users/brucef/dev/cvs/TOOLS/dev/rig/python/db2nc/test.nc datatype= 
> > > > Station
> > > >
> > > >         at 
> > > > ucar.unidata.data.point.NetcdfPointDataSource.doMakeDataset(NetcdfPointDataSource.java:247) 
> > > >
> > > >         at 
> > > > ucar.unidata.data.point.NetcdfPointDataSource.getDataset(NetcdfPointDataSource.java:212) 
> > > >
> > > >         at 
> > > > ucar.unidata.data.point.NetcdfPointDataSource.initAfterCreation(NetcdfPointDataSource.java:199) 
> > > >
> > > >         at 
> > > > ucar.unidata.data.DataManager.createDataSource(DataManager.java:1222)
> > > >         at 
> > > > ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1829) 
> > > >
> > > >         at 
> > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1748) 
> > > >
> > > >         at 
> > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1682) 
> > > >
> > > >         at 
> > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1663) 
> > > >
> > > >         at 
> > > > ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:830) 
> > > >
> > > >         at 
> > > > ucar.unidata.idv.chooser.FileChooser.selectFilesInner(FileChooser.java:631) 
> > > >
> > > >         at 
> > > > ucar.unidata.idv.chooser.FileChooser.selectFiles(FileChooser.java:500)
> > > >         at 
> > > > ucar.unidata.idv.chooser.FileChooser.doLoadInThread(FileChooser.java:522) 
> > > >
> > > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > >         at 
> > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
> > > >
> > > >         at 
> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
> > > >
> > > >         at java.lang.reflect.Method.invoke(Method.java:585)
> > > >         at ucar.unidata.util.Misc$2.run(Misc.java:983)
> > > >         at ucar.unidata.util.Misc$3.run(Misc.java:1011)
> > > >
> > > > I can't really tell from the exception if it's a problem with my 
> > > > data format or if it's a bug in the code.
> > > >
> > > > Here's the latest NetCDF Header for my file, and I've also placed 
> > > > the file at ftp://ftp.ssec.wisc.edu/pub/incoming/test.nc if you 
> > > > want to take a look at it.
> > > >
> > > > netcdf test {
> > > > dimensions:
> > > >         time = UNLIMITED ; // (14851 currently)
> > > >         station = 1 ;
> > > > variables:
> > > >         double time(time) ;
> > > >                 time:long_name = "Record epoch time offset" ;
> > > >                 time:units = "seconds since 1970-1-1 0:00:00 0:00" 
> > > > ;tation as calculated." ;
> > > >                 ACCURAIN:missing_value = -9999. ;
> > > >         double temperature(time) ;
> > > >                 temperature:resolution = 5 ;
> > > >                 temperature:units = "degC" ;
> > > >                 temperature:standard_name = "air_temperature" ;
> > > >                 temperature:description = "Outside temperature 
> > > > reading." ;
> > > >                 temperature:missing_value = -9999. ;
> > > >         double rh(time) ;
> > > >                 rh:resolution = 5 ;
> > > >                 rh:units = "%" ;
> > > >                 rh:standard_name = "relative_humidity" ;
> > > >                 rh:description = "Outside relative humidity 
> > > > reading." ;
> > > >                 rh:missing_value = -9999. ;
> > > >         double wspd(time) ;
> > > >                 wspd:resolution = 5 ;
> > > >                 wspd:units = "m/s" ;
> > > >                 wspd:standard_name = "wind_speed" ;
> > > >                 wspd:description = "RM Young Wind Monitor speed 
> > > > reading." ;
> > > >                 wspd:missing_value = -9999. ;
> > > >         double wdir(time) ;
> > > >                 wdir:resolution = 5 ;
> > > >                 wdir:units = "degrees" ;
> > > >                 wdir:standard_name = "wind_from_direction" ;
> > > >                 wdir:description = "RM Young Wind Monitor direction 
> > > > reading." ;
> > > >                 wdir:missing_value = -9999. ;
> > > >         double elevation(station) ;
> > > >                 elevation:units = "meters" ;
> > > >                 elevation:description = "Height of the base of the 
> > > > RIG Tower" ;
> > > >         double latitude(station) ;
> > > >                 latitude:units = "degrees_east" ;
> > > >                 latitude:description = "Latitiude of the RIG Tower" ;
> > > >         double longitude(station) ;
> > > >                 longitude:units = "degrees_north" ;
> > > >                 longitude:description = "Longitude of the RIG Tower" ;
> > > >
> > > > // global attributes:
> > > >                 :convention = "CF-1.0" ;
> > > >                 :description = "Meteorological Station Observations 
> > > > from the tower on the top of\n",
> > > >                         "the UW SSEC/AOS building, commonly referd 
> > > > to as the Rooftop Instrument Group (RIG)." ;
> > > >                 :contact = "University of Wisconsin\n",
> > > >                         "Space Science and Engineering Center\n",
> > > >                         "1225 W. Dayton St.\n",
> > > >                         "Madison, WI 53706\n",
> > > >                         "http://www.ssec.wisc.edu"; ;
> > > >                 :url = "http://rig.ssec.wisc.edu"; ;
> > > > }
> > > > On Dec 4, 2007, at 10:45 AM, Don Murray wrote:
> > > >
> > > > > Hi Bruce-
> > > > >
> > > > > I guess the docs have been updated as we get ready to move to
> > > > > Netcdf 4.  However, the CDL at:
> > > > >
> > > > > http://www.unidata.ucar.edu/software/netcdf-java/formats/metar.cdl.txt 
> > > > >
> > > > >
> > > > > doesn't seem to use structures but adheres to the convention.  I'm
> > > > > cc'ing John Caron who is responsible for this convention for his 
> > > > > input.
> > > > >
> > > > > Don
> > > > >
> > > > > Bruce Flynn wrote:
> > > > > > Don,
> > > > > > Most of the record formats listed involve NetCDF Structures.  I 
> > > > > > think this is obviously the best way to go for efficiency, but it 
> > > > > > seems there are not a lot of APIs that support reading 
> > > > > > Structures.  I was planing on using the "pseudo-Structure" model 
> > > > > > so the scientists can use what ever API they want.  It seems to 
> > > > > > want a station dimension though.  Is this really necessary if you 
> > > > > > only have a single station?
> > > > > > Bruce
> > > > > > On Dec 3, 2007, at 3:44 PM, Don Murray wrote:
> > > > > > >
> >
> > > > > --
> > > > > *************************************************************
> > > > > Don Murray                               UCAR Unidata Program
> > > > > address@hidden                        P.O. Box 3000
> > > > > (303) 497-8628                              Boulder, CO 80307
> > > > > http://www.unidata.ucar.edu/staff/donm
> > > > > *************************************************************