Re: NDBC buoy data aggregation

Joe McLean wrote:

> Hi John,
>
> Attached is an NcML aggregation file which is a good example of  NDBC time
> series data which has missing values.
> The netcdf elements with missing data are below.
>
>   <!-- 2002-10 has 0 hrs = 0 days -->
>   <netcdf
> uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2002_10_stdmet.nc'/>
> MISSING 744 of 744 hrs
> I could create a dummy file using all missing_value values and plug it in place
> of this uri.
>
>   <!-- 2003-04 has 184 hrs = 7.66666666666667 days -->
>   <netcdf
> uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2003_04_stdmet.nc'/>
> MISSING 536 of 720 hrs: 'Apr-08 16:00' thru 'Apr-30 23:00'
>
>   <!-- 2003-05 has 414 hrs = 17.25 days -->
>   <netcdf
> uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2003_05_stdmet.nc'/>
> MISSING 330 of 744 hrs: 'May-01 00:00` thru 'May-14 17:00'
>
> The last two are missing data on either end of the expected time axis.  In the
> previous example (email 22SEP 16:50), the missing data was in the middle of the
> expected time axis.  I'll leave this to you to see how one could use NCML to
> aggregate such files, but using agg type 'joinNew' and adding a range attribute
> to the netcdf element (then, when the missing data is in the middle of the time
> axis, the uri could be listed twice) might be an idea.  Then there is the whole
> issue of irregularly stepped time axes - I realize with NCML aggregations, one
> can hard code the coordinate values, but these aggregations are 17,000+ time
> steps - what issues would be involved then?.
>
> Joe
>  
i have an experimental version that allows unit conversion, as long as 
the individual files have correct udunits.

get a new version of the AS:

ftp://ftp.unidata.ucar.edu/pub/thredds/dodsCupdate.war

and try this in your NcML, instead of the <values> element:

 <variable name='time' type='double' shape='time'>
   <attribute name='long_name' type='string' value='time coordinate'/>
   <attribute name='units' type='string' value='hours since 2003-08-19'/>
   <units convertTo="hours since 2003-08-19" />
 </variable>

where you change your units (in both places) to whatever you want.

let me know how that works.

CAUTION: the syntax is experimental, although i will try to keep the 
functionality. so test it on one before you do a lot of work converting, 
unless it will be easy to change.