[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: handling sub-second time-series data in netCDF in a CF compliant way

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: handling sub-second time-series data in netCDF in a CF compliant way
  • Date: Fri, 02 Oct 2009 13:26:43 -0600
Brian Eaton wrote:

Hi Janine,

On Fri, Oct 02, 2009 at 10:07:05AM -0600, Janine Goldstein wrote:

Hi Brian,
Thanks for your email. We do actually have time as unlimited sometimes, and we have received requests to remove the second dimension, so we are investigating ways to do that in a CF compliant way. 

So would I do something like:

variables:
 double time(time) ;
   time:long_name = "time" ;
   time:units = "second since 1970-01-01 00:00:00" ;
data:
 Time = 0,0.04,0.08,... ;

Then one second of my data would have 25 time values like 0.00,0.04, etc and 25
GFS values. But A1DC_LWO is 1hz data, so would I put a value and then 24
_FillValues? Wouldn't that pad the file and make it MUCH bigger?


Exactly.  You could do it the way you've suggested, but the space taken for
the 1hz data would be 25 times larger than it needs to be.
Is there a CF-compliant way to put the data in at 1hz, and then map that to a 24hz time variable? I am not seeing anything in the convention that handles multi-rate data in a single file. 

Section 8.2, "Compression by Gathering", describes a technique that could
be used for this purpose.  But I'm afraid that I've never seen software
that recognizes this convention.  And I've gotten the impression from
following the mail list discussions that this convention is considered too
complex.  So I can't really recommend that you use it.  John might have a
view on this as well.

I haven't looked into this, but I think it's possible to have multiple
unlimited dimensions in netcdf4.

In netcdf3 format, in order to have both a 1hz and a 25hz time coordinate
in the same file, they'd need to be regular fixed size dimensions.  If you
want them to be unlimited dimensions then having separate files for 1hz and
25hz data would be the way to go.

Brian

yes, i agree with brian's analysis. My preference would be:

dimensions:
time1Hz = 1000; time25Hz = 25000; 

variables:
   float time1Hz (time1Hz ) ;
   float time25Hz (time25Hz ) ;

   float data1(time1Hz ) ;
   float data2(time1Hz ) ;

   float data3(time25Hz ) ;
   float data4(time25Hz ) ;
   ...