[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #TBR-569861]: how to merge asynch data using time as index

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #TBR-569861]: how to merge asynch data using time as index
  • Date: Wed, 18 Apr 2012 09:22:48 -0600
Hi Paul,

> we are looking at using NetCDF V3 for realtime data logging of marine
> sensors such as GPS, echosounders.
> 
> Each sensor will be logged to a dedicated nc file at its natural data
> rate.  it will be timestamped using a common clock (NTP).
> 
> the resulting files could be visulaised as:
> 
> sounder.nc
> Time, Depth
> 12:00:00.0, 12.123
> 12:00:00.1, 12.124
> 12:00:00.2, 12.124
> etc
> 
> position.nc
> Time, Latitude, Longitude
> 12:00:00.0, -32.0, 114.5
> 12:00:01.0, -32.1, 114.6
> 12:00:02.0, -32.2, 114.7
> etc
> 
> data will arrive at different rates.
> 
> We need to merge the positions into the sounder files, so we end up with
> 
> sounder.nc
> Time, Depth, Latitude, Longitude
> 12:00:00.0, 12.123, -32, 114.5
> 12:00:00.1, 12.124, -32.01, 114.51
> 12:00:00.2, 12.124, -32.02, 114.52
> etc
> 
> there are 2 questions here....
> 
> How would I lookup a variable such as latitide given a specific time
> (in this case the sounding time)?
> 
> If there is not an exact match, is it possible to easily interpolate to
> find the correct latitude?

The netCDF API doesn't support lookup by value for data in a disk
file.  Assuming time is a netCDF coordinate variable (which means it's
monotonically increasing and has no missing values), it's
straightforward to read the time coordinates into a memory array,
use something like a binary search algorithm to locate the time
indices identifying a time interval containing the time sought, and
use that index to access the desired latitude or interpolate a value
in the corresponding interval.

Alternatively, you could use one of the packages built on netCDF, such
as NCO, NCL, CDO, or FIMEX that supports interpolation:

  http://www.unidata.ucar.edu/netcdf/software.html#NCO
  http://www.unidata.ucar.edu/netcdf/software.html#NCL
  http://www.unidata.ucar.edu/netcdf/software.html#CDO
  http://www.unidata.ucar.edu/netcdf/software.html#fimex

Those are all open source packages, but there are also some commercial
packages that acess netCDF data and support interpolation, such as the
Environmental Workbench and ESRI software.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: TBR-569861
Department: Support netCDF
Priority: Normal
Status: Closed