Re: EOL created netcdf files

John,

Attached is an email exchange that happened more than a year ago about conventions for netcdf files.  I wanted to revisit this once again since you mentioned at the time that there were more issues that you had to think about.  I have also noticed that there is a newer CF Conventions document and we would like to follow the latest CF conventions with our created netcdf files.  Here are my questions:

1) I was wondering if you had any other comments on my created netcdf files or other issues that you had more information about since you mentioned that there were more issues for you to think about.

2) I'm not certain what type of point feature type I should be using for this data.  My data sounds most like the description to profile, but then there is stationProfile which may be best used for a file which has multiple stations vs. one station.  But when I look at profile, it seems like my data wouldn't fit into this category, since it requires values to uniquely identify the profile.  Lat and lon don't change within one file of station data, so maybe it should be trajectory?   Lat, lon and altitude, in my data file, are not technically coordinate variables, since they don't change within a file.  The only coordinate variables are time and height.  It's also not clear what data would be contained within trajectory_id or profile_id, i.e. what values would I use to uniquely identify either the trajectory or profile with my data.  Trajectory seems to apply more towards flight or ship data, since the lat and lon would be different.  What do you think is the most appropriate point feature type to use with my data?  Here is a snapshot from my netcdf file:
dimensions:
    time = 35088 ;
    station = 1 ;
    height = 2 ;
variables:
    float height(height) ;
        height:standard_name = "height" ;
        height:long_name = "Sensor Height" ;
        height:units = "m" ;
        height:positive = "up" ;
    float latitude(station) ;
        latitude:standard_name = "latitude" ;
        latitude:long_name = "Latitude" ;
        latitude:units = "degrees_north" ;
    float longitude(station) ;
        longitude:standard_name = "longitude" ;
        longitude:long_name = "Longitude" ;
        longitude:units = "degrees_east" ;
    float altitude(station) ;
        altitude:standard_name = "altitude" ;
        altitude:long_name = "Altitude" ;
        altitude:units = "m" ;
        altitude:positive = "up" ;
    double time(time) ;
        time:standard_name = "time" ;
        time:long_name = "UTC Actual Date/Time" ;
        time:units = "seconds since 1970-01-01 00:00:00" ;
    double time_nominal(time, height) ;
        time_nominal:standard_name = "time_nominal" ;
        time_nominal:long_name = "UTC Nominal Date/Time" ;
        time_nominal:units = "seconds since 1970-01-01 00:00:00" ;
    float surface_air_pressure(time, height) ;
        surface_air_pressure:standard_name = "surface_air_pressure" ;
        surface_air_pressure:long_name = "Station Pressure" ;
        surface_air_pressure:units = "Pa" ;
        surface_air_pressure:_FillValue = -999.99f ;
        surface_air_pressure:missing_value = -999.99f ;
    char surface_air_pressure_flag(time, height) ;
        surface_air_pressure_flag:standard_name = "surface_air_pressure_flag" ;
        surface_air_pressure_flag:long_name = "Station Pressure Flag" ;
        surface_air_pressure_flag:flag_value = "C M B I D G U" ;
        surface_air_pressure_flag:flag_meanings = "exceeds_field_size missing bad interpolated_or_estimated_or_gap_filled questionable good unchecked" ;
        surface_air_pressure_flag:_FillValue = "M" ;
        surface_air_pressure_flag:missing_value = "M" ;

3) Do you have some simple data files that you could point me to, that would be similar to the data files that I am creating?

4) Do I need to specify geospatial_lat_min, etc for these data files for it to properly follow standards?

Thanks in advance for any help you can offer to clarify these issues.
Janet


John Caron wrote:

Janet Scannell wrote:
  

Hi John,

Thank you for your response.  I have some more questions which I have
included below.

Thanks,
Janet

John Caron wrote:
    

Hi Janet:

My apologies for taking so long to respond. The problem is that CF does not cover point data adequately, and Ive been trying to formulate a proposal to CF about this. Im afraid you will have to decide whether its worth waiting for that to be an official standard or not. I can probably give you some reasonable interim advice, but it may not end up to be standard.

Looking at your files:

980101.PAM_Atl_met.nc looks fine as an unconnected collection of point data, using existing CF Conventions. However, you may intend it to be a trajectory, ie a connected collection?
  
      

Are there any restrictions on what should be considered a trajectory? 
According to the web site: "A  trajectory  is a collection of
observations which are connected along a one dimensional track in space,
with time increasing monotonically along the track".  The observations
in the PAM data file are made from radio towers that are mounted on the
sea ice.  The lat and long for the stations change continually because
the ice is constantly shifting.  Therefore, it doesn't fit the rule that
the observations are connected along a one dimensional track in space. 
Since the points are all from the same radio tower, I would think that
they should be considered connected in some fashion.  Would a trajectory
be the correct definition to connect all of these points together?
    

its an interesting "grey area" between a station time series, a trajectory, and a point collection. technically, its closest to a trajectory, but it would be up to you how to name it.

  

I have also fixed the latitude and longitude so that there are not
missing values for these variables.
    

good

  

BALTEX_Lindenberg_Falkenberg_20021001_20041231_sfc.nc is a time series of station data, with only one station. Will all your files have only one station, or do you want to be able to add multiple stations in the same file? If so, will all stations have the same number of observations?

  
      

We have decided that there will be only one station in each netcdf file
for this dataset.
    

that makes things easy, and the need to clarify trajectory vs point collection less important.


  

BALTEX_Lindenberg_Falkenberg_20021001_20041231_twr.nc has a couple of problems: 

1) rename variable "height" to "heights" so its a coordinate variable. (or rename dimension "heights" to "height".

2) making the time variable two dimensional time(time, heights) looks like its unneeded, ie it could be time(time)? If you really need 2D time, then you must explicitly add to each data variable:

  :coordinates = "time height"

there are still some issues that i have to think about some more.