[THREDDS #CRZ-633675]: THREDDS UNIDATA SERVER

Greetings!

There are a few reasons why variables within the GRIB collections have a 
different number of time steps. One is due to some including the zero hour 
forecast, while others do not (note, a zero hour forecast field is generally 
different than an analysis field). Another reason is due to the way the 
National Centers for Environmental Prediction (NCEP) produces GRIB messages for 
a given variable.

Very roughly, there are two types of variables that are constructed for a 
collection of GRIB messages - instantaneous variables and interval variables. 
Temperature at 2m (Temperature_height_above_ground), for example, would be an 
instantaneous variable, where as Total Precipitation Accumulation 
(Total_precipitation_surface_Mixed_intervals_Accumulation) would be an interval 
variable. Interval variables will always include something like {interval 
length}_{statistical process}, so something like 12_hour_Accumulation or 
Mixed_intervals_Accumulation. This all has to do with how netCDF-Java maps a 
collection of 2D GRIB messages into an ND netCDF-like Variable, and is 
described here:

https://docs.unidata.ucar.edu/netcdf-java/5.2/userguide/grib_files_cdm.html#mapping-a-grib-collection-into-multidimensional-variables

Now, in the case of GFS, and specifically the total precipitation variable, it 
turns out that NCEP computes the accumulation over several intervals, not just 
a fixed interval (say a 6 hour interval). This is all described in the variable 
metadata. Again, looking at the total precipitation in the case of 0.25 degree 
output from the 6Z run of the GFS on 2019-12-02 
(https://thredds.ucar.edu/thredds/catalog/grib/NCEP/GFS/Global_0p25deg/latest.html?dataset=grib/NCEP/GFS/Global_0p25deg/GFS_Global_0p25deg_20191202_0600.grib2),
 the metadata look like (as seen from 
https://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p25deg/GFS_Global_0p25deg_20191202_0600.grib2.html):

Total_precipitation_surface_Mixed_intervals_Accumulation(time, lat, lon):
    long_name: Total precipitation (Mixed_intervals Accumulation) @ Ground or 
water surface
    units: kg.m-2
    abbreviation: APCP
    missing_value: NaN
    grid_mapping: LatLon_Projection
    coordinates: reftime time lat lon 
    Grib_Statistical_Interval_Type: Accumulation
    Grib_Variable_Id: VAR_0-1-8_L1_Imixed_S1
    Grib2_Parameter: 0, 1, 8
    Grib2_Parameter_Discipline: Meteorological products
    Grib2_Parameter_Category: Moisture
    Grib2_Parameter_Name: Total precipitation
    Grib2_Level_Type: 1
    Grib2_Level_Desc: Ground or water surface
    Grib2_Generating_Process_Type: Forecast

We see the dimensions of the variable are:

time, lat, and lon

where the lengths of those dimensions are:

time = 180
lat = 721lon = 1440

We also see the "coordinates" associated with the variable are:

coordinates: reftime time lat lon 

The name of the appropriate time variable to use will always be found in the 
"coordinates" attribute (following the Climate and Forecast metadata 
conventions).

If we look at the time variable, we see:

time(time):
    units: Hour since 2019-12-02T06:00:00Z    standard_name: time
    long_name: GRIB forecast or observation time
    calendar: proleptic_gregorian
    bounds: time_bounds
    _CoordinateAxisType: Time

If we look at the actual values of the time array, we see:

(https://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p25deg/GFS_Global_0p25deg_20191202_0600.grib2.ascii?time%5B0:1:179%5D):

time[180]
3.0, 6.0, 9.0, 9.0, 12.0, 12.0, 15.0, 15.0, 18.0, 18.0,...

As you can see here, we have some repeated values (e.g. there are two 12's, 
15's, etc.), and this accounts for the different number of time steps in these 
kinds of variables. What this means, for example, is that there are two 
accumulated precipitation amounts that are valid at the 12 hour forecast. In 
order to distinguish between the two, you need to look at the metadata of the 
time variable. In that metadata, we see there is a "bounds" attribute whose 
value is time_bounds. Again, following the Climate and Forecast metadata 
conventions, this means you need to inspect the time_bounds variable for more 
information. If we look at that variable, we see:

[0], 0.0, 3.0
[1], 0.0, 6.0
[2], 0.0, 9.0
[3], 6.0, 9.0
[4], 0.0, 12.0
[5], 6.0, 12.0
[6], 0.0, 15.0
[7], 12.0, 15.0
[8], 0.0, 18.0
[9], 12.0, 18.0

Pairing the time_bounds variable up with the time variable, we get the 
following 
(https://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p25deg/GFS_Global_0p25deg_20191202_0600.grib2.ascii?time%5B0:1:179%5D,time_bounds%5B0:1:179%5D%5B0:1:1%5D):

time[0], time_bounds[0][:]:
Total precipitation valid at 3 hours into the forecast, representing the 
precipitation accumulation over the 0-3 hour period of time.

time[1], time_bounds[1][:]:
Total precipitation valid at 6 hours into the forecast, representing the 
precipitation accumulation over the 0-6 hour period of time.

time[2], time_bounds[2][:]:
Total precipitation valid at 9 hours into the forecast, representing the 
precipitation accumulation over the 0-9 hour period of time.

So far, so good. Now it gets tricky:

time[3], time_bounds[3][:]:
Total precipitation valid at 9 hours into the forecast, representing the 
precipitation accumulation over the 6-9 hour period of time.

So, the first time value of 9 (time index 2) is associated with a 9 hour 
accumulation, and the second time value of 9 (time index 3) is associated with 
a 3 hour accumulation. This is why the variable is considered a "Mixed 
interval" variable - there is not one unique accumulation period used by the 
GRIB messages produced by NCEP. This will be the case for any variable that 
includes "Mixed_intervals" in its name.

Note that the name of the time variable, time dimension, and time bounds 
variable will likely change between model runs. That is because netCDF-Java 
creates these as a collection of GRIB messages is scanned (variable names and 
dimensions are not concepts found within a GRIB message and must be 
constructed), and the order in which we receive various messages from NCEP is 
not set. The GRID found within the first message will get assigned "time", for 
example, where GRIDs that require another set of times (mixed interval 
variables, for example), will be assigned "time2", "time3", etc. However, the 
name of the appropriate time variable will always be found in the "coordinates" 
attribute of a given variable, and that's what you should use.

Cheers,

Sean

> My name is Luca and I am a meteorologist trying to access GFS data from
> the UCAR servers. One issue I have found is that when I query for the
> data, I tend to receive variables that are not the same length, which
> is concerning for me. This is because some variables for example include
> an initial time step, and some do not.
> 
> An example would be for temperature at 2 meters, if I query for all
> the times I receive a (129, 361, 720) matrix. This runs from 00z
> to 00z 16 days later. But if I query for Total Precip, my matrix is
> (138,361,720). Therefore I am unable to distinguish the time variables
> for each of these variables.
> 
> Is there a way for me to be able to distinguish times that are associated
> with each variable?
> 
> Thanks
> 
> 
> 





Ticket Details
===================
Ticket ID: CRZ-633675
Department: Support THREDDS
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.