Sorry for not responding sooner helpful gents, as I see in this
thread I was asked a question.
In looking at the below PDS info, I see no valid reason why they
would have this set as they do:
Grib2ProductDefinitionSection
Interval =
(2012-08-27T00:00:00.000Z,2012-09-27T00:00:00.000Z)
The file is/was a climate model run that ran on 2012-09-27T00:00:00.000Z,
with forecast conditions that go out MONTHS in advance. I see no
good reason for them to be referencing 2012-08-27T00:00:00.000Z--that
looks like a flat out error.
It seems to me they need to encode that setting as you surmised it
should be:
(2013-05-01T00:00:00.000Z, 2013-06-01T00:00:00.000Z)
So, should I attempt to find the "creator" of this GRIB to get
this corrected, rather than you making changes to your system
which would work if they encoded it properly?
On 9/28/2012 2:19 PM, John Caron wrote:
Hi all:
1. The CFSR dataset is known to have grib encoding issues, so
its possible we are seeing some of that and/or bugs in the
software. in particular, the translation of grib1 to grib2 had
bugs in it. ive been unable to get authoritative info on how to
fix this, but i do have machinery to do so if we can figure out
what to do.
2. The 4.3 software is completely different from 4.2. We are
only going to fix 4.3 grib problems. 4.2 doesnt handle time
interval coordinates correctly.
3. In this file:
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01/flxf.01.2012092700.201305.avrg.grib.grb2
I am seeing records like the one I have dumped below.
statistic type is "Average"
reference time is 2012-09-27T00:00:00.000Z
the "end of overall time interval" is == 2012/09/27T0:0:0
forecast time is 8 months.
we are interpreting the time interval coordinate as:
(0.000000, 1.000000) == (2012-08-27T00:00:00.000Z,
2012-09-27T00:00:00.000Z)
which seems wrong. ill have to look at this closer.
4. Note that this is an interval coordinate, meaning the "valid
time" is a time interval. From this comment:
"The forecasts from that model run are for the mean
conditions over the months of May 2013"
Im guessing the correct interval should be:
(0.000000, 1.000000) == (2013-05-01T00:00:00.000Z,
2013-06-01T00:00:00.000Z)
so:
- we do have a bug
- the dates are encoded wrong
- have to figure out how to fix systematically. if theres a
way to derive the correct answer from the info in the PDS
below, can you see it? It will be unfortunate if we have to
rely on the filename.
---------
File=0
E:/data/work/ansari/flxf.01.2012092700.201305.avrg.grib.grb2
Header=""
Grib2IndicatorSection
Discipline = (0) Meteorological products
Length = 64370
Grib2IdentificationSection
Center = (7) US National Weather Service,
National Centres for Environmental Prediction (NCEP)
SubCenter = (0) null
Master Table = 2
Local Table = 1
RefTimeSignif = 1 (Start of forecast)
RefTime = 2012-09-27T00:00:00.000Z
RefTime Fields = 2012-9-27 0:0:0
ProductionStatus = 0 (Operational products)
TypeOfProcessedData = 1 (Forecast products)
Grib2GridDefinitionSection hash=-1824839391
crc=4229097007
Length = 72
Source (3.0) = 0 (Specified in Code table 3.1)
Npts = 72960
Template (3.1) = 40
(3.40) Grid definition template 3.40 - Gaussian
latitude/longitude
1:
GDS length == 72
5:
Section == 3
6: Source
of Grid Definition (see code table 3.0) == 0 (table 3.0:
Specified in Code table 3.1)
7:
Number of data points == 72960
11: Number
of octects for optional list of numbers == 0
12:
Interpretation of list of numbers == 0 (table 3.11: There is
no appended list)
13:
Grid Definition Template Number == 40
15:
Shape of the Earth == 6 (table 3.2: Earth assumed spherical
with radius of 6 371 229.0 m)
16:
Scale factor of radius of spherical Earth == 0
17:
Scaled value of radius of spherical Earth == 0
21: Scale
factor of major axis of oblate spheroid Earth == 0
22: Scaled
value of major axis of oblate spheroid Earth == 0
26: Scale
factor of minor axis of oblate spheroid Earth == 0
27: Scaled
value of minor axis of oblate spheroid Earth == 0
31:
Ni - number of points along a parallel == 384
35:
Nj - number of points along a meridian == 190
39:
Basic angle of the initial production domain == 0
43: Subdivisions of basic angle used to define extreme
longitudes and latitudes, and direction increments == 0
47:
La1 - latitude of first grid point == 89277000
51:
Lo1 - longitude of first grid point == 0
55:
Resolution and component flags == 48
56:
La2 - latitude of last grid point == -89277000
60:
Lo2 - longitude of last grid point == 359062000
64:
Di - i direction increment == 938000
68: N - number of
parallels between a pole and the Equator == 95
72:
Scanning mode == 0
73: List of number of
points along each meridian or parallel. == -9999
Grib2ProductDefinitionSection
Interval =
(2012-08-27T00:00:00.000Z,2012-09-27T00:00:00.000Z)
(4.8) Product definition template 4.8 - average,
accumulation and/or extreme values or other
statistically-processed values at a horizontal level or in a
horizontal layer in a continuous or non-continuous time
interval
1:
PDS length == 58
5:
Section == 4
6:
Number of coordinates values after Template == 0
8:
Product Definition Template Number == 8
10:
Parameter category == 2
11:
Parameter number == 17
12:
Type of generating process == 2 (table 4.3: Forecast)
13: Background generating process
identifier (defined by originating centre) == 0 (table
ProcessId: Table ProcessId code 0 not found)
14: Analysis or forecast generating process
identifier (defined by originating centre) == 98 (table
ProcessId: Table ProcessId code 98 not found)
15:
Hours after reference time of data cutoff == 0
17:
Minutes after reference time of data cutoff == 0
18:
Indicator of unit of time range == 3 (table 4.4: Month)
19:
Forecast time in units defined by octet 18 == 8
23:
Type of first fixed surface == 1 (table 4.5: Ground or water
surface)
24:
Scale factor of first fixed surface == 0
25:
Scaled value of first fixed surface == 0
29:
Type of second fixed surface == 255 (table 4.5: Missing)
30:
Scale factor of second fixed surface == 0
31:
Scaled value of second fixed surface == 0
35:
Year - time of end of overall time interval == 2012
37:
Month - time of end of overall time interval == 9
38:
Day - time of end of overall time interval == 27
39:
Hour - time of end of overall time interval == 0
40:
Minute - time of end of overall time interval == 0
41:
Second - time of end of overall time interval == 0
42: n - number of time range specifications describing
the time intervals used to calculate the
statistically-processed field == 1
43: Total number of
data values missing in statistical process == 0
47: Statistical process used to calculate the
processed field from the field at each time increment during
the time range == 0 (table 4.10: Average)
48: Type of time increment between successive
fields used in the statistical processing == 2 (table 4.11:
Successive times processed have same start time of forecast,
forecast time is incremented)
49: Indicator of unit of time for time range
over which statistical processing is done == 3 (table 4.4:
Month)
50: Length of the time range over which statistical
processing is done, in units defined by the previous octet ==
1
54: Indicator of unit of time for the
increment between the successive fields used == 255 (table
4.4: Missing)
55: Time increment between successive
fields, in units defined by the previous octet == 0
59: As octets 47
to 58, next innermost step of processing == -9999
71: Additional time range specifications, included in
accordance with the value of n. Contents as octets 47 to 58,
repeated as necessary == -9999
Grib2SectionDataRepresentation
Template = 40 (Grid point data - JPEG 2000
code stream format)
NPoints = 72960
Grib2SectionData
Starting Pos = 196
Data Length = 64170
On 9/28/2012 11:45 AM, Steve Ansari wrote:
Hi Jack,
Thanks for bringing this up. It appears that the GRIB
decoders in the NetCDF for Java library are not detecting the
time dimension correctly. 20120927 should be the runtime and
201305 should be the time, but the runtime date is showing up
as the time dimension and the valid time of the model is not
present.
I'm cc-ing John Caron and Robb at Unidata who are the GRIB
masters at Unidata.
Thanks!
Steve
On Fri, Sep 28, 2012 at 1:03 PM, Jack
Settelmaier <address@hidden>
wrote:
I'm
poking around with displaying output from the climate
(CFS) model, and it's forecasts of mean monthly
conditions.
So, in the below files, one gets the CFS model output from
the Sep 26, 2012 18Z model run. The forecasts from that
model run are for the mean conditions over the months of
May and June, 2013, respectively.
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01/flxf.01.2012092700.201305.avrg.grib.grb2
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01/flxf.01.2012092700.201306.avrg.grib.grb2
When I Save a single KMZ after viewing each dataset, your
software apparently assigns a "time stamp" based on first
date in the filename, or from pulling the model initial
time, NOT the valid time as I'd like. So what I end up
with in these two files
http://www.srh.noaa.gov/rtimages/srh/stsd/Jack/CFS_500HAnom4.kmz
http://www.srh.noaa.gov/rtimages/srh/stsd/Jack/CFS_500HAnom5.kmz
are two images that have the same "time" in Google, even
though their valid time is a month apart. Maybe if I made
a loop over the 2 grib files it would have inserted a
"time step" that would allow for some sort of animation in
Google, but I'm still not sure it would help show me the
valid time.
This is pretty specific to the CFS files and their nuanced
forecasts and such, but is there something that could be
done to better the result?
|