Hi all:
1. The CFSR dataset is known to have grib encoding
issues, so its
possible we are seeing some of that and/or bugs in the
software. in
particular, the translation of grib1 to grib2 had bugs
in it. ive been
unable to get authoritative info on how to fix this, but
i do have
machinery to do so if we can figure out what to do.
2. The 4.3 software is completely different from 4.2. We
are only
going to fix 4.3 grib problems. 4.2 doesnt handle time
interval
coordinates correctly.
3. In this file:
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01/flxf.01.2012092700.201305.avrg.grib.grb2
I am seeing records like the one I have dumped below.
statistic type is "Average"
reference time is 2012-09-27T00:00:00.000Z
the "end of overall time interval" is == 2012/09/27T0:0:0
forecast time is 8 months.
we are interpreting the time interval coordinate as:
(0.000000, 1.000000) == (2012-08-27T00:00:00.000Z,
2012-09-27T00:00:00.000Z)
which seems wrong. ill have to look at this closer.
4. Note that this is an interval coordinate, meaning the
"valid
time"is a time interval. From this comment:
"The forecasts from that model run are for the mean
conditions over
the months of May 2013"
Im guessing the correct interval should be:
(0.000000, 1.000000) == (2013-05-01T00:00:00.000Z,
2013-06-01T00:00:00.000Z)
so:
- we do have a bug
- the dates are encoded wrong
- have to figure out how to fix systematically. if
theres a way to
derive the correct answer from the info in the PDS
below, can you see
it? It will be unfortunate if we have to rely on the
filename.
---------
File=0
E:/data/work/ansari/flxf.01.2012092700.201305.avrg.grib.grb2
Header=""
Grib2IndicatorSection
Discipline = (0) Meteorological products
Length = 64370
Grib2IdentificationSection
Center = (7) US National Weather Service,
National Centres for
Environmental Prediction (NCEP)
SubCenter = (0) null
Master Table = 2
Local Table = 1
RefTimeSignif = 1 (Start of forecast)
RefTime = 2012-09-27T00:00:00.000Z
RefTime Fields = 2012-9-27 0:0:0
ProductionStatus = 0 (Operational products)
TypeOfProcessedData = 1 (Forecast products)
Grib2GridDefinitionSection hash=-1824839391 crc=4229097007
Length = 72
Source (3.0) = 0 (Specified in Code table 3.1)
Npts = 72960
Template (3.1) = 40
(3.40) Grid definition template 3.40 - Gaussian
latitude/longitude
1: GDS length == 72
5: Section == 3
6: Source
of Grid
Definition (see code table 3.0) == 0 (table 3.0:
Specified in Code
table 3.1)
7: Number of data points == 72960
11: Number
of octects for
optional list of numbers == 0
12: Interpretation of list of numbers == 0 (table 3.11:
There is no
appended list)
13: Grid Definition Template Number == 40
15: Shape of the Earth == 6 (table 3.2: Earth assumed
spherical with
radius of 6 371 229.0 m)
16: Scale factor of radius of spherical Earth == 0
17: Scaled value of radius of spherical Earth == 0
21: Scale factor
of major axis
of oblate spheroid Earth == 0
22: Scaled value
of major axis
of oblate spheroid Earth == 0
26: Scale factor
of minor axis
of oblate spheroid Earth == 0
27: Scaled value
of minor axis
of oblate spheroid Earth == 0
31: Ni - number of points along a parallel == 384
35: Nj - number of points along a meridian == 190
39: Basic angle of the initial production domain == 0
43: Subdivisions of basic angle used to define extreme
longitudes and
latitudes, and direction increments == 0
47: La1 - latitude of first grid point == 89277000
51: Lo1 - longitude of first grid point == 0
55: Resolution and component flags == 48
56: La2 - latitude of last grid point == -89277000
60: Lo2 - longitude of last grid point == 359062000
64: Di - i direction increment == 938000
68: N - number of
parallels
between a pole and the Equator == 95
72: Scanning mode == 0
73: List of number of
points along
each meridian or parallel. == -9999
Grib2ProductDefinitionSection
Interval =
(2012-08-27T00:00:00.000Z,2012-09-27T00:00:00.000Z)
(4.8) Product definition template 4.8 - average,
accumulation and/or
extreme values or other statistically-processed values
at a horizontal
level or in a horizontal layer in a continuous or
non-continuous time
interval
1: PDS length == 58
5: Section == 4
6: Number of coordinates values after Template == 0
8: Product Definition Template Number == 8
10: Parameter category == 2
11: Parameter number == 17
12: Type of generating process == 2 (table 4.3: Forecast)
13: Background generating process
identifier
(defined by originating centre) == 0 (table ProcessId:
Table ProcessId
code 0 not found)
14: Analysis or forecast generating process
identifier
(defined by originating centre) == 98 (table ProcessId:
Table
ProcessId code 98 not found)
15: Hours after reference time of data cutoff == 0
17: Minutes after reference time of data cutoff == 0
18: Indicator of unit of time range == 3 (table 4.4:
Month)
19: Forecast time in units defined by octet 18 == 8
23: Type of first fixed surface == 1 (table 4.5: Ground
or water
surface)
24: Scale factor of first fixed surface == 0
25: Scaled value of first fixed surface == 0
29: Type of second fixed surface == 255 (table 4.5:
Missing)
30: Scale factor of second fixed surface == 0
31: Scaled value of second fixed surface == 0
35: Year - time of end of overall time interval == 2012
37: Month - time of end of overall time interval == 9
38: Day - time of end of overall time interval == 27
39: Hour - time of end of overall time interval == 0
40: Minute - time of end of overall time interval == 0
41: Second - time of end of overall time interval == 0
42: n - number of time range specifications describing
the time
intervals used to calculate the statistically-processed
field == 1
43: Total number of
data values
missing in statistical process == 0
47: Statistical process used to calculate the processed
field from
the field at each time increment during the time range
== 0 (table
4.10: Average)
48: Type of time increment between successive
fields used in
the statistical processing == 2 (table 4.11: Successive
times
processed have same start time of forecast, forecast
time is incremented)
49: Indicator of unit of time for time range
over which
statistical processing is done == 3 (table 4.4: Month)
50: Length of the time range over which statistical
processing is
done, in units defined by the previous octet == 1
54: Indicator of unit of time for the
increment between
the successive fields used == 255 (table 4.4: Missing)
55: Time increment between successive fields,
in units
defined by the previous octet == 0
59: As octets 47
to 58, next
innermost step of processing == -9999
71: Additional time range specifications, included in
accordance with
the value of n. Contents as octets 47 to 58, repeated as
necessary ==
-9999
Grib2SectionDataRepresentation
Template = 40 (Grid point data - JPEG 2000
code stream
format)
NPoints = 72960
Grib2SectionData
Starting Pos = 196
Data Length = 64170
On 9/28/2012 11:45 AM, Steve Ansari wrote:
Hi Jack,
Thanks for bringing this up. It appears that the GRIB
decoders in
the NetCDF for Java library are not detecting the time
dimension
correctly. 20120927 should be the runtime and 201305
should be the
time, but the runtime date is showing up as the time
dimension and
the valid time of the model is not present.
I'm cc-ing John Caron and Robb at Unidata who are the
GRIB masters at
Unidata.
I'm using NCJ 4.2 in the Weather and Climate Toolkit,
but I tested
several files with NCJ 4.3 and got the same results.
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01
John, Robb, thanks for your help!
Thanks!
Steve
On Fri, Sep 28, 2012 at 1:03 PM, Jack Settelmaier
<address@hidden
<mailto:address@hidden>> wrote:
I'm poking around with displaying output from the
climate (CFS)
model, and it's forecasts of mean monthly conditions.
So, in the below files, one gets the CFS model
output from the
Sep 26, 2012 18Z model run. The forecasts from
that model run
are for the mean conditions over the months of May
and June,
2013, respectively.
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01/flxf.01.2012092700.201305.avrg.grib.grb2
ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120927/00/monthly_grib_01/flxf.01.2012092700.201306.avrg.grib.grb2
When I Save a single KMZ after viewing each
dataset, your
software apparently assigns a "time stamp" based on
first date in
the filename, or from pulling the model initial
time, NOT the
valid time as I'd like. So what I end up with in
these two files
http://www.srh.noaa.gov/rtimages/srh/stsd/Jack/CFS_500HAnom4.kmz
http://www.srh.noaa.gov/rtimages/srh/stsd/Jack/CFS_500HAnom5.kmz
are two images that have the same "time" in Google,
even though
their valid time is a month apart. Maybe if I made
a loop over
the 2 grib files it would have inserted a "time
step" that would
allow for some sort of animation in Google, but I'm
still not
sure it would help show me the valid time.
This is pretty specific to the CFS files and their
nuanced
forecasts and such, but is there something that
could be done to
better the result?