[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDFJava #IUX-310433]: Aggregating with Groups in NCML
This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
- Subject: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Date: Mon, 20 Jun 2011 13:43:28 -0600
Hi Tom, HP:
I have two files, each with 3 records.
1) 1 record uses "space view perspective" (attached MSG1.png), 2 records use a
lat/lon GDS (attached MSG2.png). The first has an interesting banded pattern.
The second has -9999 as the first lat/lon point (see GDS dump below). Im
guessing that this might mean "calculate the domain using only the second
lat/lon point and nx/ny/dx/dy". But theres nothing in the GRIB spec that seems
to allow this. HP, is this the case? Is "use -9999 to mean calculate the grid
from the other info" part of GRIB standard or is it some local convention from
EUmetsat?
2) the 2 lat/lon have unknown parameters 3-1-98 and 3-1-99 (3=discipline,
1=category,98/99=parameter). These are not in the latest WMO parameter tables,
and they are not in the "local parameter" area of the table (>191). HP, can you
comment on what these are, and if EUM is using WMO parameter numbers for local
parameters ?
3) theres some interesting metadata in the local use section that we are not
picking up. Ive copied a complete dump of one of the records below.
4) because there are 2 GDS, there are 2 groups. the aggregation is failing with
groups. im not sure if i can fix this quickly, although im taking a stab at it.
4) TDS 4.3 has a complete refactoring of GRIB-2, with a new feature to specify
a collection of grib2 files, which are then correctly aggregated. So it works
correctly because NcML aggregation is not needed. I can give you an advance
copy, but this wont be released for a few months, and then theres the question
of IDV integration.
John
----
File=0
E:/work/whittaker/MSG2-SEVI-MSGMPEG-0100-0100-20090830163000.000000000Z-1002820.grb
Header="(0)(2)(1)(6)(1)(2)(0)(0)(234)u(10)(20)2(4)(5)(1)(0)!V(9)(13)(10)HeaderVersionNo
0(13)(10)FileType
2(13)(10)SubHeaderType 1(13)(10)SourceFacilityID
6(13)(10)SourceEnvId 1(13)(10)SourceInstanceId
2(13)(10)SourceSUId 60021(13)(10)SourceCPUId
10 20 50 4(13)(10)DestFaciliyId 5(13)(10)DestEnvId
1(13)(10)DataFieldLength
2184713(13)(10)(0)(171)(24)I(181)(3)(153)(10) (1)B(13)(10)SubHeaderVersionNo
0(13)(10)ServiceType 171(13)(10)ServiceSubType
24(13)(10)FileTime
18869:60361248(13)(10)SpacecraftId
322(13)(10)(0)(16)(0)(0)(1)(1)I(181)(3)(138)_@ddd(1)(0)(0)I(181)(3)(138)_@(1)(0)I(181)(3)(138)_@(0)(0)I(181)(3)(138)_@(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0
)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(0)(0)(0)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)"
Grib2IndicatorSection
Discipline = (3) Space products
Length = 2184244
Grib2IdentificationSection
Center = (254) EUMETSAT Operation Centre
SubCenter = (0) null
Master Table = 3
Local Table = 0
RefTimeSignif = 3 (Observation time)
RefTime = 2009-08-30T16:30:00.000Z
RefTime Fields = 2009-8-30 16:30:0
ProductionStatus = 1 (Operational test products)
TypeOfProcessedData = 6 (Processed satellite observations)
Grib2GridDefinitionSection hash=-1919541760 crc=3711080673
Length = 72
Source (3.0) = 0 (Specified in Code table 3.1)
Npts = 576
Template (3.1) = 0
(3.0) Grid definition template 3.0 - latitude/longitude (or equidistant
cylindrical, or Plate Carrée)
1:
GDS length == 72
5:
Section == 3
6: Source of Grid Definition (see
code table 3.0) == 0 (table 3.0: Specified in Code table 3.1)
7:
Number of data points == 576
11: Number of octects for optional
list of numbers == 0
12: Interpretation of
list of numbers == 0 (table 3.11: There is no appended list)
13: Grid Definition
Template Number == 0
15:
Shape of the Earth == 3 (table 3.2: Earth assumed oblate spheroid with major
and minor axes specified (in km) by data producer)
16: Scale factor of radius of
spherical Earth == 255
17: Scaled value of radius of
spherical Earth == -9999
21: Scale factor of major axis of
oblate spheroid Earth == 4
22: Scaled value of major axis of
oblate spheroid Earth == 63781400
26: Scale factor of minor axis of
oblate spheroid Earth == 4
27: Scaled value of minor axis of
oblate spheroid Earth == 63567550
31: Ni - number of points
along a parallel == 24
35: Nj - number of points
along a meridian == 24
39: Basic angle of the initial
production domain == 0
43: Subdivisions of basic angle used to define extreme longitudes and
latitudes, and direction increments == -9999
47: La1 - latitude of
first grid point == -9999
51: Lo1 - longitude of
first grid point == -9999
55: Resolution and
component flags == 56
56: La2 - latitude of
last grid point == 60000000
60: Lo2 - longitude of
last grid point == 60000000
64: Di - i
direction increment == 5000000
68: Dj - j
direction increment == 5000000
72:
Scanning mode == 64
73: List of number of points along each
meridian or parallel. == -9999
Grib2ProductDefinitionSection
(4.30) Product definition template 4.30 - satellite product
1:
PDS length == 24
5:
Section == 4
6: Number of coordinates
values after Template == 0
8: Product Definition
Template Number == 30
10:
Parameter category == 1
11:
Parameter number == 98
12: Type of
generating process == 8 (table 4.3: Observation)
13: Observation generating process identifier (defined by
originating centre) == 7
14: Number of contributing
spectral bands (NB) == 1
Grib2SectionDataRepresentation
Template = 0 (Grid point data - simple packing)
NPoints = 576
Grib2SectionData
Starting Pos = 2183854
Data Length = 581
> Hi John....
>
> Did the data files sample get through okay? If not, I have also posted this
> at:
> <ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/hpgrids.zip>
>
> Thanks again for looking into this...
>
> tom
>
> On Mon, Jun 13, 2011 at 2:32 PM, Tom Whittaker <address@hidden> wrote:
> > Hi John...
> >
> > Thanks....I had simply mis-typed it (guess I should use
> > cut-and-paste)....but since it does not work for me, I'm attaching a
> > zip file with the 2 nc files and the attempt at ncml...
> >
> > Thanks ahead for your help!!
> >
> > tom
> >
> > On Fri, Jun 10, 2011 at 4:11 PM, Unidata netCDF Java Support
> > <address@hidden> wrote:
> >> hi tom:
> >>
> >> jointExisting should be joinExisting.
> >>
> >> if thats not the actual problem, can you send me 2 files to check?
> >>
> >> john
> >>
> >>> I have several files that I want to aggregate over time. Each file
> >>> has two Groups, with a dimension "time" which is used for the desired
> >>> variable...for example:
> >>>
> >>> Group proj1 {
> >>> dimensions:
> >>> time = 1;
> >>> y = 3712;
> >>> x = 3712;
> >>> variables:
> >>> float Instantaneous_Rain_Rate(time=1, y=3712, x=3712);
> >>> ...
> >>>
> >>> I am trying to use NCML to accomplish this, with an "aggregation" element
> >>> like:
> >>>
> >>> <aggregation dimName="time" type="jointExisting">
> >>> ...
> >>>
> >>> But apparently "time" is not the right syntax for this. I've tried
> >>> "proj1/time" as well, but to no available. I'm hoping you can help.
> >>>
> >>> Thanks!
> >>>
> >>> tom
> >>>
> >>> --
> >>> Tom Whittaker
> >>> University of Wisconsin-Madison
> >>> Space Science & Engineering Center (SSEC)
> >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >>> 1225 W. Dayton Street
> >>> Madison, WI 53706 USA
> >>> ph: +1 608 262 2759
> >>>
> >>>
> >>
> >>
> >> Ticket Details
> >> ===================
> >> Ticket ID: IUX-310433
> >> Department: Support netCDF Java
> >> Priority: Normal
> >> Status: Open
> >>
> >>
> >
> >
> >
> > --
> > Tom Whittaker
> > University of Wisconsin-Madison
> > Space Science & Engineering Center (SSEC)
> > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > 1225 W. Dayton Street
> > Madison, WI 53706 USA
> > ph: +1 608 262 2759
> >
>
>
>
> --
> Tom Whittaker
> University of Wisconsin-Madison
> Space Science & Engineering Center (SSEC)
> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> 1225 W. Dayton Street
> Madison, WI 53706 USA
> ph: +1 608 262 2759
>
>
Ticket Details
===================
Ticket ID: IUX-310433
Department: Support netCDF Java
Priority: Normal
Status: Open
- Prev by Date: [netCDFJava #JNI-586917]: IllegalArgumentException (variable name must be unique) reading grib2
- Next by Date: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Previous by thread: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Next by thread: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Index(es):