[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Subject: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Date: Tue, 21 Jun 2011 10:32:51 -0600
If he extracted the records he wants and put them in a single file, that would
work.
> Hi John...
>
> Many thanks for looking into this. I'm not sure about the projection
> issues, since HP was only working with the
> "proj1/Instantaneous_Rain_Rate" and this display just fine in the IDV.
> His problem is he has 96 files and wants to show
> them in a loop (without having to select each of them....).
>
> So, I must conclude from your comments that the 2 GDS's are the real
> issue with the aggregation, and because of that he is out-of-luck for
> now, unless your "stab" works out...? I'll keep my fingers crossed!
>
> Thank you again...let us know if you have any success!
>
> Best wishes,
>
> tom
>
> On Mon, Jun 20, 2011 at 2:43 PM, Unidata netCDF Java Support
> <address@hidden> wrote:
> > Hi Tom, HP:
> >
> > I have two files, each with 3 records.
> >
> > 1) 1 record uses "space view perspective" (attached MSG1.png), 2 records
> > use a lat/lon GDS (attached MSG2.png). The first has an interesting banded
> > pattern. The second has -9999 as the first lat/lon point (see GDS dump
> > below). Im guessing that this might mean "calculate the domain using only
> > the second lat/lon point and nx/ny/dx/dy". But theres nothing in the GRIB
> > spec that seems to allow this. HP, is this the case? Is "use -9999 to mean
> > calculate the grid from the other info" part of GRIB standard or is it some
> > local convention from EUmetsat?
> >
> > 2) the 2 lat/lon have unknown parameters 3-1-98 and 3-1-99 (3=discipline,
> > 1=category,98/99=parameter). These are not in the latest WMO parameter
> > tables, and they are not in the "local parameter" area of the table (>191).
> > HP, can you comment on what these are, and if EUM is using WMO parameter
> > numbers for local parameters ?
> >
> > 3) theres some interesting metadata in the local use section that we are
> > not picking up. Ive copied a complete dump of one of the records below.
> >
> > 4) because there are 2 GDS, there are 2 groups. the aggregation is failing
> > with groups. im not sure if i can fix this quickly, although im taking a
> > stab at it.
> >
> > 4) TDS 4.3 has a complete refactoring of GRIB-2, with a new feature to
> > specify a collection of grib2 files, which are then correctly aggregated.
> > So it works correctly because NcML aggregation is not needed. I can give
> > you an advance copy, but this wont be released for a few months, and then
> > theres the question of IDV integration.
> >
> > John
> >
> > ----
> >
> > File=0
> > E:/work/whittaker/MSG2-SEVI-MSGMPEG-0100-0100-20090830163000.000000000Z-1002820.grb
> > Header="(0)(2)(1)(6)(1)(2)(0)(0)(234)u(10)(20)2(4)(5)(1)(0)!V(9)(13)(10)HeaderVersionNo
> > 0(13)(10)FileType
> > 2(13)(10)SubHeaderType 1(13)(10)SourceFacilityID
> > 6(13)(10)SourceEnvId
> > 1(13)(10)SourceInstanceId 2(13)(10)SourceSUId
> > 60021(13)(10)SourceCPUId 10 20 50 4(13)(10)DestFaciliyId
> > 5(13)(10)DestEnvId
> > 1(13)(10)DataFieldLength
> > 2184713(13)(10)(0)(171)(24)I(181)(3)(153)(10)
> > (1)B(13)(10)SubHeaderVersionNo 0(13)(10)ServiceType
> > 171(13)(10)ServiceSubType 24(13)(10)FileTime
> > 18869:60361248(13)(10)SpacecraftId
> > 322(13)(10)(0)(16)(0)(0)(1)(1)I(181)(3)(138)
_@ddd(1)(0)(0)I(181)(3)(138)_@(1)(0)I(181)(3)(138)_@(0)(0)I(181)(3)(138)_@(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0
> >
> > )(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(0)(0)(0)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)"
> >
> > Grib2IndicatorSection
> > Discipline = (3) Space products
> > Length = 2184244
> >
> > Grib2IdentificationSection
> > Center = (254) EUMETSAT Operation Centre
> > SubCenter = (0) null
> > Master Table = 3
> > Local Table = 0
> > RefTimeSignif = 3 (Observation time)
> > RefTime = 2009-08-30T16:30:00.000Z
> > RefTime Fields = 2009-8-30 16:30:0
> > ProductionStatus = 1 (Operational test products)
> > TypeOfProcessedData = 6 (Processed satellite observations)
> >
> > Grib2GridDefinitionSection hash=-1919541760 crc=3711080673
> > Length = 72
> > Source (3.0) = 0 (Specified in Code table 3.1)
> > Npts = 576
> > Template (3.1) = 0
> >
> > (3.0) Grid definition template 3.0 - latitude/longitude (or equidistant
> > cylindrical, or Plate Carrée)
> > 1:
> > GDS length == 72
> > 5:
> > Section == 3
> > 6: Source of Grid Definition
> > (see code table 3.0) == 0 (table 3.0: Specified in Code table 3.1)
> > 7:
> > Number of data points == 576
> > 11: Number of octects for
> > optional list of numbers == 0
> > 12:
> > Interpretation of list of numbers == 0 (table 3.11: There is no appended
> > list)
> > 13: Grid
> > Definition Template Number == 0
> > 15:
> > Shape of the Earth == 3 (table 3.2: Earth assumed oblate spheroid with
> > major and minor axes specified (in km) by data producer)
> > 16: Scale factor of
> > radius of spherical Earth == 255
> > 17: Scaled value of
> > radius of spherical Earth == -9999
> > 21: Scale factor of major axis of
> > oblate spheroid Earth == 4
> > 22: Scaled value of major axis of
> > oblate spheroid Earth == 63781400
> > 26: Scale factor of minor axis of
> > oblate spheroid Earth == 4
> > 27: Scaled value of minor axis of
> > oblate spheroid Earth == 63567550
> > 31: Ni - number of
> > points along a parallel == 24
> > 35: Nj - number of
> > points along a meridian == 24
> > 39: Basic angle of the
> > initial production domain == 0
> > 43: Subdivisions of basic angle used to define extreme longitudes and
> > latitudes, and direction increments == -9999
> > 47: La1 - latitude
> > of first grid point == -9999
> > 51: Lo1 - longitude
> > of first grid point == -9999
> > 55: Resolution
> > and component flags == 56
> > 56: La2 -
> > latitude of last grid point == 60000000
> > 60: Lo2 -
> > longitude of last grid point == 60000000
> > 64: Di - i
> > direction increment == 5000000
> > 68: Dj - j
> > direction increment == 5000000
> > 72:
> > Scanning mode == 64
> > 73: List of number of points along each
> > meridian or parallel. == -9999
> >
> > Grib2ProductDefinitionSection
> >
> > (4.30) Product definition template 4.30 - satellite product
> > 1:
> > PDS length == 24
> > 5:
> > Section == 4
> > 6: Number of coordinates
> > values after Template == 0
> > 8: Product
> > Definition Template Number == 30
> > 10:
> > Parameter category == 1
> > 11:
> > Parameter number == 98
> > 12: Type
> > of generating process == 8 (table 4.3: Observation)
> > 13: Observation generating process identifier (defined by
> > originating centre) == 7
> > 14: Number of contributing
> > spectral bands (NB) == 1
> >
> > Grib2SectionDataRepresentation
> > Template = 0 (Grid point data - simple packing)
> > NPoints = 576
> >
> > Grib2SectionData
> > Starting Pos = 2183854
> > Data Length = 581
> >
> >
> >> Hi John....
> >>
> >> Did the data files sample get through okay? If not, I have also posted
> >> this at:
> >> <ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/hpgrids.zip>
> >>
> >> Thanks again for looking into this...
> >>
> >> tom
> >>
> >> On Mon, Jun 13, 2011 at 2:32 PM, Tom Whittaker <address@hidden> wrote:
> >> > Hi John...
> >> >
> >> > Thanks....I had simply mis-typed it (guess I should use
> >> > cut-and-paste)....but since it does not work for me, I'm attaching a
> >> > zip file with the 2 nc files and the attempt at ncml...
> >> >
> >> > Thanks ahead for your help!!
> >> >
> >> > tom
> >> >
> >> > On Fri, Jun 10, 2011 at 4:11 PM, Unidata netCDF Java Support
> >> > <address@hidden> wrote:
> >> >> hi tom:
> >> >>
> >> >> jointExisting should be joinExisting.
> >> >>
> >> >> if thats not the actual problem, can you send me 2 files to check?
> >> >>
> >> >> john
> >> >>
> >> >>> I have several files that I want to aggregate over time. Each file
> >> >>> has two Groups, with a dimension "time" which is used for the desired
> >> >>> variable...for example:
> >> >>>
> >> >>> Group proj1 {
> >> >>> dimensions:
> >> >>> time = 1;
> >> >>> y = 3712;
> >> >>> x = 3712;
> >> >>> variables:
> >> >>> float Instantaneous_Rain_Rate(time=1, y=3712, x=3712);
> >> >>> ...
> >> >>>
> >> >>> I am trying to use NCML to accomplish this, with an "aggregation"
> >> >>> element like:
> >> >>>
> >> >>> <aggregation dimName="time" type="jointExisting">
> >> >>> ...
> >> >>>
> >> >>> But apparently "time" is not the right syntax for this. I've tried
> >> >>> "proj1/time" as well, but to no available. I'm hoping you can help.
> >> >>>
> >> >>> Thanks!
> >> >>>
> >> >>> tom
> >> >>>
> >> >>> --
> >> >>> Tom Whittaker
> >> >>> University of Wisconsin-Madison
> >> >>> Space Science & Engineering Center (SSEC)
> >> >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> >>> 1225 W. Dayton Street
> >> >>> Madison, WI 53706 USA
> >> >>> ph: +1 608 262 2759
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >> Ticket Details
> >> >> ===================
> >> >> Ticket ID: IUX-310433
> >> >> Department: Support netCDF Java
> >> >> Priority: Normal
> >> >> Status: Open
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Tom Whittaker
> >> > University of Wisconsin-Madison
> >> > Space Science & Engineering Center (SSEC)
> >> > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> > 1225 W. Dayton Street
> >> > Madison, WI 53706 USA
> >> > ph: +1 608 262 2759
> >> >
> >>
> >>
> >>
> >> --
> >> Tom Whittaker
> >> University of Wisconsin-Madison
> >> Space Science & Engineering Center (SSEC)
> >> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> 1225 W. Dayton Street
> >> Madison, WI 53706 USA
> >> ph: +1 608 262 2759
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: IUX-310433
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> >
> >
>
>
>
> --
> Tom Whittaker
> University of Wisconsin-Madison
> Space Science & Engineering Center (SSEC)
> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> 1225 W. Dayton Street
> Madison, WI 53706 USA
> ph: +1 608 262 2759
>
>
Ticket Details
===================
Ticket ID: IUX-310433
Department: Support netCDF Java
Priority: Normal
Status: Closed