[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDFJava #IUX-310433]: Aggregating with Groups in NCML
This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
- Subject: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Date: Tue, 21 Jun 2011 10:32:51 -0600
If he extracted the records he wants and put them in a single file, that would work. > Hi John... > > Many thanks for looking into this. I'm not sure about the projection > issues, since HP was only working with the > "proj1/Instantaneous_Rain_Rate" and this display just fine in the IDV. > His problem is he has 96 files and wants to show > them in a loop (without having to select each of them....). > > So, I must conclude from your comments that the 2 GDS's are the real > issue with the aggregation, and because of that he is out-of-luck for > now, unless your "stab" works out...? I'll keep my fingers crossed! > > Thank you again...let us know if you have any success! > > Best wishes, > > tom > > On Mon, Jun 20, 2011 at 2:43 PM, Unidata netCDF Java Support > <address@hidden> wrote: > > Hi Tom, HP: > > > > I have two files, each with 3 records. > > > > 1) 1 record uses "space view perspective" (attached MSG1.png), 2 records > > use a lat/lon GDS (attached MSG2.png). The first has an interesting banded > > pattern. The second has -9999 as the first lat/lon point (see GDS dump > > below). Im guessing that this might mean "calculate the domain using only > > the second lat/lon point and nx/ny/dx/dy". But theres nothing in the GRIB > > spec that seems to allow this. HP, is this the case? Is "use -9999 to mean > > calculate the grid from the other info" part of GRIB standard or is it some > > local convention from EUmetsat? > > > > 2) the 2 lat/lon have unknown parameters 3-1-98 and 3-1-99 (3=discipline, > > 1=category,98/99=parameter). These are not in the latest WMO parameter > > tables, and they are not in the "local parameter" area of the table (>191). > > HP, can you comment on what these are, and if EUM is using WMO parameter > > numbers for local parameters ? > > > > 3) theres some interesting metadata in the local use section that we are > > not picking up. Ive copied a complete dump of one of the records below. > > > > 4) because there are 2 GDS, there are 2 groups. the aggregation is failing > > with groups. im not sure if i can fix this quickly, although im taking a > > stab at it. > > > > 4) TDS 4.3 has a complete refactoring of GRIB-2, with a new feature to > > specify a collection of grib2 files, which are then correctly aggregated. > > So it works correctly because NcML aggregation is not needed. I can give > > you an advance copy, but this wont be released for a few months, and then > > theres the question of IDV integration. > > > > John > > > > ---- > > > > File=0 > > E:/work/whittaker/MSG2-SEVI-MSGMPEG-0100-0100-20090830163000.000000000Z-1002820.grb > > Header="(0)(2)(1)(6)(1)(2)(0)(0)(234)u(10)(20)2(4)(5)(1)(0)!V(9)(13)(10)HeaderVersionNo > > 0(13)(10)FileType > > 2(13)(10)SubHeaderType 1(13)(10)SourceFacilityID > > 6(13)(10)SourceEnvId > > 1(13)(10)SourceInstanceId 2(13)(10)SourceSUId > > 60021(13)(10)SourceCPUId 10 20 50 4(13)(10)DestFaciliyId > > 5(13)(10)DestEnvId > > 1(13)(10)DataFieldLength > > 2184713(13)(10)(0)(171)(24)I(181)(3)(153)(10) > > (1)B(13)(10)SubHeaderVersionNo 0(13)(10)ServiceType > > 171(13)(10)ServiceSubType 24(13)(10)FileTime > > 18869:60361248(13)(10)SpacecraftId > > 322(13)(10)(0)(16)(0)(0)(1)(1)I(181)(3)(138) _@ddd(1)(0)(0)I(181)(3)(138)_@(1)(0)I(181)(3)(138)_@(0)(0)I(181)(3)(138)_@(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0 > > > > )(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(0)(0)(0)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)" > > > > Grib2IndicatorSection > > Discipline = (3) Space products > > Length = 2184244 > > > > Grib2IdentificationSection > > Center = (254) EUMETSAT Operation Centre > > SubCenter = (0) null > > Master Table = 3 > > Local Table = 0 > > RefTimeSignif = 3 (Observation time) > > RefTime = 2009-08-30T16:30:00.000Z > > RefTime Fields = 2009-8-30 16:30:0 > > ProductionStatus = 1 (Operational test products) > > TypeOfProcessedData = 6 (Processed satellite observations) > > > > Grib2GridDefinitionSection hash=-1919541760 crc=3711080673 > > Length = 72 > > Source (3.0) = 0 (Specified in Code table 3.1) > > Npts = 576 > > Template (3.1) = 0 > > > > (3.0) Grid definition template 3.0 - latitude/longitude (or equidistant > > cylindrical, or Plate Carrée) > > 1: > > GDS length == 72 > > 5: > > Section == 3 > > 6: Source of Grid Definition > > (see code table 3.0) == 0 (table 3.0: Specified in Code table 3.1) > > 7: > > Number of data points == 576 > > 11: Number of octects for > > optional list of numbers == 0 > > 12: > > Interpretation of list of numbers == 0 (table 3.11: There is no appended > > list) > > 13: Grid > > Definition Template Number == 0 > > 15: > > Shape of the Earth == 3 (table 3.2: Earth assumed oblate spheroid with > > major and minor axes specified (in km) by data producer) > > 16: Scale factor of > > radius of spherical Earth == 255 > > 17: Scaled value of > > radius of spherical Earth == -9999 > > 21: Scale factor of major axis of > > oblate spheroid Earth == 4 > > 22: Scaled value of major axis of > > oblate spheroid Earth == 63781400 > > 26: Scale factor of minor axis of > > oblate spheroid Earth == 4 > > 27: Scaled value of minor axis of > > oblate spheroid Earth == 63567550 > > 31: Ni - number of > > points along a parallel == 24 > > 35: Nj - number of > > points along a meridian == 24 > > 39: Basic angle of the > > initial production domain == 0 > > 43: Subdivisions of basic angle used to define extreme longitudes and > > latitudes, and direction increments == -9999 > > 47: La1 - latitude > > of first grid point == -9999 > > 51: Lo1 - longitude > > of first grid point == -9999 > > 55: Resolution > > and component flags == 56 > > 56: La2 - > > latitude of last grid point == 60000000 > > 60: Lo2 - > > longitude of last grid point == 60000000 > > 64: Di - i > > direction increment == 5000000 > > 68: Dj - j > > direction increment == 5000000 > > 72: > > Scanning mode == 64 > > 73: List of number of points along each > > meridian or parallel. == -9999 > > > > Grib2ProductDefinitionSection > > > > (4.30) Product definition template 4.30 - satellite product > > 1: > > PDS length == 24 > > 5: > > Section == 4 > > 6: Number of coordinates > > values after Template == 0 > > 8: Product > > Definition Template Number == 30 > > 10: > > Parameter category == 1 > > 11: > > Parameter number == 98 > > 12: Type > > of generating process == 8 (table 4.3: Observation) > > 13: Observation generating process identifier (defined by > > originating centre) == 7 > > 14: Number of contributing > > spectral bands (NB) == 1 > > > > Grib2SectionDataRepresentation > > Template = 0 (Grid point data - simple packing) > > NPoints = 576 > > > > Grib2SectionData > > Starting Pos = 2183854 > > Data Length = 581 > > > > > >> Hi John.... > >> > >> Did the data files sample get through okay? If not, I have also posted > >> this at: > >> <ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/hpgrids.zip> > >> > >> Thanks again for looking into this... > >> > >> tom > >> > >> On Mon, Jun 13, 2011 at 2:32 PM, Tom Whittaker <address@hidden> wrote: > >> > Hi John... > >> > > >> > Thanks....I had simply mis-typed it (guess I should use > >> > cut-and-paste)....but since it does not work for me, I'm attaching a > >> > zip file with the 2 nc files and the attempt at ncml... > >> > > >> > Thanks ahead for your help!! > >> > > >> > tom > >> > > >> > On Fri, Jun 10, 2011 at 4:11 PM, Unidata netCDF Java Support > >> > <address@hidden> wrote: > >> >> hi tom: > >> >> > >> >> jointExisting should be joinExisting. > >> >> > >> >> if thats not the actual problem, can you send me 2 files to check? > >> >> > >> >> john > >> >> > >> >>> I have several files that I want to aggregate over time. Each file > >> >>> has two Groups, with a dimension "time" which is used for the desired > >> >>> variable...for example: > >> >>> > >> >>> Group proj1 { > >> >>> dimensions: > >> >>> time = 1; > >> >>> y = 3712; > >> >>> x = 3712; > >> >>> variables: > >> >>> float Instantaneous_Rain_Rate(time=1, y=3712, x=3712); > >> >>> ... > >> >>> > >> >>> I am trying to use NCML to accomplish this, with an "aggregation" > >> >>> element like: > >> >>> > >> >>> <aggregation dimName="time" type="jointExisting"> > >> >>> ... > >> >>> > >> >>> But apparently "time" is not the right syntax for this. I've tried > >> >>> "proj1/time" as well, but to no available. I'm hoping you can help. > >> >>> > >> >>> Thanks! > >> >>> > >> >>> tom > >> >>> > >> >>> -- > >> >>> Tom Whittaker > >> >>> University of Wisconsin-Madison > >> >>> Space Science & Engineering Center (SSEC) > >> >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS) > >> >>> 1225 W. Dayton Street > >> >>> Madison, WI 53706 USA > >> >>> ph: +1 608 262 2759 > >> >>> > >> >>> > >> >> > >> >> > >> >> Ticket Details > >> >> =================== > >> >> Ticket ID: IUX-310433 > >> >> Department: Support netCDF Java > >> >> Priority: Normal > >> >> Status: Open > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Tom Whittaker > >> > University of Wisconsin-Madison > >> > Space Science & Engineering Center (SSEC) > >> > Cooperative Institute for Meteorological Satellite Studies (CIMSS) > >> > 1225 W. Dayton Street > >> > Madison, WI 53706 USA > >> > ph: +1 608 262 2759 > >> > > >> > >> > >> > >> -- > >> Tom Whittaker > >> University of Wisconsin-Madison > >> Space Science & Engineering Center (SSEC) > >> Cooperative Institute for Meteorological Satellite Studies (CIMSS) > >> 1225 W. Dayton Street > >> Madison, WI 53706 USA > >> ph: +1 608 262 2759 > >> > >> > > > > > > Ticket Details > > =================== > > Ticket ID: IUX-310433 > > Department: Support netCDF Java > > Priority: Normal > > Status: Open > > > > > > > > -- > Tom Whittaker > University of Wisconsin-Madison > Space Science & Engineering Center (SSEC) > Cooperative Institute for Meteorological Satellite Studies (CIMSS) > 1225 W. Dayton Street > Madison, WI 53706 USA > ph: +1 608 262 2759 > > Ticket Details =================== Ticket ID: IUX-310433 Department: Support netCDF Java Priority: Normal Status: Closed
- Prev by Date: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Next by Date: [netCDFJava #JNI-586917]: IllegalArgumentException (variable name must be unique) reading grib2
- Previous by thread: [netCDFJava #IUX-310433]: Aggregating with Groups in NCML
- Next by thread: [netCDFJava #JNI-586917]: IllegalArgumentException (variable name must be unique) reading grib2
- Index(es):