[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #IUX-310433]: Aggregating with Groups in NCML



forgot attachments

> Hi Tom, HP:
> 
> I have two files, each with 3 records.
> 
> 1) 1 record uses "space view perspective" (attached MSG1.png), 2 records use 
> a lat/lon GDS (attached MSG2.png). The first has an interesting banded 
> pattern. The second has -9999 as the first lat/lon point (see GDS dump 
> below). Im guessing that this might mean "calculate the domain using only the 
> second lat/lon point and nx/ny/dx/dy". But theres nothing in the GRIB spec 
> that seems to allow this. HP, is this the case? Is "use -9999 to mean 
> calculate the grid from the other info" part of GRIB standard or is it some 
> local convention from EUmetsat?
> 
> 2) the 2 lat/lon have unknown parameters 3-1-98 and 3-1-99 (3=discipline, 
> 1=category,98/99=parameter). These are not in the latest WMO parameter 
> tables, and they are not in the "local parameter" area of the table (>191). 
> HP, can you comment on what these are, and if EUM is using WMO parameter 
> numbers for local parameters ?
> 
> 3) theres some interesting metadata in the local use section that we are not 
> picking up. Ive copied a complete dump of one of the records below.
> 
> 4) because there are 2 GDS, there are 2 groups. the aggregation is failing 
> with groups. im not sure if i can fix this quickly, although im taking a stab 
> at it.
> 
> 4) TDS 4.3 has a complete refactoring of GRIB-2, with a new feature to 
> specify a collection of grib2 files, which are then correctly aggregated. So 
> it works correctly because NcML aggregation is not needed. I can give you an 
> advance copy, but this wont be released for a few months, and then theres the 
> question of IDV integration.
> 
> John
> 
> ----
> 
> File=0 
> E:/work/whittaker/MSG2-SEVI-MSGMPEG-0100-0100-20090830163000.000000000Z-1002820.grb
> Header="(0)(2)(1)(6)(1)(2)(0)(0)(234)u(10)(20)2(4)(5)(1)(0)!V(9)(13)(10)HeaderVersionNo
>                    0(13)(10)FileType                          
> 2(13)(10)SubHeaderType                     1(13)(10)SourceFacilityID          
>         6(13)(10)SourceEnvId                       1(13)(10)SourceInstanceId  
>                 2(13)(10)SourceSUId                    
> 60021(13)(10)SourceCPUId          10  20  50   4(13)(10)DestFaciliyId         
>             5(13)(10)DestEnvId                         
> 1(13)(10)DataFieldLength             
> 2184713(13)(10)(0)(171)(24)I(181)(3)(153)(10) (1)B(13)(10)SubHeaderVersionNo  
>               0(13)(10)ServiceType                     
> 171(13)(10)ServiceSubType                   24(13)(10)FileTime             
> 18869:60361248(13)(10)SpacecraftId                    
> 322(13)(10)(0)(16)(0)(0)(1)(1)I(181)(3)(138)_@ddd(1)(0)(0)I(181)(3)(138)_@(1)(0)I(181)(3)(138)_@(0)(0)I(181)(3)(138)_@(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)
 
(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(0)(0)(0)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)"
> 
> Grib2IndicatorSection
> Discipline = (3) Space products
> Length     = 2184244
> 
> Grib2IdentificationSection
> Center        = (254) EUMETSAT Operation Centre
> SubCenter     = (0) null
> Master Table  = 3
> Local Table   = 0
> RefTimeSignif = 3 (Observation time)
> RefTime       = 2009-08-30T16:30:00.000Z
> RefTime Fields = 2009-8-30 16:30:0
> ProductionStatus      = 1 (Operational test products)
> TypeOfProcessedData   = 6 (Processed satellite observations)
> 
> Grib2GridDefinitionSection hash=-1919541760 crc=3711080673
> Length             = 72
> Source  (3.0)      = 0 (Specified in Code table 3.1)
> Npts               = 576
> Template (3.1)     = 0
> 
> (3.0) Grid definition template 3.0 - latitude/longitude (or equidistant 
> cylindrical, or Plate Carrée)
> 1:                                                                            
>      GDS length == 72
> 5:                                                                            
>         Section == 3
> 6:                                             Source of Grid Definition (see 
> code table 3.0) == 0 (table 3.0: Specified in Code table 3.1)
> 7:                                                                      
> Number of data points == 576
> 11:                                             Number of octects for 
> optional list of numbers == 0
> 12:                                                          Interpretation 
> of list of numbers == 0 (table 3.11: There is no appended list)
> 13:                                                            Grid 
> Definition Template Number == 0
> 15:                                                                         
> Shape of the Earth == 3 (table 3.2: Earth assumed oblate spheroid with major 
> and minor axes specified (in km) by data producer)
> 16:                                                  Scale factor of radius 
> of spherical Earth == 255
> 17:                                                  Scaled value of radius 
> of spherical Earth == -9999
> 21:                                        Scale factor of major axis of 
> oblate spheroid Earth == 4
> 22:                                        Scaled value of major axis of 
> oblate spheroid Earth == 63781400
> 26:                                        Scale factor of minor axis of 
> oblate spheroid Earth == 4
> 27:                                        Scaled value of minor axis of 
> oblate spheroid Earth == 63567550
> 31:                                                     Ni - number of points 
> along a parallel == 24
> 35:                                                     Nj - number of points 
> along a meridian == 24
> 39:                                               Basic angle of the initial 
> production domain == 0
> 43: Subdivisions of basic angle used to define extreme longitudes and 
> latitudes, and direction increments == -9999
> 47:                                                         La1 - latitude of 
> first grid point == -9999
> 51:                                                        Lo1 - longitude of 
> first grid point == -9999
> 55:                                                             Resolution 
> and component flags == 56
> 56:                                                          La2 - latitude 
> of last grid point == 60000000
> 60:                                                         Lo2 - longitude 
> of last grid point == 60000000
> 64:                                                                 Di - i 
> direction increment == 5000000
> 68:                                                                 Dj - j 
> direction increment == 5000000
> 72:                                                                           
>    Scanning mode == 64
> 73:                                  List of number of points along each 
> meridian or parallel. == -9999
> 
> Grib2ProductDefinitionSection
> 
> (4.30) Product definition template 4.30 - satellite product
> 1:                                                                            
>      PDS length == 24
> 5:                                                                            
>         Section == 4
> 6:                                                Number of coordinates 
> values after Template == 0
> 8:                                                         Product Definition 
> Template Number == 30
> 10:                                                                         
> Parameter category == 1
> 11:                                                                           
> Parameter number == 98
> 12:                                                                 Type of 
> generating process == 8 (table 4.3: Observation)
> 13:                  Observation generating process identifier (defined by 
> originating centre) == 7
> 14:                                                 Number of contributing 
> spectral bands (NB) == 1
> 
> Grib2SectionDataRepresentation
> Template           = 0 (Grid point data - simple packing)
> NPoints            = 576
> 
> Grib2SectionData
> Starting Pos       = 2183854
> Data Length        = 581
> 
> 
> > Hi John....
> >
> > Did the data files sample get through okay?  If not, I have also posted 
> > this at:
> > <ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/hpgrids.zip>
> >
> > Thanks again for looking into this...
> >
> > tom
> >
> > On Mon, Jun 13, 2011 at 2:32 PM, Tom Whittaker <address@hidden> wrote:
> > > Hi John...
> > >
> > > Thanks....I had simply mis-typed it (guess I should use
> > > cut-and-paste)....but since it does not work for me, I'm attaching a
> > > zip file with the 2 nc files and the attempt at ncml...
> > >
> > > Thanks ahead for your help!!
> > >
> > > tom
> > >
> > > On Fri, Jun 10, 2011 at 4:11 PM, Unidata netCDF Java Support
> > > <address@hidden> wrote:
> > >> hi tom:
> > >>
> > >> jointExisting should be joinExisting.
> > >>
> > >> if thats not the actual problem, can you send me 2 files to check?
> > >>
> > >> john
> > >>
> > >>> I have several files that I want to aggregate over time.  Each file
> > >>> has two Groups, with a dimension "time" which is used for the desired
> > >>> variable...for example:
> > >>>
> > >>> Group proj1 {
> > >>> dimensions:
> > >>> time = 1;
> > >>> y = 3712;
> > >>> x = 3712;
> > >>> variables:
> > >>> float Instantaneous_Rain_Rate(time=1, y=3712, x=3712);
> > >>> ...
> > >>>
> > >>> I am trying to use NCML to accomplish this, with an "aggregation" 
> > >>> element like:
> > >>>
> > >>> <aggregation dimName="time" type="jointExisting">
> > >>> ...
> > >>>
> > >>> But apparently "time" is not the right syntax for this.  I've tried
> > >>> "proj1/time" as well, but to no available.  I'm hoping you can help.
> > >>>
> > >>> Thanks!
> > >>>
> > >>> tom
> > >>>
> > >>> --
> > >>> Tom Whittaker
> > >>> University of Wisconsin-Madison
> > >>> Space Science & Engineering Center (SSEC)
> > >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > >>> 1225 W. Dayton Street
> > >>> Madison, WI  53706  USA
> > >>> ph: +1 608 262 2759
> > >>>
> > >>>
> > >>
> > >>
> > >> Ticket Details
> > >> ===================
> > >> Ticket ID: IUX-310433
> > >> Department: Support netCDF Java
> > >> Priority: Normal
> > >> Status: Open
> > >>
> > >>
> > >
> > >
> > >
> > > --
> > > Tom Whittaker
> > > University of Wisconsin-Madison
> > > Space Science & Engineering Center (SSEC)
> > > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > > 1225 W. Dayton Street
> > > Madison, WI  53706  USA
> > > ph: +1 608 262 2759
> > >
> >
> >
> >
> > --
> > Tom Whittaker
> > University of Wisconsin-Madison
> > Space Science & Engineering Center (SSEC)
> > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > 1225 W. Dayton Street
> > Madison, WI  53706  USA
> > ph: +1 608 262 2759
> >
> >
> 


Ticket Details
===================
Ticket ID: IUX-310433
Department: Support netCDF Java
Priority: Normal
Status: Open

Attachment: MSG1.png
Description: PNG image

Attachment: MSG2.png
Description: PNG image