Re: space products

On Tue, 5 Dec 2006, HansPeter Roesli wrote:

> Hi Rob and Don
>
> Sorry for my earlier email. There must have gone something wrong here.
> Now you should get my response loud and clear.
>
> I am sorry to have confused you with my naive cropping of the grb files
> with a word processor. Obviously the second set was okay. I just checked
> them with IDV and the outcome looks quite reasonable(even with the
> correct table being applied as hinted in Simon's email). In particular
> Simon and me looked at the CLAI product which stand for CLoud Analysis
> Image. The content reflects its intent, but geographical-wise the image
> is flipped vertically (W<->E) and does not project on the selected

HP,

I fixed this yesterday. the scanning mode process was not invoked on the
image, thus it was flipped. the projection part still needs fixed, John is
out of town for a couple of weeks and he's the one who works on that code.

robb...

> projection (it depicts just a disk). May I assume that this still is
> work in progress?
>
> Enjoy the snow (still no traces over here!), HP
>
>
> Robb Kambic wrote:
> > On Fri, 17 Nov 2006, Simon Elliott wrote:
> >
> >> Hi all,
> >>
> >> WHen I look at the file attached by HansPeter (HP GRIB.001) it looks
> >> fine to me.  The GRIB part can be found after the first 899 bytes.  The
> >> remaining part (GRIB to 7777, inclusive) is 803521 bytes long, which is
> >> as indicated in octets 9 to 16 of Section 0.
> >
> > Hi Simon,
> >
> > First, let me thank you for your time. There's no problem decoding the
> > (HP GRIB.001) file with my decoder that is  803521 bytes long. The file
> > that was originally sent was not 803521 bytes, but  804420 bytes long.
> > When the file was edited, the editor inserted extra bytes into the
> > file. This caused the decoder to get confused and it could not extract the
> > data. As you know the GRIB format is very rigid, so one extra byte can
> > change the course of action of the decoder. There was some confusion about
> > the acquistion of the file and I was unawared that the file was edited, so
> > I assumed that there was some EUMETSAT changes in the file. Since the
> > EUMETSAT datasets are very important to our community, I made an
> > effort to decode the file with the extra bytes. When debugging, I added
> > some code to accomodate the extra bytes but the decoder still got confused
> > about the data extraction process. When I got an uncorrupted file, my
> > decoder work without a problem.
> >
> >
> > There is one question, the level information is using level 0, is that
> > correct?
> >
> >            First Surface Type : 0 Reserved
> >            First Surface value : 0.0
> >            Second Surface Type : 0 Reserved
> >           Second Surface value : 0.0
> >
> > I have never run across a model using these values before, can you give
> > some insight?
> >
> > Thanks,
> > Robb...
> >
> > Here's a dump of the Header information of the file, is there an problem
> > here?
> >
> > --------------------------------------------------------------------
> >                         Header : GRIB2
> >                     Discipline : 3 Space products
> >                   GRIB Edition : 2
> >                    GRIB length : 803521
> >             Originating Center : 254 EUMETSAT Operation Centre
> >         Originating Sub-Center : 0
> > Significance of Reference Time : 3 Observation time
> >                 Reference Time : 2006-01-02T11:45:00Z
> >                 Product Status : 1 Operational test products
> >                   Product Type : 6 Processed satellite observations
> >          Number of data points : 1530169
> >                      Grid Name : 90 Space View Perspective or Orthographic
> >                      Grid Shape: 3 Earth oblate spheroid with axes
> > specified by
> > producer
> >         Oblate earth major axis: 6378.14
> >         Oblate earth minor axis: 6356.7554
> > Number of points along parallel: 1237
> > Number of points along meridian: 1237
> > Latitude of sub-satellite point: 0.0
> >   Longitude of sub-satellite pt: 0.0
> >   Resolution & Component flags : 0
> >          i direction increment : 1188.0
> >          j direction increment : 1187.0
> >  X-coordinate of sub-satellite : 619.0
> >  Y-coordinate of sub-satellite : 619.0
> >                  Scanning mode : 192
> >                    Basic angle : 0
> >         Altitude of the camera : 7.4533146E8
> >         X-coordinate of origin : 0.0
> >         Y-coordinate of origin : 0.0
> >             Product Definition : 30 Satellite product
> >             Parameter Category : 1 Quantitative_products
> >                 Parameter Name : 2 Cloud_Top_Height
> >                Parameter Units : m
> >        Generating Process Type : 8 Observation
> >                   ForecastTime : 0
> >             First Surface Type : 0 Reserved
> >            First Surface value : 0.0
> >            Second Surface Type : 0 Reserved
> >           Second Surface value : 0.0
> > --------------------------------------------------------------------
> >                         Header : GRIB2
> >                     Discipline : 3 Space products
> >                   GRIB Edition : 2
> >                    GRIB length : 803521
> >             Originating Center : 254 EUMETSAT Operation Centre
> >         Originating Sub-Center : 0
> > Significance of Reference Time : 3 Observation time
> >                 Reference Time : 2006-01-02T11:45:00Z
> >                 Product Status : 1 Operational test products
> >                   Product Type : 6 Processed satellite observations
> >          Number of data points : 1530169
> >                      Grid Name : 90 Space View Perspective or Orthographic
> >                      Grid Shape: 3 Earth oblate spheroid with axes
> > specified by
> > producer
> >         Oblate earth major axis: 6378.14
> >         Oblate earth minor axis: 6356.7554
> > Number of points along parallel: 1237
> > Number of points along meridian: 1237
> > Latitude of sub-satellite point: 0.0
> >   Longitude of sub-satellite pt: 0.0
> >   Resolution & Component flags : 0
> >          i direction increment : 1188.0
> >          j direction increment : 1187.0
> >  X-coordinate of sub-satellite : 619.0
> >  Y-coordinate of sub-satellite : 619.0
> >                  Scanning mode : 192
> >                    Basic angle : 0
> >         Altitude of the camera : 7.4533146E8
> >         X-coordinate of origin : 0.0
> >         Y-coordinate of origin : 0.0
> >             Product Definition : 30 Satellite product
> >             Parameter Category : 1 Quantitative_products
> >                 Parameter Name : 3 Cloud_Top_Height_Quality_Indicator
> >                Parameter Units :
> >        Generating Process Type : 8 Observation
> >                   ForecastTime : 0
> >             First Surface Type : 0 Reserved
> >            First Surface value : 0.0
> >            Second Surface Type : 0 Reserved
> >           Second Surface value : 0.0
> >
> >
> >> The users who have been looking at the data have received it via
> >> EUMETCast, and also via ftp from me.  Maybe some got data from the
> >> UMARF, but I don't get to hear about them.  I think that as long as a
> >> decoder looks through a file and pulls out GRIB (or BUFR) messages
> >> before decoding, then it doesn't matter how they are packed into files.
> >>
> >> Simon
> >>
> >> Dr Simon Elliott
> >> Product Implementation Manager
> >> Operations Department
> >> EUMETSAT
> >> tel: (+49) 6151 807385
> >> fax: (+49) 6151 807304
> >>
> >>
> >>>>> HansPeter Roesli <address@hidden> 17/11/2006 09:37:39 >>>
> >> Hi Simon and Robb
> >>
> >> After having read Simon's first reaction I got a slight doubt about
> >> what
> >> I did. Actually, the file Robb is mentioning is not the original file
> >> as
> >> retrieved from EUMETSAT's archive U-MARF (you find that now attached).
> >>
> >> The original file is refused right away by IDV. Therefore, I chopped
> >> off
> >> the ASCII header with an editor and that is the file Robb is using.
> >> Might be that my editor changed things inside the GRIB-7777 clause when
> >>
> >> I was pruning the file. So, Robb please check again with the attached
> >> file which has leading bytes (EUMETSAT's ASCII header) before the bytes
> >>
> >> spelling GRIB.
> >>
> >> Also, Simon, when you are saying that other centres are not having
> >> decoding problems, are they getting the data from EUMETCast/GTS or from
> >>
> >> U-MARF. I am using exclusively U-MARF retrieved products. Might be
> >> there
> >> is a problem with archived data?
> >>
> >> HP
> >>
> >> Simon Elliott wrote:
> >>> Hello Robb,
> >>>
> >>> HansPeter has asked me to contact you directly as I am responsible
> >> for
> >>> the generation of the GRIB2 data from EUMETSAT.  I am interested to
> >> read
> >>> about the extra characters you found inside hte message.  As I'm
> >> sure
> >>> you can imagine, we have exchanged the data with other centres using
> >>> different software and they have yet to spot this problem.
> >>>
> >>> I suspect either (i) the GRIB data have been chopped up for
> >>> dissemination and not been properly re-assembled into the original
> >>> message, or (ii) at some stage the data have been transferred with
> >> ASCII
> >>> FTP ... this happens quite often.
> >>>
> >>> If you could send me the file you mention, or make it availabel for
> >> FTP
> >>> transfer, I would be happy to investigate.
> >>>
> >>> Regards,
> >>>
> >>> Simon
> >>>
> >>> Dr Simon Elliott
> >>> Product Implementation Manager
> >>> Operations Department
> >>> EUMETSAT
> >>> tel: (+49) 6151 807385
> >>> fax: (+49) 6151 807304
> >>>
> >>>
> >>>>>> Robb Kambic <address@hidden> 14/11/2006 22:50:35 >>>
> >>> HansPeter,
> >>>
> >>> i investigated the problem with Grib2 decoder not being able to
> >> decode
> >>> the
> >>> space products, ie
> >>> MSG1-SEVI-MSGCLTH-0100-0100-20060102114500.000000000Z-12774_GRIB.grb
> >>>
> >>> at first it seemed to be a table problem but that's not the case.
> >>> the products do not follow the standard Grib encoding, there are
> >>> extra characters inserted between the Grib format sections that's
> >> the
> >>> problem. i had the same problem with the Grib 1 format from some of
> >>> the
> >>> european centers. i tried the Grib 1 format fix then the decoder was
> >>> able to
> >>> decode the first 5 sections of the product but then the 6th section
> >>> didn't begin at the correct position. at this time, i could some
> >> help
> >>> in
> >>> finding out the format of the product, so i can modify the decoder.
> >> i
> >>> did a
> >>> quick look for the product information but was unable to get the
> >>> information
> >>> i needed. Could you contact the data provider to get the structure
> >> of
> >>> the
> >>> product?  or get a contact person for this product?
> >>>
> >>> thanks,
> >>> robb...
> >>>
> >>>
> >>>
> >>>
> >>>
> >
> > ===============================================================================
> > Robb Kambic                            Unidata Program Center
> > Software Engineer III                          Univ. Corp for Atmospheric 
> > Research
> > address@hidden                 WWW: http://www.unidata.ucar.edu/
> > ===============================================================================
> >
> >
>
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================