Re: space products

Simon,

Do you have newer documentation? Because both the NCEP and EUMETSAT docs
for template 4.3 have bytes 23-34 designated for 1st and 2nd fixed
surfaces. I understand that these values are probably not used in space
products.

robb...




On Tue, 5 Dec 2006, Simon Elliott wrote:

> Hello Robb,
>
> I am glad you are getting further with decoding the data.
>
> >From your question about the first and second surface types I get the
> impression
> you are using Product Definition Template 4.0 to interpret the data.
> In fact, when
> you detected "Product Definition : 30 Satellite product", this should
> trigger you to
> use Product Definition Template 4.30 to interpret Section 4.  There you
> will find
> that there are no references to first and second surfaces, but rather
> other information
> describing the satellite data.
>
> You mention that you have never seen a model using these values, but I
> guess this
> is a simple misunderstanding, as we do not generate these values in any
> case.
>
> I hope this helps.
>
> Regards,
>
> Simon
>
> >>> Robb Kambic <address@hidden> 04/12/2006 18:54:38
> >>>
> On Fri, 17 Nov 2006, Simon Elliott wrote:
>
> > Hi all,
> >
> > WHen I look at the file attached by HansPeter (HP GRIB.001) it looks
> > fine to me.  The GRIB part can be found after the first 899 bytes.
> The
> > remaining part (GRIB to 7777, inclusive) is 803521 bytes long, which
> is
> > as indicated in octets 9 to 16 of Section 0.
>
> Hi Simon,
>
> First, let me thank you for your time. There's no problem decoding the
> (HP GRIB.001) file with my decoder that is  803521 bytes long. The
> file
> that was originally sent was not 803521 bytes, but  804420 bytes long.
> When the file was edited, the editor inserted extra bytes into the
> file. This caused the decoder to get confused and it could not extract
> the
> data. As you know the GRIB format is very rigid, so one extra byte can
> change the course of action of the decoder. There was some confusion
> about
> the acquistion of the file and I was unawared that the file was edited,
> so
> I assumed that there was some EUMETSAT changes in the file. Since the
> EUMETSAT datasets are very important to our community, I made an
> effort to decode the file with the extra bytes. When debugging, I
> added
> some code to accomodate the extra bytes but the decoder still got
> confused
> about the data extraction process. When I got an uncorrupted file, my
> decoder work without a problem.
>
>
> There is one question, the level information is using level 0, is that
> correct?
>
>            First Surface Type : 0 Reserved
>            First Surface value : 0.0
>            Second Surface Type : 0 Reserved
>           Second Surface value : 0.0
>
> I have never run across a model using these values before, can you
> give
> some insight?
>
> Thanks,
> Robb...
>
> Here's a dump of the Header information of the file, is there an
> problem
> here?
>
> --------------------------------------------------------------------
>                         Header : GRIB2
>                     Discipline : 3 Space products
>                   GRIB Edition : 2
>                    GRIB length : 803521
>             Originating Center : 254 EUMETSAT Operation Centre
>         Originating Sub-Center : 0
> Significance of Reference Time : 3 Observation time
>                 Reference Time : 2006-01-02T11:45:00Z
>                 Product Status : 1 Operational test products
>                   Product Type : 6 Processed satellite observations
>          Number of data points : 1530169
>                      Grid Name : 90 Space View Perspective or
> Orthographic
>                      Grid Shape: 3 Earth oblate spheroid with axes
> specified by
> producer
>         Oblate earth major axis: 6378.14
>         Oblate earth minor axis: 6356.7554
> Number of points along parallel: 1237
> Number of points along meridian: 1237
> Latitude of sub-satellite point: 0.0
>   Longitude of sub-satellite pt: 0.0
>   Resolution & Component flags : 0
>          i direction increment : 1188.0
>          j direction increment : 1187.0
>  X-coordinate of sub-satellite : 619.0
>  Y-coordinate of sub-satellite : 619.0
>                  Scanning mode : 192
>                    Basic angle : 0
>         Altitude of the camera : 7.4533146E8
>         X-coordinate of origin : 0.0
>         Y-coordinate of origin : 0.0
>             Product Definition : 30 Satellite product
>             Parameter Category : 1 Quantitative_products
>                 Parameter Name : 2 Cloud_Top_Height
>                Parameter Units : m
>        Generating Process Type : 8 Observation
>                   ForecastTime : 0
>             First Surface Type : 0 Reserved
>            First Surface value : 0.0
>            Second Surface Type : 0 Reserved
>           Second Surface value : 0.0
> --------------------------------------------------------------------
>                         Header : GRIB2
>                     Discipline : 3 Space products
>                   GRIB Edition : 2
>                    GRIB length : 803521
>             Originating Center : 254 EUMETSAT Operation Centre
>         Originating Sub-Center : 0
> Significance of Reference Time : 3 Observation time
>                 Reference Time : 2006-01-02T11:45:00Z
>                 Product Status : 1 Operational test products
>                   Product Type : 6 Processed satellite observations
>          Number of data points : 1530169
>                      Grid Name : 90 Space View Perspective or
> Orthographic
>                      Grid Shape: 3 Earth oblate spheroid with axes
> specified by
> producer
>         Oblate earth major axis: 6378.14
>         Oblate earth minor axis: 6356.7554
> Number of points along parallel: 1237
> Number of points along meridian: 1237
> Latitude of sub-satellite point: 0.0
>   Longitude of sub-satellite pt: 0.0
>   Resolution & Component flags : 0
>          i direction increment : 1188.0
>          j direction increment : 1187.0
>  X-coordinate of sub-satellite : 619.0
>  Y-coordinate of sub-satellite : 619.0
>                  Scanning mode : 192
>                    Basic angle : 0
>         Altitude of the camera : 7.4533146E8
>         X-coordinate of origin : 0.0
>         Y-coordinate of origin : 0.0
>             Product Definition : 30 Satellite product
>             Parameter Category : 1 Quantitative_products
>                 Parameter Name : 3 Cloud_Top_Height_Quality_Indicator
>                Parameter Units :
>        Generating Process Type : 8 Observation
>                   ForecastTime : 0
>             First Surface Type : 0 Reserved
>            First Surface value : 0.0
>            Second Surface Type : 0 Reserved
>           Second Surface value : 0.0
>
>
> >
> > The users who have been looking at the data have received it via
> > EUMETCast, and also via ftp from me.  Maybe some got data from the
> > UMARF, but I don't get to hear about them.  I think that as long as
> a
> > decoder looks through a file and pulls out GRIB (or BUFR) messages
> > before decoding, then it doesn't matter how they are packed into
> files.
> >
> > Simon
> >
> > Dr Simon Elliott
> > Product Implementation Manager
> > Operations Department
> > EUMETSAT
> > tel: (+49) 6151 807385
> > fax: (+49) 6151 807304
> >
> >
> > >>> HansPeter Roesli <address@hidden> 17/11/2006 09:37:39 >>>
> > Hi Simon and Robb
> >
> > After having read Simon's first reaction I got a slight doubt about
> > what
> > I did. Actually, the file Robb is mentioning is not the original
> file
> > as
> > retrieved from EUMETSAT's archive U-MARF (you find that now
> attached).
> >
> > The original file is refused right away by IDV. Therefore, I chopped
> > off
> > the ASCII header with an editor and that is the file Robb is using.
> > Might be that my editor changed things inside the GRIB-7777 clause
> when
> >
> > I was pruning the file. So, Robb please check again with the
> attached
> > file which has leading bytes (EUMETSAT's ASCII header) before the
> bytes
> >
> > spelling GRIB.
> >
> > Also, Simon, when you are saying that other centres are not having
> > decoding problems, are they getting the data from EUMETCast/GTS or
> from
> >
> > U-MARF. I am using exclusively U-MARF retrieved products. Might be
> > there
> > is a problem with archived data?
> >
> > HP
> >
> > Simon Elliott wrote:
> > > Hello Robb,
> > >
> > > HansPeter has asked me to contact you directly as I am responsible
> > for
> > > the generation of the GRIB2 data from EUMETSAT.  I am interested
> to
> > read
> > > about the extra characters you found inside hte message.  As I'm
> > sure
> > > you can imagine, we have exchanged the data with other centres
> using
> > > different software and they have yet to spot this problem.
> > >
> > > I suspect either (i) the GRIB data have been chopped up for
> > > dissemination and not been properly re-assembled into the original
> > > message, or (ii) at some stage the data have been transferred with
> > ASCII
> > > FTP ... this happens quite often.
> > >
> > > If you could send me the file you mention, or make it availabel
> for
> > FTP
> > > transfer, I would be happy to investigate.
> > >
> > > Regards,
> > >
> > > Simon
> > >
> > > Dr Simon Elliott
> > > Product Implementation Manager
> > > Operations Department
> > > EUMETSAT
> > > tel: (+49) 6151 807385
> > > fax: (+49) 6151 807304
> > >
> > >
> > >>>> Robb Kambic <address@hidden> 14/11/2006 22:50:35 >>>
> > > HansPeter,
> > >
> > > i investigated the problem with Grib2 decoder not being able to
> > decode
> > > the
> > > space products, ie
> > >
> MSG1-SEVI-MSGCLTH-0100-0100-20060102114500.000000000Z-12774_GRIB.grb
> > >
> > > at first it seemed to be a table problem but that's not the case.
> > > the products do not follow the standard Grib encoding, there are
> > > extra characters inserted between the Grib format sections that's
> > the
> > > problem. i had the same problem with the Grib 1 format from some
> of
> > > the
> > > european centers. i tried the Grib 1 format fix then the decoder
> was
> > > able to
> > > decode the first 5 sections of the product but then the 6th
> section
> > > didn't begin at the correct position. at this time, i could some
> > help
> > > in
> > > finding out the format of the product, so i can modify the
> decoder.
> > i
> > > did a
> > > quick look for the product information but was unable to get the
> > > information
> > > i needed. Could you contact the data provider to get the structure
> > of
> > > the
> > > product?  or get a contact person for this product?
> > >
> > > thanks,
> > > robb...
> > >
> > >
> > >
> > >
> > >
> >
>
> ===============================================================================
> Robb Kambic                              Unidata Program Center
> Software Engineer III                    Univ. Corp for Atmospheric
> Research
> address@hidden                   WWW:
> http://www.unidata.ucar.edu/
> ===============================================================================
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================