[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: space products



On Fri, 17 Nov 2006, Simon Elliott wrote:

> Hi all,
>
> WHen I look at the file attached by HansPeter (HP GRIB.001) it looks
> fine to me.  The GRIB part can be found after the first 899 bytes.  The
> remaining part (GRIB to 7777, inclusive) is 803521 bytes long, which is
> as indicated in octets 9 to 16 of Section 0.

Hi Simon,

First, let me thank you for your time. There's no problem decoding the
(HP GRIB.001) file with my decoder that is  803521 bytes long. The file
that was originally sent was not 803521 bytes, but  804420 bytes long.
When the file was edited, the editor inserted extra bytes into the
file. This caused the decoder to get confused and it could not extract the
data. As you know the GRIB format is very rigid, so one extra byte can
change the course of action of the decoder. There was some confusion about
the acquistion of the file and I was unawared that the file was edited, so
I assumed that there was some EUMETSAT changes in the file. Since the
EUMETSAT datasets are very important to our community, I made an
effort to decode the file with the extra bytes. When debugging, I added
some code to accomodate the extra bytes but the decoder still got confused
about the data extraction process. When I got an uncorrupted file, my
decoder work without a problem.


There is one question, the level information is using level 0, is that
correct?

           First Surface Type : 0 Reserved
           First Surface value : 0.0
           Second Surface Type : 0 Reserved
          Second Surface value : 0.0

I have never run across a model using these values before, can you give
some insight?

Thanks,
Robb...

Here's a dump of the Header information of the file, is there an problem
here?

--------------------------------------------------------------------
                        Header : GRIB2
                    Discipline : 3 Space products
                  GRIB Edition : 2
                   GRIB length : 803521
            Originating Center : 254 EUMETSAT Operation Centre
        Originating Sub-Center : 0
Significance of Reference Time : 3 Observation time
                Reference Time : 2006-01-02T11:45:00Z
                Product Status : 1 Operational test products
                  Product Type : 6 Processed satellite observations
         Number of data points : 1530169
                     Grid Name : 90 Space View Perspective or Orthographic
                     Grid Shape: 3 Earth oblate spheroid with axes
specified by
producer
        Oblate earth major axis: 6378.14
        Oblate earth minor axis: 6356.7554
Number of points along parallel: 1237
Number of points along meridian: 1237
Latitude of sub-satellite point: 0.0
  Longitude of sub-satellite pt: 0.0
  Resolution & Component flags : 0
         i direction increment : 1188.0
         j direction increment : 1187.0
 X-coordinate of sub-satellite : 619.0
 Y-coordinate of sub-satellite : 619.0
                 Scanning mode : 192
                   Basic angle : 0
        Altitude of the camera : 7.4533146E8
        X-coordinate of origin : 0.0
        Y-coordinate of origin : 0.0
            Product Definition : 30 Satellite product
            Parameter Category : 1 Quantitative_products
                Parameter Name : 2 Cloud_Top_Height
               Parameter Units : m
       Generating Process Type : 8 Observation
                  ForecastTime : 0
            First Surface Type : 0 Reserved
           First Surface value : 0.0
           Second Surface Type : 0 Reserved
          Second Surface value : 0.0
--------------------------------------------------------------------
                        Header : GRIB2
                    Discipline : 3 Space products
                  GRIB Edition : 2
                   GRIB length : 803521
            Originating Center : 254 EUMETSAT Operation Centre
        Originating Sub-Center : 0
Significance of Reference Time : 3 Observation time
                Reference Time : 2006-01-02T11:45:00Z
                Product Status : 1 Operational test products
                  Product Type : 6 Processed satellite observations
         Number of data points : 1530169
                     Grid Name : 90 Space View Perspective or Orthographic
                     Grid Shape: 3 Earth oblate spheroid with axes
specified by
producer
        Oblate earth major axis: 6378.14
        Oblate earth minor axis: 6356.7554
Number of points along parallel: 1237
Number of points along meridian: 1237
Latitude of sub-satellite point: 0.0
  Longitude of sub-satellite pt: 0.0
  Resolution & Component flags : 0
         i direction increment : 1188.0
         j direction increment : 1187.0
 X-coordinate of sub-satellite : 619.0
 Y-coordinate of sub-satellite : 619.0
                 Scanning mode : 192
                   Basic angle : 0
        Altitude of the camera : 7.4533146E8
        X-coordinate of origin : 0.0
        Y-coordinate of origin : 0.0
            Product Definition : 30 Satellite product
            Parameter Category : 1 Quantitative_products
                Parameter Name : 3 Cloud_Top_Height_Quality_Indicator
               Parameter Units :
       Generating Process Type : 8 Observation
                  ForecastTime : 0
            First Surface Type : 0 Reserved
           First Surface value : 0.0
           Second Surface Type : 0 Reserved
          Second Surface value : 0.0


>
> The users who have been looking at the data have received it via
> EUMETCast, and also via ftp from me.  Maybe some got data from the
> UMARF, but I don't get to hear about them.  I think that as long as a
> decoder looks through a file and pulls out GRIB (or BUFR) messages
> before decoding, then it doesn't matter how they are packed into files.
>
> Simon
>
> Dr Simon Elliott
> Product Implementation Manager
> Operations Department
> EUMETSAT
> tel: (+49) 6151 807385
> fax: (+49) 6151 807304
>
>
> >>> HansPeter Roesli <address@hidden> 17/11/2006 09:37:39 >>>
> Hi Simon and Robb
>
> After having read Simon's first reaction I got a slight doubt about
> what
> I did. Actually, the file Robb is mentioning is not the original file
> as
> retrieved from EUMETSAT's archive U-MARF (you find that now attached).
>
> The original file is refused right away by IDV. Therefore, I chopped
> off
> the ASCII header with an editor and that is the file Robb is using.
> Might be that my editor changed things inside the GRIB-7777 clause when
>
> I was pruning the file. So, Robb please check again with the attached
> file which has leading bytes (EUMETSAT's ASCII header) before the bytes
>
> spelling GRIB.
>
> Also, Simon, when you are saying that other centres are not having
> decoding problems, are they getting the data from EUMETCast/GTS or from
>
> U-MARF. I am using exclusively U-MARF retrieved products. Might be
> there
> is a problem with archived data?
>
> HP
>
> Simon Elliott wrote:
> > Hello Robb,
> >
> > HansPeter has asked me to contact you directly as I am responsible
> for
> > the generation of the GRIB2 data from EUMETSAT.  I am interested to
> read
> > about the extra characters you found inside hte message.  As I'm
> sure
> > you can imagine, we have exchanged the data with other centres using
> > different software and they have yet to spot this problem.
> >
> > I suspect either (i) the GRIB data have been chopped up for
> > dissemination and not been properly re-assembled into the original
> > message, or (ii) at some stage the data have been transferred with
> ASCII
> > FTP ... this happens quite often.
> >
> > If you could send me the file you mention, or make it availabel for
> FTP
> > transfer, I would be happy to investigate.
> >
> > Regards,
> >
> > Simon
> >
> > Dr Simon Elliott
> > Product Implementation Manager
> > Operations Department
> > EUMETSAT
> > tel: (+49) 6151 807385
> > fax: (+49) 6151 807304
> >
> >
> >>>> Robb Kambic <address@hidden> 14/11/2006 22:50:35 >>>
> > HansPeter,
> >
> > i investigated the problem with Grib2 decoder not being able to
> decode
> > the
> > space products, ie
> > MSG1-SEVI-MSGCLTH-0100-0100-20060102114500.000000000Z-12774_GRIB.grb
> >
> > at first it seemed to be a table problem but that's not the case.
> > the products do not follow the standard Grib encoding, there are
> > extra characters inserted between the Grib format sections that's
> the
> > problem. i had the same problem with the Grib 1 format from some of
> > the
> > european centers. i tried the Grib 1 format fix then the decoder was
> > able to
> > decode the first 5 sections of the product but then the 6th section
> > didn't begin at the correct position. at this time, i could some
> help
> > in
> > finding out the format of the product, so i can modify the decoder.
> i
> > did a
> > quick look for the product information but was unable to get the
> > information
> > i needed. Could you contact the data provider to get the structure
> of
> > the
> > product?  or get a contact person for this product?
> >
> > thanks,
> > robb...
> >
> >
> >
> >
> >
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================