[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [netcdf-java] Grib File Dump not correct?/Grib Code to read Grib Parameter Names & Values



Chad,

I've been answering many Grib questions lately, it seems that most programmers can use the CDM layer above the grib layer and remove themselves from the details of the Grib file, ie such as offsets, etc.
The CDM layer should have all the information that you need.

On Tue, 6 Oct 2009, Chad Saxon wrote:

Robb,
i have been looking at the doc and been trying out some code. By the way thanks
for the updates and quick replies. 

The crux of what i want to do is really read in and store:

- the unit parameter of the data (i.e. temperature)
- the data coupled with the lat/lon in each record
- the offset of lat/lon(resolution) so i can loop through the data and store
appropriate parameter values.

Basically you would open the file as GridDataset, this will perform all the low level Grib actions for you.

Here's a tutorial to help get you get started: http://www.unidata.ucar.edu/software/netcdf-java/tutorial/

 I would look at GridDatatype, NetcdfFile, and NetcdfDatasets

There are also javaDocs at: http://www.unidata.ucar.edu/software/netcdf-java/v2.2.20/javadoc/index.html

Sample code: http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2009/msg00233.html
http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2009/msg00235.html


The above messages have urls to javaDocs about GridDatasets, if you do some searching I believe you will be able to get all the information that you need. On the netcdf-java home page:
http://www.unidata.ucar.edu/software/netcdf-java

There is a reference to a program ToolsUI GUI that will permit you to look at the files. I would launch it by webstart so you can get the feel for it. There isn't any documentation but it intutitive. Hope this gets you out the low levels of Grib.

Robb...


right now i have this:

*testfile = whateverfile.grb
    List<GridRecord> gribRecords;

ucar.unidata.io.RandomAccessFile gribfile = new
ucar.unidata.io.RandomAccessFile(testfile, "rw");
   index = new GribReadIndex();
   gridindex = new GridIndex();
   gridindex = index.open(testfile);

   gribRecords = gridindex.getGridRecords();

   i wanted to declare gribRecords to the class GribGridRecord since it seems i
have access to the offset parameters i would need to do my seek that you 
suggested
a while back for looking at all the data but the getGridRecords only returns of
type GridRecord which, according to the doc, doesnt have any functionality to 
look
at the offsets, etc.

   Once i get the list of records i just want to be able to do a seek on each 
one,
loop through its data and start on the next record. The outdated Index class
seemed to have that capability before. Maybe i am just overlooking something.
thanks for your help...

Chad




On Sep 14, 2009, at 10:07 AM, Robb Kambic wrote:

      Chad,

      Thanks for pointing out that the Java Docs need to be updated on the
      Decoders page, I'll do it sometime this afternoon.

      Robb...



      On Mon, 14 Sep 2009, Chad Saxon wrote:

            Robb,

            thanks for the input and feedback. i went ahead and got
            the grib-8.0 library

            and have to work through some of the depreciated stuff in
            my code now but i

            was wondering the the javadoc reflects the grib 8.0
            changes? Some of the

            functionality like, for
            example, Grib1GridDefinitionSection.getGdsVars();

            doesnt appear in the API javadoc that i am looking at.

            thanks...

            Chad

            On Sep 8, 2009, at 2:03 PM, Robb Kambic wrote:


                 On Mon, 7 Sep 2009, Chad Saxon wrote:


                       I am writing some low level code(because i need
            to),

                       to read grib files and all the values that i
            print out

                       on the console do not match the values when i
            dump

                       them out to a file. The grib file that i am
            testing

                       against is including in this e-mail for those
            that

                       want to take a look.


                       Also i seem to be having problems getting
            actual

                       String literals for certain codes.. like the
            parameter

                       name for code 11 is Temperature and the Unit
            type is K

                       in this case but it seems that the way to get
            these

                       are not very intuitive to get using Index and
            my

                       GribRecords variable in the code below. Maybe
            someone

                       could shed some light on this??


                       In advance, i want to thank all those that can
            take

                       the time to look through this.


                       --Chad


                 Chad,


                 The Grib library usually reads the records of the
            grib file in

                 sequence of the sections, there are 5 sections to
            Grib1. See:

                 http://www.nco.ncep.noaa.gov/pmb/docs/on388/


                 To read a particular section, the filePointer must be
            moved to the

                 start of the section to have a proper read. There is
            an index file

                 that has some of the offsets to the sections, such as
            the GDS

                 section. Here's the index of your sample file:


            
     ------------------------------------------------------------------

                 10 0 -1 11 96 1 0.0 255 0.0 2009-09-07T06:00:00Z 24
            284494432 36

                 68 1 false 7 0 2

                 10 0 -1 11 96 1 0.0 255 0.0 2009-09-07T06:00:00Z 48
            284494432 390

                 422 1 false 7 0 2

                 10 0 -1 11 96 1 0.0 255 0.0 2009-09-07T06:00:00Z 72
            284494432 744

                 776 1 false 7 0 2

            
     ------------------------------------------------------------------


                 The offsets 36, 390, and 744 are the GDS offsets for
            the 3 records

                 in the file. If you would modify you code to do the
            seeks and then

                 read the GDS, then the values will be correct.


                 gribfile.seek( 36 );

                 Grib1GridDefinitionSection gds =

                 Grib1GridDefinitionSection(gribfile);


                 Here are some files that will help you understand the
            how the

                 index works.

                 IndexFormat.txt  locate in the root directory of the
            package.

                 ucar/grib/grib1 directory:

                 Grib1WriteIndex.java

                 Grib1Dump.java

                 Grib1GetData.java

                 Grib1Data.java

                 Grib1Input.java


                 also look at the following files on how to extract
            particular PDS

                 and GDS values;


                 GribReadIndex.java

                 ShowGribIndex.java


                 I must warn you that the Grib library has been
            refactored, so the

                 Index api will be slightly different then the 6.0
            release. I would

                 suggest that you get the latest 8.0 release on the
            decoders page

                 located at:


                 http://www.unidata.ucar.edu/software/decoders/


                 Robb...


                       here is my code below with output:


                       public class GribPlugin {


                       java.io.PrintStream ps;


                       String[] fileDump;


                       Grib1GridDefinitionSection gridSection;


                       String testfile = "gfs20090907162501123.grb";


                       public GribPlugin() throws IOException,

                       NoValidGribException, NotSupportedException{


                       ucar.unidata.io.RandomAccessFile gribfile = new

                       ucar.unidata.io.RandomAccessFile(testfile,
            "rw");


                       type = file.getEdition(gribfile);


                       System.out.println("File read in: " + testfile
            + ",

                       Type of Grib File is " + type);


                       fileDump = new String[3];


                       fileDump[0] = "gfs20090907162501123.grb";


                       fileDump[1] = "gribdump.out";


                       fileDump[2] = "true";


                       try{


                       gridSection = new

                       Grib1GridDefinitionSection(gribfile);


                       System.out.println("defining a new
            gridsection");


                       }


                       catch(IOException e){


                       throw new IOException("Grid Section " + e);


                       }


                       System.out.println("grid definition section");


                       System.out.println("dx: " + gridSection.getDx()
            + ",

                       dy: " + gridSection.getDy() + ", La1: " +

                       gridSection.getLa1() + ", La2: " +

                       gridSection.getLa2() + ", Lo1: " +


                       gridSection.getLo1() + ", Lo2: " +

                       gridSection.getLo2() + ", Nx: " +
            gridSection.getNx()

                       + ", Ny: " + gridSection.getNy() + ", Grid
            Section

                       Name: " + gridSection.getName());


                       System.out.println("File Dump Information");


                       gribDump.main(fileDump);


                       System.out.println("Dump GridFile Info Out To
            File");


                       ps = new java.io.PrintStream("grib.out");


                       index = gribIndex.writeFileIndex(gribfile, ps,
            true);


                       //this is printing out the grib records


                       System.out.println("Grib Records");


                       gribRecords = index.getGribRecords();


                       System.out.println("Number of Grib Records: " +

                       gribRecords.size());


                       for(int i = 0; i < gribRecords.size(); i++){


                       System.out.println(gribRecords.get(i).category
            + " " +


                       gribRecords.get(i).center + " " +


                       gribRecords.get(i).discipline + " " +


                              gribRecords.get(i).paramNumber + " " +
            <--how

                       do you convert this to the equivalent param
            name??


                       gribRecords.get(i).productType + " " +


                       gribRecords.get(i).decimalScale + " " +


                              gribRecords.get(i).gdsKey + " " +


                       gribRecords.get(i).typeGenProcess + " " +


                       gribRecords.get(i).forecastTime + " " +


                              gribRecords.get(i).refTime + " " +


                       gribRecords.get(i).subCenter + " " +

                       gribRecords.get(i).bmsExists);


                       } //end for


                       } //end public class constructor


                       } // end class GribPlugin


                       Console Output:


                       File read in: gfs20090907162501123.grb, Type of
            Grib

                       File is 1


                       GribPDSLevel: Table 3 level 96 is not
            implemented yet


                       PDS: Time Unit 9 is not yet supported


                       Using default
            table:resources/grib/tables/wmo_2_v1.tab

                       (0:0:-1)


                       defining a new gridsection


                       Grid 28 not configured yet


                       grid definition section


                       dx: 2.632, dy: 0.0, La1: -8355.851, La2:
            591.622, Lo1:

                       65.536, Lo2: 0.256, Nx: 540, Ny: 24583, Grid
            Section

                       Name: Latitude/Longitude Grid


                       File Dump Information


                       gds exception was caught


                       Dump GridFile Info Out To File


                       Grib Records


                       Number of Grib Records: 3


                       -1 7 0 11 10 1 2350466037 96 24
            2009-09-07T06:00:00Z 0

                       false


                       -1 7 0 11 10 1 2350466037 96 48
            2009-09-07T06:00:00Z 0

                       false


                       -1 7 0 11 10 1 2350466037 96 72
            2009-09-07T06:00:00Z 0

                       false


                       *****


                       Now you will notice that my grid definition
            section

                       does not match any of the parameters from the
            grid

                       dump below(except for the grid section name).
            My

                       question is, is why is this and/or what am i
            doing

                       wrong here??? Those numbers that i get from the

                       Console output just do not make any sense to me
            at

                       all.


                       *****


                       GridDump Output:


                       Header : GRIB1


                                        Discipline : 0 Meteorological

                       Products


                                      GRIB Edition : 1


                                       GRIB length : 354


                                Originating Center : 7 US National
            Weather

                       Service (NCEP)


                            Originating Sub-Center : 0 WMO Secretariat


                                Product Definition : 10 product valid
            at

                       reference time P1


                                Parameter Category : -1 Meteorological

                       Parameters


                                    Parameter Name : 11 TMP
            Temperature


                                   Parameter Units : K


                                    Reference Time :
            2009-09-07T06:00:00Z


                                        Time Units : hour


                              Time Range Indicator : product valid at
            RT + P1


                                       Time 1 (P1) : 24


                                       Time 2 (P2) : 0


                           Generating Process Type : 96 Global
            Forecast

                       System Model (formerly known as the Aviation)


                                        Level Type : 1 surface


                                     Level Value 1 : 0.0


                                     Level Value 2 : 0.0


                                        GDS Exists : true


                                        BMS Exists : false


                             Number of data points : 3447


                                         Grid Name :
            Latitude/Longitude Grid


                                         Grid Shape: 0 spherical


                             Spherical earth radius: 6367.47


                                                Nx : 16


                                                Ny : 15


                                               La1 : 0.0


                                               Lo1 : -100.0


                       Resolution & Component flags : 128


                                             Winds : True


                                               La2 : 29.0


                                               Lo2 : -70.0

            
=============================================================================

                 ==

                 Robb Kambic           Unidata Program Center

                 Software Engineer III   Univ. Corp for Atmospheric
            Research

                 address@hidden   WWW:
            http://www.unidata.ucar.edu/

            
=============================================================================

                 ==



      
===============================================================================
      Robb Kambic           Unidata Program Center
      Software Engineer III   Univ. Corp for Atmospheric Research
      address@hidden   WWW: http://www.unidata.ucar.edu/
      
===============================================================================





===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================