[IDD #WFA-619667]: Request for CONDUIT feed

Hi Randy,

re:
> in /data/conduit/GEFS we now have about 1887 individual grib files. I
> created a gribm.csh which runs wgrib2 on each file and piped the output
> to gribout.txt in the same directory. There were no ERRORS found in this
> file.

VERY interesting!

> I also wgrib2 the GFS_Global_1p0deg_Ensemble_12_2009022400.grib2 file
> located in /data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/.
> 
> I get the *** FATAL ERROR: rd_grib2_msg, missing end section ('7777') ***  
> after
> just a 126 records.
> 
> Does this mean there is an issue with cating the files together?

Perhaps.  It could also mean that 'wgrib2' has a problem that manifests itself
when looking through all of the GRIB2 messages in the monolithic output
file.  It could also mean that there is some problem related to the LDM
or OS that results in one ore more GRIB2 messages being stepped on in
the process of writing them to the end of the monolithic output file.

Here is what I would do to test this:

- get a full listing of the files that got put into one of the monolithic
  output files ** in the same order that they were written by the LDM into
  the file created by the FILE action **

- create a new monolithic output file by 'cat'ting each GRIB2 file to the
  end ** in the exact same order as in the file created by the LDM FILE
  action **

- run 'wgrib2' on the newly created output file

If 'wgrib2' does not show an error for the new file you create, then there
is some problem with the LDM appending new GRIB2 messages to the end of
the output file.  If this is the case, I would add the '-flush' flag to
the FILE action:

CONDUIT data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
    FILE -flush 
/data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\2_\100.grib2

By the way, I just noticed that the copy of this action that you included
in a previous email contained the '-close' flag:

CONDUIT data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
    FILE -close 
/data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\2_\100.grib2

I would remove the '-close' flag!

> I did notice that some individual files are named: 
> UREL;VREL_925Pa_20090224_1200_378.grib2
> Because of the ; in the file name one must put quotes "" around the
> file name in order for wgrib2 to work correctly. I dont think this is the
> issue but wanted to throw that out.
> 
> comments?

I was a bit concerned about the product IDs that contained the 'UREL;VREL' field
as the semi-colon has a special meaning for the *nix shell.  I ran a test that
proved to myself that one can create a valid output file name, and I stopped
worrying.  I am glad you recognized that you had to quote the file name
to get 'wgrib2' to work.  You are correct in your assumption that this should
_not_ be a problem.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: WFA-619667
Department: Support IDD
Priority: Normal
Status: Closed