[IDD #WFA-619667]: Request for CONDUIT feed

Hi Randy and Martha,

Good news!

We found the error causing problems with the monolithic files written
for the GEFS  ensemble fields.  The problem was not in the LDM, but,
rather, in the action that I extracted from the THREDDS server on
motherlode.ucar.edu and sent along to you.  In short, the problem
was in the pattern being used to select the products that would be
FILEd.

The pattern I grabbed from motherlode was:

CONDUIT data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
     FILE 
/data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\2_\100.grib2

It should have been:

CONDUIT ^data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
     FILE 
/data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\1_\200.grib2

-- OR --

CONDUIT ^data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
     FILE   -flush   
/data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\1_\200.grib2

The differences are the '^' (anchor) at the beginning of the pattern
to be matched and the changing of the order of use of the '\1' and '\2'.
fields.  Without the inclusion of the '^', the ASCII manifest (status) files
included in the GEFS set was written to the monolithic output file along with
the GRIB2 messages.  'wgrib2' correctly reported an error when it ran into
these products.  The switching of the '\1' and '\2' fields is more cosmetic
than anything (it didn't make any sense to add a trailing '00' to the
end of the date, and it didn't make much sense to put the model runtime before
the date in the output file name).

The other thing that needed correction (for future debugging) is the pattern
used to write GRIB2 products into individual disk files.  The pattern I
sent originally was:

CONDUIT gefs.*grib2/ncep/SPEC62MRF/#000/(........)(....)F(...)/(.*)/(.*) (Pa)
    FILE    -close    /data/conduit/GEFS/\4_\5\6_\1_\2_\3.grib2

This pattern would ignore a number of the GRIB2 products in the datastream 
namely
those that did not end in (Pa).  A better pattern to use is:

CONDUIT 
gefs\........./../....../(.....).*grib2/ncep/SPEC62MRF/#000/(........)(....)F(...)/([^/]+)/([^
 ]+)
        FILE    -close
        
data/CONDUIT/NCEP/GFS/gribs/Global_1p0deg_Ensemble/\2_\3/\5_\6_\1_\2_\3_\4.grib2

The last couple of extended regular expression patterns say:

[^/]+   ->  one or more occurrences of anything _except_ a '/'
[^ ]+   ->  one or more occurrences of anything _except_ a ' ' (space)

I made the appropriate modifications to both of the actions that needed changing
(even though the one that writes individual files was commented out) and
forced your LDM to use the updated pqact.thredds.tasc_conduit entries.

I also made a slight modification to your entry for the Grid #212
(SREF_CONUS_40km) fields (same sort of thing as above).

Please verify that newly written GEFS output files can be correctly and
fully read by 'wgrib2'.


Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: WFA-619667
Department: Support IDD
Priority: Normal
Status: Closed