[IDD #WFA-619667]: Request for CONDUIT feed

Hi Randy,

Sorry for the silence yesterday; there were too many things going on that
prevented me from responding...

re:
> Martha and I changed the pqact file according to some documentation online we
> found on how to file each grib message separately. We also added the SREF 
> since
> the 12 GEFS had already come in. Unfortunately this syntax did not work
> correctly (see below) as it created a single file called:
> 
> 20090223

The output file name specification you included in your 
pqact.thredds.tasc_conduit
pattern-action file:

   FILE  /data/conduit/\1

only specifies the use of one field from the product ID, and this one
field is the date.  If you want to write out each GRIB2 message uniquely,
you need to form an output file name that is unique for each product.

Here is some (old) 'notifyme' output I saved on my workstation for GEFS
data from CONDUIT:

 ...
Oct 24 18:22:49 notifyme[24123] INFO:    52582 20081024164116.711 CONDUIT 004  
data/nccf/com/gens/prod/gefs.20081024/12/pgrb2/gep12.t12z.pgrb2aanl 
!grib2/ncep/SPEC62MRF/#000/200810241200F000/HGHT/250 Pa PRES! 000004
Oct 24 18:22:49 notifyme[24123] INFO:    21404 20081024164116.711 CONDUIT 005  
data/nccf/com/gens/prod/gefs.20081024/12/pgrb2/gep12.t12z.pgrb2aanl 
!grib2/ncep/SPEC62MRF/#000/200810241200F000/TMPK/250 Pa PRES! 000005
Oct 24 18:22:49 notifyme[24123] INFO:    31950 20081024164116.719 CONDUIT 014  
data/nccf/com/gens/prod/gefs.20081024/12/pgrb2/gep12.t12z.pgrb2aanl 
!grib2/ncep/SPEC62MRF/#000/200810241200F000/RELH/700 Pa PRES! 000014
Oct 24 18:22:49 notifyme[24123] INFO:    52652 20081024164116.730 CONDUIT 004  
data/nccf/com/gens/prod/gefs.20081024/12/pgrb2/gep11.t12z.pgrb2aanl 
!grib2/ncep/SPEC62MRF/#000/200810241200F000/HGHT/250 Pa PRES! 000004
Oct 24 18:22:49 notifyme[24123] INFO:    51611 20081024164116.757 CONDUIT 000  
data/nccf/com/gens/prod/gefs.20081024/12/pgrb2/gep11.t12z.pgrb2aanl 
!grib2/ncep/SPEC62MRF/#000/200810241200F000/HGHT/200 Pa PRES! 000000
Oct 24 18:22:49 notifyme[24123] INFO:   119365 20081024164116.730 CONDUIT 015  
data/nccf/com/gens/prod/gefs.20081024/12/pgrb2/gep12.t12z.pgrb2aanl 
!grib2/ncep/SPEC62MRF/#000/200810241200F000/UREL;VREL/700 Pa PRES! 000015
 ...

Notice that the the portion of the product ID in the section that starts with 
!grib2/ncep/...
contains information that will allow one to write uniquely named, individual 
GRIB2
messages:

 ...
!grib2/ncep/SPEC62MRF/#000/200810241200F000/HGHT/250 Pa PRES! 000004
!grib2/ncep/SPEC62MRF/#000/200810241200F000/TMPK/250 Pa PRES! 000005
!grib2/ncep/SPEC62MRF/#000/200810241200F000/RELH/700 Pa PRES! 000014
!grib2/ncep/SPEC62MRF/#000/200810241200F000/HGHT/250 Pa PRES! 000004
!grib2/ncep/SPEC62MRF/#000/200810241200F000/HGHT/200 Pa PRES! 000000
!grib2/ncep/SPEC62MRF/#000/200810241200F000/UREL;VREL/700 Pa PRES! 000015
 ...

Here is one idea for an entry in your pqact.thredds.tasc_conduit pattern-action
file that will use this information to write out uniquely named, individual
GRIB2 messages:

CONDUIT gefs.*grib2/ncep/SPEC62MRF/#000/(........)(....)F(...)/(.*)/(.*) (Pa)
    FILE    -close /data/conduit/GEFS/\4_\5\6_\1_\2_\3.grib2

NB:

- this is only one idea for the output naming; there can be LOTS more!

- as always, be careful to use tabs where needed:

  - between CONDUIT and gefs in line #1
  - before and after FILE in line #2

A similar pattern will need to be fashioned for the SREF data.

re:
> Incidently the SREF file which was created is also corrupted. Getting the same
> error message. I suspect there still may be some firewall issues messing with
> the data as it comes in. But  your test to file the messages individually will
> help to determine that.

All the SREF data?  Or some of the GRIB2 messages in the output file of
concatenated messages?

> CONDUIT data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
> FILE    -close 
> /data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\2_\100.grib2
> 
> CONDUIT data/nccf/com/gens/prod/gefs\.(........)/(..)/pgrb2a
> FILE    -close /data/conduit/\1
> #
> #
> # Grid #212 SREF_CONUS_40km for ensprod_biasc and pgrb_biasc
> CONDUIT data/nccf/com/sref/prod/sref.(........)/(..)/(.*)/.*212.(.*).grib2
> FILE    
> /data/pub/native/grid/NCEP/SREF/CONUS_40km/\3/SREF_CONUS_40km_\3_\1_\200.grib2
> #
> CONDUIT data/nccf/com/sref/prod/sref.(........)/(..)/(.*)/.*212.(.*).grib2
> FILE    -close /data/conduit/SREF/\1

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: WFA-619667
Department: Support IDD
Priority: Normal
Status: Closed