Re: pqsurf.conf

On Mon, 29 Nov 1999 address@hidden wrote:

> 
> 
> I have spent a frustrating two days trying to come up with a work around.
> I have two possibilities:
> 
> 1) file the products as they come in a file more like what pqsurf would be
> expecting and then re-queue those files into the ldm.pq.  I can store the
> files with a filename of: NWSTG TEXT SAUS70 KDEN 281536 1999333154000765
> Getting that inserted into the queue is the next trick and I have succeeded
> at that by using pqinsert as follows:
>     pqinsert -f WMO 'NWSTG TEXT SAUS70 KDEN 281536 1999333154000765'
> pqsurf still balks with Can't handle MESSAGE_TYPE_UNKNOWN.  I have also

Alan,

That's a good idea to re-queue the products but you got the header wrong,
it should start with the "SAUS70 KDEN 281536 1999333154000765".  Here's
the output from ldmadmin watch on the products:

Nov 29 20:48:02 pqutil:     1271 19991129204801.533 IDS|DDPLUS 420  SRUS58
KWBC 292044 /pRRSMT
Nov 29 20:48:02 pqutil:      147 19991129204801.542 IDS|DDPLUS 421  SRUS51
KWBC 292044 /pRRSHOU
Nov 29 20:48:02 pqutil:      826 19991129204801.553 IDS|DDPLUS 422  SRUS57
KWBC 292044 /pRRSMS
Nov 29 20:48:02 pqutil:      197 19991129204801.558 IDS|DDPLUS 423  SRUS60
KWBC 292044 /pRRSMHX

I would also check surf_split.c to see what the header it's expecting.
I think the routine to look at is: pqing/wmo_header.c:get_wmo_header

I think this will work with a little more investigation.


Robb...




> attempted to use the filename with the last 16 character date/time stamp
> omitted at the end still get the same results.
> I can get this work if I type in the pqinsert command manually, but if I
> put into pqact.conf I get 'child exited with status 127' and I can't figure
> out what that means and the product never gets inserted.
> 
> 2) modify the ingest code to insert it the right way from the begining.  I
> am still investigating this, but with the tests from above I can't get
> pqsurf to work even if I do have it inserted into the queue as it is
> expecting it.
> 
> If I could just get pqsurf to work so I can see what it's going to do, I
> could make all the code changes needed.   I am completely blind about what
> pqsurf is expecting, what it will do, what the pqsurf.conf file should look
> like, what the 'standard WMO type of header' is supposed to look like?
> 
> As you can see I'm getting very frustrated.
> 
> Alan.
> 
> 
> 
> 
> 
> 
> Robb Kambic <address@hidden> on 11/23/99 03:36:30 PM
>                                                               
>                                                               
>                                                               
>  To:      Alan Hall/NCDC                                      
>                                                               
>  cc:      support-ldm <address@hidden>          
>                                                               
>                                                               
>                                                               
>  Subject: Re: pqsurf.conf                                     
>                                                               
> 
> 
> 
> 
> 
> 
> 
> 
> Alan,
> 
> I looked into the pqsurf code, here's the scoop. The pqsurf code is
> looking for the standard WMO type of header before it can split the
> bulletin into individual reports. There are 2 solutions:
> 
> - modify your ingest code to produce standard WMO type headers, ie
> eliminate the NOAAPORT prefix and change the "." to spaces.
> 
> - modify the surf_split.c code in the LDM pqsurf directory to accept your
> current type headers.  At this time, I have many other top priority
> projects so I could not get to this for months. Maybe you have the
> resources to do the modifications.
> 
> I know this is not what you wanted to hear but that's about it in a
> nutshell. Maybe Leslie could help you out.
> 
> Robb...
> 
> On Mon, 22 Nov 1999 address@hidden wrote:
> 
> >
> >
> > I think we are making progress.  I'm getting some entries in the ldmd.log
> > that tell me we are getting things to pqsurf.  Here is a sample of the
> log:
> >
> > Nov 22 13:30:15 npfos pqsurf[9283]: Starting Up (9279)
> > Nov 22 13:30:15 npfos pqsurf[9283]: lastmatch:
> > 6453cb112c44cf491c21769dee1f4138      987 19991122133008.749     WMO 000
> > NOAAPORT.NW
> > Nov 22 13:30:15 npfos pqsurf[9283]: 19991122123015.215 TS_ENDT {{WMO,
> > "^NOAAPORT\.NWSTG\.TEXT\.S[AIMNP]"}}
> > Nov 22 13:30:15 npfos pqact[9284]: Starting Up
> > Nov 22 13:30:15 npfos pqact[9284]: Successfully read pattern file
> > "/usr/local/ldm/etc/pqsurf.conf"
> > Nov 22 13:30:15 npfos pqact[9284]: TS_ZERO TS_ENDT {{WMO,  ".*"}}
> > Nov 22 13:30:15 npfos pqact[9284]:        0 19991122133015.238     ANY
> 000
> >   _BEGIN_
> > Nov 22 13:30:17 npfos localhost[9291]: Connection from localhost
> > Nov 22 13:30:17 npfos localhost[9291]: Connection reset by peer
> > Nov 22 13:30:17 npfos localhost[9291]: Exiting
> > Nov 22 13:30:30 npfos pqsurf[9283]:      103 19991122133016.828     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPCN51.CWAO.221327.1999326133016827
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      121 19991122133016.830     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPCN31.CWAO.221326.1999326133016830
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      139 19991122133016.832     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPCN41.CWAO.221327.1999326133016832
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      111 19991122133016.841     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPUS43.KWBC.221328.1999326133016840
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      586 19991122133016.843     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPUS70.KWBC.221328.1999326133016843
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      144 19991122133016.845     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPXX60.KWBC.221328.1999326133016844
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      115 19991122133019.927     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SAPL61.KRAK.221330.1999326133019926
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      545 19991122133024.005     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SNVB20.RJTD.221300.1999326133024005
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      107 19991122133026.080     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SPCN48.CWAO.221326.1999326133026080
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> > Nov 22 13:30:30 npfos pqsurf[9283]:      133 19991122133026.083     WMO
> 000
> >   NOAAPORT.NWSTG.TEXT.SACN87.CWAO.221325.1999326133026083
> > Nov 22 13:30:30 npfos pqsurf[9283]: surface_split: Can't handle
> > MESSAGE_TYPE_UNKNOWN
> >
> > What is MESSAGE_TYPE_UNKNOWN and how can I correct it?
> >
> >
> >
> >
> >
> > Robb Kambic <address@hidden> on 11/19/99 04:11:06 PM
> >
> >
> >
> >  To:      Alan Hall/NCDC
> >
> >  cc:      support-ldm <address@hidden>
> >
> >
> >
> >  Subject: Re: pqsurf.conf
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Fri, 19 Nov 1999 address@hidden wrote:
> >
> > >
> > >
> > > I have the following entry in my pqsurf.conf file (this very line works
> > > perfectly in pqact.conf):
> > >
> > > WMO
> > >
> >
> ^NOAAPORT\.NWSTG\.(TEXT)\.([A-Z]...).[0-9]\.[A-Z]..[A-Z]|[0-9]\.([0-3][0-9]
> > > )....\.
> > >      FILE -strip
> > >      /npraid/noaaport/data/NWSTG/\1/SURF/\2(\3:yyyy)(\3:mmm)(\3:dd)
> >
> > Alan,
> >
> > I reread the man page, it seems there are flags to set the feedtype and
> > pattern. use the following flags on the pqsuf line in ldmd.conf.
> >
> > exec pqsurf -f WMO -p "^NOAAPORT\.NWSTG\.(TEXT)\.S[AIMNP]"
> >
> > Robb...
> >
> >
> >
> > >
> > > tabs separate all the fields.  I can't get anything from pqsurf.  I am
> > > completely lost on what this is supposed to do...
> > > What should an entry in pqsurf look like?
> > >
> > > Alan.
> > >
> > >
> >
> >
> ===========================================================================
> > ====
> > Robb Kambic                      Unidata Program Center
> > Software Engineer III                 Univ. Corp for Atmospheric Research
> > address@hidden         WWW: http://www.unidata.ucar.edu/
> >
> ===========================================================================
> > ====
> >
> >
> >
> >
> 
> ===========================================================================
> ====
> Robb Kambic                      Unidata Program Center
> Software Engineer III                 Univ. Corp for Atmospheric Research
> address@hidden         WWW: http://www.unidata.ucar.edu/
> ===========================================================================
> ====
> 
> 
> 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================