Re: 20021114: pqsurf in LDM 5.2

On Tue, 19 Nov 2002, Neil R. Smith wrote:

> Robb Kambic wrote:
> > 
> > On Mon, 18 Nov 2002, Neil R. Smith wrote:
> > 
> > > Hi Robb,
> > > I sent you the log files from interactive pqsurf as requested.
> > > Did you receive it? Does it provide any clues? Do I need
> > > to provide more info? Thanks, -Neil
> > >
> > > Robb Kambic wrote:
> > > >
> > > > On Thu, 14 Nov 2002, Unidata Support wrote:
> > > >
> > > > >
> > > > > ------- Forwarded Message
> > > > >
> > > > > >To: address@hidden
> > > > > >From: "Neil R. Smith" <address@hidden>
> > > > > >Subject: pqsurf crashing in ldm 5.2
> > > > > >Organization: UCAR/Unidata
> > > > > >Keywords: 200211142352.gAENq6L05743
> > > > >
> > > > > Our pqsurf is crashing in our recently upgraded ldm 5.2 (from
> > > > > 5.0.6). We are useing the same pqsurf.conf file.
> > > > >
> > > > > Here are the ldmd.conf errors:
> > > > > Nov 14 23:02:01 5Q:coriolis rpc.ldmd[188942]: Starting Up (built: Nov 
> > > > > 13
> > > > > 2002 21:54:31)
> > > > > Nov 14 23:02:01 5Q:coriolis pqbinstats[162465]: Starting Up (188942)
> > > > > Nov 14 23:02:01 5Q:coriolis pqexpire[187333]: Starting Up
> > > > > Nov 14 23:02:01 5Q:coriolis pqact[181011]: Starting Up
> > > > > Nov 14 23:02:01 5Q:coriolis pqsurf[169954]: Starting Up (188942)
> > > > > Nov 14 23:02:01 5Q:coriolis pqact[187141]: Starting Up
> > > > > .
> > > >
> > > > Neil,
> > > >
> > > > A couple things to do is to make a few startup changes:
> > > >
> > > > -Comment out the pqexpire line with a "#" in the etc/ldmd.conf file. 
> > > > That
> > > > process is not needed any more and it might be causing some problems.
> > > > - remake the ldmqueue and pqsurfqueue. ie
> > > >
> > > > % ldmadmin stop
> > > > % ldmadmin delqueue
> > > > % ldmadmin mkqueue
> > > > % ldmadmin delsurfqueue
> > > > % ldmadmin mlsurfqueue
> > > > % ldmadmin start
> > > >
> > > > > . (no intervening proc.#169954 messages)
> > > > > .
> > > > > Nov 14 23:03:33 3Q:coriolis pqsurf[169954]: surface_split: Can't 
> > > > > handle
> > > > > MESSAGE_TYPE_UNKNOWN
> > > > > Nov 14 23:03:33 3Q:coriolis pqsurf[169954]: HDR + PIL: SXHI
> > > > > Nov 14 23:03:34 3Q:coriolis pqsurf[169954]: surface_split: Can't 
> > > > > handle
> > > > > MESSAGE_TYPE_UNKNOWN
> > > > > Nov 14 23:03:34 3Q:coriolis pqsurf[169954]: HDR + PIL: SXHI
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: child 187141 terminated by
> > > > > signal 10
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Exiting
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]:   Queue usage (bytes):
> > > > > 72784
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]:            (nregions):
> > > > > 377
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of products 27
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of observations 135
> > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of dups 2
> > > > >
> > > >
> > > > - another idea is to run the pqsurf in verbose mode from the command 
> > > > line.
> > > >
> > > > - comment out pqsurf in the etc/ldmd.conf file
> > > > - run pqsurf from command line as user ldm
> > > >
> > > > % pqsurf -v -l -
> > > >
> > > > This send output to the terminal.  If you replace -l - with -l log it 
> > > > will
> > > > put the output in file log
> > > >
> > > >
> > > > See comment below first
> > > >
> > > > > Here is the ldmd.conf entry:
> > > > > exec    "pqsurf -p ^S[APX] -Q /data/pqsurf.pq 
> > > > > /unidata/ldm/etc/pqsurf.conf"
> > > >
> > > > I would change the above line to
> > > >
> > > > exec    "pqsurf -p ^S(A....|P....|XUS8.|XUS91) -q /data/pqsurf.pq
> > > > /unidata/ldm/etc/pqsurf.conf"
> > > >
> > > > You might be getting some radar products or some other products wrongly
> > > > causing pqsurf to crash.
> > > >
> > > > If none of the above works, send me info about your machine and the 
> > > > pqsurf
> > > > log file from the command line.
> > > >
> > > > Robb...
> > > >
> > > > >
> > > > > Here are the pqsurf.conf entries:
> > > > > WMO     ^sao .. (...) ([0-3][0-9])([0-2][0-9])
> > > > >         DBFILE  /data/ddplus/db_sao/(\2:yy)(\2:mm)\2\3.SA_DB \1
> > > > > WMO     ^sao ... (...) ([0-3][0-9])([0-2][0-9])
> > > > >         DBFILE  /data/ddplus/db_sao/(\2:yy)(\2:mm)\2\3.SA_DB \1
> > > > > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> > > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB \1
> > > > > WMO     ^speci (....) ([0-3][0-9])([0-2][0-9])
> > > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB \1
> > > > >
> > > > > Is any of the above config. antiquated and causing the
> > > > > problem?
> > 
> > Neil,
> > 
> > We had a power outage today, it's a pita to get back your environment etc.
> > Anyhow, I look at the logs today. It seems the problem isn't with pqsurf
> > per se, it's the pqact running on the pqsurf.conf that exited.  Since
> > pqact usuallly doesn't fail unless a entry in the file causes a problem.
> > I notice all your entires are DBFILE ones.  I wondering if you could
> > comment those out with a '#' and make a entry that writes to a file?  This
> > way we could determine if it's the DBFILE entries or not.  How about using
> > a entry like this:
> > 
> > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> >         file    /data/ddplus/(\2:yy)(\2:mm)\2\3.wmo

Neil,

My mistake, file action must be uppercase, ie FILE.

WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
        FILE    /data/ddplus/(\2:yy)(\2:mm)\2\3.wmo

The metars products arrive at a rate of 1 a sec. You can scan
the queue looking for product via pqcat, ie 

% pqcat -vl -  -q <yourPQSURFqueue>  > /dev/null

For more info on pqcat look at:
http://www.unidata.ucar.edu/cgi-bin/man-cgi?pqcat+1

Robb...
> > 
> > This will file all the metars in a file like:  /data/ddplus/02111812.wmo
> > Rememeber the white spaces needs to be tabs.
> > 
> > Let me know what you find out.
> > 
> > Robb...
> 
> Hi Robb,
> Nope. That didn't produce any file output. Is there a diagnostic
> I can look at or generate for analysis? Also, what is the frequency
> of these products so I can know when to expect them in the queue?
> Thanks, -Neil
> -- 
> Neil R. Smith, Comp. Sys. Mngr.               address@hidden
> Dept. Atmospheric Sci., Texas A&M Univ.       979/845-6272 FAX:979/862-4466
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================