Re: 20021114: pqsurf in LDM 5.2

Neil,

Here's some compile hints at:

http://www.unidata.ucar.edu/glimpse/ldm/3185

Hope this helps,
Robb...



On Tue, 19 Nov 2002, Neil R. Smith wrote:

> Robb Kambic wrote:
> > 
> > On Tue, 19 Nov 2002, Neil R. Smith wrote:
> > 
> > > Robb Kambic wrote:
> > > >
> > > > On Mon, 18 Nov 2002, Neil R. Smith wrote:
> > > >
> > > > > Hi Robb,
> > > > > I sent you the log files from interactive pqsurf as requested.
> > > > > Did you receive it? Does it provide any clues? Do I need
> > > > > to provide more info? Thanks, -Neil
> > > > >
> > > > > Robb Kambic wrote:
> > > > > >
> > > > > > On Thu, 14 Nov 2002, Unidata Support wrote:
> > > > > >
> > > > > > >
> > > > > > > ------- Forwarded Message
> > > > > > >
> > > > > > > >To: address@hidden
> > > > > > > >From: "Neil R. Smith" <address@hidden>
> > > > > > > >Subject: pqsurf crashing in ldm 5.2
> > > > > > > >Organization: UCAR/Unidata
> > > > > > > >Keywords: 200211142352.gAENq6L05743
> > > > > > >
> > > > > > > Our pqsurf is crashing in our recently upgraded ldm 5.2 (from
> > > > > > > 5.0.6). We are useing the same pqsurf.conf file.
> > > > > > >
> > > > > > > Here are the ldmd.conf errors:
> > > > > > > Nov 14 23:02:01 5Q:coriolis rpc.ldmd[188942]: Starting Up (built: 
> > > > > > > Nov 13
> > > > > > > 2002 21:54:31)
> > > > > > > Nov 14 23:02:01 5Q:coriolis pqbinstats[162465]: Starting Up 
> > > > > > > (188942)
> > > > > > > Nov 14 23:02:01 5Q:coriolis pqexpire[187333]: Starting Up
> > > > > > > Nov 14 23:02:01 5Q:coriolis pqact[181011]: Starting Up
> > > > > > > Nov 14 23:02:01 5Q:coriolis pqsurf[169954]: Starting Up (188942)
> > > > > > > Nov 14 23:02:01 5Q:coriolis pqact[187141]: Starting Up
> > > > > > > .
> > > > > >
> > > > > > Neil,
> > > > > >
> > > > > > A couple things to do is to make a few startup changes:
> > > > > >
> > > > > > -Comment out the pqexpire line with a "#" in the etc/ldmd.conf 
> > > > > > file. That
> > > > > > process is not needed any more and it might be causing some 
> > > > > > problems.
> > > > > > - remake the ldmqueue and pqsurfqueue. ie
> > > > > >
> > > > > > % ldmadmin stop
> > > > > > % ldmadmin delqueue
> > > > > > % ldmadmin mkqueue
> > > > > > % ldmadmin delsurfqueue
> > > > > > % ldmadmin mlsurfqueue
> > > > > > % ldmadmin start
> > > > > >
> > > > > > > . (no intervening proc.#169954 messages)
> > > > > > > .
> > > > > > > Nov 14 23:03:33 3Q:coriolis pqsurf[169954]: surface_split: Can't 
> > > > > > > handle
> > > > > > > MESSAGE_TYPE_UNKNOWN
> > > > > > > Nov 14 23:03:33 3Q:coriolis pqsurf[169954]: HDR + PIL: SXHI
> > > > > > > Nov 14 23:03:34 3Q:coriolis pqsurf[169954]: surface_split: Can't 
> > > > > > > handle
> > > > > > > MESSAGE_TYPE_UNKNOWN
> > > > > > > Nov 14 23:03:34 3Q:coriolis pqsurf[169954]: HDR + PIL: SXHI
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: child 187141 
> > > > > > > terminated by
> > > > > > > signal 10
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Exiting
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]:   Queue usage (bytes):
> > > > > > > 72784
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]:            (nregions):
> > > > > > > 377
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of products 27
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of 
> > > > > > > observations 135
> > > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of dups 2
> > > > > > >
> > > > > >
> > > > > > - another idea is to run the pqsurf in verbose mode from the 
> > > > > > command line.
> > > > > >
> > > > > > - comment out pqsurf in the etc/ldmd.conf file
> > > > > > - run pqsurf from command line as user ldm
> > > > > >
> > > > > > % pqsurf -v -l -
> > > > > >
> > > > > > This send output to the terminal.  If you replace -l - with -l log 
> > > > > > it will
> > > > > > put the output in file log
> > > > > >
> > > > > >
> > > > > > See comment below first
> > > > > >
> > > > > > > Here is the ldmd.conf entry:
> > > > > > > exec    "pqsurf -p ^S[APX] -Q /data/pqsurf.pq 
> > > > > > > /unidata/ldm/etc/pqsurf.conf"
> > > > > >
> > > > > > I would change the above line to
> > > > > >
> > > > > > exec    "pqsurf -p ^S(A....|P....|XUS8.|XUS91) -q /data/pqsurf.pq
> > > > > > /unidata/ldm/etc/pqsurf.conf"
> > > > > >
> > > > > > You might be getting some radar products or some other products 
> > > > > > wrongly
> > > > > > causing pqsurf to crash.
> > > > > >
> > > > > > If none of the above works, send me info about your machine and the 
> > > > > > pqsurf
> > > > > > log file from the command line.
> > > > > >
> > > > > > Robb...
> > > > > >
> > > > > > >
> > > > > > > Here are the pqsurf.conf entries:
> > > > > > > WMO     ^sao .. (...) ([0-3][0-9])([0-2][0-9])
> > > > > > >         DBFILE  /data/ddplus/db_sao/(\2:yy)(\2:mm)\2\3.SA_DB \1
> > > > > > > WMO     ^sao ... (...) ([0-3][0-9])([0-2][0-9])
> > > > > > >         DBFILE  /data/ddplus/db_sao/(\2:yy)(\2:mm)\2\3.SA_DB \1
> > > > > > > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> > > > > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB 
> > > > > > > \1
> > > > > > > WMO     ^speci (....) ([0-3][0-9])([0-2][0-9])
> > > > > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB 
> > > > > > > \1
> > > > > > >
> > > > > > > Is any of the above config. antiquated and causing the
> > > > > > > problem?
> > > >
> > > > Neil,
> > > >
> > > > We had a power outage today, it's a pita to get back your environment 
> > > > etc.
> > > > Anyhow, I look at the logs today. It seems the problem isn't with pqsurf
> > > > per se, it's the pqact running on the pqsurf.conf that exited.  Since
> > > > pqact usuallly doesn't fail unless a entry in the file causes a problem.
> > > > I notice all your entires are DBFILE ones.  I wondering if you could
> > > > comment those out with a '#' and make a entry that writes to a file?  
> > > > This
> > > > way we could determine if it's the DBFILE entries or not.  How about 
> > > > using
> > > > a entry like this:
> > > >
> > > > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> > > >         file    /data/ddplus/(\2:yy)(\2:mm)\2\3.wmo
> > 
> > Neil,
> > 
> > My mistake, file action must be uppercase, ie FILE.
> > 
> > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> >         FILE    /data/ddplus/(\2:yy)(\2:mm)\2\3.wmo
> > 
> > The metars products arrive at a rate of 1 a sec. You can scan
> > the queue looking for product via pqcat, ie
> > 
> > % pqcat -vl -  -q <yourPQSURFqueue>  > /dev/null
> > 
> > For more info on pqcat look at:
> > http://www.unidata.ucar.edu/cgi-bin/man-cgi?pqcat+1
> > 
> > Robb...
> > > >
> > > > This will file all the metars in a file like:  /data/ddplus/02111812.wmo
> > > > Rememeber the white spaces needs to be tabs.
> > > >
> > > > Let me know what you find out.
> > > >
> > > > Robb...
> > >
> > > Hi Robb,
> > > Nope. That didn't produce any file output. Is there a diagnostic
> > > I can look at or generate for analysis? Also, what is the frequency
> > > of these products so I can know when to expect them in the queue?
> > > Thanks, -Neil
> 
> OK. The telltail seems to be in your second bit of info - how to 
> look at the queue with pqcat. When I do
> coriolis{ldm}52%  pqcat -vl -  -q /data/surf.pq > /dev/null
> Nov 20 00:43:57 pqcat: Starting Up (28721)
> Nov 20 00:43:57 pqcat: pq_open failed: /data/surf.pq: No such file or
> directory
> Nov 20 00:43:57 pqcat: Exiting
> Nov 20 00:43:57 pqcat: Number of products 0
> 
> Looks like an empty queue. To review the ldmd.conf entry:
> 
> exec    "pqsurf -p ^S(A....|P....|XUS8.|XUS91) -q /data/pqsurf.pq
> /unidata/ldm/etc/pqsurf.conf"
> which I copied from your email ... which cited pqsurf.pq with -q instead
> of -Q. 
> OK. My fault. Well, correcting the q/Q issue, I get a pqsurf.pq with 
> products I can see qith pqcat. With 'file' changed to 'FILE' in the 
> recommended step in pqsurf.conf, I do indeed get the files created in
> the
> expected directory. 
> 
> Now, is this a dbm issue? What did I need to pay attention to during
> ldm package build?
> -Neil
> -- 
> Neil R. Smith, Comp. Sys. Mngr.               address@hidden
> Dept. Atmospheric Sci., Texas A&M Univ.       979/845-6272 FAX:979/862-4466
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================