[LDM #YEW-253326]: LDM 6.13.5 dumps core with missing product queue

Gilbert,

The SIGSEGV was received while the LDM program was trying to log the message
"SIGTERM received" at the NOTICE level. Very odd. I've never seen this behavior
before.

If you can, would you please have the debugger go to frame #2 and print the
values of the following variables: tm, now, ident, level, *loc, msglen, and msg.

I'm a little concerned that the bottom of the stack-trace appears to be from
the dynamic linker. Could there be a problem with it, the dynamic libraries, or
their search order?

Why is the product-queue being deleted? Who's doing that?

> I am filing this on behalf of the College of Dupage meteorology program
> 
> As you might remember, COD's NOAAport ingesters simultaneously, randomly
> crash when the ldm.pq file is mysteriously deleted. We don't know how it
> is being deleted, but I do know that both ingesters simultaneously crash
> when the product queue is mysteriously deleted.
> 
> BUT...
> 
> The last time this happened, I enabled core file read/write for
> the chance that this would happen again. And it did the other
> day. Furthermore, inspecting noaaport2.cod.edu (which is behind a
> firewall), I got me a nice core file. Does this help you *at all*
> in figuring out if this is a bug, or on COD?
> 
> gdb /home/ldm/bin/ldmd /home/ldm/core
> 
> GNU gdb (Ubuntu 7.10-1ubuntu2) 7.10
> Copyright (C) 2015 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> 
> Reading symbols from /home/ldm/bin/ldmd...done.
> [New LWP 41654]
> [Thread debugging using libthread_db enabled]
> 
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Core was generated by `ldmd -I 0.0.0.0 -P 388 -M 256 -m 3600 -o 3600 -q
> /dev/shm/ldm.pq /home/ldm/etc/'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  ___fprintf_chk (fp=0x0, flag=flag@entry=1,
> format=format@entry=0x7f788f6de800 "%04d%02d%02dT%02d%02d%02d.%06ldZ
> %s[%d] %s %s:%d:%s() %.*s\n") at fprintf_chk.c:30
> 30      fprintf_chk.c: No such file or directory.
> 
> (gdb) where
> 
> #0  ___fprintf_chk (fp=0x0, flag=flag@entry=1,
> format=format@entry=0x7f788f6de800 "%04d%02d%02dT%02d%02d%02d.%06ldZ
> %s[%d] %s %s:%d:%s() %.*s\n") at fprintf_chk.c:30
> #1  0x00007f788f6921f8 in fprintf (
> __fmt=0x7f788f6de800 "%04d%02d%02dT%02d%02d%02d.%06ldZ %s[%d] %s
> %s:%d:%s() %.*s\n", __stream=<optimized out>)
> at /usr/include/x86_64-linux-gnu/bits/stdio2.h:97
> #2  stream_log (dest=0x7f788f9143c0 <dest>, level=<optimized out>,
> loc=<optimized out>, msg=0x2563af0 "SIGTERM received") at
> log2slog.c:204
> #3  0x00007f788f690d20 in flush (level=LOG_LEVEL_NOTICE) at log.c:534
> #4  0x00007f788f691680 in logl_vlog (loc=<optimized out>, level=<optimized
> out>,
> format=<optimized out>, args=args@entry=0x7ffd455b9970) at log.c:777
> #5  0x00007f788f69170f in logl_log (loc=loc@entry=0x7ffd455b9a50,
> level=level@entry=LOG_LEVEL_NOTICE,
> format=format@entry=0x40a91f "SIGTERM received") at log.c:812
> #6  0x0000000000407e40 in signal_handler (sig=<optimized out>) at
> ldmd.c:306
> #7  <signal handler called>
> #8  elf_machine_fixup_plt (map=<optimized out>, t=<optimized out>,
> reloc=<optimized out>, value=140155772755872,
> reloc_addr=0x7f788c75db58 <address@hidden>)
> at ../sysdeps/x86_64/dl-machine.h:211
> #9  _dl_fixup (l=<optimized out>, reloc_arg=<optimized out>)
> at ../elf/dl-runtime.c:148
> #10 0x00007f788f940f30 in _dl_runtime_resolve ()
> at ../sysdeps/x86_64/dl-trampoline.S:87
> #11 0x00007f788c46b8f3 in ?? () from
> /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> ---Type <return> to continue, or q <return> to quit---
> #12 0x00007ffd455ba250 in ?? ()
> #13 0x00007f788f93abc7 in _dl_fini () at dl-fini.c:252
> Backtrace stopped: frame did not save the PC
> (gdb)

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: YEW-253326
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.