[LDM #VVA-689466]: LDM eats 24GB of RAM in less than a day :: Memory Leak?

Phil,

> We recently installed LDM v. 6.9.8 on a new box running RHEL 6.  I noticed
> this afternoon when, after about 4 days of running non-stop, the LDM had
> consumed our entire 24GB of RAM (save about 40 MB)!  Is this expected
> behavior? Our old machine only had 1 GB of RAM, so I never thought much
> of it, but now I think I may have something configured incorrectly.

I think you encountered a "feature" of the Linux kernel: it will use as much 
physical memory as possible to make things faster (mostly for system buffers 
and cached pages).

> I also noticed that when I stopped the LDM, very little of the memory
> was actually released.  After rebooting the machine (without starting
> the LDM), we were using about 1GB of RAM and this remained stable.
> Since I restarted the LDM at around 4PM this afternoon (and this is
> the only user-level process running on this box outside of root), the
> physical RAM usage has slowly but steadily increased to now over 17GB
> (span of about 7.5 hours)  (after the expected initial jump to about
> 5GB as our Product Queue is 4GB)
> 
> Here are the top few processes when I run "top" -- nothing looks even
> remotely excessive.
> 
> top - 00:49:56 up  8:47,  1 user,  load average: 0.01, 0.02, 0.00
> Tasks: 493 total,   1 running, 492 sleeping,   0 stopped,   0 zombie
> Cpu(s):  0.0%us,  0.0%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Mem:  24592160k total, 17379852k used,  7212308k free,   268112k buffers
> Swap: 26836984k total,        0k used, 26836984k free, 14821172k cached
> 
> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 3936 ldm       20   0 3944m 129m 127m S  0.3  0.5   0:15.41 ldmd
> 3940 ldm       20   0 3942m 761m 760m S  0.3  3.2   0:30.77 ldmd

Note that the virtual memory usage of the LDM processes is reasonable given 
that your product-queue is about 4 GB.

> 8366 ldm       20   0 35136 9776 1264 S  0.3  0.0   0:00.30 dcgrib2
> 8379 ldm       20   0 15284 1532  928 R  0.3  0.0   0:00.11 top
> 1 root      20   0 19328 1508 1212 S  0.0  0.0   0:01.49 init
> 2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
> 3 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
> 4 root      20   0     0    0    0 S  0.0  0.0   0:00.05 ksoftirqd/0
> 5 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
> 6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
> 7 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/1
> 8 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/1
> 9 root      20   0     0    0    0 S  0.0  0.0   0:00.19 ksoftirqd/1
> 10 root      RT   0     0    0    0 S  0.0  0.0   0:00.02 watchdog/1
> 11 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/2
> 12 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/2
...

> Thanks in advance for your help or insight on this… We appreciate it…

As long as you don't start using significant amounts of swap space, I wouldn't 
worry.

> -- Phil  Birnie --
> Department of Geography
> The Ohio State University
> (614)519-6176
> 
> Cc. Jim DeGrand, Jens Blegvad

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: VVA-689466
Department: Support LDM
Priority: Normal
Status: Closed