This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
> Greg, > > The memory leak is in the underlying C library, not in the C++ > layer. It only occurs when opening netCDF-4 files, not > equivalent netCDF-3 files, so it might even be in the underlying > HDF5 layer, but that seems unlikely. We'll continue to > investigate ... > > --Russ > > Russ Rew UCAR Unidata Program > address@hidden http://www.unidata.ucar.edu > > Howdy Russ and Greg! I believe that there is no memory leak. I think the problem is that we are misinterpreting the fields of the mallinfo struct. I found some documentation here: http://www.gnu.org/software/libtool/manual/libc/Statistics-of-Malloc.html Here's some relevant parts: int arena This is the total size of memory allocated with sbrk by malloc, in bytes. int uordblks This is the total size of memory occupied by chunks handed out by malloc. int fordblks This is the total size of memory occupied by free (not in use) chunks. When reduced to the simplest test case, your test program came down to reading the same test file twice, which resulted in different arena values after file closes. This seems to imply that memory leaked. But a carefull reading of the fordblks documentation leads me to think that the GNU malloc is doing some last-minute harvesting of memory (i.e. when the program exits). The mallinfo structure shows the following after the two file closes: {arena = 679936, ordblks = 258, smblks = 472, hblks = 0, hblkhd = 0, usmblks = 0, fsmblks = 21232, uordblks = 227344, fordblks = 452592, keepcost = 78960} {arena = 1261568, ordblks = 141, smblks = 28, hblks = 0, hblkhd = 0, usmblks = 0, fsmblks = 1776, uordblks = 243872, fordblks = 1017696, keepcost = 117568} Note that in the second case, the arena is higher, leading us to think there is a leak, but the fordblks is also higher. So the program may have freed the memory properly, but that's not yet reflected in the mallinfo.arena field. Since the program is still running at this point, we have not given the program time to clean up. On the other hand, when I run the program with valgrind I get it's assurance that "no memory leaks are possible." I like that kind of confidence! ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 1) ==20527== malloc/free: in use at exit: 0 bytes in 0 blocks. ==20527== malloc/free: 339,048 allocs, 339,048 frees, 79,875,278 bytes allocated. ==20527== For counts of detected errors, rerun with: -v ==20527== All heap blocks were freed -- no leaks are possible. (I also think this is a crazy amount of mallocs and frees. Is the HDF5 library doing all this? I don't know.) Also, using valgrind seems to turn off the fancy memory management too, because the arena number remains unchanged no matter how many times I open and close the file. That is, valgrind makes this entire problem go away! ;-) So my conclusion is that there is no memory leak here, and what we are seeing with the mallinfo.arena field is an artifact of memory management. Please let me know if this doesn't seem right. Thanks, Ed Ticket Details =================== Ticket ID: ZBS-432143 Department: Support netCDF Priority: Urgent Status: Closed