This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Justin, > You have a good memory :-) . Yes, you helped our ESA with a similar > problem on a "virtual" machine a month or so ago and I guess you all > arrived at that conclusion...that there is a problem with virtualization. > > Anyhow, I can't answer whether these "cloud" systems are real or > virtual. Supposedly, these servers use pooled resources for disk space, > processor cores, memory!!!, etc., and therefore, can be increased or > decreased as needed. Sounds good in writing...not sure how good it > actually works and therein may be the problem. > > However, this may not be a problem after all. I misspoke somewhat in my > email in that I just assumed that as the memory utilization by > ldmd/pqact increased, eventually the machine would crash, because that > what the "virtual" machine did. However, after composing that email > over the weekend, we decided to test the theory and actually send data > to it to see what would happen. We are currently ingesting several > NEXRAD3 sites from Vortex, some NEXRAD2 sites from Texas Tech and the > entire text feed from Vortex. Anyhow, when free memory gets very low > (8-16 MB), the memory increases stop and the machine continues to run > fine (at least that's what we have seen over the last several hours). > The machine has used some swap space, but very little. > > Feel free to pop in and look around, but maybe I was wasting your time > and things will operate normally?? I logged onto the system in question and looked around but didn't see anything untoward. (Didn't look very long, though.) You should know that Linux systems allow processes to accumulate memory up to the limit of physical memory before resorting to use of swap space. Also, LDM processes dynamically allocate memory for the data-product buffer based on to the largest data-product seen, so you should expect memory usage to increase until the largest data-product is seen. These two factors might be the basis for your "memory leak". I modified your crontab(1) table: I added entries for accumulating LDM and system metrics and for purging the metrics files. I suggest you install the gnuplot(1) package and then use the "ldmadmin plotmetrics" command to visualize the health of your system. > XXXXXX.com > user: XXXXXX > pass: XXXXXX > > Thanks, Steve, always appreciate your insight. > > Justin > > "You can easily judge the character of a man by how he treats those who can > do nothing for him." > -James D. Miles > > Justin Weaver > Meteorologist-in-Charge > National Weather Service > US Government/DOC/NOAA > Lubbock, Texas > 806.745.3916 x222 > weather.gov/lubbock > > Disclaimer: The views expressed in this email are my own and do not > necessarily reflect those of the National Weather Service, NOAA, > Department of Commerce nor the Executive Branch of the Federal > Government. Regards, Steve Emmerson Ticket Details =================== Ticket ID: OJL-403914 Department: Support LDM Priority: Normal Status: Closed