This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
"James R. Frysinger" wrote: > > Good morning! > > When it rains it pours. I seem to have solved some of the problems that > were keeping data from being decoded and stored on our local computer, > weather.cofc.edu. Now, the user hard drive partition keeps filling up -- > even after an "ldmadmin stop". It seems that stopping ldm doesn't stop > data ingestion! > > The pqact.conf file has the standard WMO data line that was included in > the distribution. I'm planning on modifying it to limit the WMO data > input, probably by specifying only "US" for the 3rd and 4th characters > (and as appropriate for Canada, the Gulf, etc.). I expect that at first, > at least, North American area data will be of the greatest interest to > our users. More could be added later in additional lines, I suppose. > > The pqact file also has the standard MCIDAS request lines, cut and > pasted from the sample web page specified in the ldm manual. I > installed the ldm-mcidas decoder and provided a symbolic link to its bin > address in the decoder section of ldm. That stopped the problem of > getting frequent error messages about pnga2area not existing. I'll do > the same with SATANNOT and SATBAND in the etc directory of ldm-mcidas > to clear the errors I'm getting on those. (Or just copy them into the > decoder section.) Fortunately, your archived emails have been very > educational --- to the point where I'm probably dangerous, as indicated > by my shrinking free disk space. ;-) > > On Saturday, after spending some time doing the tutorial in the MCIDAS > Learning Guide on using real time data (during which I downloaded, > saved, and executed the RTSERVER.BAT file) I discovered this problem > with the user partition on the hard disk of weather.cofc.edu filling up > completely! I stopped ldm with "ldmadmin stop", revised my scour.conf > file to add a line > ~ldm/data 1 *.wmo > to keep my .wmo files to a minimum and I managed to recover about a > gigabyte of space by manually initating scour. It's filling back up, > though, even though ldm is supposedly not running! Also, this makes the > scour routine hard to complete in a 20 minute window! > On Mon, 15 Jan 2001, James R. Frysinger wrote: > Good morning! .... > though, even though ldm is supposedly not running! Also, this makes the > scour routine hard to complete in a 20 minute window! /***brain spasm--->> scour is on a 3 h cycle, not 20 min.***/ > Can you give me some ideas on what else to look for to control this > "in-leak" of data? Is MCIDAS bringing in data via ADDE on its own? > (That's not what I glean from the manual, though.) How can I control my > disk usage better? What is a reasonable size for the data storage needs > of WMO and MCIDAS data? Any comments on my actions and intentions above? > > Another thought... Right now, my WMO data is going into > /export/home/mcdata/ldm/ > and I also have the following directories in there: > /export/home/mcdata/ldm/campus/ > /export/home/mcdata/ldm/decoded/ > /export/home/mcdata/ldm/forecasts/ > /export/home/mcdata/ldm/gempak/ > /export/home/mcdata/ldm/ldm/ > /export/home/mcdata/ldm/logs/ > /export/home/mcdata/ldm/mcidas/ > /export/home/mcdata/ldm/severe/ > /export/home/mcdata/ldm/surface/ > /export/home/mcdata/ldm/upperair/ > /export/home/mcdata/ldm/warnings/ > as well as my ldm.pq (500 MB). Most of those directories are empty > except for the appropriate .scour file. Any suggestions on a better way > to organize my data directories would be appreciated. > > I've added my sysadmin-ldm address above, since that's the easiest > account to read when I'm on the console locally. Please include that as > well as my regular (frysingerj) address. > > Don't you hate Mondays? Sorry.... > > Jim > > -- > James R. Frysinger University/College of Charleston > 10 Captiva Row Dept. of Physics and Astronomy > Charleston, SC 29407 66 George Street > 843.225.0805 Charleston, SC 29424 > http://www.cofc.edu/~frysingj address@hidden > Cert. Adv. Metrication Specialist 843.953.7644 "James R. Frysinger" wrote: > > Partial answer?... When I execute "ldmadmin stop" it appears that the > crontabbed ldmfail kicks in to restart rpc.ldmd, which of course sires > pqact. This sounds like a "shoot yourself in the foot" configuration. > Could it be that I have things set up wrong? I thought ldmadmin worked > off of ldmping and not on whether data was coming in or not. > > Jim > Morning, Jim! So now you have TOO MUCH data!!! :-) First, you should always confirm that the ldm has actually stopped when you do 'ldmadmin stop'. Do 'ldmadmin ps' and also do 'ps -ef | grep ldm' to verify that all the ldm processes have died. But, keep in mind that after stopping the ldm it is very common for ldm processes to hang around for a while. If an rpc.ldmd process is writing to the queue or transmitting a product, it will finish its task then die gracefully. Similar for pqact. If a process is working on a large product or if there's a slow connection this could take a while. For this reason we tell people to always do 'ldmadmin ps' and 'ps -ef | grep ldm' before restarting the ldm. If you restart the ldm but still have some old processes hanging around from a previous invocation it could get into a confused state. This makes me wonder if you have some rogue processes left over from previous invocations of the ldm. 'ps -ef | grep ldm' will give you the parent PID of each process. If you see more than one parent for your processes then you have rogue processes that probably should be assasinated. pqact is the program that files products to the disk and is thus part of this problem you're having. If somehow pqact is still running it will continue processing everything in the queue until all products have been processed, then it will idle until more products become available. Could you have a rogue pqact process running? Still, there would need to be a rpc.ldmd process running to be putting more products into the queue. (FYI, if you must annihilate an rpc.ldmd process, thus not allowing it to die gracefully, you may corrupt your product queue. In this case I would recommend deleting the queue and remaking it. The cost is having to retrieve the past hour's worth of data, which is usually bearable.) Anyway, in summary, the ldm should stop when you tell it to. You can and should confirm that all processes are stopped after you stop the ldm. This should stop all requests for data, and also stop pqact so that no products are being written to the disk. I'm sorry but I can't answer McIDAS questions for you. Our McIDAS person is at AMS this week, but if you send your McIDAS questions to support@unidata, someone will try to help you out. Having said this, I don't think that the ADDE server will be retrieving data unless you iniate that operation. Also, it is the case that if ldmfail is invoked while the ldm is down it will attempt to restart it. But, its invocation is not linked to 'ldmadmin stop'. If you've scheduled ldmfail to run every 20 minutes and your ldm is down longer than that, then ldmfail will be invoked and try to restart it. Perhaps you should comment out that line in your crontab until you get this other stuff straightened out. Hope this helps! Anne -- *************************************************** Anne Wilson UCAR Unidata Program address@hidden P.O. Box 3000 Boulder, CO 80307 ---------------------------------------------------- Unidata WWW server http://www.unidata.ucar.edu/ ****************************************************