This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi Martha, What version of Java, Tomcat, and TDS are you running? > I think part of the problem is due to the THREDDS server that we are > running on NOAAPXCD. When I restarted it I reclaimed a lot of disk space. > I did notice that there were a lot of files marked "deleted" when I > ran the UNIX command "lsof" on our data file system. Do you have a list of file that were marked "deleted"? Were they all data files? > The lsof command shows files that are opened by the OS and what process > has the files open; "java" which is attached to THREDDS had a lot of > files in our /data/pub area opened, many of which had been deleted by > a regularly schedule cron job cleanup. I don't know how often THREDDS > refreshes > itself; certainly IDV instructs it to refresh but how it refreshes > locally is something I'll have to research. The TDS keeps information about recently requested datasets in memory. I'm not sure if that includes an open file handle or not. If so, that might cause the deletion to not happen until the TDS releases the file handle. The TDS scours this pool of dataset objects. That scouring is configured in the content/thredds/threddsConfig.xml file in the NetcdfFileCache, NetcdfDatasetCache, and HTTPFileCache elements. On our server we scour all of these every 5 minutes and keep 50-100 in the first two and 5-20 in the third but the settings should depend on how much memory you are running with and maybe your cron scour schedule. The TDS uses the cached dataset information each time a request for that dataset comes in. So an access might cause the TDS to check the file status and release it if it has been deleted. I'm going to ask John, who is more familiar with the caching code, about this one. Ethan > > > Otherwise, this is our setup for file cleanup: > > ldmadmin scour - /data/pub (the area where the files are written > for THREDDS) > > I also noted that in our pqact file for THREDDS, we have not "close" > directive with FILE, just "flush". Do you recommend changing that; I > thought that you had set up "flush" rather than "close" for some reason > when you were helping us with CONDUIT. Ticket Details =================== Ticket ID: PZR-552966 Department: Support THREDDS Priority: Normal Status: Open