[netCdfJava #PZR-552966]: Missing file space reappears upon THREDDS restart

Hi Martha,

What version of Java, Tomcat, and TDS are you running?

> I think part of the problem is due to the THREDDS server that we are
> running on NOAAPXCD.  When I restarted it I reclaimed a lot of disk space.
> I did notice that there were a lot of files marked "deleted" when I
> ran the UNIX command "lsof" on our data file system.

Do you have a list of file that were marked "deleted"? Were they all data files?

> The lsof command shows files that are opened by the OS and what process
> has the files open; "java" which is attached to THREDDS had a lot of
> files in our /data/pub area opened, many of which had been deleted by
> a regularly schedule cron job cleanup.  I don't know how often THREDDS 
> refreshes
> itself; certainly IDV instructs it to refresh but how it refreshes
> locally is something I'll have to research.

The TDS keeps information about recently requested datasets in memory. I'm not 
sure if that includes an open file handle or not. If so, that might cause the 
deletion to not happen until the TDS releases the file handle. The TDS scours 
this pool of dataset objects. That scouring is configured in the 
content/thredds/threddsConfig.xml file in the NetcdfFileCache, 
NetcdfDatasetCache, and HTTPFileCache elements. On our server we scour all of 
these every 5 minutes and keep 50-100 in the first two and 5-20 in the third 
but the settings should depend on how much memory you are running with and 
maybe your cron scour schedule.

The TDS uses the cached dataset information each time a request for that 
dataset comes in. So an access might cause the TDS to check the file status and 
release it if it has been deleted.

I'm going to ask John, who is more familiar with the caching code, about this 
one.

Ethan



> 
> 
> Otherwise, this is our setup for file cleanup:
> 
> ldmadmin scour - /data/pub (the area where the files are written
> for THREDDS)
> 
> I also noted that in our pqact file for THREDDS, we have not "close"
> directive with FILE, just "flush".  Do you recommend changing that; I
> thought that you had set up "flush" rather than "close" for some reason
> when you were helping us with CONDUIT.


Ticket Details
===================
Ticket ID: PZR-552966
Department: Support THREDDS
Priority: Normal
Status: Open