Re: File Limit? Tomcat settings?

Sorry, of course the docs I point you to are out-of date!

here is the current scoop:

File Handles and Caching 

The TDS caches file handles to minimize OS overhead. Currently this is hardwired to allow 400 - 600 open files for OPeNDAP and WCS, and 20 - 40 for HTTP file serving. This means that your OS must allow at least that many file handles per process, else you can get a "out of file handles" message, and the server may freeze up. 

These numbers limit performance, but not functionality. For example, the number of files in an aggregation is not limited by these file handle limits. 

We will make these limits user-settable in a future version of the TDS. 



Roy Mendelssohn wrote:
> On those pages you state:
>
> *File Handles
> *The OPeNDAP-NetCDF server will keep up to 100 NetCDF files open at a 
> time. This is set in* {tomcat_home}/webapps/thredds/WEB-INF/web.xml* 
> file, in the parameter called/* maxNetcdfFilesCached.*/ You can change 
> it, but beware that it gets overwritten when a new thredds.war is installed.
>
>
> Does this mean that performance will be affected because only 100 can be 
> open at a time, or that no more than 100 can be accessed in a request?
>
> -Roy
>
> > Roy Mendelssohn wrote:
> >
> > > Hi John:
> > >
> > > In the TDS aggregating data, is there any limit to the number of 
> > > files that can aggregated together and still be accessed.
> >
> >
> > no, just practical limits like memory etc.
> >
> > > Also, are you aware of any interplay between THREDDS stability and 
> > > performance and setting for the Tomcat server  (i.e. are there ways 
> > > we can set up Tomcat that will optimize the performance of the TDS)?)
> >
> >
> > give it as much memory as possible. We are recommending Tomcat 5.5, 
> > are you using?
> >
> > also see:
> >
> > http://motherlode.ucar.edu:8080/thredds/docs/
> >
> > Performance, Tomcat FAQ sections
> >
> > > Our Tomcat has been crashing.  It may or may not have something to do 
> > > with THREDDS.  Just trying to get info to both start looking fo 
> > > causes and to improve stability.
> >
> >
> >
> >
> > Getting any error messages?
> >
> > > TIA,
> > >
> > > -Roy
>
>
> --
>
> **********************
> "The contents of this message do not reflect any position of the U.S. 
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division   
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: address@hidden (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."