[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: IOOS test data set goes away?

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: IOOS test data set goes away?
  • Date: Thu, 28 May 2009 11:16:18 -0400
Hi Rich,

Rich Signell wrote:

Steve,

We had a problem where periodically we would show up in the morning to
find that our bathy_catalog.xml would only have 1 entry instead of the
30 or so that it should have.   And restarting tomcat fixed the
problem.   Eventually we discovered that by blocking web crawlers our
problem went away, and the reaso speculated by our sys admin was that
the crawlers were gobbling up all the tomcat memory.
If so, it sounds like a tomcat memory leak. That's one thing that I've always disliked about Java. Even though it is object-oriented and it should be easy for deconstructors to free up the memory it never seems to work that way.
I remember you sent an email to the IOOS model data interoperability group about this. I'll give it a try and see if that helps. Also, I uncommented the logging section of the tomcat conf/server.xml to start logging requests. 


So yes, we've seen problems where only one THREDDS catalog seems to be
affected, perhaps by memory issues, and restarting Tomcat helps.

Perhaps your attempt to aggregate 3 years of forecast data into the
FMRC aggregation is blowing things out?
Is there a rule of thumb for knowing how many elements is reasonable for aggregation? Is 1000 too much? 10000? 


I think we need a tutorial (webinar?) on caching, aggregation and memory...
That would be very helpful now that I have a better idea about what is happening. 

Steve


-Rich


On Thu, May 28, 2009 at 9:49 AM, Steve Cousins <address@hidden> wrote:

Hi Rich,

Do you have problems with THREDDS data sets going away? I came in this
morning and the Monthly Average data set was available but the IOOS Test one
was not available. I didn't see anything to speak of in the logs about it.
Restarting Tomcat brought it back. I remember tomcat being a problem in LAS
a number of years ago but usually the whole thing would be hosed not just a
single data set. Is this to be expected?

Thanks,

Steve