[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Thredds out of memory

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: Thredds out of memory
  • Date: Wed, 06 Apr 2005 15:03:52 +1000
For anyone who's interested, I am now quite happily serving large data sets via HTTP to thredds. The key was to re-write my java servlet in python. I think object creation overhead was killing me because of the sheer number of independant requests made by thredds. I used a default window size of 65,536 bytes, and was able to get 1.1Mb/s throughput downloading from thredds. My test file was 590Mb.
I like this approach, because my custom data source is neatly de-coupled from thredds code. All I have to do is deal with the HTTP protocol, and everything is handled for me.
If anyone else is having difficulty integrating a new data source into thredds, feel free to contact me for my code. 

Thanks for everyone's help...

Cheers,
-Tennessee