Re: Netcdf based web application and a question

Mark Rehbein wrote:

> Hi John,
>  
> About a year ago you helped me with using the Netcdf java API you are 
> developing.
>  
> We finally have our web application which uses the netcdf api 
> available for use so if you would like to see it then please visit the 
> following URL:
>  
> http://138.7.120.8:9555/sstatlas/do/gotoWelcome.do
>  
> We have not advertised the url yet so please do NOT publish the url to 
> the site just yet.
>  
>  
>  
> I also have a question about the NCML dataset that we are using and 
> its scalability.
>  
> We have a netcdf file for each month of SST data which we aggregate 
> with NCML and use as a dataset in the Netcdf java API. I have found 
> that when the dataset is used, a file handle is created to each netcdf 
> file defined in the aggregate. Obviously, if we have too many of these 
> netcdf files in the aggregate the system will run out of file handles.
>  
> I think our unix server can handle about 1024 file handles per process 
> but these could potentially be used up if we are running multiple 
> netcdf type applications on the single tomcat process.
>  
>  
> Thanks
>  
> Mark

Hi Mark:

thanks for the update. looks pretty cool. have you studied the 
performance bottlenecks at all?

i am working on a new version of ncml aggregation that will let you 
specify number of  open file handles. wont be done till next year 
sometime. its will be part of version 2.2 I may try to multithread it.

i'd like to hear about what features in ncml you are using, since i am 
making a new version and i am planning on dropping or modifying some of 
the tags.