Re: Updating Aggregations in TDS

When a file is added to a directory, a flag is set in a table that  
says it has been added.   We would rather poll that table  (much  
faster as it doesn't need to open a read a slew of netcdf files).  I  
can have Bob write a program in Java to do the required recheck  
operations if you can point us to the appropriate steps to be taken  
for the netcdf-java library.

In the near future, both the information to generate our catalog.xml  
and what files are present and/or have been added will be in a database.

-Roy

On Oct 30, 2008, at 9:55 AM, John Caron wrote:

> We are going to add this kind of functionality to 4.0, currently it  
> doesnt exist.
>
> How do you detect when data changes?
>
> Roy Mendelssohn wrote:
> > Hi John:
> >
> > I am glad we understand how recheckEvery works.  It is consistent  
> > with
> > what we were experiencing.
> >
> > > basically you want to do the recheck in the background, and then
> > > accept whatever is there when a request comes in? Or you want to
> > > detect whenever the directory changes?
> >
> > Because other things, such as web page displays, need to be updated  
> > when
> > data is added, we already have a system where we know when data have
> > been added to a dataset.  What we would like to do is at that point
> > force a recheck for that dataset, and hopefully only for that  
> > dataset,
> > rather than the entire catalog.  If we can do that, we probably will
> > turn off the recheckEvery because we know every dataset will be up  
> > to date.
> >
> > Does this makes sense?  If need be, we can write a script in Java  
> > to do
> > the above, if we can get advice on the best way to do this.
> >
> > Thanks,
> >
> > -Roy
> >
> > **********************
> > "The contents of this message do not reflect any position of the U.S.
> > Government or NOAA."
> > **********************
> > Roy Mendelssohn
> > Supervisory Operations Research Analyst
> > NOAA/NMFS
> > Environmental Research Division
> > Southwest Fisheries Science Center
> > 1352 Lighthouse Avenue
> > Pacific Grove, CA 93950-2097
> >
> > e-mail: address@hidden (Note new e-mail address)
> > voice: (831)-648-9029
> > fax: (831)-648-8440
> > www: http://www.pfeg.noaa.gov/
> >
> > "Old age and treachery will overcome youth and skill."
> > "From those who have been given much, much will be expected"

**********************
"The contents of this message do not reflect any position of the U.S.  
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"