Re: occasional weird timestamps and bad data with THREDDS

On 4/7/2011 10:25 AM, tom cook wrote:

John,

The upgrade seems to working no problem, but my threddsServlet.log hasn't updated since March 30. Is this something that only updates when there is an error? Or should it be updating often?

Hi Tom:

threddsServlet.log should get new messages every time a request comes in. If thats not happening:

stop server.
remove/copy all files from {tomcat}/logs and {tomcat}/content/thredds/logs
restart the server

and make sure its starts to update. if still not happening, send me all the content of both directories.

its probably a good idea to clan out those logs, so that old messages dont confuse us as we try to debug this problem.

Thanks,

Tom

On Wed, Apr 6, 2011 at 10:06 AM, tom cook <address@hidden> wrote:

Thanks John,

I've upgraded to 4.2.5 and will take a look at the errors later today.

Tom

On Tue, Apr 5, 2011 at 4:29 PM, John Caron <address@hidden> wrote:

Hi Tom:

There are some errors happening

ERROR - ucar.nc2.ncml.Aggregation - Cant make cache directory= /usr/share/tomcat6/content/thredds/cache/agg/HFRNet-AKNS-6km-hourly-RTV

that are fixed in the latest version of 4.2.5

Can you upgrade and see if its still happens?

John

On 4/4/2011 10:47 AM, tom cook wrote:

HI John,
I was wondering, when I see this again, what files and logs should I
gather before I empty the cache?
Any other info you have would be appreciated.
Thanks,
Tom

On Wed, Mar 30, 2011 at 1:17 PM, tom cook<address@hidden>  wrote:

John, thanks for the reply

On Wed, Mar 30, 2011 at 11:12 AM, John Caron<address@hidden>  wrote:

Hi Tom:

1) what version TDS?

Version 4.2.1 - 20101201.2028

2) is this an aggregaton? (you can send me the config catalog)

yes, threddsConfig.xml&  catalog.xml attached, let me know if you want
more (we have an enhancedCatalog.xml and wmsConfig.xml too)

3) are there errors in threddsServlet.log?

yes, I've attached a text file with the errors from the logs

4) when you "update the netcdf files multiple times over 5-6 hours ", do you
overwrite? extend?

I believe the the files are always overwritten.  I'll double check and
get back to you if I'm wrong.


THANKS
Tom

its hard when its intermittent, we may have to figure out how to
consistently reproduce.

john

On 3/30/2011 11:21 AM, tom cook wrote:

HI John,
I have been tasked with maintaining the HF Radar national network
THREDDS server here at SIO. Rich has helped us set things up, and has
been a valuable resource for learning this system. We have been having
a frequent problem that Rich has been unable to help us with, which is
occasional bad timestamps and occasional bad data served through
THREDDS.
An example of bad data being served through THREDDS: we are working
with ASA&    USCG to get data into their SAROPS product. They have
complained that many of the data samples have had data in the wrong
geographical location (ie ocean currents over land). I have noticed
this as well using the Godiva viewer, and when I compare the Godiva
map with the source netcdf file, it is not plotting the data in the
correct location. This makes me think it is a problem with the data
being served by THREDDS. The problem goes away after some time, as we
typically update the netcdf files multiple times over 5-6 hours after
they are created. I'm not sure if the updating of the netcdf files is
the cause of the problem, but it is not something that is easy to
reproduce.

The problem we have with weird timestamps happens occasionally, and in
random locations throughout the time index. It usually is just one
sample here or there, and not successive samples. It almost always
goes away when I empty the cache.

I'm not sure if these issues are setup related, or issues we can
script some workarounds to help. But I wanted to check with you to see
if you have any idea of what is going on.

Thanks for your time,

Tom Cook
Programmer/Analyst
Coastal Observation Research&    Development Center
Scripps Institution of Oceanography