[THREDDS #FJC-813079]: truncated data download

Fan,

Well there's definitely an exception there--looks like it's accessing a remote 
dataset over DODS. Can you send me the configuration catalog for this 
aggregation?

Thanks,

Ryan

> Hi Ryan,
> 
> I just sent the second URL in my original report again and sorted out the 
> logs you wanted (attached).  I got the following truncated response.
> 
> This particular aggregation is for TRMM_3A25 monthly data, 203 time steps 
> from year 1998 to late 2014.
> 
> -Fan
> 
> date,lat[unit="degrees_north"],lon[unit="degrees_east"],bbPixNum1[unit=""]
> 190097-12-16T12:00:00Z,26.0,132.0,2699.0
> 190098-01-16T12:00:00Z,26.0,132.0,4256.0
> 190098-02-15T00:00:00Z,26.0,132.0,6364.0
> 190098-03-16T12:00:00Z,26.0,132.0,4634.0
> 190098-04-16T00:00:00Z,26.0,132.0,6216.0
> 190098-05-16T12:00:00Z,26.0,132.0,10090.0
> 190098-06-16T00:00:00Z,26.0,132.0,10004.0
> 190098-07-16T12:00:00Z,26.0,132.0,3976.0
> 190098-08-16T12:00:00Z,26.0,132.0,343.0
> 190098-09-16T00:00:00Z,26.0,132.0,3445.0
> 190098-10-16T12:00:00Z,26.0,132.0,7464.0
> 190098-11-16T00:00:00Z,26.0,132.0,8957.0
> 190098-12-16T12:00:00Z,26.0,132.0,4054.0
> 190099-01-16T12:00:00Z,26.0,132.0,1033.0
> 190099-02-15T00:00:00Z,26.0,132.0,235.0
> 190099-03-16T12:00:00Z,26.0,132.0,12310.0
> 190099-04-16T00:00:00Z,26.0,132.0,8380.0
> 190099-05-16T12:00:00Z,26.0,132.0,10625.0
> 190099-06-16T00:00:00Z,26.0,132.0,8768.0
> 190099-07-16T12:00:00Z,
> ________________________________________
> From: Unidata THREDDS Support [address@hidden]
> Sent: Wednesday, March 04, 2015 5:36 PM
> To: Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> Cc: address@hidden; Fang, Fan (GSFC-610.2)[ADNET SYSTEMS INC]
> Subject: [THREDDS #FJC-813079]: truncated data download
> 
> Fan,
> 
> I just noticed that when I refreshed the CSV link (still OSX and Chrome 41), 
> I got truncated data, but it responded *very* quickly. Refreshing again 
> quickly gave me a different amount of truncated data. This behavior strikes 
> me as odd. It's clearly not a timeout, which leads me to believe it's a 
> server issue, not a client one.
> 
> I'd be very curious to know if those logs show anything interesting.
> 
> Ryan
> 
> > Fan,
> >
> > The TDS will not time out the data transfer--that's got to be a client 
> > issue.
> >
> > It's technically possible that the slow, or even non-responsive, behavior 
> > of the server could be caused by a bug. The logs (threddsServlet.log and 
> > catalina.out) may shed some light on that. 4.3.19-SNAPSHOT is a bit out of 
> > date as well, the latest 4.3 series release is 4.3.23, released last fall.
> >
> > I don't know how big the aggregations are, but in general aggregations are 
> > known to be inefficient and have problems scaling to large collections.
> >
> > Ryan
> >
> > > Hi Ryan,
> > >
> > > This is curious.  I just tried Safari 8.0.3 (10600.3.18) on OSX - it does 
> > > return the full csv data.  In the mean time, netcdf request downloaded an 
> > > .html showing server error.  Attached.
> > >
> > > My request with truncated data was with firefox 31.5.0 on a CentOS 6 
> > > machine.  It still does that.
> > >
> > > My Chrome 40.0.2214.115 on OSX shows just two lines for csv; below 
> > > (showing what I mean by truncated).
> > >
> > > I used wget and curl also (on either OSX or CentOS box) with the csv URL. 
> > >  Both are returning me incomplete data, though with many more lines than 
> > > Chrome 40 does.
> > >
> > > Occasionally I got server timeouts also.  How do configure TDS to avoid 
> > > the timeouts?
> > >
> > > I just want to make sure it is not a server behavior.  But at the moment 
> > > I cannot exclude that possibility.
> > >
> > > Thanks.
> > >
> > > -Fan
> > >
> > > date,lat[unit="degrees_north"],lon[unit="degrees_east"],bbPixNum1[unit=""]
> > > 190097-12-16T12:00:00Z,
> 
> 
> Ticket Details
> ===================
> Ticket ID: FJC-813079
> Department: Support THREDDS
> Priority: Normal
> Status: Closed
> 
> 
> 


Ticket Details
===================
Ticket ID: FJC-813079
Department: Support THREDDS
Priority: Normal
Status: Closed