Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]

Hi Dan, Glenn:

java.net.SocketException: Connection reset

probably means that the socket was closed on the server.

To go further with this we need to look at the threddsServlet.log files 
and see if we are getting any error messages in there that correspond to 
what Gerald is seeing. this may be hard without knowing the exact URL 
and a small time window during which the errors happened, and maybe the 
client IP address. Gerald do you have any of that?

this can also happen when the connection gets very slow, eg if one of 
the IP routes is flaky. Gerald can you ask your network guys (if you 
have them) to test to see if your network routing to nomads has any 
problems?

On 4/23/2010 6:59 AM, Dan Swank wrote:
> John,
> This addy is for NCDC-NOMADS servers nomads.ncdc.* only.
> For NCEP-NOMADS ~ you should contact their HelpDesk: @
> http://nomads.ncep.noaa.gov/txt_descriptions/Help_Desk_doc.shtml
>
> -Dan
>
>
> address@hidden wrote the following on 4/22/2010 8:41 PM:
>    
> > hmm-
> > John
> > can you assist here?
> >
> > thx, glenn
> >
> > ----- Original Message -----
> > From: Gerald Spreitzhofer<address@hidden>
> > Date: Thursday, April 22, 2010 4:28 pm
> > Subject: Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]
> >
> >      
> > > Thanks, John.
> > > Glenn, could you please answer my mail or explain me the error
> > > messages I
> > > am receiving from time to time (see below)? I think there were similar
> > > messages some days in fall, some people from the mailing list
> > > complainedand then you were able to fix them.
> > > Best regards, Gerald
> > >
> > > On Do, 22.04.2010, 22:18, John Caron wrote:
> > >        
> > > > gerald, send reports to
> > > >
> > > > address@hidden
> > > >
> > > >
> > > >
> > > > On 4/22/2010 12:54 PM, Gerald Spreitzhofer wrote:
> > > >          
> > > > > John,
> > > > > I am having the same problems as before and the same error
> > > > >            
> > > message (see
> > >        
> > > > > below, java.net.SocketException) most of the day, using
> > > > >            
> > > DODS/OPENDAP.>>  Also
> > >        
> > > > > just clicked again on the motherlode-link, there I can download GFS
> > > > > 0.5deg
> > > > > data (well, some gridpoints, I did not check for big amounts of
> > > > >            
> > > data).>>  The
> > >        
> > > > > amount of data I am downloading is max. 20 MB (like always), I
> > > > >            
> > > don't>>  think
> > >        
> > > > > this is a problem. The error usually ocurs some time during the
> > > > > downloading, not just at the beginning.
> > > > > Can you forward this mail to persons who might help?
> > > > > Thanks, Gerald
> > > > >
> > > > > On Mo, 19.04.2010, 16:44, john caron wrote:
> > > > >
> > > > >            
> > > > > > gerald, can you try hitting datasets on
> > > > > > http://motherlode.ucar.edu/thredds/ and see if you have any
> > > > > >              
> > > problems?>>>
> > >        
> > > > > > Gerald Spreitzhofer wrote:
> > > > > >
> > > > > >              
> > > > > > > Hi John,
> > > > > > > Thanks for answering my mail last Friday that quickly.
> > > > > > >                
> > > Problems still
> > >        
> > > > > > > persist, but it looks like they are not permanent (sometimes in
> > > > > > > between
> > > > > > > GFS data can be read without problems). Maybe the error
> > > > > > >                
> > > messages below
> > >        
> > > > > > > help (they were produced this morning around 8 UTC).
> > > > > > > Best Regards, Gerald
> > > > > > >
> > > > > > > ===== Reading 3d-variable 58: pressfc=====
> > > > > > > java.net.SocketException: Connection reset
> > > > > > >         at java.net.SocketInputStream.read(SocketInputStream.java:168)
> > > > > > >         at
> > > > > > >                
> > > java.io.BufferedInputStream.fill(BufferedInputStream.java:183)>>>>
> > >         at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
> > >        
> > > > > > >         at
> > > > > > >                
> > > java.io.BufferedInputStream.read(BufferedInputStream.java:277)>>>>
> > >         at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:741)
> > >        
> > > > > > >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> > > > > > >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:712)
> > > > > > >         at
> > > > > > >
> > > > > > >                
> > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:583)>>>>
> >         at dods.dap.DConnect.openConnection(DConnect.java:174)
> >      
> > > > > > >         at dods.dap.DConnect.getDataFromUrl(DConnect.java:380)
> > > > > > >         at dods.dap.DConnect.getData(DConnect.java:339)
> > > > > > >         at dods.dap.DConnect.getData(DConnect.java:522)
> > > > > > >         at
> > > > > > >                
> > ucar.nc2.dods.DODSNetcdfFile.readDODSData(DODSNetcdfFile.java:536)>>>>
> > at ucar.nc2.dods.DODSVariable.readDODSData(DODSVariable.java:497)
> >      
> > > > > > >         at ucar.nc2.dods.DODSVariable.read(DODSVariable.java:393)
> > > > > > >         at SEVIS.NetcdfUtil.readNcArrayLayer(NetcdfUtil.java:1846)
> > > > > > >         at SEVIS.NetcdfUtil.readNcArray(NetcdfUtil.java:1708)
> > > > > > >         at
> > > > > > >                
> > > SEVIS.NetcdfUtil.readNcArray_0MedAdapted(NetcdfUtil.java:1659)>>>>
> > >         at SEVIS.DODSReader.writeNcVariables(DODSReader.java:825)
> > >        
> > > > > > >         at SEVIS.DODSReader.createNetcdfFile(DODSReader.java:676)
> > > > > > >         at SEVIS.DODSReader.<init>(DODSReader.java:159)
> > > > > > >         at SEVIS.SuperDataFile.getDodsFile(SuperDataFile.java:2526)
> > > > > > >         at
> > > > > > >
> > > > > > >                
> > SEVIS.SuperDataFile.getSpecifiedNetcdfFile(SuperDataFile.java:2344)>>>>     at
> > SEVIS.MainFrame.forecastUpdate(MainFrame.java:2131)
> >      
> > > > > > >         at
> > > > > > >
> > > > > > >                
> > > SEVIS.MainFrame.jButtonDataUpdate_actionPerformed(MainFrame.java:2163)>>>>
> > >        
> > > > > > > ------------------------ Ursprüngliche Nachricht
> > > > > > > ------------------------
> > > > > > > Betreff: Re: problems reading GFS data from NOMADS-HD; DConnect
> > > > > > > Von:     "Glenn Rutledge"<address@hidden>
> > > > > > > Datum:   Fr, 16.04.2010, 19:41
> > > > > > > An:      "John Caron"<address@hidden>
> > > > > > > Cc:      "Gerald Spreitzhofer"<address@hidden>
> > > > > > >            address@hidden
> > > > > > > ---------------------------------------------------------------
> > > > > > >                
> > > -----------
> > >        
> > > > > > > Thanks John,
> > > > > > > We don't seem to be experiencing any issues.  I've cc:ed Dan
> > > > > > >                
> > > Swank>>>>  here
> > >        
> > > > > > > and he might be able to track down this issue.
> > > > > > > Regards, Glenn
> > > > > > >
> > > > > > > John Caron said the following on 4/16/2010 1:01 PM:
> > > > > > >
> > > > > > >
> > > > > > >                
> > > > > > > > Hi gerald: im not having any trouble with that URL. is it still
> > > > > > > > happening (11 am MST) ?
> > > > > > > >
> > > > > > > > On 4/16/2010 10:15 AM, Gerald Spreitzhofer wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >                  
> > > > > > > > > Hi John,
> > > > > > > > > I am operationally reading GFS 0.5deg data from the NOMADS-
> > > > > > > > >                    
> > > HD server
> > >        
> > > > > > > > > via
> > > > > > > > > DODS and the Java/NetCDF library. I did not change anything
> > > > > > > > >                    
> > > in my
> > >        
> > > > > > > > > code
> > > > > > > > > during the last couple of months, but recently (for instance
> > > > > > > > >                    
> > > the>>>>>>  last
> > >        
> > > > > > > > > 20
> > > > > > > > > hours) there have been some errors reading data from there
> > > > > > > > >                    
> > > (see the
> > >        
> > > > > > > > > protocol below). I think I remember that there was a similar
> > > > > > > > >                    
> > > problem>>>>>>  last
> > >        
> > > > > > > > > fall, and the reason was a change in some security or firewall
> > > > > > > > > setting on
> > > > > > > > > NOMADS, just concerning this protocol. Do you know what the
> > > > > > > > >                    
> > > reason>>>>>>  for the
> > >        
> > > > > > > > > messages can be or who might be responsible? Thanks, Gerald
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > ===== Reading 3d-variable 28: hgtsfc=====
> > > > > > > > > DConnect getData failed; retry (1,100)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (2,200)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (3,400)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (4,800)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (5,1600)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (1,100)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (2,200)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (3,400)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (4,800)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>
> >      
> > > > > > > > > DConnect getData failed; retry (5,1600)
> > > > > > > > >
> > > > > > > > >                    
> > http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>
> >      
> > > > > > > > >                    
> > > > > > > > > > > -----------------------------------------
> > > > > > > > > > > Dr. Gerald Spreitzhofer, Researcher
> > > > > > > > > > > Institute of Meteorology and Geophysics
> > > > > > > > > > > University of Vienna
> > > > > > > > > > > A-1090 Vienna, Althanstrasse 14, UZA2
> > > > > > > > > > > Tel.: +43-1-4277-537 33
> > > > > > > > > > > email: address@hidden
> > > > > > > > > > > -----------------------------------------
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >                        
> > > > > > >                
> > > > > >
> > > > > >              
> > > >
> > > >          
> > >
> > >        
>