[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]

Subject: Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]
Date: Thu, 29 Apr 2010 07:52:12 -0600

Hi Glenn, Dan:

Can you send me your threddsServlet.log files for the period geraldshows below? they arre in {tomcat}/content/thredds/logs/


thanks

Gerald Spreitzhofer wrote:

Hi John,
The problem with SocketException still persists, looking like a network
problem. Sometimes it occurs only sporadically, sometimes we cannot read
GFS data for several hours (with exactly the same code that worked without
problems during the past months). Today for instance between 4:50 and
10:30 UTC in a number of tries from different computers (IP addresses
131.130.157.14, 131.130.157.15, 131,130,157.49), all of them got stuck at
some point reading the GFS data. The data were searched at:
"dods://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100429/gfs_hd_00z"
According to my system administrator our connections to the rest of the
world (like ECMWF, etc.) work fine, so there is a reason to assume that
the error could be on your side (especially in times of network
congestion). By the way, the problems ocur since the first day of the
volcano eruption in Island, and seem to be more frequent in times when a
new GFS run is available (and many users try to get the data at the same
time).
If you need more exact information to check your log files, please let me
know.
Hope some of you can help. Best Regards, Gerald

On Fr, 23.04.2010, 17:02, John Caron wrote:

Hi Dan, Glenn:

java.net.SocketException: Connection reset

probably means that the socket was closed on the server.

To go further with this we need to look at the threddsServlet.log files
and see if we are getting any error messages in there that correspond to
what Gerald is seeing. this may be hard without knowing the exact URL
and a small time window during which the errors happened, and maybe the
client IP address. Gerald do you have any of that?

this can also happen when the connection gets very slow, eg if one of
the IP routes is flaky. Gerald can you ask your network guys (if you
have them) to test to see if your network routing to nomads has any
problems?

On 4/23/2010 6:59 AM, Dan Swank wrote:

John,
This addy is for NCDC-NOMADS servers nomads.ncdc.* only.
For NCEP-NOMADS ~ you should contact their HelpDesk: @
http://nomads.ncep.noaa.gov/txt_descriptions/Help_Desk_doc.shtml

-Dan


address@hidden wrote the following on 4/22/2010 8:41 PM:

hmm-
John
can you assist here?

thx, glenn

----- Original Message -----
From: Gerald Spreitzhofer<address@hidden>
Date: Thursday, April 22, 2010 4:28 pm
Subject: Re: [Fwd: Re: problems reading GFS data from NOMADS-HD;
DConnect]

Thanks, John.
Glenn, could you please answer my mail or explain me the error
messages I
am receiving from time to time (see below)? I think there were similar
messages some days in fall, some people from the mailing list
complainedand then you were able to fix them.
Best regards, Gerald

On Do, 22.04.2010, 22:18, John Caron wrote:

gerald, send reports to

address@hidden



On 4/22/2010 12:54 PM, Gerald Spreitzhofer wrote:

John,
I am having the same problems as before and the same error

message (see

below, java.net.SocketException) most of the day, using

DODS/OPENDAP.>>  Also

just clicked again on the motherlode-link, there I can download GFS
0.5deg
data (well, some gridpoints, I did not check for big amounts of

data).>>  The

amount of data I am downloading is max. 20 MB (like always), I

don't>>  think

this is a problem. The error usually ocurs some time during the
downloading, not just at the beginning.
Can you forward this mail to persons who might help?
Thanks, Gerald

On Mo, 19.04.2010, 16:44, john caron wrote:

gerald, can you try hitting datasets on
http://motherlode.ucar.edu/thredds/ and see if you have any

problems?>>>

Gerald Spreitzhofer wrote:

Hi John,
Thanks for answering my mail last Friday that quickly.

Problems still

persist, but it looks like they are not permanent (sometimes in
between
GFS data can be read without problems). Maybe the error

messages below

help (they were produced this morning around 8 UTC).
Best Regards, Gerald

===== Reading 3d-variable 58: pressfc=====
java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:168)
        at

java.io.BufferedInputStream.fill(BufferedInputStream.java:183)>>>>
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)

at

java.io.BufferedInputStream.read(BufferedInputStream.java:277)>>>>
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:741)

        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:712)
        at

sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:583)>>>>
        at dods.dap.DConnect.openConnection(DConnect.java:174)

        at dods.dap.DConnect.getDataFromUrl(DConnect.java:380)
        at dods.dap.DConnect.getData(DConnect.java:339)
        at dods.dap.DConnect.getData(DConnect.java:522)
        at

ucar.nc2.dods.DODSNetcdfFile.readDODSData(DODSNetcdfFile.java:536)>>>>
at ucar.nc2.dods.DODSVariable.readDODSData(DODSVariable.java:497)

        at ucar.nc2.dods.DODSVariable.read(DODSVariable.java:393)
        at SEVIS.NetcdfUtil.readNcArrayLayer(NetcdfUtil.java:1846)
        at SEVIS.NetcdfUtil.readNcArray(NetcdfUtil.java:1708)
        at

SEVIS.NetcdfUtil.readNcArray_0MedAdapted(NetcdfUtil.java:1659)>>>>
        at SEVIS.DODSReader.writeNcVariables(DODSReader.java:825)

        at SEVIS.DODSReader.createNetcdfFile(DODSReader.java:676)
        at SEVIS.DODSReader.<init>(DODSReader.java:159)
        at SEVIS.SuperDataFile.getDodsFile(SuperDataFile.java:2526)
        at

SEVIS.SuperDataFile.getSpecifiedNetcdfFile(SuperDataFile.java:2344)>>>>
        at
SEVIS.MainFrame.forecastUpdate(MainFrame.java:2131)

at

SEVIS.MainFrame.jButtonDataUpdate_actionPerformed(MainFrame.java:2163)>>>>

------------------------ Ursprüngliche Nachricht
------------------------
Betreff: Re: problems reading GFS data from NOMADS-HD; DConnect
Von:     "Glenn Rutledge"<address@hidden>
Datum:   Fr, 16.04.2010, 19:41
An:      "John Caron"<address@hidden>
Cc:      "Gerald Spreitzhofer"<address@hidden>
           address@hidden
---------------------------------------------------------------

-----------

Thanks John,
We don't seem to be experiencing any issues.  I've cc:ed Dan

Swank>>>>  here

and he might be able to track down this issue.
Regards, Glenn

John Caron said the following on 4/16/2010 1:01 PM:

Hi gerald: im not having any trouble with that URL. is it still
happening (11 am MST) ?

On 4/16/2010 10:15 AM, Gerald Spreitzhofer wrote:

Hi John,
I am operationally reading GFS 0.5deg data from the NOMADS-

HD server

via
DODS and the Java/NetCDF library. I did not change anything

in my

code
during the last couple of months, but recently (for instance

the>>>>>>  last

20
hours) there have been some errors reading data from there

(see the

protocol below). I think I remember that there was a similar

problem>>>>>>  last

fall, and the reason was a change in some security or firewall
setting on
NOMADS, just concerning this protocol. Do you know what the

reason>>>>>>  for the

messages can be or who might be responsible? Thanks, Gerald


===== Reading 3d-variable 28: hgtsfc=====
DConnect getData failed; retry (1,100)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>

DConnect getData failed; retry (2,200)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>

DConnect getData failed; retry (3,400)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>

DConnect getData failed; retry (4,800)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>

DConnect getData failed; retry (5,1600)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[5:5][250:299][704:719]>>>>>>

DConnect getData failed; retry (1,100)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>

DConnect getData failed; retry (2,200)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>

DConnect getData failed; retry (3,400)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>

DConnect getData failed; retry (4,800)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>

DConnect getData failed; retry (5,1600)

http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20100415/gfs_hd_18z.dods?hgtsfc[0:0][250:299][0:66]>>>>>>

-----------------------------------------
Dr. Gerald Spreitzhofer, Researcher
Institute of Meteorology and Geophysics
University of Vienna
A-1090 Vienna, Althanstrasse 14, UZA2
Tel.: +43-1-4277-537 33
email: address@hidden
-----------------------------------------

References:
- Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]
  - From: John Caron
- Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]
  - From: Gerald Spreitzhofer

Prev by Date: [Fwd: Re: [NCEP.List.Nomads-Users] problem with OpenDAP access to daily rtofs data]
Next by Date: Re: NCDC Datasets and COARDS Conventions
Previous by thread: Re: [Fwd: Re: problems reading GFS data from NOMADS-HD; DConnect]
Next by thread: http://coast-enviro.er.usgs.gov/thredds/catalog.html
Index(es):
- Date
- Thread