[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20041129: monster latencies from atm.geo.nsf.gov (cont.)
- Subject: 20041129: monster latencies from atm.geo.nsf.gov (cont.)
- Date: Mon, 29 Nov 2004 15:17:01 -0700
>From: "Patrick L. Francis" <address@hidden>
>Organization: Bowling Green State University
>Keywords: 200411291316.iATDGZlI028344 IDD latency
Hi Patrick,
First, my earlier email should have suggested that you were requesting
everything from the UNIDATA feed from atm.geo.nsf.gov. This means that
your request line would look like:
request UNIDATA ".*" atm.geo.nsf.gov
'UNIDATA' is a compound feed mnemonic. It is actually IDS|DDPLUS|HDS|UNIWISC.
>traceroutes into bgnet are throttled... and as mentioned i examined
>the internal network for difficulties. For example, we are receiving
>data from OU (stokes) with minimal delay.
>
>http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NNEXRAD+unidata.bgsu.edu
>
>however, over the same time period IDD data from atm shows monster latencies.
>
>http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?HDS+unidata.bgsu.edu
Yes, but the volume of NNEXRAD data you are receiving from OU is small:
http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?NNEXRAD+unidata.bgsu.edu
and the volume of the request from atm.geo.nsf.gov would be much higher
_if_ your request is for everything in UNIDATA. I just did a quick
check of the connection messages in the ~ldm/logs/ldmd.log.* files on
atm.geo.nsf.gov, and I see that my assumption is correct:
Nov 28 03:01:34 atm.geo.nsf.gov unidata(feed)[3309]: up6.c:331: Starting
Up(6.0.14/6): 20041128030119.632 TS_ENDT {{UNIDATA, ".*"}}
Nov 28 03:01:34 atm.geo.nsf.gov unidata(feed)[3309]: topo: unidata.bgsu.edu
UNIDATA
>this would seem to indicate a problem outside of the BG Network.
Not necessarily. If there is a some sort of volume limitation on a
feed (like through "packet shaping" being done on a BGSU firewall or
through a problem on a router), and if your request for data from
atm.geo.nsf.gov is like:
request UNIDATA ".*" atm.geo.nsf.gov
then the requested volume of data from atm.geo.nsf.gov is up to 60
times more than the NEXRAD Level III data you are getting from OU. The
fact that the latency plots for IDS|DDPLUS, HDS, and UNIWISC are
identical (not similar, _identical_) is what tipped me off that me that
the feed request to atm.geo.nsf.gov is for everything from UNIDATA.
>Also notice
>the significant drop in latencies to unidata.bgsu from atm.geo since the time
>of my first message.
This drop is dramatic indeed, and the traceroute from atm.geo.nsf.gov
to unidata.bgsu.edu indicates that the problem at the BGSU machine
mentioned previously has disappeared:
/local/ldm% traceroute unidata.bgsu.edu
traceroute to unidata.bgsu.edu (129.1.112.220), 30 hops max, 40 byte packets
1 colima.nsf.gov (198.181.231.1) 1 ms 1 ms 1 ms
2 arlg-nsf.maxgigapop.net (206.196.177.137) 2 ms 2 ms 1 ms
3 dcne-so3-0-0.maxgigapop.net (206.196.178.42) 2 ms 10 ms 18 ms
4 abilene-rtr.maxgigapop.net (206.196.177.2) 25 ms * 4 ms
5 nycmng-washng.abilene.ucaid.edu (198.32.8.84) 6 ms 6 ms 9 ms
6 chinng-nycmng.abilene.ucaid.edu (198.32.8.82) 27 ms 48 ms 28 ms
7 iplsng-chinng.abilene.ucaid.edu (198.32.8.77) 249 ms 252 ms 241 ms
8 clmbq-r2-po0-0.bb.oar.net (192.88.192.133) 34 ms 35 ms 35 ms
9 dytnq-r3-at1-0s16.bb.oar.net (192.88.191.101) 37 ms 37 ms 37 ms
10 bgsu2-r1-at2-0s103.cpe.oar.net (192.88.192.174) 62 ms 62 ms 61 ms
11 129.1.8.252 (129.1.8.252) 62 ms 67 ms 62 ms
12 129.1.8.125 (129.1.8.125) 63 ms 62 ms 63 ms
13 * * *
14 * * *
15 unidata.bgsu.edu (129.1.112.220) 76 ms 106 ms 93 ms
The problem you were seeing earlier was not at or "near"
atm.geo.nsf.gov since other machines feeding the same data as
unidata.bgsu.edu from atm.geo.nsf.gov showed no appreciable latencies
over the time where you were observing problems. The machine at the
University of North Carolina-Ashville is a good example:
IDS|DDPLUS latency:
http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?IDS|DDPLUS+storm2.atms.unca.edu
HDS latency:
http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?HDS+storm2.atms.unca.edu
UNIWISC latency:
http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?UNIWISC+storm2.atms.unca.edu
>my concerns are that since other universities feed off of us, usually with
>great success, any delays from the top would snowball.
Your concern is a valid one!
In a way, it is almost too bad (but not really :-) that the slowdown is
not still occurring. If it was, I was going to ask you to split your
single UNIDATA feed request to three separate ones:
change:
request WMO ".*" atm.geo.nsf.gov
to:
request IDS|DDPLUS ".*" atm.geo.nsf.gov
request HDS ".*" atm.geo.nsf.gov
request UNIWISC ".*" atm.geo.nsf.gov
After a change like this, I would expect that even though there was
still some sort of problem, the latency for the IDS|DDPLUS feed would
fall to levels you were seeing in your NNEXRAD request from OU since
the volume in the IDS|DDPLUS feed is equivalent to the volume of
NNEXRAD data you are requesting. I would expect that the latencies for
UNIWISC would also decrease, but the ones for HDS would probably not
since it comprises the bulk of the volume in the combined request.
>Hope you had a wonderful thanksgiving :)
It was grand, thanks. I hope yours was as much fun as mine!!
Cheers,
Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web. If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.
>From address@hidden Mon Nov 29 16:37:18 2004
>Please let us know the results of your 'traceroute'.
forgot these earlier... will reply to your last message tomorrow ...
it's almost by bedtime :)
------------------[ from weather.bgsu.edu ]------------------------
[pfrancis@weather pfrancis]$ /usr/sbin/traceroute atm.geo.nsf.gov
traceroute to atm.geo.nsf.gov (198.181.231.53), 30 hops max, 38 byte packets
1 router1-112 (129.1.112.252) 0.307 ms 0.255 ms 0.232 ms
2 10.9.2.1 (10.9.2.1) 0.525 ms 0.431 ms 0.468 ms
3 10.9.10.12 (10.9.10.12) 0.354 ms 0.297 ms 0.237 ms
4 129.1.8.126 (129.1.8.126) 0.533 ms 0.454 ms 0.442 ms
5 gw-bgsu (129.1.8.253) 1.604 ms 1.558 ms 2.110 ms
6 dytnq-r3-at2-0s6.bb.oar.net (192.88.192.173) 25.119 ms 25.761
ms 25.201 ms
7 clmbq-r2-at2-0s13.bb.oar.net (192.88.191.102) 28.993 ms 27.149
ms 27.254 ms
8 abilene-IPLSng.ohio-gigapop.oar.net (192.88.192.134) 33.395
ms 32.685 ms 33.648 ms
9 chinng-iplsng.abilene.ucaid.edu (198.32.8.76) 44.590 ms 38.635
ms 48.238 ms
10 nycmng-chinng.abilene.ucaid.edu (198.32.8.83) 58.707 ms 84.901
ms 62.806 ms
11 washng-nycmng.abilene.ucaid.edu (198.32.8.85) 61.339 ms 61.714
ms 64.282 ms
12 dcne-abilene-oc48.maxgigapop.net (206.196.177.1) 61.604 ms 60.610
ms 62.031 ms
13 arlg-so3-1-0.maxgigapop.net (206.196.178.41) 61.171 ms 60.459
ms 61.173 ms
14 nsf-rtr.maxgigapop.net (206.196.177.138) 61.883 ms 61.537 ms 61.390 ms
15 atm.geo.nsf.gov (198.181.231.53) 61.655 ms 61.956 ms 62.473 ms
[pfrancis@weather pfrancis]$
---------------------[ from unidata.bgsu.edu ]--------------------------
[pfrancis@unidata pfrancis]$ /usr/sbin/traceroute atm.geo.nsf.gov
traceroute to atm.geo.nsf.gov (198.181.231.53), 30 hops max, 38 byte packets
1 router1-112 (129.1.112.252) 0.286 ms 0.233 ms 0.225 ms
2 10.9.2.1 (10.9.2.1) 0.493 ms 0.404 ms 0.390 ms
3 10.9.10.12 (10.9.10.12) 0.293 ms 0.246 ms 0.229 ms
4 129.1.8.126 (129.1.8.126) 0.395 ms 0.385 ms 0.387 ms
5 gw-bgsu (129.1.8.253) 3.244 ms 2.377 ms 2.759 ms
6 dytnq-r3-at2-0s6.bb.oar.net (192.88.192.173) 25.752 ms 24.496
ms 26.233 ms
7 clmbq-r2-at2-0s13.bb.oar.net (192.88.191.102) 28.529 ms 27.223
ms 28.123 ms
8 abilene-IPLSng.ohio-gigapop.oar.net (192.88.192.134) 32.394
ms 33.580 ms 36.879 ms
9 chinng-iplsng.abilene.ucaid.edu (198.32.8.76) 42.701 ms 44.354
ms 72.150 ms
10 nycmng-chinng.abilene.ucaid.edu (198.32.8.83) 64.432 ms 69.913
ms 57.960 ms
11 washng-nycmng.abilene.ucaid.edu (198.32.8.85) 61.733 ms 67.642
ms 70.044 ms
12 dcne-abilene-oc48.maxgigapop.net (206.196.177.1) 60.852 ms 62.229
ms 62.586 ms
13 arlg-so3-1-0.maxgigapop.net (206.196.178.41) 60.994 ms 61.581
ms 60.775 ms
14 nsf-rtr.maxgigapop.net (206.196.177.138) 62.236 ms 64.352 ms 62.337 ms
15 atm.geo.nsf.gov (198.181.231.53) 70.127 ms 70.208 ms 73.690 ms
[pfrancis@unidata pfrancis]$
------------------------------------------------------------------------------
>From address@hidden Tue Nov 30 07:20:25 2004
Pat,
I have given a higher traffic priority in the packet shaper to all
traffic to and from weather.bgsu.edu and unidata.bgsu.edu.
Let me know if this improves anything.
Thanks
-Mike