This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
>From: "Patrick L. Francis" <address@hidden> >Organization: Bowling Green State University >Keywords: 200411291316.iATDGZlI028344 IDD latency Hi Patrick, First, my earlier email should have suggested that you were requesting everything from the UNIDATA feed from atm.geo.nsf.gov. This means that your request line would look like: request UNIDATA ".*" atm.geo.nsf.gov 'UNIDATA' is a compound feed mnemonic. It is actually IDS|DDPLUS|HDS|UNIWISC. >traceroutes into bgnet are throttled... and as mentioned i examined >the internal network for difficulties. For example, we are receiving >data from OU (stokes) with minimal delay. > >http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NNEXRAD+unidata.bgsu.edu > >however, over the same time period IDD data from atm shows monster latencies. > >http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?HDS+unidata.bgsu.edu Yes, but the volume of NNEXRAD data you are receiving from OU is small: http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?NNEXRAD+unidata.bgsu.edu and the volume of the request from atm.geo.nsf.gov would be much higher _if_ your request is for everything in UNIDATA. I just did a quick check of the connection messages in the ~ldm/logs/ldmd.log.* files on atm.geo.nsf.gov, and I see that my assumption is correct: Nov 28 03:01:34 atm.geo.nsf.gov unidata(feed)[3309]: up6.c:331: Starting Up(6.0.14/6): 20041128030119.632 TS_ENDT {{UNIDATA, ".*"}} Nov 28 03:01:34 atm.geo.nsf.gov unidata(feed)[3309]: topo: unidata.bgsu.edu UNIDATA >this would seem to indicate a problem outside of the BG Network. Not necessarily. If there is a some sort of volume limitation on a feed (like through "packet shaping" being done on a BGSU firewall or through a problem on a router), and if your request for data from atm.geo.nsf.gov is like: request UNIDATA ".*" atm.geo.nsf.gov then the requested volume of data from atm.geo.nsf.gov is up to 60 times more than the NEXRAD Level III data you are getting from OU. The fact that the latency plots for IDS|DDPLUS, HDS, and UNIWISC are identical (not similar, _identical_) is what tipped me off that me that the feed request to atm.geo.nsf.gov is for everything from UNIDATA. >Also notice >the significant drop in latencies to unidata.bgsu from atm.geo since the time >of my first message. This drop is dramatic indeed, and the traceroute from atm.geo.nsf.gov to unidata.bgsu.edu indicates that the problem at the BGSU machine mentioned previously has disappeared: /local/ldm% traceroute unidata.bgsu.edu traceroute to unidata.bgsu.edu (129.1.112.220), 30 hops max, 40 byte packets 1 colima.nsf.gov (198.181.231.1) 1 ms 1 ms 1 ms 2 arlg-nsf.maxgigapop.net (206.196.177.137) 2 ms 2 ms 1 ms 3 dcne-so3-0-0.maxgigapop.net (206.196.178.42) 2 ms 10 ms 18 ms 4 abilene-rtr.maxgigapop.net (206.196.177.2) 25 ms * 4 ms 5 nycmng-washng.abilene.ucaid.edu (198.32.8.84) 6 ms 6 ms 9 ms 6 chinng-nycmng.abilene.ucaid.edu (198.32.8.82) 27 ms 48 ms 28 ms 7 iplsng-chinng.abilene.ucaid.edu (198.32.8.77) 249 ms 252 ms 241 ms 8 clmbq-r2-po0-0.bb.oar.net (192.88.192.133) 34 ms 35 ms 35 ms 9 dytnq-r3-at1-0s16.bb.oar.net (192.88.191.101) 37 ms 37 ms 37 ms 10 bgsu2-r1-at2-0s103.cpe.oar.net (192.88.192.174) 62 ms 62 ms 61 ms 11 129.1.8.252 (129.1.8.252) 62 ms 67 ms 62 ms 12 129.1.8.125 (129.1.8.125) 63 ms 62 ms 63 ms 13 * * * 14 * * * 15 unidata.bgsu.edu (129.1.112.220) 76 ms 106 ms 93 ms The problem you were seeing earlier was not at or "near" atm.geo.nsf.gov since other machines feeding the same data as unidata.bgsu.edu from atm.geo.nsf.gov showed no appreciable latencies over the time where you were observing problems. The machine at the University of North Carolina-Ashville is a good example: IDS|DDPLUS latency: http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?IDS|DDPLUS+storm2.atms.unca.edu HDS latency: http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?HDS+storm2.atms.unca.edu UNIWISC latency: http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?UNIWISC+storm2.atms.unca.edu >my concerns are that since other universities feed off of us, usually with >great success, any delays from the top would snowball. Your concern is a valid one! In a way, it is almost too bad (but not really :-) that the slowdown is not still occurring. If it was, I was going to ask you to split your single UNIDATA feed request to three separate ones: change: request WMO ".*" atm.geo.nsf.gov to: request IDS|DDPLUS ".*" atm.geo.nsf.gov request HDS ".*" atm.geo.nsf.gov request UNIWISC ".*" atm.geo.nsf.gov After a change like this, I would expect that even though there was still some sort of problem, the latency for the IDS|DDPLUS feed would fall to levels you were seeing in your NNEXRAD request from OU since the volume in the IDS|DDPLUS feed is equivalent to the volume of NNEXRAD data you are requesting. I would expect that the latencies for UNIWISC would also decrease, but the ones for HDS would probably not since it comprises the bulk of the volume in the combined request. >Hope you had a wonderful thanksgiving :) It was grand, thanks. I hope yours was as much fun as mine!! Cheers, Tom -- NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us. >From address@hidden Mon Nov 29 16:37:18 2004 >Please let us know the results of your 'traceroute'. forgot these earlier... will reply to your last message tomorrow ... it's almost by bedtime :) ------------------[ from weather.bgsu.edu ]------------------------ [pfrancis@weather pfrancis]$ /usr/sbin/traceroute atm.geo.nsf.gov traceroute to atm.geo.nsf.gov (198.181.231.53), 30 hops max, 38 byte packets 1 router1-112 (129.1.112.252) 0.307 ms 0.255 ms 0.232 ms 2 10.9.2.1 (10.9.2.1) 0.525 ms 0.431 ms 0.468 ms 3 10.9.10.12 (10.9.10.12) 0.354 ms 0.297 ms 0.237 ms 4 129.1.8.126 (129.1.8.126) 0.533 ms 0.454 ms 0.442 ms 5 gw-bgsu (129.1.8.253) 1.604 ms 1.558 ms 2.110 ms 6 dytnq-r3-at2-0s6.bb.oar.net (192.88.192.173) 25.119 ms 25.761 ms 25.201 ms 7 clmbq-r2-at2-0s13.bb.oar.net (192.88.191.102) 28.993 ms 27.149 ms 27.254 ms 8 abilene-IPLSng.ohio-gigapop.oar.net (192.88.192.134) 33.395 ms 32.685 ms 33.648 ms 9 chinng-iplsng.abilene.ucaid.edu (198.32.8.76) 44.590 ms 38.635 ms 48.238 ms 10 nycmng-chinng.abilene.ucaid.edu (198.32.8.83) 58.707 ms 84.901 ms 62.806 ms 11 washng-nycmng.abilene.ucaid.edu (198.32.8.85) 61.339 ms 61.714 ms 64.282 ms 12 dcne-abilene-oc48.maxgigapop.net (206.196.177.1) 61.604 ms 60.610 ms 62.031 ms 13 arlg-so3-1-0.maxgigapop.net (206.196.178.41) 61.171 ms 60.459 ms 61.173 ms 14 nsf-rtr.maxgigapop.net (206.196.177.138) 61.883 ms 61.537 ms 61.390 ms 15 atm.geo.nsf.gov (198.181.231.53) 61.655 ms 61.956 ms 62.473 ms [pfrancis@weather pfrancis]$ ---------------------[ from unidata.bgsu.edu ]-------------------------- [pfrancis@unidata pfrancis]$ /usr/sbin/traceroute atm.geo.nsf.gov traceroute to atm.geo.nsf.gov (198.181.231.53), 30 hops max, 38 byte packets 1 router1-112 (129.1.112.252) 0.286 ms 0.233 ms 0.225 ms 2 10.9.2.1 (10.9.2.1) 0.493 ms 0.404 ms 0.390 ms 3 10.9.10.12 (10.9.10.12) 0.293 ms 0.246 ms 0.229 ms 4 129.1.8.126 (129.1.8.126) 0.395 ms 0.385 ms 0.387 ms 5 gw-bgsu (129.1.8.253) 3.244 ms 2.377 ms 2.759 ms 6 dytnq-r3-at2-0s6.bb.oar.net (192.88.192.173) 25.752 ms 24.496 ms 26.233 ms 7 clmbq-r2-at2-0s13.bb.oar.net (192.88.191.102) 28.529 ms 27.223 ms 28.123 ms 8 abilene-IPLSng.ohio-gigapop.oar.net (192.88.192.134) 32.394 ms 33.580 ms 36.879 ms 9 chinng-iplsng.abilene.ucaid.edu (198.32.8.76) 42.701 ms 44.354 ms 72.150 ms 10 nycmng-chinng.abilene.ucaid.edu (198.32.8.83) 64.432 ms 69.913 ms 57.960 ms 11 washng-nycmng.abilene.ucaid.edu (198.32.8.85) 61.733 ms 67.642 ms 70.044 ms 12 dcne-abilene-oc48.maxgigapop.net (206.196.177.1) 60.852 ms 62.229 ms 62.586 ms 13 arlg-so3-1-0.maxgigapop.net (206.196.178.41) 60.994 ms 61.581 ms 60.775 ms 14 nsf-rtr.maxgigapop.net (206.196.177.138) 62.236 ms 64.352 ms 62.337 ms 15 atm.geo.nsf.gov (198.181.231.53) 70.127 ms 70.208 ms 73.690 ms [pfrancis@unidata pfrancis]$ ------------------------------------------------------------------------------ >From address@hidden Tue Nov 30 07:20:25 2004 Pat, I have given a higher traffic priority in the packet shaper to all traffic to and from weather.bgsu.edu and unidata.bgsu.edu. Let me know if this improves anything. Thanks -Mike