20011008: motherlode feed slow? (cont.)

>From: "Arthur A. Person" <address@hidden>
>Organization: Penn State
>Keywords: 200110061836.f96IaT125601 IDD

Art,

>Our latencies have quickly popped back up to about 55 minutes now from
>motherlode.  SSEC NEXRAD is still running within seconds.  I'm working
>with our networking people now to see if they can discover anything.

We see a jump in traceroute times at our abilene connection right at the
moment, but I don't know if this is enough to slow the delivery from
motherlode to downstream sites to the extent you are seeing:

motherlode.ucar.edu% traceroute sunny.atmos.washington.edu
traceroute to sunny.atmos.washington.edu (128.95.89.38), 30 hops max, 40 byte 
packets
 1  mlra-n13 (128.117.13.253)  0.511 ms  0.542 ms  0.310 ms
 2  vbnsr-n2 (128.117.2.252)  0.463 ms  0.591 ms  0.335 ms
 3  internetr-n243-104 (128.117.243.106)  1.107 ms  1.042 ms  0.873 ms
 4  denv-abilene (128.117.243.126)  2.196 ms  2.739 ms  2.150 ms
 5  sttl-dnvr.abilene.ucaid.edu (198.32.8.49)  29.941 ms  30.317 ms  33.031 ms
 6  hnsp1-wes-S5-0-0.pnw-gigapop.net (198.48.91.77)  30.270 ms  30.288 ms  
29.983 ms
 7  uwbr1-GE5-0.cac.washington.edu (198.107.151.51)  29.922 ms  29.917 ms  
30.412 ms
 8  dustbuster-V49.cac.washington.edu (140.142.150.2)  30.968 ms  30.772 ms  
30.594 ms
 9  sunny.atmos.washington.edu (128.95.89.38)  30.325 ms  30.527 ms  30.436 ms

I am CCing this to Anne for further comment/action.

Tom

>On Mon, 8 Oct 2001, Arthur A. Person wrote:
>
>> On Sun, 7 Oct 2001, Unidata Support wrote:
>>
>> > >From: "Arthur A. Person" <address@hidden>
>> > >Organization: UCAR/Unidata
>> > >Keywords: 200110061836.f96IaT125601
>> >
>> > Art,
>> >
>> > Sorry I didn't see this before this morning.
>> >
>> > >Our feed from motherlode is about 40 minutes behind...  this has been
>> > >happening for several hours according to our users.  Is motherlode having
>> > >a problem, or some networking issue?  Our feed from SSEC and WSI (for
>> > >example) are on-time.
>> >
>> > 'traceroute's from motherlode to three different machines show a significa
> nt
>> > increase at the abilene connection in Denver, but these are not (by an
>> > order of magnitude) as bad as what we were seeing last week:
>> >
>> > motherlode.ucar.edu% windfall.evsc.virginia.edu (128.143.142.246), 30 hops
>  max, 40 byte packets
>> >  1  mlra-n13 (128.117.13.253)  0.393 ms  0.406 ms  0.321 ms
>> >  2  vbnsr-n2 (128.117.2.252)  0.719 ms  0.808 ms  0.462 ms
>> >  3  internetr-n243-104 (128.117.243.106)  0.835 ms  0.750 ms  0.687 ms
>> >  4  denv-abilene (128.117.243.126)  1.745 ms  1.435 ms  2.519 ms
>> >  5  kscy-dnvr.abilene.ucaid.edu (198.32.8.14)  12.757 ms  12.745 ms  12.61
> 0 ms
>> >  6  ipls-kscy.abilene.ucaid.edu (198.32.8.6)  21.673 ms  21.931 ms  22.044
>  ms
>> >  7  clev-ipls.abilene.ucaid.edu (198.32.8.26)  28.459 ms  27.756 ms  28.04
> 2 ms
>> >  8  nycm-clev.abilene.ucaid.edu (198.32.8.30)  39.994 ms  42.397 ms  40.91
> 6 ms
>> >  9  uva-internet.misc.Virginia.EDU (192.35.48.41)  48.781 ms  48.051 ms  4
> 8.335 ms
>> > 10  carruthers1-all-router.acc.Virginia.EDU (128.143.99.2)  48.793 ms  50.
> 587 ms  48.993 ms
>> > 11  garrett-router.acc.Virginia.EDU (128.143.227.55)  48.427 ms  55.669 ms
>   56.705 ms
>> > 12  windfall.evsc.Virginia.EDU (128.143.142.246)  49.113 ms *  51.171 ms
>> > motherlode.ucar.edu% traceroute ldm.meteo.psu.edu
>> > traceroute to ldm.meteo.psu.edu (128.118.28.12), 30 hops max, 40 byte pack
> ets
>> >  1  mlra-n13 (128.117.13.253)  0.397 ms  0.293 ms  0.260 ms
>> >  2  vbnsr-n2 (128.117.2.252)  0.497 ms  0.723 ms  0.714 ms
>> >  3  internetr-n243-104 (128.117.243.106)  0.985 ms  0.655 ms  0.593 ms
>> >  4  denv-abilene (128.117.243.126)  2.250 ms  1.694 ms  1.895 ms
>> >  5  kscy-dnvr.abilene.ucaid.edu (198.32.8.14)  13.265 ms  12.135 ms  12.27
> 0 ms
>> >  6  ipls-kscy.abilene.ucaid.edu (198.32.8.6)  21.351 ms  21.388 ms  21.532
>  ms
>> >  7  clev-ipls.abilene.ucaid.edu (198.32.8.26)  27.697 ms  27.944 ms  27.63
> 4 ms
>> >  8  abilene.psc.net (192.88.115.122)  31.044 ms  31.789 ms  30.856 ms
>> >  9  penn-state.psc.net (198.32.224.66)  34.750 ms  35.134 ms  35.304 ms
>> > 10  * * *
>> > 11  * * *
>> > 12  * * *
>> > 13 ^C
>> >
>> > motherlode.ucar.edu% traceroute sunny.atmos.washington.edu
>> > traceroute to sunny.atmos.washington.edu (128.95.89.38), 30 hops max, 40 b
> yte packets
>> >  1  mlra-n13 (128.117.13.253)  0.531 ms  0.237 ms  0.376 ms
>> >  2  vbnsr-n2 (128.117.2.252)  0.525 ms  0.578 ms  0.611 ms
>> >  3  internetr-n243-104 (128.117.243.106)  1.091 ms  0.748 ms  0.603 ms
>> >  4  denv-abilene (128.117.243.126)  1.559 ms  1.391 ms  1.361 ms
>> >  5  sttl-dnvr.abilene.ucaid.edu (198.32.8.49)  30.196 ms  30.232 ms  30.05
> 6 ms
>> >  6  hnsp1-wes-S5-0-0.pnw-gigapop.net (198.48.91.77)  29.871 ms  30.391 ms 
>  30.313 ms
>> >  7  uwbr1-GE5-0.cac.washington.edu (198.107.151.51)  30.106 ms  30.904 ms 
>  30.213 ms
>> >  8  dustbuster-V49.cac.washington.edu (140.142.150.2)  31.096 ms  31.053 m
> s  30.981 ms
>> >  9  sunny.atmos.washington.edu (128.95.89.38)  30.621 ms  31.133 ms  30.48
> 3 ms
>> >
>> > Are you still seeing the slowness today?
>>
>> Our traceroutes this morning (Monday) to motherlode don't look bad (except
>> for a few blips now-and-then) but we've seen data slowness all weekend I'm
>> told.  Looking at things this morning, the DDPLUS/HDS stuff is running
>> about 3-4 minutes behind which isn't bad, but normally this stuff is
>> within a minute, usually less.  The NEXRAD stuff from SSEC, for
>> comparison, is within a few seconds of transmission.  Since we're in a
>> non-peak load time, I'm guessing delays from motherlode will increase as
>> the data rate increases... we'll see.  Since the traceroutes look pretty
>> good, it would appear that the data are either being throttled or hitting
>> severe bottlenecks.  I'll continue to monitor it today.
>>
>>                                      Art.
>>
>> >
>> > Tom Yoksas
>
>Arthur A. Person
>Research Assistant, System Administrator
>Penn State Department of Meteorology
>email:  address@hidden, phone:  814-863-1563