[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20010924: UCAR network slowness?



"Arthur A. Person" wrote:
> 
> >
> > If it's easy to do, it would be nice to allow us in.  It would be
> > helpful to get all the way to your site just as a data point when we're
> > doing such stuff.
> 
> I'll check on this.  Is there a specific machine or subnet that you would
> be doing this from?  This is to minimize our exposure to DOS
> attacks, etc...
> 

Hi Art,

Yes, let's use our Unidata subnet, 128.117.140.  motherlode is on a
different subnet, but this is close enough.


> > Regarding today, I can only identify Abilene as being relatively slow.
> > How long have you been seeing such latencies?  And, I'd be interested to
> > know when you see these latencies go away (assuming that does actually
> > happen).
> 
> I don't believe I've seen anything like this before, although I'm not
> always paying that close an attention.  Our users are usually the first to
> complain about the real-time stuff though, especially obs and model stuff,
> and I'm sure I've never seen it max out to an hour.  This seems quite
> unusual.
> 
> I'm checking back through our stats files and see an episode starting
> during 15Z and subsiding during 16Z the 24th (mild), and then starting
> again during 18Z the 24th and continuing through 08Z the 25th after which
> it abruptly stops.  Here's the stats grep'd for HDS:
> 
> 20010924125048     HDS    ingestor.wunderground.com         30     210813     
>   0.31    7@3542 20010924125044
> 20010924125956     HDS    noaaport.unidata.ucar.edu       1302    5094970     
>   0.55   15@3532 20010924125956
> 20010924125958     HDS       sunshine.ssec.wisc.edu        864    4099772     
>   0.81   18@3529 20010924125959
> 20010924135953     HDS    ingestor.wunderground.com         29     281391     
>   2.99   30@2223 20010924135949
> 20010924135959     HDS    noaaport.unidata.ucar.edu       3708   39883168     
>   0.95   11@3809 20010924135959
> 20010924135949     HDS       sunshine.ssec.wisc.edu       1136    9063820     
>   1.29   10@3812 20010924135949
> 20010924140043     HDS    ingestor.wunderground.com          5      49728     
>  37.48   45@0043 20010924140128
> 20010924145959     HDS    noaaport.unidata.ucar.edu      11727  176893104     
>   2.51   20@3752 20010924150000
> 20010924145947     HDS       sunshine.ssec.wisc.edu       1640   27062572     
>   2.58   18@3755 20010924145948
> 20010924155958     HDS    noaaport.unidata.ucar.edu      19048  178188962     
> 129.24  571@5958 20010924160930
> 20010924155956     HDS       sunshine.ssec.wisc.edu        976   14898380     
>  15.97 1754@5900 20010924162907
> 20010924165957     HDS    noaaport.unidata.ucar.edu      12023  108540708     
> 489.71  723@0454 20010924165957
> 20010924165950     HDS       sunshine.ssec.wisc.edu        150    1405888     
> 121.92 1745@0003 20010924165951
> 20010924175956     HDS    ingestor.wunderground.com        202    1046427     
>   0.10    7@4400 20010924175951
> 20010924175950     HDS    noaaport.unidata.ucar.edu       4141   27054001     
>   1.77   17@3828 20010924175950
> 20010924175959     HDS       sunshine.ssec.wisc.edu       1921   10272797     
>   1.68   18@3945 20010924180001
> 20010924185917     HDS    ingestor.wunderground.com         92     619190     
>  72.60 1155@5917 20010924191832
> 20010924185956     HDS    noaaport.unidata.ucar.edu       3336   32377973     
> 172.82 1185@5956 20010924191941
> 20010924185956     HDS       sunshine.ssec.wisc.edu        996    5079590     
> 101.86 1188@5956 20010924191944
> 20010924193703     HDS    ingestor.wunderground.com         19     299854    
> 1014.76 1250@0201 20010924195345
> 20010924195959     HDS    noaaport.unidata.ucar.edu       3270   63881768    
> 1760.56 3662@4926 20010924210052
> 20010924194204     HDS       sunshine.ssec.wisc.edu        195    1125066    
> 1022.67 3605@4204 20010924204208
> 20010924205959     HDS    noaaport.unidata.ucar.edu       5366   56158135    
> 3624.37 3666@1112 20010924220002
> 20010924215958     HDS    noaaport.unidata.ucar.edu       9717   61521753    
> 3608.69 3660@5332 20010924225956
> 20010924225957     HDS    noaaport.unidata.ucar.edu       7907   52735513    
> 3468.64 3660@1027 20010924234441
> 20010924235956     HDS    noaaport.unidata.ucar.edu       3347   20548242    
> 1681.42 2681@0002 20010925002717
> 20010925005956     HDS    noaaport.unidata.ucar.edu       2621   10798097    
> 1241.42 1714@0339 20010925010802
> 20010925015959     HDS    noaaport.unidata.ucar.edu       5031   50200558     
>  32.73  482@0000 20010925015959
> 20010925011136     HDS       sunshine.ssec.wisc.edu          1       1521    
> 3025.22 3025@1136 20010925020202
> 20010925025959     HDS    noaaport.unidata.ucar.edu      12896  199432662    
> 1635.18 3511@5547 20010925035625
> 20010925035959     HDS    noaaport.unidata.ucar.edu      18137  175819860    
> 3336.56 3660@5247 20010925050026
> 20010925045957     HDS    noaaport.unidata.ucar.edu      12354  110313912    
> 3243.46 3660@0152 20010925053514
> 20010925055955     HDS    noaaport.unidata.ucar.edu       6080   36373335    
> 1124.67 2116@0001 20010925055959
> 20010925065954     HDS    noaaport.unidata.ucar.edu       4396   37255221     
> 163.91  594@5811 20010925070944
> 20010925074430     HDS    ingestor.wunderground.com          1      49167    
> 2316.08 2316@4430 20010925082306
> 20010925075959     HDS    noaaport.unidata.ucar.edu       8881  101765228     
> 792.24 1023@4521 20010925081214
> 20010925075119     HDS       sunshine.ssec.wisc.edu          2      40403    
> 2572.36 3166@2530 20010925082417
> 20010925085630     HDS    ingestor.wunderground.com          7     629202    
> 3450.34 3602@5630 20010925095632
> 20010925085958     HDS    noaaport.unidata.ucar.edu      13069  184533424     
>  88.92  732@0004 20010925085959
> 20010925084743     HDS       sunshine.ssec.wisc.edu         16     265830    
> 1077.03 2003@0231 20010925090038
> 20010925095957     HDS    noaaport.unidata.ucar.edu      12219   79920935     
>   0.86    9@5547 20010925095958
> 20010925095949     HDS       sunshine.ssec.wisc.edu        923    5395727     
>   2.62  293@0148 20010925095951
> 20010925105956     HDS    noaaport.unidata.ucar.edu       5432   43936551     
>   1.69   20@3642 20010925105956
> 20010925105959     HDS       sunshine.ssec.wisc.edu        778    4342190     
>   1.27   15@3650 20010925110000
> 20010925115343     HDS    ingestor.wunderground.com         23     490525     
>   2.22    6@4221 20010925115343
> 20010925115949     HDS    noaaport.unidata.ucar.edu       1923   15462930     
>   0.74   13@3658 20010925115949
> 20010925115933     HDS       sunshine.ssec.wisc.edu        954    6507860     
>   0.96    9@3706 20010925115933
> 20010925121924     HDS    ingestor.wunderground.com         19      47940     
>   0.66    2@1907 20010925121924
> 20010925121928     HDS    noaaport.unidata.ucar.edu        278     925520     
>   0.39    6@1828 20010925121929
> 20010925121938     HDS       sunshine.ssec.wisc.edu        303    1057802     
>   0.58   10@1131 20010925121938
> 
> Here are the complete stats log for comparisons:
> 
...
> 
> BTW, our network folks here pointed me to an Abilene network page at:
> 
> http://hydra.uits.iu.edu/~abilene/traffic/abilene.html
> 
> which showed "red" from NYC all the way to the west coast through Denver.
> I don't know whether "red" in this case means anything, however.  The key
> indicates red as < 10-30% utilization, but there are 3 reds together that
> go up through 70% so it could be confusing also.  I'm sure I did see one
> darker red yesterday somewhere between DENV and NYCM which would take it
> up to 30-60% utilization.
> 
>                                  Art.
> 
> 
> Arthur A. Person
> Research Assistant, System Administrator
> Penn State Department of Meteorology
> email:  address@hidden, phone:  814-863-1563


Thank you for the stats - this is useful.  We are seeing increasing
latencies all over the network and are trying to figure out what's going
on.  At the high latency periods, your stats files also show high
latencies from ssec and wunderground, also implicating the network. 

Thanks for the Abilene pointer. I'm not sure exactly what it means
either, but it's still somewhat informative.

I'll let you know when we come to some conclusion.

If you see this happening again will you please drop me an email?

Thanks!

Anne


-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************