Re: 19990716: CONDUIT latencies

Chiz,
> 
> There are some things we can do to tune the LDM requests,
> and we can look at other upstreams. 
> 
> There is currently 1 bottle neck that drives the initial latencies up.
> This is the T-1 line from NCEP to GSFC which is roughly capable of
> delivering 550MB per hour. Since the AVN (now out to 84 hours) can exceed
> 600MB, and the MRF is also pushing 500MB then things can get backlogged-
> especially when NCEP has Cray crashes and then come back up and post
> the data all at once.

Not much we can do about that one (at least for now).
> 
> I have found I can get better throughput to the upstream using
> seaparte request lines like:
> 
> request NMC2    ".*[036]$"       128.183.252.98
> request NMC2    ".*[147]$"       nora.gsfc.nasa.gov
> request NMC2    ".*[^013467]$"   conduit_a


I'll try that. I'll have to figure out the best
way to do it though. Right now my request reads:

request NMC2 "(.status)|(.*erl/icwf)|(.*mrf/(mrf|ens))|(.*ens/ens.*/tar)" flood.
atmos.uiuc.edu

which I *believe* should get me only status messages, eta, mrf and ensemble 
forecasts.
(I don't really need the other stuff right now, so, since I don't
have a downstream node for the NMC2 feed I restricted what I
requested in an attempt to get better throughput)

I guess I would just put two entries:

request NMC2 
"(.status)|(.*erl/icwf)|(.*mrf/(mrf|ens))|(.*ens/ens.*/tar).*[02468]$" 
flood.atmos.uiuc.edu
request NMC2 
"(.status)|(.*erl/icwf)|(.*mrf/(mrf|ens))|(.*ens/ens.*/tar).*[13579]$" 
128.174.80.47

Or, actually, maybe these should read ??? :

request NMC2 
"(.status.*[02468]$)|(.*erl/icwf.*[02468]$)|(.*mrf/(mrf|ens).*[02468]$)|(.*ens/ens.*/tar.*[02468]$)"
 flood.atmos.uiuc.edu
request NMC2 
"(.status.*[13579])|(.*erl/icwf.*[13579])|(.*mrf/(mrf|ens).*[13579])|(.*ens/ens.*/tar.*[13579])"
 128.174.80.47

I've used this trick in the past to split up the Unidata feed into FOS and 
MCIDAS.


> 
> I provided an alias in our hosts table for the third entry above.
> By using different names, the LDM actually will create a seperate
> rpc.ldmd connection for each subset.
> 
> The idea here is that all the conduit products have a sequence number
> tagged on to the end, and the split requests seem to get more
> round robin time from the router than a single request.
> 
> Unfortunately PSU did not feel like they had a good connection to gsfc,
> so the feed from back here same as UIUC, but I was wondering what your
> traceroutes to profhorn.meteor.wisc.edu and nora.gsfc.nasa.gov look like.

traceroute to wisc is OK, but not great:

traceroute to profhorn.meteor.wisc.edu (144.92.131.143), 30 hops max, 40 byte 
packets
 1  be102 (169.226.4.1)  4 ms  1 ms  1 ms
 2  169.226.13.1 (169.226.13.1)  1 ms  4 ms  1 ms
 3  at-gw3-alb-1-0-T3.appliedtheory.net (169.130.23.5)  2 ms  2 ms  2 ms
 4  at-gw2-alb-0-0.appliedtheory.net (169.130.20.2)  3 ms  3 ms  3 ms
 5  at-gw2-syr-1-1-0-T3.appliedtheory.net (169.130.1.22)  5 ms  5 ms  5 ms
 6  at-bb1-pen-5-0-0-T3.appliedtheory.net (169.130.1.121)  42 ms  39 ms  39 ms
 7  sl-bb10-pen-4-2.sprintlink.net (144.232.5.109)  62 ms  33 ms  48 ms
 8  sl-bb10-nyc-6-0-622M.sprintlink.net (144.232.9.101)  37 ms  27 ms  34 ms
 9  sl-bb10-chi-6-0.sprintlink.net (144.232.9.150)  37 ms  40 ms  37 ms
10  sl-gw14-chi-8-0-0.sprintlink.net (144.232.0.194)  287 ms  300 ms  275 ms
11  sl-napnet-2-0-0-T3.sprintlink.net (144.228.159.18)  43 ms  42 ms  44 ms
12  chi1-oc3.wisc.edu (207.227.0.202)  45 ms  52 ms  56 ms
13  r-peer-a4-0-0-3.net.wisc.edu (216.56.1.18)  47 ms *  75 ms
14  r-macc.net.wisc.edu (144.92.128.129)  70 ms  83 ms  87 ms
15  ProfHorn.meteor.wisc.edu (144.92.131.143)  120 ms  81 ms  75 ms

This seems to be about typical.

However, a tracroute to nasa looks really good!

traceroute to nora.gsfc.nasa.gov (128.183.104.227), 30 hops max, 40 byte packets
 1  be102 (169.226.4.1)  6 ms  1 ms  1 ms
 2  169.226.13.1 (169.226.13.1)  1 ms  1 ms  1 ms
 3  at-gw3-alb-1-0-T3.appliedtheory.net (169.130.23.5)  30 ms  2 ms  161 ms
 4  at-gw2-alb-0-0.appliedtheory.net (169.130.20.2)  3 ms  4 ms  3 ms
 5  at-gw2-syr-1-1-0-T3.appliedtheory.net (169.130.1.22)  6 ms  5 ms  5 ms
 6  at-bb1-pen-5-0-0-T3.appliedtheory.net (169.130.1.121)  37 ms  16 ms  37 ms
 7  128.161.250.1 (128.161.250.1)  29 ms  24 ms  26 ms
 8  128.161.10.37 (128.161.10.37)  23 ms  20 ms  20 ms
 9  rtr-wan1-ef.gsfc.nasa.gov (192.43.240.33)  25 ms  22 ms  25 ms
10  rtr-cbr5a-a.gsfc.nasa.gov (198.119.59.4)  21 ms  25 ms  21 ms
11  nora.gsfc.nasa.gov (128.183.104.227)  23 ms  24 ms  29 ms

again these numbers are consistant for several tries.


> 
> For the past month we have also been downgraded from out 100baseT
> connection to a 10baseT connection while our building is rewired.
> When our switch gets moved to the new wiring closet it could improve
> things as well.
> 
> 
> Steve Chiswell
> Unidata User Support
Thanks
David

David Knight
Department of Earth and Atmospheric Sciences   Tel: (518)-442-4204
SUNYA   ES-228                                 Fax: (518)-442-4494
Albany, NY  12222                              Email: address@hidden