Re: [conduit] Slower transfer rates from NCEP

Oh, and to answer your other question, the slowdowns (I don't know if they were to boulder or ncep) seemed to start around Thursday of last week and got to the point where I noticed them on Friday.

The attached screenshot is my latency graph taken at 17:08 UTC on Friday, Feb 19. So the right side of the graph is that time, and each of the peaks represents a 6 hourly NCEP model suite. So the peak at the far right is the 12 UTC run from Friday 2/19, working to the left you see the 06 UTC, 00 UTC 2/19, and 18 UTC 2/18 that go off the scale. The previous 12, 06, 00 and 18, and even a bit 12 UTC on the 17th (if I'm interpreting correctly, going from left to right, there are two spikes near the left axis, and then relatively low numbers, then a spike not quite a quarter of the way across from the left to right - I think that's 12 UTC on the 17th)

I am still confused as to why when I was feeding conduit from idd.unidata.ucar.edu and ncepldm4.woc.noaa.gov there were three lines on that graph (red, green blue) but when I feed from conduit.ncep.noaa.gov and ncepldm4.woc.noaa.gov, there is only one line (blue) both last Friday, and now again at the far right of the current graph.

Tom or Steve, perhaps you have a sense as to why this is?

Pete

--
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - address@hidden

From: Carissa Klemmer - NOAA Federal <address@hidden>
Sent: Thursday, February 25, 2016 11:50 AM
To: Pete Pokrandt
Cc: address@hidden; address@hidden; Gerry Creager - NOAA Affiliate
Subject: Re: [conduit] Slower transfer rates from NCEP

Pete,

Can you confirm that the slow downs to Boulder also started on Friday? And we'd appreciate testing conduit.ncep again. You are the first who has said they haven't seen an improvement. Before I report this I'd like to be 100% sure that both Boulder and conduit.ncep (Maryland) are bad.

Carissa Klemmer
NCEP Central Operations
Dataflow Team Lead
301-683-3835

On Thu, Feb 25, 2016 at 12:15 PM, Pete Pokrandt <address@hidden> wrote:

Here's an mtr output from about the last 16 hours or so, and also a traceroute from idd.aos.wisc.edu to conduit.ncep.noaa.gov. No real change as far as I can see..

mtr conduit.ncep.noaa.gov

Host Loss% Snt Last Avg Best Wrst StDev

1. r-cssc-b280c-1-core-vlan-510-primary.net.wisc.edu 0.0% 69823 3.3 0.6 0.5 67.2 5.2

2. internet2-ord-600w-100G.net.wisc.edu 0.0% 69823 18.0 18.0 18.0 92.5 5.3

3. et-10-0-0.107.rtr.clev.net.internet2.edu 0.0% 69823 27.9 27.6 27.6 102.2 5.3

4. et-11-3-0-1276.clpk-core.maxgigapop.net 0.0% 69823 37.4 37.1 37.1 112.2 5.7

5. noaa-i2.demarc.maxgigapop.net 0.0% 69823 37.5 37.5 37.3 430.0 22.2

6. 140.90.111.36 0.0% 69823 40.6 39.7 37.4 505.4 43.2

7. 140.90.76.69 0.1% 69823 37.9 37.7 37.5 161.9 8.7

8. ???

traceroute conduit.ncep.noaa.gov

raceroute to conduit.ncep.noaa.gov (140.90.101.42), 30 hops max, 60 byte packets

1 r-cssc-b280c-1-core-vlan-510-primary.net.wisc.edu (144.92.130.3) 1.115 ms 2.274 ms 1.083 ms

2 internet2-ord-600w-100G.net.wisc.edu (144.92.254.229) 21.264 ms 21.261 ms 21.233 ms

3 et-10-0-0.107.rtr.clev.net.internet2.edu (198.71.45.9) 32.370 ms 32.091 ms 31.255 ms

4 et-11-3-0-1276.clpk-core.maxgigapop.net (206.196.177.4) 37.971 ms 38.678 ms 38.066 ms

5 noaa-i2.demarc.maxgigapop.net (206.196.177.118) 37.997 ms 38.160 ms 38.298 ms

6 140.90.111.36 (140.90.111.36) 41.625 ms 41.252 ms 41.509 ms

7 140.90.76.69 (140.90.76.69) 41.837 ms 43.261 ms 42.738 ms

8 * * *

9 * * *

10 * * *

The latency graph for our conduit data hasn't changed either - we still see the ekg looking lags coming through ncep-ldm0.ncep.boulder_v_idd.uni

http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

There are no times on that graph, but it covers roughly 3 days - each latency peak corresponds with a 6 hourly NCEP model run suite.

I have not tried actually getting CONDUIT data from conduit.ncep.noaa.gov since the slowness started last Friday. I'll try again pulling data from there and let you know what happens.

Pete

--
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - address@hidden