[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IDD #ZAJ-969398]: IDD Data is too old
- Subject: [IDD #ZAJ-969398]: IDD Data is too old
- Date: Tue, 06 Nov 2018 15:01:03 -0700
Hi,
> The hostname in the registry is "irads-ingest0.net.ou.edu".
That's the name that appears on the rtstats(1) webpage,
<http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex>, which is the
*only* thing that name is used for, which I why we need the IP address.
> In the registry, we have 'time-offset' lowered to 600 seconds. Even with that
> low of a
> period to backfill, it never catches up. In fact, it gets further behind.
If, on average, products want to come in faster than the LDM can insert them
into the queue, then latency will increase regardless of the size of the queue.
Something is slowing how fast your LDM can insert data-products. Possibilities
include 1) a really slow system trying to do too much (not likely, IMHO); and
2) some network problem that's reducing the effective bandwidth of the TCP
connections (far more likely, IMHO).
> Yesterday evening, I tested idd.meteo.psu.edu. The same issue occurred. We
> were able to
> get a bit better performance by splitting the request into two, but it still
> doesn't
> perform well.
That's consistent with both hypotheses above.
> While we've been focused on one host, irads-ingest0, at 156.110.246.56, we
> are seeing the
> same issue from echo-ingestA.services.ou.edu (129.15.2.32). It has been
> tested against
> idd.unidata.ucar.edu and idd.aos.wisc.edu and behaves the same as
> irads-ingest0. To add
> to the issue, both hosts have started having issues receiving the NL2 TDS
> feed as well.
That's consistent with the network hypothesis but not with the slow system
hypothesis.
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NEXRAD3+echo-ingesta.services.ou.edu
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NEXRAD2+echo-ingesta.services.ou.edu
>
> The two hosts are in spate networks
spate?
> on separate campuses, managed by different parts of
> the IT department. With that, we have put in a ticket with our ISP, OneNet.
> They have
> requested traceroutes from our upstream sources to our hosts, if possible, to
> trace the
> data path.
>
> Would it be possible to get traceroutes from a host behind
> idd.unidata.ucar.edu to
> 129.15.2.32 and 156.110.246.56?
Here you go:
~: traceroute 129.15.2.32
traceroute to 129.15.2.32 (129.15.2.32), 30 hops max, 60 byte packets
1 flr-n140.unidata.ucar.edu (128.117.140.251) 0.660 ms 1.127 ms 1.572 ms
2 ml2core-fl2core.unet.ucar.edu (128.117.243.194) 1.101 ms 1.104 ms 1.496
ms
3 corel3-ml2core-i2.unet.ucar.edu (128.117.243.141) 1.519 ms 1.553 ms
1.541 ms
4 v3454.rtr-chic.frgp.net (192.43.217.222) 23.119 ms 23.145 ms 23.130 ms
5 et-2-1-0.4079.rtsw.chic.net.internet2.edu (162.252.70.116) 23.693 ms
23.691 ms 23.806 ms
6 ae-3.4079.rtsw.kans.net.internet2.edu (162.252.70.141) 34.436 ms 34.440
ms 34.475 ms
7 et-7-0-0.4079.rtsw.tuls.net.internet2.edu (162.252.70.35) 38.431 ms
38.370 ms 38.334 ms
8 198.71.46.45 (198.71.46.45) 38.658 ms 38.659 ms 38.642 ms
9 164.58.244.44 (164.58.244.44) 38.614 ms 38.718 ms 38.448 ms
10 164.58.244.15 (164.58.244.15) 41.176 ms 41.325 ms 41.317 ms
11 164.58.245.55 (164.58.245.55) 40.658 ms 40.803 ms 164.58.245.53
(164.58.245.53) 40.984 ms
12 164.58.245.56 (164.58.245.56) 41.402 ms 164.58.245.58 (164.58.245.58)
40.972 ms 40.969 ms
13 164.58.244.33 (164.58.244.33) 41.475 ms 41.507 ms 41.496 ms
14 164.58.10.98 (164.58.10.98) 41.416 ms 41.593 ms 41.586 ms
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
~: traceroute 156.110.246.56
traceroute to 156.110.246.56 (156.110.246.56), 30 hops max, 60 byte packets
1 flr-n140.unidata.ucar.edu (128.117.140.251) 0.714 ms 0.873 ms 0.865 ms
2 ml1core-flacore.unet.ucar.edu (128.117.243.78) 0.949 ms 0.951 ms 1.346 ms
3 ml2core-ml1core.unet.ucar.edu (128.117.243.99) 0.915 ms
ml2core-fl2core.unet.ucar.edu (128.117.243.194) 1.368 ms 1.363 ms
4 corel3-ml2core-i2.unet.ucar.edu (128.117.243.141) 1.710 ms 1.710 ms
1.690 ms
5 v3454.rtr-chic.frgp.net (192.43.217.222) 23.139 ms 23.077 ms 23.075 ms
6 et-2-1-0.4079.rtsw.chic.net.internet2.edu (162.252.70.116) 23.555 ms
23.538 ms 23.594 ms
7 ae-3.4079.rtsw.kans.net.internet2.edu (162.252.70.141) 34.683 ms 34.676
ms 34.463 ms
8 et-7-0-0.4079.rtsw.tuls.net.internet2.edu (162.252.70.35) 38.283 ms
40.679 ms 40.648 ms
9 198.71.46.45 (198.71.46.45) 39.418 ms 39.410 ms 39.389 ms
10 164.58.244.44 (164.58.244.44) 39.205 ms 164.58.244.46 (164.58.244.46)
38.648 ms 164.58.244.44 (164.58.244.44) 38.710 ms
11 164.58.244.217 (164.58.244.217) 38.561 ms 38.554 ms 38.532 ms
12 164.58.16.26 (164.58.16.26) 38.372 ms 38.370 ms 38.885 ms
13 156.110.254.97 (156.110.254.97) 46.317 ms 45.003 ms 45.072 ms
14 156.110.254.50 (156.110.254.50) 41.374 ms 156.110.254.62 (156.110.254.62)
44.831 ms 156.110.254.50 (156.110.254.50) 41.488 ms
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
~:
It should also be possible to run traceroute on those systems to
idd.unidata.ucar.edu.
Regards,
Steve Emmerson
Ticket Details
===================
Ticket ID: ZAJ-969398
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.