Re: [Ncep.list.pmb-dataflow] [conduit] NCEP CONDUIT Outage planned

On Fri, Jun 19, 2020 at 3:40 PM Pete Pokrandt <address@hidden> wrote:

Just an FYI the CONDUIT latency times continue to slowly increase. Unidata is seeing the same slowly increasing lags, so that suggests to me it is an issue with the source. I don't think we're actually getting data that much later, I think the clock has a slow drift on conduit.ncep.noaa.gov.

We are not experiencing any real problems with this right now, but if the lag grows larger than 3600 seconds, I am concerned we will begin losing data, since (as I understand it) the ldm doesn't ask for data more than an hour old.

Can you please have someone look into this at some point?

Thanks,

Pete

-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - address@hidden

From: Dustin Sheffler - NOAA Federal <address@hidden>
Sent: Friday, June 5, 2020 1:50 PM
To: Pete Pokrandt <address@hidden>
Cc: Carissa Klemmer - NOAA Federal <address@hidden>; Anne Myckow - NOAA Federal <address@hidden>; _NCEP.List.pmb-dataflow <address@hidden>
Subject: Re: [Ncep.list.pmb-dataflow] [conduit] NCEP CONDUIT Outage planned

Hi Pete,

Thank you for passing along the information about the increasing lag times your seeing for CONDUIT ingest. That is a very interesting and telling graph for a slowly worsening issue. We will investigate and let you know what we find.

-Dustin

On Tue, Jun 2, 2020 at 5:15 PM 'Pete Pokrandt' via _NCEP list.pmb-dataflow <address@hidden> wrote:

Carissa, Anne,

Thanks so much for keeping us updated on these outages.

FYI I think there is a slow clock drift on the CONDUIT source machine conduit.ncep.noaa.gov. I'm seeing the lags slowly increasing at my ingest site, and I see the same slow increase on conduit.unidata.ucar.edu. I don't see the same drift on data feeds from other sources.

Maybe the conduit.ncep.noaa.gov isn't running ntpd to keep the time accurate?

https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd-agg.aos.wisc.edu

https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+conduit.unidata.ucar.edu

Thanks,

Pete

-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086 - address@hidden

From: conduit <address@hidden> on behalf of Anne Myckow - NOAA Federal via conduit <address@hidden>
Sent: Monday, June 1, 2020 1:00 PM
To: Carissa Klemmer - NOAA Federal <address@hidden>
Cc: address@hidden <address@hidden>; _NCEP.List.pmb-dataflow <address@hidden>
Subject: Re: [conduit] [Ncep.list.pmb-dataflow] NCEP CONDUIT Outage planned

Hello,

Wanted to let you know that the 6/3 outage has been moved to 6/8, same time.

Please let us know if you have any questions, thanks.

Anne

On Thu, May 28, 2020 at 8:23 PM Carissa Klemmer - NOAA Federal <address@hidden> wrote:

Hello -

We have 2 dates of maintenance that I'd like to make you aware of, both of which may impact your ability to pull conduit data.

6/1 between 13-15Z

6/3 between 13-15Z

Expect that the outage will last the entire 2 hours. Data will be queued up to backfill anything that was missed during the outage. If you have any questions please ask the dataflow team.

Thanks,

Carissa Klemmer
NCEP Central Operations
IDSB Branch Chief

301-683-3835

_______________________________________________
Ncep.list.pmb-dataflow mailing list
address@hidden
https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.pmb-dataflow

--

Anne Myckow

Dataflow Team Lead

NWS/NCEP/NCO

301-683-3825

_______________________________________________
Ncep.list.pmb-dataflow mailing list
address@hidden
https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.pmb-dataflow

--

Dustin Sheffler

NCEP Central Operations - Dataflow
5830 University Research Court, Rm 1030
College Park, Maryland 20740

Office: (301) 683-3827