[IDD #TNF-892711]: idd.cise-nsf.gov connectivity marginal

Hi Art,

I apologize for not responding to your email before now...

re:
> We noticed a slowdown in getting CONDUIT data from idd.cise-nsf.gov back
> in mid-September and since then we've been working with our network folks
> (who have been working with Unidata and probably others) to try and remedy
> the situation.  Some progress seems to have been made, however the
> bandwidth coming out of idd.cise-nsf.gov still appears to be marginal.

Part of the problem with feeds out of idd.cise-nsf.gov are likely related
to how we are doing clustering there, not with the available network
bandwidth at NSF.  We are set to update the systems at NSF in the not
too distant future, and those problems should go away or be minimized.

re:
> In evaluating the situation, I notice that both idd.cise-nsf.gov and
> ncepldm.woc.noaa.gov (the CONDUIT source for the tier-1 machines) are both
> behind the MAX Gigapop domain.  However, none of the tier-1 machines
> feeding CONDUIT from ncepldm.woc.noaa.gov through the MAX are seeing
> delays, whereas the sites that I've checked (idd-ingest.meteo.psu.edu,
> sun1.envsci.rutgers.edu, emo.unidata.ucar.edu) feeding CONDUIT from
> idd.cise-nsf.gov through the MAX *are* seeing CONDUIT delays, and they all
> appear identical.  This leads me to believe that the connectivity
> constraint is very close to the NSF feed source inside the MAX rather than
> with the MAX itself or at other points in-between.

Again, we believe this to be a result of the method we are using to cluster
machines at NSF.  The upgrade that we will perform will better align the NSF
system with the toplevel IDD relay we operate here at UCAR, idd.ucar.edu.

re:
> Given that this problem has been going on for about 3 months now with no
> resolution (which leads me to think this is the best we can do), I'm
> wondering if this is a good time to consider switching the tier-1 service
> from idd.cise-nsf.gov to ldm.meteo.psu.edu?

We have no concern with switching top level IDD relay duties PSU other than
your constraint for only feeding .edu sites via I2.  

Questions:

- if PSU became the toplevel relay for CONDUIT in the East, would 
idd.cise-nsf.gov
  be allowed to feed from it?

- what about other non-.edu sites that we want to be fed?  For instance, 
idd.cise-nsf.gov
  is currently feeding several international, U.S. government, U.S. .orgs
  and at least one U.S. .mil sites.

- what about all of the other feeds available in the IDD?  For instance, NEXRAD
  Level II data now routinely exceeds CONDUIT in bandwidth use.

re:
> Based on the performance I
> observe from the other 3 tier-1 sites pulling CONDUIT from
> ncepldm.woc.noaa.gov, it leads me to believe that we should be able to
> provide similar performance and eliminate the bottleneck that seems to be
> occuring getting data from idd.cise-nsf.gov.

We agree, but we have to be able to address the concerns behind the
questions above.

re:
> Let me know what you think...

We don't want you to get the impression that we do not want PSU to
take a/the leading role in IDD relay for the Eastern U.S.; we do.
We just need to make sure that that sites that are already being
fed data continue to be provided for, and new, unanticipated synergies
be possible.

Some relevant and ancillary information:

- idd.cise-nsf.gov currently has about 200 downstream connections

- idd.cise-nsf.gov is currently serving about 120 Mbps of data on
  average

- the upgrade that we will be doing will include a motherlode.ucar.edu
  clone that will provide remote access to data via THREDDS/ADDE, etc.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: TNF-892711
Department: Support IDD
Priority: Normal
Status: Open