20030723: LSU LDM Operations and status

>From: Robert Leche <address@hidden>
>Organization: LSU
>Keywords: 200307231714.h6NHEELd002848 IDD topology

Hi Bob,

>I want to let you know about a time error problem on Datoo that was 
>corrected today, as well as solicit your thoughts on Seistan's 
>performance and status.

Thanks for the update.

>The NTP process on Datoo failed to resume normal operation after the 
>last restart. I made changes to the order of NTP uplink hosts and this 
>restored time services. The time skew of 70 - 100 seconds was 
>propagating to all down stream LDM hosts.

OK.  The good news is that the great majority of data that is being
relayed through seistan originates from the Unidata NOAAPORT ingest
machine, jackie.unidata.ucar.edu.  The reason for this is that the
UPC-developed NOAAPORT ingest system is considerably faster at getting
a new NOAAPORT product into its LDM queue than the SSEC SDI system.  In
this context, "considerably" means on the order of one to a few
seconds.

As an example, all IDS|DDPLUS and HDS data is relayed from jackie to
thelma.ucar.edu, then to undata2.ssec.wisc.edu, then to sirocco, and
finally to seistan.  This entire process occurs faster than the time it
takes to get the same IDS|DDPLUS or HDS product from the SDI on datoo
to sirocco or seistan.

>How long do you want to run Seistan in test mode? All indications are 
>the latency problem we experienced three weeks ago have cleared up. If 
>you agree the problem is resolved, then is it time to remove the test 
>feeds and move to a more permanent operational mode?

I have been monitoring the latencies for hosts downsteam of seistan (at
least for those reporting latency numbers) since our conference call,
and I have been very pleased with what I have seen.  I had been
thinking about moving out of the test mode (where several UPC machines
are ingesting data from seistan) and into a more normal operational
mode for a couple of weeks, but things have been so busy here that I
kept putting off initiating the change.  Given all of this, your email
is very timely.

>One consideration 
>in timing the feed changes is our telecommunications people have 
>requested that we make and finalize the changes in LDM topology before 
>the week of August the 4th. This is so LSU network specialist are 
>available to help monitor traffic and/or trouble shoot problems if 
>problems are detected (while changes to LDM topology are being made). 

The only problem in this concept is that the set of machines that an
IDD toplevel relay machine will feed is never fixed.  Downstream hosts
get added and removed as the need arises.  I understand that it might
seem to make sense to fix a downstream topology to the network folks,
but that is not how the IDD works/should work.  By the way, I would be
worried if the IT folks are looking for a fixed list of sites seistan
will be feeding since it would seem like their next move would be to
reintroduce packet shaping to hosts not on the fixed list.  A return to
packet shaping should be resisted with as much enthusiasm as can be
mustered so that the IDD topology can be adjusted as needed.

One last comment on maintaining IDD topology flexibility:  one of the
research projects here at the UPC is the development of an adaptive
routing scheme for IDD-transmitted data.  What this will mean in the
future is that the set of sites feeding from any one IDD relay would be
dynamic, and the dynamicism could be vary in the time range of one
to several hours.

>The August 4th time frame is a time when our students return to campus 
>and a time when the telecommunications department resources are 
>stretched to the limit handling the students needs.

I understand their concerns, but I was left with the impression that
the IT folks had created a large pipe that was to be used only for LDM
traffic, and that the lifetime of this pipe was limited.  Is your
understanding different from this?

>From my stand point, I am not under pressure to make topology changes, 
>but if you are planning to make topology changes, making the changes 
>before the week of August 4th makes sense.

I will discuss this with the LDM group here at the UPC and see what
should be done.

>Best regards,

Cheers,

Tom

>From address@hidden Wed Jul 23 12:32:27 2003

Tom,
The LSU Office of Telecommunications (OTC) has dedicated a 20MB virtual 
pipe to I2. There are no plans to change this. The concern was a man 
power resource issue within OTC, not a network resource issue. OTC 
requested we perform the change(s) while OTC had the man power to 
monitor the connection.

You and I are free to make changes in LDM topology at any time, as the 
need araises, regardless of OTC's schedule.

Bob
-- 
----------------------------------------------------------------
Robert Leche
System Administrator
Louisiana State University - Southern Regional Climate Center
260 Howe-Russell Building
Baton Rouge, La. 70803
address@hidden
225 578 5023
----------------------------------------------------------------