Re: mlong time no see - and multicast

> From: "David E. Wilensky" <address@hidden>
> To: address@hidden
> Cc: Joe Fahle <address@hidden>, "Wilensky, David" <address@hidden>
> Subject: mlong time no see - and multicast

Hi David,

> It's been a long time.  I left LSU a little over two years ago and
> am working for SeaSpace now, with Joe Fahle.
> 
> We're looking at high volume data distribution issues lately and I
> recalled being at an ATAC meeting three or so years ago where
> multicast was discussed for a potential new LDM architecture, along
> with a presentation by a recent PhD from (I think) U.C. Berkeley.

Yes, I remember that too.  There are now many research and commercial
implementations of "reliable multicast", e.g.:

  http://research.ivv.nasa.gov/RMP/links.html

including a link to RFC 2357, "IETF Criteria for Evaluating Reliable
Multicast Transport and Application Protocols":
 
  ftp://ftp.isi.edu/in-notes/rfc2357.txt

which discusses the issues and argues that a single solution that solves all
the demands for reliable multicast is likely to be infeasible:

  As we discuss in Section 3, the issues raised by reliable multicast
  are considerably more complex than those related to reliable unicast.
  In particular, in today's Internet, reliable multicast protocols
  could do great damage through causing congestion disasters if they
  are widely used and do not provide adequate congestion control.

I believe NWS has decided to use StarBurst for an experimental
distribution of NIDS products to all the weather forecast offices

  http://www.starburstcom.com/patches/mcastres.htm

> I'm wondering if you've made any decisions along those lines, and if
> so, if you've solved the technical issues at the time;
> 
> a) router tranmission of multicast packets
> b) reliability of the data transmission, espcially dealing with the fact
>    that multicast protocols are build on UDP.
> c) developer sources / toolkits

The only decision we've made is that all the current research and
development on "reliable" multicast seems to not be very relevant to
our IDD work.  For the kind of reliability we need, we have to deal
with outages where a node can be down for periods from minutes to an
hour, so thousands of products comprising hundreds of megabytes need
to be kept for retransmission later.  The reliable multicast research
seems to be aimed at solving a different problem, where packets are
buffered up in routers to deal with outages of milliseconds to seconds.

The way we might make use of multicast would be to let it deal with
very short outages and dropped packets, but to have the system make
regular and automatic use of archive sites to request missed data from
longer outages.  This would require some fairly basic changes to the
design, and we're currently leaning toward incorporating some kind of
automatic routing into the system to handle failures instead, rather
than using multicast.  However, we've just hired a new person (Anne
Wilson) who will be helping to design new platform-independent
software for the IDD infrastructure, and some of these basic design
decisions are still pending.

> I hope things are going well with you folks up there.
>
> I was pretty sad to hear about Glenn.
>
> Regards to Ben Domenico and Dave Fulker.

Thanks.  Hope you are doing well at SeaSpace also, and regards to Joe
Fahle.

--Russ