20051109: theoretical ldm behavior / adding a feedtype

>From: Rob Cermak <address@hidden>
>Organization: UAF
>Keywords: 200511100150.jAA1oK7s020348 LDM

Hi Rob,

re:
>I hope this is a simple ldm question.  I want to ask you first before I 
>attempt it and crash all my systems.
>
>If we have 3 machines with the ldm installed.
>
>A:
>  Allow all to request MISC from A
>  Request MISC from B,C
>
>B:
>  Allow all to request MISC from B
>  Request MISC from A,C
>
>C:
>  Allow all to request MISC from C
>  Request MISC from A,B
>
>Essentially a redundant star topology...
>
>If machine B inserts a product using the MISC feedtype, will it send the
>topology into an endless loop?

No _if_ the queues on all machines are large enough so that the product
will still be in the queue when offered from the other machines.

>Will copies forever be transmitted?  Or
>does the checksum prevent endless propogation?

Duplicate product detection and rejection works when the duplicate
product(s) offered match a product already in the queue.

>Will B get a single copy back or does the checksum prevent A and C from 
>sending a copy back to it?

The checksum will prevent the loop as long as the product remains in B's
queue.

>How difficult is it to add custom feedtypes to the LDM?  

This is not possible at this time.  The LDM supports 31 feed types and
they are essentially all in use.  We recommend use of the EXP feed type
for site data transmissions.  Appropriate creation of product headers
coupled with LDM-6.4 capabilities to limit limit the allow of data
from downstreams in many ways eliminates the need for multiple feed
types.  A simple example of this might better explain what I mean.

Currently there are defined set of products flowing in the HDS feed
type.  These could be included in an EXP feed and still be recognized
as being a set if the product headers included a mnemonic that
reflected the set:

EXP HDS original header

A downstream cound request the stream of HDS products using:

request EXP     "HDS"   hostname

In LDM-6.4, the upstream can limit a downstrea to a portion of a
feed:

allow   EXP     host(s) pattern_to_allow        pattern_to_NOT_allow

I have been trying to get folks to think hard about how they create
headers for products they insert in the queue.  With appropriate
structuring you can support a virtually unlimited set of streams within
a single feed type _if_ you are using LDM-6.4.

>If the LDM prevents loopback, then I would recommend adding a generic OOS
>feedtype to the next version of the LDM so that I can try to see if the
>emerging Ocean Observing System community is willing to deploy the LDM for
>dissemination near real time of unique datasets.

Please use the EXP feed type and specify 'OOS' as the first piece of
the headers for the products you want to relay.

>Maybe this is a bigger task than I think with may pitfalls you have
>overcome in the past...  Any other suggestions will also be appreciated.

Please let me know if you have questions about the above approach.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.