20050901: IDD connectivity

>From:  Alan Czarnetzki <address@hidden>
>Organization:  UNI
>Keywords:  200509011932.j81JWAjo013962 IDD feed host

Hi Alan,

I hope you are well...

>I'm hoping you can point me to the right person to talk to.  We (STORM 
>Project at University of Northern Iowa) have an opportunity to provide 
>radar imagery to our city cable system.  We would essentially be the 
>provider of a community weather channel.  It's a great chance to 
>heighten our visibility in Cedar Falls.

Sounds interesting.

>There are a number of technical 
>issues that we'll need to work through.  The one I'm hoping you can help 
>with is our upstream feed.  Over time, we have experienced latencies and 
>outages that would likely prevent us from being a reliable partner with 
>our city cable system.  Would it be possible for UNI to feed from 
>another site, one that would alleviate this problem?  Any assistance 
>with this is greatly appreciated.

Right now, you are redundantly feeding from the top level IDD relay
node that we operate here at Unidata/UCAR, idd.unidata.ucar.edu, and
the relay at UNL, papagayo.unl.edu.  I know that UNL's relay machine
lost a disk drive yesterday, and the folks there are struggling to get
the machine back up running all of the software that they had before
the failure (the system disk failed).

Since it is our opinion that one can't get a better upstream feed site
than idd.unidata.ucar.edu, and since I see some very high latencies on
thunder.storm.uni.edu in the past couple of hours, I am left with the
impression that the feed request(s) from thunder to idd.unidata are not
of type PRIMARY (e.g., they are SECONDARY aka ALTERNATE).  If this is
the case (please send us the contents of your ~ldm/etc/ldmd.conf file
so we can verify this), you could dramatically improve your data
reception by doing the following:

1) changing your feed request(s) to idd.unidata from SECONDARY
   to PRIMARY

2) upgrading to a current release of the LDM

If you upgrade to LDM-6.4.1, the latest LDM release, (you are currently
running LDM-6.0.14), the feed requests you currently have in place will
automatically promote to PRIMARY from SECONDARY when the preponderance
of products being received from the stream are written into the queue
(i.e., not rejected because a feed from a different source got the
products to your sooner).  Similarly, the feed request to the slower
upstream will demote to SECONDARY from PRIMARY when the preponderance
of products that connection receives are not written to the queue since
some other request was faster.  With 6.4.1, having redundant feed sites
does not result in excessive use of network bandwidth since the slower
of the feed sites will be demoted to SECONDARY while the faster will be
promoted to PRIMARY.

Just as a reminder, a PRIMARY feed request is where the downstream site
(your machine, for instance) tells the upstream machine (e.g., papagayo
and idd.undata) to send the data in a stream whenever it has anything;
in a SECONDARY feed request, the downstream tells the upstream to ask
if it wants each product before sending it.  As you can imagine, a
SECONDARY feed will result in slower delivery of products since there
will be an ask/answer transaction for each product (and the IDD
contains LOTS of products).

>By the way, Patrick O'Reilly left UNI 
>in May.  Currently, my tech person is Alex Duzik (grad student in 
>computer science).

I just updated our site contact page to indicate that Alex is the
appropriate contact now, but the phone number listed is still for Patrick.
Do you have a phone number I can use for Alex?  Or, would it
be more appropriate to list you as the site contact?

Cheers,

Tom

>From address@hidden  Fri Sep  2 07:56:15 2005

Tom -

Thanks for your response.  I'll ask Alex to make these changes.  I think 
you can list Alex as the site contact.  His phone is the same as 
Patrick's was (319-273-3789).

Alan

>From address@hidden  Fri Sep  2 15:01:22 2005

Alan,

I went ahead and made those changes yesterday after receiving this 
email.  Things seem to be moving much more smoothly now.  I'll 
investigate upgrading LDM on thunder to 6.4.1 next week -- automated 
promotion of the most reliable feed would be a very good thing.

Alex