20030321: ldm 6 : alternate feeds, foxfire problems and rtstats

>From: =?ISO-8859-1?Q?Christian_Pag=E9?= <address@hidden>
>Organization: UQAM
>Keywords: 200303211407.h2LE7CB2001473 LDM-6 PRIMARY ALTERNATE

Christian,

>In my newly LDM 6 configurations ldmd.conf, I specified several 
>ALTERNATE sites. Is it ok to have more than one alternate site for one 
>particular feed?

Yes, you can have as many as you like.  There is a price to pay for
this, however.

>Which one is chosen when the primary one is not 
>available or feeds bad?

What you are really specifying by PRIMARY and ALTERNATE is the following:

o PRIMARY says that this is the host you are telling to "just send the
  products as soon as you get them"

o ALTERNATE says to inform that upstream host to ask you if you want
  a product before it is sent

The concept is to set PRIMARY for the host that is most likely to have
the products specified first.  You specify ALTERNATE for those hosts
that might be slower to have the products in the stream.  In both
cases, a rpc.ldmd will run on your system AND the upstream system to
provide that data.  The result of correctly specifying a PRIMARY feed
site and ALTERNATEs is that you will get all of the prodcuts requested
from the PRIMARY and none from the ALTERNATEs even though they will ask
your receiving rpc.ldmd if you want each product.

The more ALTERNATE feeds you have, the more LDM rpc.ldmd processes you
will have running, and the more CPU and network bandwidth you will
consume.  The aount of CPU and network bandwidth should be small,
however, since the transaction that asks you if you want a product is
small.

One other comment: you can have as many PRIMARY feeds as you want
also.  Network use in this case will be a lot higher since duplicate
products will be only be rejected once the are received at your
machine.

>Why do my LDM ask all the primary sites and the alternate sites for 
>feeding at the same time?

Since each request line in your ldmd.conf file results in a separate
rpc.ldmd, and since these rpc.ldmds don't talk to each other, they are
all acting atonomously.

>Also, I desactivated ldmfail, is it correct?

You could, but if your PRIMARY feed went away, the ALTERNATE feeds
would be slower since they will be asking your LDM if you want each 
product.

>Also, in my logs, I have:
>
>Mar 21 13:42:23 5Q:io foxfire[25456191]: Desired product class: 
>20030320144203.477 TS_ENDT {{UNIWISC|HDS,  ".*"}}
>Mar 21 13:42:24 5Q:io foxfire[23917630]: Desired product class: 
>20030320144203.473 TS_ENDT {{IDS|DDPLUS,  ".*"}}
>Mar 21 13:44:56 5Q:io foxfire[25456191]: Couldn't create version 6 
>client handle: Port mapper failure - Timed out
>Mar 21 13:44:57 5Q:io foxfire[23917630]: Couldn't create version 6 
>client handle: Port mapper failure - Timed out
>Mar 21 13:47:28 5Q:io foxfire[25456191]: Couldn't create version 5 
>client handle: Port mapper failure - Timed out
>Mar 21 13:47:29 5Q:io foxfire[23917630]: Couldn't create version 5 
>client handle: Port mapper failure - Timed out

The way the LDM works is to first request a connection on port 388
using LDM-6 protocols.  If that doesn't work, I believe that it tries
to connect using LDM-5 protocols on port 388.  I will check with with
the developer.  If that connection doesn't work, it asks the upstream
LDM's port mapper for a LDM-6 connection.  If that doesn't work, it
asks the upstream LDM-s port mapper for an LDM-5 connection. At each
failure point, a message should be written to the log file.

>Mar 21 13:33:22 3Q:io rtstats[25584898]: 
>clnt_create(rtstats.unidata.ucar.edu, 300029, 5, "tcp"): 
>rtstats.unidata.ucar.edu: Port mapper failure - Timed out

This is telling us that the connection back to our realtime statistics
server, rtstats.unidata.ucar.edu, failed.  We have seen lots of these
things for a variety of reasons.  One of the main reasons for failure
is rtstats.unidata.ucar.edu's LDM may be down or overloaded.  We will
be moving the realtime statistics gathering to a more robust machine
soon, so most of these problems should go away.  You will not have to
do anything on your side since 'rtstats.unidata.ucar.edu' is an alias
for the real machine doing the work.  We will simply change the alias
to point at the new machine.  We will do this when LDM 6.0.3 is ready.

Tom Yoksas