20030509: ldmd.conf from syrah.atd.ucar.edu (cont.)

>From:  Chris Burghart <address@hidden>
>Organization:  National Center for Atmospheric Research
>Keywords:  200305081646.h48Gk37U015345 LDM-6 IDD install tuning

Hi Chris,

>Joe VanAndel forwarded your message regarding our LDM
>on syrah.atd.ucar.edu (a.k.a. ldm.atd.ucar.edu).

This machine is showing up in our LDM log files as 
syrah-78.atd.ucar.edu.  Forward and reverse name lookup do not
give the same results:

(laraine.unidata.ucar.edu) 608 % nslookup syrah.atd.ucar.edu
Server:         128.117.140.62
Address:        128.117.140.62#53

Non-authoritative answer:
Name:   syrah.atd.ucar.edu
Address: 128.117.78.2
Name:   syrah.atd.ucar.edu
Address: 128.117.80.2

(laraine.unidata.ucar.edu) 609 % nslookup 128.117.78.2
Server:         128.117.140.62
Address:        128.117.140.62#53

Non-authoritative answer:
2.78.117.128.in-addr.arpa       name = syrah-78.atd.ucar.edu.

Authoritative answers can be found from:
117.128.in-addr.arpa    nameserver = dns2.itd.umich.edu.
117.128.in-addr.arpa    nameserver = dns.ucar.edu.


So, what is its _real_ name (result of a 'hostname') of your machine?

>I've attached the ldmd.conf file, and you can see that we are
>currently requesting only NEXRAD feeds.  We need all of
>the radars from regions 3 and 4, and six additional
>radars outside those regions.  If you have any suggestions
>for improving the form of our requests, please let
>me know.

OK.  Here goes.

1) We are asking all sites participating in the IDD (like syrah is)
   to report real time statistics.  This is easily done by the
   addition of one 'exec' line in ~ldm/etc/ldmd.conf:

exec    "rtstats -h rtstats.unidata.ucar.edu"

   This exec line should be the last 'exec' line at the top of your file.

   If the result of a 'uname -n' on your machine does not return a
   fully qualified hostname (like syrah.atd.ucar.edu), then you need to
   edit your LDM-6 'ldmadmin' script to set the hostname correctly
   _before_ you start running LDM-6:

change:

chop($hostname = `uname -n`);
# $hostname = "your.hostname.here";

to:

# chop($hostname = `uname -n`);
$hostname = "syrah.atd.ucar.edu";

   We need the fully qualified hostname of each IDD participant in
   order to correctly process the real time stats that they be send
   in.

   For reference, real time statistics for LDM-6 sites participating
   in the IDD can be found in:

http://www.unidata.ucar.edu/staff/chiz/rtstats

   Click on the 'Statistics by Host' link, and then the link for your
   machine (after it starts reporting stats, of course), and then
   either 'latency', 'log(latency)', 'volume', 'products', or
   'topology'.  The 'topology' link will show you how the data
   gets to your machine.  The links that are machine names in that
   page give access to plots of differential latencies for the
   products in that feed (i.e., the latency from one machine to the
   next as opposed to the 'latency' link which gives overall latency).

2) You are requesting an NNEXRAD feed with two different request lines.
   In LDM-5 these two requests would be combined into a single request
   and sent to the upstream host.  In LDM-6, however, these are not
   combined.  This results in two rpc.ldmd processes on both your
   machine (probably no big deal) AND on the upstream feeder (could be
   a big deal depending on that system's load).  It also results in the
   data being sent twice unless the request regular expressions are
   mutually exclusive (yours are).

   Your request lines are:

request NEXRAD "^SDUS5[34]" thelma.ucar.edu
request NEXRAD "^SDUS.. .* .* /pN..(CLE|ILN|RLX|GSP|FFC|JGX)" thelma.ucar.edu

   These can be combined into a single request as follows:

request NNEXRAD "(SDUS5[34]|/pN..(CLE|ILN|RLX|GSP|FFC|JGX))" thelma.ucar.edu
   
   I always test regular expressions like this by using notifyme to the
   upstream host:

   notifyme -vxl- -o 3600 -f NNEXRAD -h thelma.ucar.edu -p 
"(SDUS5[34]|/pN..(CLE|ILN|RLX|GSP|FFC|JGX))"

   A review of the listing returned by notifyme will tell you that your
   request pattern is what you want or not.

Please let me know if you have any questions about the information above.

Tom

>From address@hidden Fri May  9 13:05:11 2003

Hi Tom,

Tom Yoksas wrote:

> Hi Chris,
> 
> 
>>Joe VanAndel forwarded your message regarding our LDM
>>on syrah.atd.ucar.edu (a.k.a. ldm.atd.ucar.edu).
> 
> 
> This machine is showing up in our LDM log files as 
> syrah-78.atd.ucar.edu.  Forward and reverse name lookup do not
> give the same results:
> 
> [..nslookup results...]
> 
> So, what is its _real_ name (result of a 'hostname') of your machine?

The machine's hostname is 'syrah', but it's on two nets and also
has a cname entry in DNS, so syrah has three fully qualified
names:

        syrah.atd.ucar.edu      128.117.80.2
        syrah-78.atd.ucar.edu   128.117.78.2
        ldm.atd.ucar.edu        -> syrah-78.atd.ucar.edu

The current ldmadmin has:

        #chop($hostname = `uname -n`);
        $hostname = "ldm.atd.ucar.edu";

so you will see IP 128.117.78.2; a reverse lookup will give
"syrah-78.atd.ucar.edu" and not "ldm.atd.ucar.edu".

The idea was that with a cname, we could move the LDM arbitrarily
to a different machine here and just change our cname entry, and
not have to worry about upstream LDMs changing their "allow" entries.
It violates the assumption that a forward then reverse lookup
will leave you with the same name, though.

For our current purposes, thelma is the only upstream server, so
I've just changed to using "syrah-78.atd.ucar.edu".

I've also turned on rtstats.

> 2) You are requesting an NNEXRAD feed with two different request lines.
>    In LDM-5 these two requests would be combined into a single request
>    and sent to the upstream host.  In LDM-6, however, these are not
>    combined.  This results in two rpc.ldmd processes on both your
>    machine (probably no big deal) AND on the upstream feeder (could be
>    a big deal depending on that system's load).  It also results in the
>    data being sent twice unless the request regular expressions are
>    mutually exclusive (yours are).
> 
>    Your request lines are:
> 
> request NEXRAD "^SDUS5[34]" thelma.ucar.edu
> request NEXRAD "^SDUS.. .* .* /pN..(CLE|ILN|RLX|GSP|FFC|JGX)" thelma.ucar.edu
> 
>    These can be combined into a single request as follows:
> 
> request NNEXRAD "(SDUS5[34]|/pN..(CLE|ILN|RLX|GSP|FFC|JGX))" thelma.ucar.edu

I'm all for keeping the number of processes down.  The one-line request
with your suggested regexp is in place.

Thanks for the pointers,

Chris