20031001: LDM data feed failover and an LDM upgrade request

>From: David Yeung <address@hidden>
>Organization: Hong Kong University of Science and Technology
>Keywords: 200309300045.h8U0jvk1023216 IDD LDM upgrade

Hi David,

Your email is very timely.  We were going to be contacting you anyway
with a request for you to upgrade to LDM-6.  More below...

>We have been receiving LDM data feed from iita.rap.ucar.edu which was
>our primary LDM server. However, as iita is becoming a local LDM server.
>May I request a primary LDM server for our University? We have the
>atm.geo.nsf.gov as our backup server but if possible, we would like to have
>one more data feed for our backup suppose.

Please use atm.geo.nsf.gov as your primary LDM server for now, and
use chisel.rap.ucar.edu as your failover.

We would like you to upgrade your LDM-5.2 installation to the latest
UPC release, LDM-6.0.14.  LDM-6.0.14 is available only as a source
distribution; you have to build and install the executables yourself.
This is a simple exercise, however:

<login as 'ldm'>
cd ~ldm
ftp ftp.unidata.ucar.edu
  <user> anonymous
  <pass> address@hidden
  cd pub/ldm
  binary
  get ldm-6.0.14.tar.Z
  quit

zcat ldm-6.0.14.tar.Z | tar xvf -
cd ldm-6.0.14/src
./configure
make && make install
sudo make install_setuids

cd ../bin
- adjust entries in ldmadmin to match your setup/desired setup:

  - if 'uname -n' does not return a fully qualified hostname, you
    must define $hostname:

change:

chop($hostname = `uname -n`);
# $hostname = "your.hostname.here";

to:

# $chop($hostname = `uname -n`);
hostname = "rcz006.ust.hk";

  - set $pq_size to the size of your current LDM queue, ~ldm/data/ldm.pq

  - set $numlogs to the number of ~ldm/logs/ldmd.log.* files you are now
    keeping

cd ~ldm
ldmadmin stop

- wait for all LDM-5 processes to exit

- adjust your ~ldm/etc/ldmd.conf request lines:

  - we worked with you before to split the data requests.  This was to get
    around LDM-5 limitations that were eliminated in LDM-6.  We are
    now delivering data to remote locations (e.g., UK, Brazil) using
    LDM-6 with very little (zero to a few seconds) latencies

    Here are some suggestions for for your requests:

request IDS|DDPLUS      ".*"    atm.geo.nsf.gov PRIMARY
request HDS     ".*"    atm.geo.nsf.gov PRIMARY

    Actually, I seem to remember that your HDS feed request limited
    the model output you are getting to just the global grids. If
    this is true, use the same pattern for the request, but you
    should only need to use one request line.

  - if you were requesting data by IP address previously, change that/those
    entries to use the fully qualified hostname.  This is needed so that
    we can trace back the data paths using our real time statistics
    displays

rm runtime
ln ldm-6.0.14 runtime
ldmadmin start

>Furthermore, our ldm server log keeps displaying the following messages
>recently:
>
>Oct 01 00:30:16 rcz006 rtstats[20991]: 
>clnt_create(rtstats.unidata.ucar.edu, 300029, 5, "tcp"): 
>rtstats.unidata.ucar.edu: RPC: Port mapper failure - RPC: Timed out
>Oct 01 00:31:10 rcz006 rtstats[1789]: 
>clnt_create(rtstats.unidata.ucar.edu, 300029, 5, "tcp"): 
>rtstats.unidata.ucar.edu: RPC: Port mapper failure - RPC: Timed out
>Oct 01 00:31:16 rcz006 rtstats[20991]: 
>clnt_create(rtstats.unidata.ucar.edu, 300029, 5, "tcp"): 
>rtstats.unidata.ucar.edu: RPC: Port mapper failure - RPC: Timed out
>
>Any idea what's wrong?

Not all real time statistics are getting back to our real time stats
server, rtstats.unidata.ucar.edu.  Let's worry about this if the situation
continues after you upgrade to LDM-6.0.14.

>We have the following line in our ldmd.conf file
>as you suggested to add some time ago. I guess that it cause these error 
>messages.
>
>    exec    "rtstats -h rtstats.unidata.ucar.edu"

This is correct.  Please do not delete it.

>Thanks

No worries.

>David Yeung
>Hong Kong University of Science and Technology

By the sway, one of the primary reasons that LDM-6 was developed was to
be able to feed remote sites like yours.  Our tests in the UK and Brazil
have proven that LDM-6 is _much_ better able to supply real time data
to remote sites that have decent Internet connections than LDM-5.  The
traceroute tests I have run to rcz006 show the same sorts of latencies
as the primary IDD redistribution node that we have setup collaboratively
with the Universidade Federal do Rio de Janeiro, so you should be able
to get as reliable a reception as them.

Please keep us informed about your progress of upgrading to LDM-6.0.14,
and let us know if you need help.  We are moving away from LDM-5 as
fast as we can, so it is a good idea for you to as soon as possible

Cheers,

Tom Yoksas