RE: 20030227: LDM - Linux - Running LDM on clustered systems

Hi Carl,

Thanks for your reply. What I'm mostly interested in is feeding data
from LDM running on nodes within a cluster to LDM running on other
systems outside the cluster. The cluster would of course be located
at this site, but the other LDM systems could be located here or at
any of our other offices. The problem we are trying to solve is this.
We currently have a primary and a backup LDM system receiving Noaaport
data. There are several other LDM systems (here and at some of our
other offices) that request certain types of data from the primary LDM
system. Data transfers and processing are required 24/7 but the local
and remote systems are not monitored (by humans) during extended
periods nights and weekends. If the primary LDM system fails (LDM
crashes, disk failure, motherboard failure, network card failure, etc)
and the system cannot be brought back quickly, the remote systems are
manually failed-over to request data from the backup system. What we
would like to do is cluster the primary and backup LDM systems in such
a way that they look like a single system to the remote systems; that
is, if the one system in the cluster is down for some reason, requests
by LDM on the remote systems for a data connection would be passed to
the working system in the cluster without any human intervention on the
remote systems. We were wondering if anyone had used LDM with clusters
in this way.

Best regards,
Herm


-----Original Message-----
From: Carl Sinclair [mailto:address@hidden]
Sent: Thursday, February 27, 2003 10:18 PM
To: Herm Wojdylak
Cc: Unidata Support; address@hidden
Subject: Re: 20030227: LDM - Linux - Running LDM on clustered systems


Herm,

We are running LDM on a clustered system.  The way we are doing it is
this:

Each node runs it's own instance of LDM with the queue on that node's
local hard drive.  The data that is brought in is written to a raid
shared by the cluster.

That's the basic rundown. If you'd like a further breakdown I can
provide that as well.

Carl



Unidata Support wrote:
> 
> >From: "Herm Wojdylak" <address@hidden>
> >Organization: WNI
> >Keywords: 200302272059.h1RKx3G09148 LDM Linux cluster
> 
> Herm,
> 
> >Institution: Weathernews Americas
> >Package Version: 5.1.x / 5.2.x
> >Operating System: Linux
> >Hardware Information: PC (intel)
> >
> >Inquiry:
> >
> >I've tried to search the FAQ's about this with no luck. Do you know if
> >anyone has successfully tried running LDM on clustered machines?
> 
> Yes, the folks at OU/CAPS have setup a system on a Linux cluster.
> 
> >We
> >receive NWSTG data via a Noaaport system which feeds an LDM main system
> >and a backup LDM system. The main system then feeds LDM running on
> >various downstream systems which do the actual data processing. If the
> >main LDM system dies, we manually failover the downtream systems to
> >receive data from the backup system. If possible, we would like to
> >cluster the main and backup systems so that if a downstream system lost
> >connection with the main LDM system, when it tried to reestablish the
> >data feed and the main system was down, the cluster load handler would
> >connect it to the backup machine without manual intervention.
> 
> Since I don't know the pertinent details of how the folks at CAPS
> setup the LDM on their cluster, I will refer you to the technical
> contact there: Carl Sinclair <address@hidden> (who I also CCed on
> this reply.
> 
> Tom Yoksas

-- 
_______________________
Carl Sinclair
OU/CAPS Research Associate
(405)325-5938