[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #HRQ-388363]: Replacing LDM with rsync

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #HRQ-388363]: Replacing LDM with rsync
  • Date: Thu, 09 Jun 2011 11:26:19 -0600
Leonard,

> Our usage of LDM appears to be very simple in configuration.  pqact only logs 
> the filename.  scour doesn't do anything.  I then think that the benefits of 
> LDM would be implicit it its default operation.
> 
> Is there anything implicit in LDM which makes it more reliable than 
> repeatedly running rsync?

Not really. The LDM is designed so that data older than a configurable amount 
of time is unimportant and won't be transferred. rsync(1), on the other hand, 
will try to transfer a file regardless of how old it is or how long the network 
has been down.

> I think another advantage of LDM is the architecture of point to multipoint 
> distribution, to send a stream of data to multiple recipients.  We have only 
> one point to one other point, Hawaii to here.
> 
> Is there anything else I should consider in this comparison?

Let me try to break this down.

Things that favor the LDM:
    Distribution to multiple sites
    High volume
    Rapid delivery (i.e., low latency) is important
    Selection of subsets of data streams
    New data generated routinely
    Data older than, say, one hour is unimportant
    A receiving site might become a relay site

Things that favor rsync(1):
    Distribution to a single site
    Low volume
    Everything in a directory tree must be transferred
    No data file can be ignored -- regardless of how old

I hope this helps.

> Thanks.
> 
> ==Leonard E. Sitongia
> High Altitude Observatory
> National Center for Atmospheric Research
> P.O. Box 3000 Boulder CO 80307  USA
> address@hidden  voice: (303)497-2454  fax: (303)497-1589

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: HRQ-388363
Department: Support LDM
Priority: Normal
Status: Closed