[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NLDM LDM future... 32 types, server side restriction.

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: NLDM LDM future... 32 types, server side restriction.
  • Date: Fri, 23 Jan 2004 16:39:43 -0700
Peter Silva wrote:
> 
> I had another question... I don't want to insult anyone, so I just want
> to ask simply...
> we did some performance tests... ran ftp with an mput of a thousand
> (biggish) files, and arranged for
> the same transfers, between the same end points (on a LAN) to be done
> using LDM
> LDM5 was 4x slower than ftp, LDM6 was 1.5x slower.
> 
> This is not a slam on LDM, there are a lot of great things that LDM does
> that FTP doesn't.
> I am just looking at raw point to point performance in this case.   The
> results were kind of what I would expect.
> 
> Does this sound outrageous, or about what you would expect too ?
> 

You're not the first to raise this issue.  Other people have compared
LDM relay to, say, scp for example.  And, your results are not
surprising to me.  As you say, the LDM does lots of other stuff besides
simply get data (so 1.5x FTP sounds relatively good to me!). 

Your question is timely for the following reason.   One test I'm
considering is to compare INN relay with LDM relay in as similar a
manner as possible.  That would mean relaying the same data streams
between the same two hosts at the same time with the same number of
connections.  The problem is in getting reasonable LDM latency numbers
from which to measure.  (NLDM stats are more precise and frequent that
LDM stats, if I do say so myself.)

LDM5 was heavily RPC based, incurring all the additional costly
overhead.  LDM6 either removed blocking RPC (i.e., stopped waiting for a
response before sending the data), or removed RPC altogether.  I thought
it was the former, but today I was talking with someone else here at
Unidata who thought it was the latter.  The code maintainer was out, so
I'll have to wait until next week to confirm what really happened. 
Since INN is strictly socket based, I had thought that it should be at
least a little faster than LDM nonblocking RPC based relay.   

So, I'm not sure right now whether it's worth setting up such a test. 
But I certainly wish I could say something about this aspect of the two
approaches.

(Btw, LDM6 also increased the block size plus a few other things that
also helped to improve the efficiency.)  

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                  P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************