[IDD #FFC-953121]: Main system memory v. LDM queue size

Hi Justin,

re:
> What is the appropriate ratio of main system memory to ldm queue size?

It depends on what you are trying to do with your LDM installation.  For
instance, if you are trying to feed 150 downstream connections any/all of
the data available in the IDD while keeping at least an hour's worth of data
in the queue, then you should have at least an 8 GB queue.  If you are
just receiving data from one or more upstream nodes, then the size that
your queue should be will be a function of the volume of data you are
requesting, what processing you are attempting, and the speed of your machine
(CPU and disk I/O subsystem).  Most sites that are receiving _lots_ of
data and relaying some data are using queues on the order of 2-4 GB.
IDD leaf nodes, sites that are not relaying data and are requesting
"reasonable" volumes are using queues anywhere between 400 MB to 1.5 GB.

> Its looking like we might need a 6-8GB prodcut queue and have heard that a
> ratio of 2:1 is what we should be using.

Again, this will depend on whether or not you are relaying data to multiple
downstream sites.

> We are a bit concerned both at
> the costs of all that memory plus the additional cost of finding a machine
> which can support that much memory.

We are using Sun SunFire V20Zs configured with either 12 or 16 GB of RAM
in our toplevel IDD relay cluster.  We sized the CPU spead (dual Opteron)
and RAM with the aim of being able to feed up to 250 downstream connections
from each of the 3 cluster data servers.  We created this cluster to be
able to service every IDD data request in the world.  Other installations
will settle for something less grandeous :-)

> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Justin R. Davis, Ph.D.
> address@hidden
> Post-Doctoral Research Associate
> Civil and Coastal Engineering
> 365 Weil Hall
> University of Florida
> Gainesville, FL 32611-6580
> 352-392-1436 x1528


Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: FFC-953121
Department: Support IDD
Priority: Normal
Status: Closed