Re: 20000210: Re: CONDUIT Web site (fwd)

>To: address@hidden
>From: Luis Cano <address@hidden>
>Subject: Re: 20000210: CONDUIT Web sitesubject
>Organization: NCEP
>Keywords: LDM product queue

Hi Luis,

> During an unrelated meeting a couple of weeks ago, Dan Starosta
> briefly mentioned to me that a test was done using the Internet
> directly from Silver Spring (bypassing the T1 between OSO and NASA),
> and message throughput looked better. This is about all I know
> ... other that it is being looked at by Dan's group.

Sorry, I don't know anything about that.

> On a related subject, I like to ask a general question of the ldm
> queue size in relations to the growing size of model data. I heard
> that the queue was being reworked due to a 2gig limitation, is there
> any news on this?  Availability?

There are several developments going on that are related to this:

 - The most significant current LDM limitation is the number of
   products in the queue rather than the aggregate size of all the
   products in the queue, since the operations of inserting and
   deleting products from the queue become much slower when there are
   already a large number of products in the queue.  We're rewriting
   the product queue library to fix this problem, but it may be a
   couple of months before this is ready.

 - It appears to be possible to use 4 Gbyte product queues on systems
   that define the C type size_t as unsigned.  Permitting queues
   larger than that would require a lot of work.

 - There is a new version of the IDD on the horizon that may have no
   such limitations, based on a technology that can deliver on the
   order of 100 Gbytes per day, given sufficient bandwidth.  We may
   have a prototype that uses this technology within the next year,
   assuming there's no unanticipated problems ...

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu