[IDD #KOV-575304]: NEXRAD2 data

Hi John,

re:
> I think we're getting a handle on it.  I borrowed a Dell PowerEdge R510 
> server from
> another department, and am running this as our primary ingestor for the 
> moment.  I've
> got the latest build of LDM running, and am using the ldmd.conf and 
> pqact.conf (empty)
> from our original server on it, though I did increase the queue size from 2 
> GB on the
> old machine to 10 GB on the new one (2 GB was the most the old one could do, 
> 32 bit
> issue I'm told).  HUGE difference so far: load factors usually less than 2.0, 
> and Leon
> reports much better data flow.

Excellent!  But, I have a comment based on information you include below:  
Running
with an LDM queue that is larger than the amount of physical RAM available on
the machine is, in general, a mistake.  If/how much of a mistake it may be
depends on exactly what the machine is doing now and what you want it to
be able to do in the future.  For a machine with 8 GB of RAM, I would suggest
a maximum queue size somewhere between 4 and 6 GB.

re:
> What do you recommend for a primary server?  Here's the specs on the borrowed 
> box:
> 
> 8 core 2.4 GHz CPU (dual quad-core chips), 8 GB RAM, 1 TB SATA hard drive (2 
> in mirrored
> RAID array), gigabit Ethernet to our core routers, Redhat Linux 5.
> 
> Is that overkill, underkill, or just right?

Without knowing exactly what your server is doing AND may do in the future, it
is hard to say with certainty.  If all the machine is doing is bringing in 
NEXRAD2 data
and then FILEing it to disk, this configuration is an overkill.  If
you want this machine to bring in all of the data in the IDD, then it is not
overkill at all.  A dual-quadcore machine running is certainly overkill
as far as an ingest machine goes (especially for Linux installations where
hyperthreading will result in 16 cores).

So, the question to you is what you want this ingest machine to do?

re:
> Thanks for your help and patience, I'll continue running the new system for 
> testing, and
> we may have found the money to buy a comparable system if that was the 
> problem.

No worries for any help we can provide.  Continued testing sounds good.

One additional question:

- is it the job of the ingester to reassemble Level II volume scans?

  If yes, the procedure for doing this is not as simple as it may
  seem.  Since there is no guarantee that the pieces of a volume
  scan will be received in order, there is no guarantee that the
  receipt of the last piece of a volume scan will indicate that all
  other pieces have already been received.  We have put together
  a procedure for reassembling a volume of Level II data that is
  bit more complicated than worked well a few years ago.  I can
  outline this procedure for you if you are interested...

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: KOV-575304
Department: Support IDD
Priority: Normal
Status: Closed