20030328: pqact causes unusually high CPU usage on LDM 6/Soalris Intel (cont.)

>From: Unidata Support <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200303250413.h2P4D9B2010509 LDM-6 pqact

Robert,

re: pqact causing high CPU load under LDM-6.0.2 but not LDM-5.2

As far as I am concerned, the verdict is in.  When you use LDM-5.2, the
ingest of HDS and NNEXRAD is being combined in one rpc.ldmd, and the
ingest of those data is measurably slower than with LDM-6.

For the following, please refer to the realtime statistics for
wxmcidas.nsbf.nasa.gov:

http://www.unidata.ucar.edu/staff/chiz/rtstats/siteindex.shtml

and then click on the 'wxmcidas.nsbf.nasa.gov' link.  The page you will
be looking at will contain liks for the latency for each feed received
by wxmcidas.

The test I ran to investigage your problem was:

LDM-6.0.2: Julian day 88 -> 89.1      (I switched to LDM-5 at 01:45Z on day 89)
LDM-5:     Julian day 89.1 -> present

You can see from the latency plots for HDS and NNEXRAD that LDM-6.0.2
was able to receive all of the data in these streams without losing any
data (i.e., before the latency got up to 3600 seconds -- the peak at
Julian day 88.6 doesn't fall into this, but all the data was received
nonetheless).  After switching to use of LDM-5, the latency for HDS hit
3600 seconds every few hours (presumably corresponding to the
availability of model runs in HDS), and you lost data.  The upshot of
all of this is that LDM-6 brought more data to wxmcidas faster than
LDM-5, and this caused pqact and the decoders -- especially dcgrib2 --
to have to work harder in a shorter period of time.

So, what to do?  I suggest that the LDM-6 is working better than LDM-5,
so you should be using it.  If you need to keep your CPU load down, you
might consider requesing less HDS data.  The ultimate "solution" may be
as we both suspect: replace wxmcidas with a faster machine, perhaps
even a dual processor box (our old dual 550 Mhz machine is able to keep
up with a lot more data ingest that what you are receiving).  As per
your last email, this may well be what you were expecting to hear.

Please let me know if you disagree with my interpretation of the
realtime latency plots for wxmcidas.

Tom
Unidata WWW Service              http://my.unidata.ucar.edu/content/support    
  

>From address@hidden Tue Apr  8 09:26:08 2003

I have gone in and cut back on the amount of data that we are
decoding.  I cut out McIDAS decoding completely and have pointed at
uldbldm.nsbf.nasa.gov in Palestine, and will use adde.ucar.edu or one
of the other public ADDE servers.

I also cut way back on the amount of model data that I was decoding
with GEMPAK..  and stopped some McIDAS decoders that we don't really
use.

In normal use it isn't too bad..when model data is arriving it does
slow down quite a bit, also if I stop/restart the LDM it is bogged
while catching back up.

This wouldn't be a real issue if we didn't also use the box for
GARP/McIDAS display as well...but in any event it is manageable now.

The potential good news is that apparently we are goinf to be approved
for new hardware...but I have heard this before and it's never come to
pass.

Thanks for your assistance on this.

Robert Mullenax