Re: 20030423: motherlode data request changes

Tom,

That was the high water mark for awhile.  thelma has been running
with 148 total all night and now has 149 -- and the load average
is hanging between 20 and 25.  I've been keeping an eye on the
kernel locking stats and see that "reclock" has been climbing
and is currently about 75-80%.  I think thelma can keep this up
while in steady state, but I have growing concern about recovery
after stopping and restarting the LDM especially if we had to do
it other than every six hours at the lull.

Hey that brings up another question -- how easy would it be to
add back the function to re-read the configuration file on the fly?

Anyway, it makes sense to start aggregating downstream feeds and
tuning those systems or shifting loads to atm or unidata2 soon!
Along those lines, we could remove either desi or jackie now that
atm is feeding.

Also, I've been thinking about making different plans about hardware
and moving what was to be the new motherload to thelma permanently,
maybe making the current thelma the new laraine.  Sun has their
matching grant program running again and along with the recent price
drop, we can buy a machine that's twice the 480 for less than $50K.

mike

On Apr 23, 12:15pm, Tom Yoksas wrote:
> Subject: 20030423: motherlode data request changes
>
> ...
> After making these changes, thelma has 139 rpc.ldmds running:
>
> Time            L1   L5    L15    Down Up Total Age
> 20030423.1814   9.68  9.65  9.98  115  24 139   7469
>
> Time   - YYYYMMDD.HHMM (UTC)
> L1     - load average for the past minute
> L5     - load average for the past five minutes
> L15    - load average for the past fifteen minutes
> Down   - downstream feed service rpc.ldmds
> Up     - upstream feed request rpc.ldmds
> Total  - total rpc.ldmds
> Age    - age of the oldest product in the queue in seconds
>
> 139 rpc.ldmds is a new high water mark on thelma!
>
> Tom

>From address@hidden Thu Apr 24 10:56:02 2003

re: how easy would it be to add back the function to re-read the
configuration file on the fly?

It would be pretty major.  Several months work probably.

Regards,
Steve Emmerson