20030227:ASU-Unidata update

Hi Nancy, 

Yes, I will leave the allow on thelma until we resolve what we want to
do..not a problem.

However, 

<snip>
I still don't quite understand why that outgoing message would be 4 times
the size of the incoming data stream,
<snip>

This makes no sense unless you are feeding other sites..Do you have any
allows in your ldmd.conf?


<snip> 
Another curiousity that's poking up is that the number of data packets
being received by mothra exceeds the number sent by thelma.  This is
resulting in a "Number of periods not equal" message at the LDM comparison
site.
<snip>


This is also quite strange...is mothra requesting anything else from
another site..i.e. NLDN from striker..?

Please send me your ldmd.conf file, I think that is the best way for me to
investigate...


I went to ASU for a year in 1993 to get some physics behind me before
going to grad school here at CU, and I know how ASU and U of A are like
oil and water..:)  Just a comic aside to the issues at hand of getting
prompt data delivery from U of A..  Go Devils!

Cheers,

-Jeff
____________________________                  _____________________
Jeff Weber                                    address@hidden
Unidata Support                               PH:303-497-8676 
COMET Case Study Library                      FX:303-497-8690
University Corp for Atmospheric Research      3300 Mitchell Ln
http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000
________________________________________      ______________________

On Thu, 27 Feb 2003, Nancy Selover wrote:

> Jeff,
>       Our system administrator has been monitoring the traffic into and
> out of mothra as he has been directing the Datacomm people to try and solve
> our network slowness issues, particularly with getting the LDM data.  Here
> is his latest note.  Is it possible to stay on thelma for a week to see if
> it is stable.  Also, do you know why we are getting more products than
> thelma (per the upstream-downstream comparison website)?  I would guess it
> is a function of what products we keep rather than what's in the queue.
> 
> Thanks,
> Nancy  
> 
> Nancy J. Selover
> Asst. State Climatologist
> Office of Climatology           tel:  480-965-0580
> Arizona State University      fax: 480-965-1473
> Tempe, AZ  85287-1508      e-mail: address@hidden
> 
> >  -----Original Message-----
> > From:       Talbot Brooks  
> > Sent:       Thursday, February 27, 2003 11:05 AM
> > To: Nancy Selover
> > Cc: Joseph Zehnder
> > Subject:    RE: Unidata update
> > 
> > Well, it looks like you're getting your all of your data now.  It's still
> > a bit slow at times, but it looks like everything is coming through.  Even
> > if mothra is sending a message back to thelma, I still don't quite
> > understand why that outgoing message would be 4 times the size of the
> > incoming data stream, but I'll roll with it.  Another curiousity that's
> > poking up is that the number of data packets being received by mothra
> > exceeds the number sent by thelma.  This is resulting in a "Number of
> > periods not equal" message at the LDM comparison site.  
> > 
> > Regardless, can you let things run from thelma for a week or two?  If it
> > stays "up", then I think our culprit is definitely at UofA or the link
> > between here and there and not on the shoulders of our DataComm folks.  If
> > this does prove to be the case, I think the path of least resistance would
> > be to leave our source as thelma and bag getting stuff from U of A - it's
> > gonna be a nightmare trying to get that fixed...
> > 
> > Thanks,
> > 
> > Talbot Brooks
> > ASU Dept. of Geography
> > 600 E Orange St.
> > Tempe, AZ 85287-0104
> > 
> > (480) 965-7832
> > address@hidden
> > 
> >  -----Original Message-----
> > From:       Nancy Selover  
> > Sent:       Thursday, February 27, 2003 10:00 AM
> > To: Talbot Brooks
> > Cc: Joseph Zehnder
> > Subject:    RE: Unidata update
> > 
> > Talbot,
> >     LDM has to constantly send messages to thelma (or whatever data
> > server is upstream) requesting data.  The upstream server responds to each
> > request with a data stream, which then ends, and mothra asks for another
> > one, etc.  It's a continual series of processes that start and stop.
> > Hourly there is a statistical report mothra sends out to ucar on
> > latencies.  As mothra gets the data in, it also starts and stops a series
> > of decoders which decode various data types, as soon as it detects that
> > they are in the queue.
> > 
> > Nancy  
> > 
> > Nancy J. Selover
> > Asst. State Climatologist
> > Office of Climatology           tel:  480-965-0580
> > Arizona State University      fax: 480-965-1473
> > Tempe, AZ  85287-1508      e-mail: address@hidden
> > 
> >      -----Original Message-----
> >     From:   Talbot Brooks  
> >     Sent:   Wednesday, February 26, 2003 5:16 PM
> >     To:     Joseph Zehnder; Nancy Selover
> >     Subject:        Unidata update
> > 
> >     Howdy - 
> > 
> >     As of 5 pm, the discrepency report from Unidata between mothra and
> > thelma shows minimal data missing and latencies under 500 seconds.  Using
> > eHealth to track network traffic to and from mothra shows a substantial
> > increase in data transmission.  So, it looks like you've been getting most
> > of your product for at least the last 4-5 hours.  
> > 
> >     However, there is something puzzling (at least to me) going on.  The
> > percentage of bandwidth used inbound is running at about 4%.  The
> > percentage of bandwidth outbound is as high as 18%.  Any guess why there
> > would be more outbound than inbound traffic?  
> > 
> >     My take/educated guess:
> > 
> >     This suggests some sort of large-scale read activity taking place on
> > the drive.  If it were transmitting the results of that read back out, it
> > could potentially account for the high outbound network traffic.  If this
> > is the case, I would single it out as the primary suspect for latencies as
> > high as 500 seconds - the drive in mothra is at best a 10,000 rpm disk
> > with a transfer rate around 9ms - the box is simply choking on rapid I/O.
> > 
> >     To kinda confirm, I went in and physically touched the box and
> > watched the green drive light flicker on and off rapidly.  This drive is
> > in almost constant action.  I can affirm that this trend is related to
> > LDM, since all traffic dropped to zero when Nancy had LDM stopped earlier
> > in the week.  
> > 
> >     Thanks,
> > 
> >     Talbot Brooks
> >     ASU Dept. of Geography
> >     600 E Orange St.
> >     Tempe, AZ 85287-0104
> > 
> >     (480) 965-7832
> >     address@hidden
> > 
>