[SCOOP #CIL-628524]: Re: 20060607: Do you have a minute

Hi Gerry,

re:
> Could I call later today? Discuss some of these?

I will be in my office all day after about 10:30 (dropping my truck
off for service).  Since there are no meetings scheduled, I should
be free to discuss this situation at length.

Tom

> We're still testing things related to the model runs for SCOOP, and
> sasquatch is the main machine on LDM for that.
> 
> Last evening at 2300z, we initiated a test run starting with a set of
> 69-70MB netcdfs from UF (sura-uf-pe6600-1.coastal.ufl.edu) of which we
> received 9 of 11.  If you look around on sasquatch, the matching in
> pqact is for those starting with WANA,  and the various data can be
> found in ~ldm/data/SCOOP/ANAwinds/* where we have specifically been
> concentrating on Katrina data, ca 20050826.
> 
> Justin Davis at UFl inserts, then sees what he's getting on his own ldm
> stream to verify the inserts.  He then reports out the files, sizes and
> times, pretty much as gospel.
> 
> I'm also told another machine, at the Virginia Institute of Marine
> Sciences (VIMS) (sura-vims-pe6600-1.vims.edu) received all the files,
> but ran out of disk space before writing them all out.
> 
> I saw no particular indication we got all the files via LDM here.  I'm
> getting overloaded and trying to figure this one out.  I've got a window
> running notifyme running here at home now.  I'd like to capture whenever
> we see these files and have some indication of success.  So far, I've
> not seen 'em all.
> 
> I'm also trying to recraft our regex matching to make sure we're not
> shooting ourselves in the foot.  We do a fair bit of processing for
> database entry of the files and their associated metadata when we see a
> new file come in.  We're taking some bigtime database hits because of
> the number of products we're seeing and the number of duplicates that
> come through the system.  I'm thinking we may have to approach the
> database issues differently from our current approach, and we're working
> on it.
> 
> Finally, while the load, memory and idle stats are pretty good, since
> we're both relaying and processing on sasquatch, might we be binding
> things up ourselves?  Looking last night at the stats from your site,
> we're moving a lot of data back and forth.  Here's a current 'top':
> 
> top - 09:19:40 up 34 min,  2 users,  load average: 0.26, 0.60, 0.70
> Tasks: 161 total,   1 running, 160 sleeping,   0 stopped,   0 zombie
> Cpu0  :  0.0% us,  0.3% sy,  0.0% ni, 98.0% id,  0.3% wa,  0.3% hi,  1.0% si
> Cpu1  :  2.3% us,  1.3% sy,  0.0% ni, 94.7% id,  0.0% wa,  0.3% hi,  1.3% si
> Mem:   8163060k total,  5151236k used,  3011824k free,    43956k buffers
> Swap: 15647288k total,        0k used, 15647288k free,  3422820k cached
> 
> I'm gonna propose SCOOP host a 1-day meeting for the folks running LDM
> and that we invite you or Steve Emmerson out to help us tweak things.
> We've several cooperative folks and a few rugged individualists, so
> getting agreement and concensus has proven difficult.  I'm thinking that
> getting the experts in would help a lot there.
> 
> Could I call later today? Discuss some of these?
> 
> Thanks, Gerry
> Tom Yoksas wrote:
> >>From:  Gerry Creager <address@hidden>
> >>Organization:  TAMU
> >>Keywords:  200606072219.k57MJCSZ012682 IDD-SCOOP
> >
> >
> > Hi Gerry,
> >
> > re:
> >
> >>to perhaps get on sasquatch and make sure things are reasonably happy?
> >
> >
> > I logged on a couple of minutes ago, and did not see anything amiss
> > (but I deleted some old core files in ~ldm).
> >
> > Was there something in particular that you wanted me to look at?
> >
> > Cheers,
> >
> > Tom
> 
> --
> Gerry Creager -- address@hidden
> Texas Mesonet -- AATLT, Texas A&M University
> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
> 
> 

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: CIL-628524
Department: Support IDD SCOOP
Priority: Normal
Status: Closed