20030829: 20030828: feed and processing Latencies (cont.)

>From: Gerry Creager N5JXS <address@hidden>
>Organization: Texas A&M University -- AATLT
>Keywords: 200308281358.h7SDw0Ld016206 IDD latency pqact

Hi Gerry,

>Thanks for the time today, and sorry about the abrupt termination. 

No worries.

>Cingular decided to drop cellular service for about 30-40 min starting 
>with our call...
>
>I'm going to try to rearrange the data I'm getting to grab surface data 
>from roughly NM/Tx/OK/AR/LA, grids supporting that area, upper air for 
>that area, and satellite and radar for CONUS and adjacent areas.

The observational data is one of the smallest portions of the IDD
feed, so cutting down on it will have a small impact on your data
ingestion and processing.

>If you have some suggestions for honing my feed specs down that'd be 
>greatly appreciated.

I wouldn't touch ingestion of observational data.  I would, instead,
concentrate on NNEXRAD and HDS (model output) data as the volumes and
number of products in those streams is considerable.

>I'll be looking at this over the weekend.

Only three more working days until Tuesday ;-)

>I guess my summary of the problem, based on your input is: We apparently 
>started grabbing something that slowed down my system's processing.

I would bet that this is either the NNEXRAD or HDS data feeds.

>The 
>element started while we were still feeding from Coriolis, OR the worm 
>problems on campus were affecting things... except this isn't supported 
>by your latency figures/graphs.  SO I've got to recall the delta's and 
>see if there's a fallback position.  I don't think the increase in NIDS 
>data requested would do that alone.

It is hard to tell exactly what is causing your system slowness without
being able to poke around to see what could be wrong.

>Thanks for all the support!

Glad to help!

>Hope you have a nice weekend.

I'll try, but the weather is not (yet) cooperating. One thing though, I
will not complain about rain after the 4 previous years of severe drought!

>-- 
>Gerry Creager -- address@hidden
>Network Engineering -- AATLT, Texas A&M University     
>Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
>Page: 979.228.0173
>Office: 903A Eller Bldg, TAMU, College Station, TX 77843

Cheers,

Tom

>From address@hidden Sat Aug 30 18:30:43 2003

Howdy!

re: I wouldn't touch ingestion of observational data.  I would, instead,
concentrate on NNEXRAD and HDS (model output) data as the volumes and
number of products in those streams is considerable.

>I modified what I was processing in pqact for NNEXRAD.  That "fixed" 
>things for a while.  So, I've changed it back.  I'll start doing a US 
>montage and a Texas Mesonet montage now and see where things go.  I'd 
>also restarted most of my other product scripts: contours for temp, Td, 
>Pbaro, sustained winds, 24 hr delta temp, etc.  We'll see if they still 
>work...

re: I would bet that this is either the NNEXRAD or HDS data feeds.

>I've got to get smarter on HDS as I'm not using it yet...

re: hard to tell what is wrong without poking around

>I can arrange that, if you'd like.

Later, gerry
-- 
Gerry Creager -- address@hidden
Network Engineering -- AATLT, Texas A&M University      
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
Page: 979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843