[IDD #JQW-816102]: LDM for AMS

Hi Brad,

I see a single feed request from 74.174.46.241 to oliver.unidata.ucar.edu:

Jan 18 12:57:01 oliver 74.174.46.241(feed)[3173] NOTE: Starting Up(6.8.1/6): 
20100118185658.768 TS_ENDT 
{{NXRDSRC|NEXRAD2|NEXRAD3|NPORT|NIMAGE|GEM|FNMOC|GPS|FAA604|DIFAX|NMC|FSL5|FSL
4|FSL3|FSL2|UNIDATA|SPARE|0x80000000,  ".*"}}, SIG=NONE, Primary
Jan 18 12:57:01 oliver 74.174.46.241(feed)[3173] NOTE: topo:  74.174.46.241 
{{NXRDSRC|NEXRAD2|NEXRAD3|NPORT|NIMAGE|GEM|FNMOC|GPS|FAA604|DIFAX|NMC|FSL5|FSL4|FSL3|FSL2|UNIDATA|SPARE|0x
80000000, (.*)}}

Given that the pipe into the exhibit hall is likely to not be large, I
strongly suggest that the single request be split into mutually exclusive
subsets.  For instance, the following simple split will help to minimize
latencies:

request IDS|DDPLUS|FSL2 ".*" oliver.unidata.ucar.edu
request NIMAGE ".*" oliver.unidata.ucar.edu
request GEM ".*" oliver.unidata.ucar.edu
request HDS ".*" oliver.unidata.ucar.edu
request NGRID ".*" oliver.unidata.ucar.edu
request NEXRAD3 ".*" oliver.unidata.ucar.edu
request NEXRAD2 ".*" oliver.unidata.ucar.edu

Beyond this simple split, you may want to consider requesting a subset
of products from the NEXRAD2 and NEXRAD3 datastreams (i.e., request
only the stations that you want to show).

NB: I did not include the CONDUIT datafeed in the split-out list.  If you
want to try to get CONDUIT (not a NOAAPort datastream you know), I strongly
recommend splitting it 5-ways:

# CONDUIT from Unidata/oliver
request CONDUIT "[09]$" oliver.unidata.ucar.edu
request CONDUIT "[18]$" oliver.unidata.ucar.edu
request CONDUIT "[27]$" oliver.unidata.ucar.edu
request CONDUIT "[36]$" oliver.unidata.ucar.edu
request CONDUIT "[45]$" oliver.unidata.ucar.edu

My overall comment is that we have never been successful in requesting
all of the data that is available in the IDD at the AMS.  The reason
is that the network pipe to our booth was always too small to
permit the volume of data from making it there with small latencies.
For demo purposes, we have dropped back to being able to show:

- realtime observational data (IDS|DDPLUS)
- realtime satellite imagery (NIMAGE or UNIWISC (UNIWISC is not a NOAAPort 
stream))
- HDS (aka HRS, the lower resolution model data in NOAAPort)
- some select NEXRAD3 stations (we tend to look to see where the action
  is happening around the country, and then request a subset of stations)
- some select NEXRAD2 stations (same comment as for NEXRAD3)
- as much of NGRID as the connection will allow (which is usually not much)

The BIG users of bandwidth can be seen by the following current snapshot
from one of the nodes in our IDD cluster (and the same as oliver):

Data Volume Summary for uni3.unidata.ucar.edu

Maximum hourly volume   9907.190 M bytes/hour
Average hourly volume   4028.101 M bytes/hour

Average products per hour     182836 prods/hour

Feed                           Average             Maximum     Products
                     (M byte/hour)            (M byte/hour)   number/hour
CONDUIT                1625.843    [ 40.363%]     5921.915    46799.227
NEXRAD2                 880.598    [ 21.861%]     1082.304    41463.773
NGRID                   623.889    [ 15.488%]     1290.246    14704.273
HDS                     236.415    [  5.869%]      452.493    17914.909
NEXRAD3                 187.454    [  4.654%]      227.804    28467.227
FNMOC                   175.850    [  4.366%]     1106.499     1820.273
NIMAGE                  138.496    [  3.438%]      255.250      177.591
FNEXRAD                  52.805    [  1.311%]       60.988       51.955
EXP                      43.483    [  1.079%]       75.127      433.500
IDS|DDPLUS               33.996    [  0.844%]       43.347    30759.273
UNIWISC                  20.995    [  0.521%]       28.182       26.364
DIFAX                     3.766    [  0.093%]       13.088        4.773
GEM                       2.794    [  0.069%]       29.473      191.318
FSL2                      1.717    [  0.043%]        1.862       21.455

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: JQW-816102
Department: Support IDD
Priority: Normal
Status: Closed