[Support #XQH-733254]: LDM latency issue between NWS MAX and ERC client

Hi Gini,

> LDM Feed for some NWS sites has transitioned from  NWS Regional
> Aggregators to the ROC/NWS National Aggregator at HQ

Do the transitioned sites also feed the same data to their regional HQ?

> The ROC Aggregator feeds the UMD MAX.
> 
> ERC tried to establish feed from the UMD MAX today to pick up these
> transitioned sites  in addition it is requesting feed from NWS Regional
> Aggregators..
> 
> It does receive data -- but in bursts, and it receives  the ROC
> Aggregator site data with a 50-60 minute delay.
> 
> It is pulling other Regional data fine.
> 
> ERC is LDM 6.6.3 MAX is LDM 6.1.0
> ERC Ldmd.conf is:
> 
> 
> 
> 
> > # If the same feedtype and pattern is requested from multiple hosts, then
> > # th*e host of the first such request will be the initial primary source*
> > # of data-products (i.e., data-products will be rapidly sent using the
> > # HEREIS message) and the other hosts will initially be alternate
> > sources of
> > # data-products (i.e., data will be sent using the COMMINGSOON and BLKDATA
> > # messages).  The primary host will probably change over time --
> > depending on
> > # which host can deliver the data-products most quickly on average.
> > # Request feed from Eastern Region
> > request NEXRAD2 ".*" 216.38.95.9 ##THIS server
> > # Request feed from Southern Region
> > request NEXRAD2 "(.*)" 216.38.80.15
> > # Request feed from Central Region
> > #request NEXRAD2 ".*" 164.113.226.3
> > #request NEXRAD2 ".*" 204.227.126.60 #LDM2
> > # Request feed from Western Region
> > #request NEXRAD2 ".*" 205.124.249.245
> > #request NNEXRAD ".*" 205.167.25.177
> > request ANY "^wsr88d" reflect.ncdc.noaa.gov
> >
> > # University of Maryland MAXGIGAPOP (The Gateway)
> > REQUEST NEXRAD2 ".*" 140.90.113.165

The request for NEXRAD2 data from the MAX Gigapop has the same feedtype and 
pattern as the request for NEXRAD2 data from the Eastern Region HQ.  
Consequently, the LDM that made the requests will assume that these data 
streams are identical (i.e., have exactly the same data-products).  This won't 
cause problems AS LONG AS the datasets are completely disjoint.  I suspect that 
there's some overlap, however, in which case the MAX request should be modified 
to be unique, say

    REQUEST NEXRAD2 ((.*)) 140.90.113.165

The double parentheses will make the feedtype/pattern unique without causing 
any problems.

> MAX ldmd.conf  allows for ERC are:
> 
> # ERC Requests from MAX
> # Teo Lavis address@hidden 828-350-2012
> allow NEXRD2 ^64\.147\.208\.198$
> allow NEXRD2 ^64\.147\.208\.203$
> allow NEXRD2 ^64\.147\.208\.204$
> 
> 
> ERC Logs show MAX 140.90.113.165  as alternate feeder
> >
> > Feb 22 15:30:37 NWS-LDM1 140.90.113.165[26316] NOTE: Starting
> > Up(6.6.3): 140.90.113.165:388 20100222143037.939 TS_ENDT {{NEXRAD2,
> > ".*"}}
> > Feb 22 15:30:37 NWS-LDM1 140.90.113.165[26316] NOTE: Previous
> > product-information file ".1b7a59102630c3ec7b1416a033271735.info"
> > doesn't exist
> > Feb 22 15:30:37 NWS-LDM1 140.90.113.165[26316] NOTE: LDM-6 desired
> > product-class: 20100222143037.940 TS_ENDT {{NEXRAD2,  ".*"},{NONE,
> > "SIG=dbaf7ad58b4092de22967c1ea8f64355"}}
> > Feb 22 15:30:38 NWS-LDM1 140.90.113.165[26316] NOTE: Product
> > reclassification by upstream LDM: 20100222143037.940 TS_ENDT
> > {{NEXRAD2,  ".*"},{NONE,  "SIG=dbaf7ad58b4092de22967c1ea8f64355"}} ->
> > 20100222143037.940 TS_ENDT {{NEXRAD2,  ".*"}}
> > Feb 22 15:30:38 NWS-LDM1 140.90.113.165[26316] NOTE: Upstream LDM-6 on
> > 140.90.113.165 is willing to be an alternate feeder
> > Feb 22 15:31:39 NWS-LDM1 140.90.113.165[26316] NOTE: Switching
> > data-product transfer-mode to primary
> > Feb 22 15:31:39 NWS-LDM1 140.90.113.165[26316] NOTE: LDM-6 desired
> > product-class: 20100222143139.850 TS_ENDT {{NEXRAD2,  ".*"},{NONE,
> > "SIG=a8e5bc5e90abdd702a5f8c2098f424f4"}}
> > Feb 22 15:31:39 NWS-LDM1 140.90.113.165[26316] NOTE: Product
> > reclassification by upstream LDM: 20100222143139.850 TS_ENDT
> > {{NEXRAD2,  ".*"},{NONE,  "SIG=a8e5bc5e90abdd702a5f8c2098f424f4"}} ->
> > 20100222143139.850 TS_ENDT {{NEXRAD2,  ".*"}}
> > Feb 22 15:31:39 NWS-LDM1 140.90.113.165[26316] NOTE: Upstream LDM-6 on
> > 140.90.113.165 is willing to be a primary feeder
> > Feb 22 15:33:08 NWS-LDM1 140.90.113.165[26316] NOTE: Switching
> > data-product transfer-mode to alternate
> > Feb 22 15:33:08 NWS-LDM1 140.90.113.165[26316] NOTE: LDM-6 desired
> > product-class: 20100222143308.566 TS_ENDT {{NEXRAD2,  ".*"},{NONE,
> > "SIG=888cdd188513240a9b5d6e961d66cf2e"}}
> > Feb 22 15:33:08 NWS-LDM1 140.90.113.165[26316] NOTE: Product
> > reclassification by upstream LDM: 20100222143308.566 TS_ENDT
> > {{NEXRAD2,  ".*"},{NONE,  "SIG=888cdd188513240a9b5d6e961d66cf2e"}} ->
> > 20100222143308.566 TS_ENDT {{NEXRAD2,  ".*"}}
> > Feb 22 15:33:08 NWS-LDM1 140.90.113.165[26316] NOTE: Upstream LDM-6 on
> > 140.90.113.165 is willing to be an alternate feeder
> > Feb 22 15:38:32 NWS-LDM1 140.90.113.165[26316] NOTE: Switching
> > data-product transfer-mode to primary
> > Feb 22 15:38:32 NWS-LDM1 140.90.113.165[26316] NOTE: LDM-6 d
> >
> At the MAX ldmd.logs show nws-ldm1.hpcc.ercbroadband.org  COMINGSOON and
> send failures
> 
> > eb 22 17:36:42 nws1 nws-ldm1(feed)[12326]: up6.c:210: COMINGSOON: RPC:
> > Unable to receive; errno = Connection reset by peer
> > Feb 22 17:36:42 nws1 nws-ldm1(feed)[12326]: up6.c:435: Product send
> > failure: Input/output error
> > Feb 22 17:36:42 nws1 rpc.ldmd[10608]: child 12326 exited with status 6
> > Feb 22 17:36:43 nws1 nws-ldm1[12334]: ldm6_server.c:136: Restricting
> > request: 20100222163642.867 TS_ENDT {{NEXRAD2,  ".*"},{NONE,
> > "SIG=f5164203d5f9b2b040311e6d6e781c70"}} -> 20100222163642.867 TS_ENDT
> > {{NEXRAD2,  ".*"}}
> > Feb 22 17:36:44 nws1 nws-ldm1(feed)[12334]: up6.c:339: Starting
> > Up(6.1.0/6): 20100222163642.867 TS_ENDT {{NEXRAD2,  ".*"}}
> > Feb 22 17:36:44 nws1 nws-ldm1(feed)[12334]: topo:
> > nws-ldm1.hpcc.ercbroadband.org NEXRAD2
> > ~
> > ~
> 
> Can you explain the latency of the MAX data to ERC?
> Is it internal decsion of ldmd  to pull from Regions primarily since
> same data is requested ?
> Can you suggest solution?
> 
> Gini Galvin
> 301 713 0882 x 176
> 240 393 3348c

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: XQH-733254
Department: Support LDM
Priority: Normal
Status: Closed