[IDD #JWQ-821786]: rtstats graphs poor when using primary/secondary feeds

Hi Art,

re:
> The rtstats graphs seem to have problems when IDD data are sourced from more
> than one upstream site in primary/secondary modes.

In general, we don't agree with this observation.  The reason I say this is
because the toplevel IDD relays for data delivered in the NOAAPort SBN
feed from multiple injection sites, and latencies/volume plots for
those feeds look OK.

re:
> Since we've switched to the
> two woc sources for conduit, the graph has become almost useless (see
> http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd-ingest.meteo.psu.edu).
> I'm assuming this is a known problem...?

I am not quite sure what you are referring to with the plot referenced.
Can you elaborate?

NB: I see clearly that the latencies you are seeing from the Boulder WOC
are too high.  The question that immediately comes to mind is what
your ~ldm/etc/ldmd.conf REQUEST lines look like?  We use a 5-way split
REQUEST setup, and this helps keep the latencies as low as possible.

The other thing I think may be happening is that the LDM queue size
on your ingest machine is not large enough to house CONDUIT products
long enough so that products from the slower upstream can be rejected
because they are still in the receiving queue.

re:
> Is the cause known and/or is there a way to fix it?

I can not respond without knowing exactly what you are referring to.

re:
> On the other hand, I'm concerned that by having
> primary/secondary sources of data on two unbalanced paths (i.e. our path to
> Silver Spring is probably much faster than our path to Boulder), our feed
> delays will increase due to the flip/flopping between faster and slower
> paths... but this is hard to confirm with the graph messed up. Please 
> advise...

This should not happen in the scenario you present.  The flip-flopping occurs
most when the latency from the two upstream sites is nearly identical, AND
this should not pose any problems with the contents of the datastream(s)
on the two upstream hosts are identical.  When the latencies measurably differ,
the downstream should continue to feed in PRIMARY mode from the path with least
latency and not flip to the other feed.

If you would like to discuss this further by phone, I will be out of User 
Committee
meetings by 12:30 MDT.  You could call me at 1 pm our time (3 pm your time)
and we could discuss the issues in detail.  My office phone number is
303.497.8642.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: JWQ-821786
Department: Support IDD
Priority: Normal
Status: Closed