20050519: IDD latency ULM->SRCC (cont.)

>From: "Adam Taylor" <address@hidden>
>Organization: ULM
>Keywords: 200505182044.j4IKirP3020384 IDD latency

Hi Adam,

re:
>Here are my old and new ldmd.conf (with pavan instead of seistan)  As far as
>I see there are no errors in it.

There are no errors in your ldmd.conf 'request' lines.  The problem you
were/are seeing is a result of a machine that stokes.metr.ou.edu is
feeding from injecting data into its LDM queue tagged as 'WMO'
products.

Tagging products with WMO makes request lines for IDS, DDS, PPS, HDS
and DDPLUS all match those products.  Each simple feedtype is
represented by a single bit in a 32-bit word; compound feed types
(DDPLUS in this case) contain all bits from the feedtypes it
represents.  WMO is a compound feed type that represents the union of
IDS, DDS, PPS, and HDS; DDPLUS is a compound feed type that represents
DDS and PPS.

Since lots of products being ingested on stokes are labeled as WMO,
they "advertise" that they are IDS, DDS, PPS, and HDS products all at
the same time.  Given this, your request lines IDS|DDPLUS and HDS data
match each product.  Because of this match, you are receiving each
product in your request lines:

from ldmd.conf.old:

#
# Global observational data
#
request IDS|DDPLUS      ".*"    seistan.srcc.lsu.edu PRIMARY
request IDS|DDPLUS      ".*"    stokes.metr.ou.edu ALTERNATE
#
# NOAAPORT model output
#
request HDS     ".*"    stokes.metr.ou.edu ALTERNATE
request HDS     ".*"    seistan.srcc.lsu.edu PRIMARY

The huge latencies you were seeing are a result of requesting the large
volume of data data using the ALTERNATE feedtype.  The system inserting
data into the machine upstream of stokes was not calcualting MD5
checksums in the same way as the machines "officially" supplying
NOAAPORT data to the IDD.

If you look at your IDS|DDPLUS latencies now, you will see that you are
getting the "real" IDS|DDPLUS products from pavan.srcc.lsu.edu with no
latency.  You continue, however, to get all of the products labeled as
WMO from stokes with an ALTERNATE feed, and this is still showing high
latencies.  In order to eliminate the high latency, mislabeled products
from your ingest, you should turn off your request lines for IDS|DDPLUS
and HDS to stokes and then stop/restart your LDM.

We will be working with OU to rectify the situation today, but I don't
know how long this will take to get fixed.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.