20030705: upgrade to LDM-6 at NMT (cont.)

>From: "D. J. Raymond" <address@hidden>
>Organization: NMT
>Keywords: 200307041809.h64I9TLd018509 IDD LDM-6

Hi Dave,

> > request     IDS|DDPLUS      ".*"    yin.engin.umich.edu
> > request     HDS     ".*"    yin.engin.umich.edu
> > 
> > If your setup is like a number of others, this split will at least drop
> > the latencies for the IDS|DDPLUS observational data down to low values.

re: request lines in ldmd.conf
>Here is what I am doing now:
>
>request WMO|HRS "(^[A-OQ-X])|(^[YZ].[^AHIJRU])" yin.engin.umich.edu

OK, two things here:

- since WMO is the union of IDS|DDPLUS and HDS so WMO|HDS is
  redundant.  I see that you are subsetting your request most likely
  for the HDS data, which is what a lot of folks have done.  I
  recommend splitting this request so that the ingestion of HDS doesn't
  slow down the ingestion of IDS|DDPLUS:

request IDS|DDPLUS      ".*"    yin.engin.umich.edu
request HDS "(^[A-OQ-X])|(^[YZ].[^AHIJRU])" yin.engin.umich.edu

  Even with potential packet shaping (comment below), this split should
  really help drop the latencies for IDS|DDPLUS data.

- we are not seeing any real time statistics for the UNIWISC feed
  (aka MCIDAS). This most likely means that your request to
  iita.rap.ucar.edu is being denied or that iita.rap.ucar.edu is
  not currently running an LDM.  A quick 'ldmping' from a machine
  here in the UPC indicates that the latter is the case:

% ldmping iita.rap.ucar.edu
Jul 06 16:32:55      State    Elapsed Port   Remote_Host           rpc_stat
Jul 06 16:32:55  ADDRESSED   0.035560    0   iita.rap.ucar.edu  RPC: Unable to 
receive; errno = Connection reset by peer

Given that iita is not running, I setup an allow for any .nmt.edu
machine on rainbow.al.noaa.gov.  Also, your request pattern for UNIWISC
imagery can be cleaned up just a bit.  I would advise you to change
your request for MCIDAS imagery to:

request UNIWISC  "(^pnga2area Q1)"       rainbow.al.noaa.gov

The Unidata-Wisconsin datastream will most likely be undergoing a
transition later this summer, so you will probably want to revisit this
request later.

re: clock synchronization

>I am using ntp, and I am pretty sure the clock is correct.

OK, I just threw that out because we are seeing a lot of sites not
maintaining their clock accurately.  This is a big problem for data
reception when the offset gets to be on the order of an hour.

re: high latencies being seen on huron
>I have been fighting with the local packetshaper implementation, and this
>may be the problem.

This would have been my next question.  We are finding that more and
more university sites are running packet shapers.  We have had great
success in convincing networking support folks that they should bore a
hole for LDM/IDD traffic through their shaper.

>PS -- I am off to Japan for a week for the IUGG meeting.  Back in the
>office on 14 July.

Color me envious!  Have a great time!

Tom