20050411: [address@hidden: nexrad2 data on rasp]

>From: Celia Chen <address@hidden>
>Organization: NCAR/RAP
>Keywords: 200504111650.j3BGoJv2002156 IDD latenccy

Hi Celia,

>We see a latency problem on the NEXRAD2 data on rasp at RAL. Could you please 
>check to see if thelma is having the similar problem?

The confusion here is most likely due to our transition of thelma from
an ingester and relay to a front end for a cluster of data servers, and
from those data servers not showing up in our real time statistics
pages as 'thelma.ucar.edu'.  The current set of data servers that
represent 'thelma.ucar.edu' (and 'idd.unidata.ucar.edu') are
idd2.unidata.ucar.edu, idd3.unidata.ucar.edu, and
idd4.unidata.ucar.edu.

The quickest thing to do in troublshooting an ingest problem of any kind
is to look at the Realtime statistics pages on our web site:

Unidata HomePage
http://www.unidata.ucar.edu
  LDM
  http://www.unidata.ucar.edu/content/software/ldm
     Real Time IDD/LDM Statistics
     http://my.unidata.ucar.edu/content/software/idd/rtstats/index.html
       Statistics by Host
       http://my.unidata.ucar.edu/content/software/idd/rtstats/siteindex.php

Look down the list and find the machine you are having ingest problems on:

edu.ucar.rap

  rasp.rap.ucar.edu [6.1.0]
    NEXRAD2 topology
    
Then click on the thelma.ucar.edu link.  The resultant plot is the latency
from thelma.ucar.edu to rasp.rap.ucar.edu.  This plot will take a long time
to generate given the number of NEXRAD stations contained in the feed.

Alternately, you an look at the latency for NEXRAD2 on one of the
cluster machines that is the back end for thelma.ucar.edu:

uni3.unidata.ucar.edu [6.2.0]
http://my.unidata.ucar.edu/content/software/idd/rtstats/siteindex.php?uni3.unidata.ucar.edu
  NEXRAD2 latency
  
http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NEXRAD2+uni3.unidata.ucar.edu
  
The last plot indicates that there is no problem getting the data on
the cluster nodes that are thelma's backend.  The latency you are
seeing, therefore, is likely due to the processing you are trying to do
with the data you are ingesting.

>Thanks.

>----- Forwarded message from Dan Megenhardt <address@hidden> -----
>
>From: Dan Megenhardt <address@hidden>
>To: address@hidden
>Cc: address@hidden, address@hidden
>Subject: nexrad2 data on rasp
>X-Virus-Scanned: amavisd-new at ucar.edu
>
>Celia,
>
>The NEXRAD2 data on rasp is coming in 45-60 minutes late. Looks like 
>this has been happening for over an hour now. We saw this same thing 
>occur on level last week. Do you know why this may be happening? We are 
>relying on this data to run a realtime system for the Dallas National 
>Weather Service forecast office. If you have any suggestions on 
>improving our data reliability please let me know.
>
>Thanks,
>Dan

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.