[IDD #RTY-317977]: Recent troubles with delayed 18 UTC METAR data over the IDD

Hi Daryl,

It is likely that the anomalies you are reporting from NOAAPort
ingest machines we operate here in Boulder are a result of the twice
a year solar interference that satellite ingest systems are plagued
with.

Here is a snippet from our Novra S300N log file on leno that shows the
marked drop in C/N (Carrier to Noise) around 18Z from 16.0dBm to as low
as 7.4 dBm and then back to 16.0 dBm:

leno.unidata.ucar.edu

20200305.1752 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.0dB -30dBm  -0.035  -89.001  155.64  40.72  50.63
20200305.1753 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.1dB -30dBm  -0.035  -89.001  155.64  40.72  58.69
20200305.1754 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.4dB -30dBm  -0.035  -89.001  155.64  40.72  70.42
20200305.1755 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
11.3dB -30dBm  -0.035  -89.001  155.64  40.72  71.83
20200305.1756 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On >255/s 7.0600e-03 
9.6dB -30dBm  -0.036  -89.001  155.64  40.72  32.41
20200305.1757 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 3.6300e-03 8.3dB 
-29dBm  -0.036  -89.000  155.64  40.72   0.00
20200305.1758 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 3.6300e-03 7.6dB 
-29dBm  -0.036  -89.000  155.64  40.72   0.00
20200305.1759 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 3.6300e-03 7.4dB 
-29dBm  -0.036  -89.000  155.64  40.72   0.00
20200305.1800 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 3.6300e-03 7.9dB 
-29dBm  -0.036  -89.000  155.64  40.72   0.00
20200305.1801 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 3.6300e-03 8.9dB 
-30dBm  -0.036  -89.000  155.64  40.72   0.00
20200305.1802 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
10.5dB -30dBm  -0.036  -89.000  155.64  40.72  33.47
20200305.1803 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.2dB -30dBm  -0.037  -89.000  155.64  40.72  67.84
20200305.1804 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
14.0dB -30dBm  -0.037  -89.000  155.64  40.72  70.45
20200305.1805 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.2dB -31dBm  -0.037  -88.999  155.64  40.72  33.83
20200305.1806 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.7dB -32dBm  -0.037  -88.999  155.64  40.72  59.07
20200305.1807 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.0dB -32dBm  -0.037  -88.999  155.64  40.72   8.44
20200305.1808 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.2dB -31dBm  -0.037  -88.999  155.64  40.72  31.62
20200305.1809 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.3dB -31dBm  -0.037  -88.999  155.64  40.72  65.35
20200305.1810 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.2dB -31dBm  -0.038  -88.999  155.64  40.72  68.88
20200305.1811 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.3dB -31dBm  -0.038  -88.999  155.64  40.72  48.17
20200305.1812 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.3dB -31dBm  -0.038  -88.999  155.64  40.72  47.79
20200305.1813 DVBS2 1110.0 29.999 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
16.1dB -31dBm  -0.038  -88.998  155.64  40.72  51.48

The NOAAPort ingest machine at LSU/SRCC did experience solar interference
around but not exactly at the same time since they are geographically distant 
from
us.  Here is the same time interval snippet from the LSU/SRCC NOAAPort ingest
machine:

mistral.srcc.lsu.edu

20200305.1752 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.3dB -34dBm  -0.035  -89.001  155.64  40.72  50.19
20200305.1753 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.2dB -34dBm  -0.035  -89.001  155.64  40.72  58.68
20200305.1754 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.4dB -34dBm  -0.035  -89.001  155.64  40.72  70.21
20200305.1755 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.3dB -35dBm  -0.035  -89.001  155.64  40.72  71.53
20200305.1756 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
15.1dB -34dBm  -0.036  -89.001  155.64  40.72  70.77
20200305.1757 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
14.6dB -34dBm  -0.036  -89.000  155.64  40.72  39.43
20200305.1758 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
14.3dB -34dBm  -0.036  -89.000  155.64  40.72  63.39
20200305.1759 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
14.2dB -34dBm  -0.036  -89.000  155.64  40.72  47.00
20200305.1800 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.9dB -34dBm  -0.036  -89.000  155.64  40.72  69.41
20200305.1801 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.6dB -34dBm  -0.036  -89.000  155.64  40.72  67.22
20200305.1802 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.4dB -34dBm  -0.036  -89.000  155.64  40.72  34.82
20200305.1803 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 2/s 0.0000e+00 
10.9dB -34dBm  -0.037  -89.000  155.64  40.72  64.01
20200305.1804 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 3.1500e-03 9.3dB 
-33dBm  -0.037  -89.000  155.64  40.72   0.00
20200305.1805 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 3.1500e-03 8.0dB 
-34dBm  -0.037  -88.999  155.64  40.72   0.00
20200305.1806 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 3.1500e-03 7.4dB 
-33dBm  -0.037  -88.999  155.64  40.72   0.00
20200305.1807 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 3.1500e-03 7.3dB 
-33dBm  -0.037  -88.999  155.64  40.72   0.00
20200305.1808 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 3.1500e-03 7.7dB 
-33dBm  -0.037  -88.999  155.64  40.72   0.00
20200305.1809 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 3.1500e-03 8.8dB 
-34dBm  -0.037  -88.999  155.64  40.72   0.00
20200305.1810 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 242/s 7.8000e-03 
10.4dB -34dBm  -0.038  -88.999  155.64  40.72  60.73
20200305.1811 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.1dB -34dBm  -0.038  -88.999  155.64  40.72  46.33
20200305.1812 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.5dB -35dBm  -0.038  -88.999  155.64  40.72  48.45
20200305.1813 DVBS2 1110.0 29.998 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
14.0dB -35dBm  -0.038  -88.998  155.64  40.72  42.34

The LSU solar interference period starts a little later, 1759Z, and lasts
till later, 1813Z.

Here is the same snippet from one of the NOAAPort ingest machines at UW/SSEC:

20200305.1752 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.8dB 100dBm  -0.035  -89.001  155.64  40.72  50.21
20200305.1753 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.7dB 100dBm  -0.035  -89.001  155.64  40.72  58.60
20200305.1754 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.6dB 100dBm  -0.035  -89.001  155.64  40.72  70.26
20200305.1755 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.7dB 100dBm  -0.035  -89.001  155.64  40.72  71.67
20200305.1756 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.6dB 100dBm  -0.036  -89.001  155.64  40.72  71.13
20200305.1757 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.6dB 100dBm  -0.036  -89.000  155.64  40.72  39.95
20200305.1758 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.5dB 100dBm  -0.036  -89.000  155.64  40.72  61.46
20200305.1759 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.6dB 100dBm  -0.036  -89.000  155.64  40.72  48.40
20200305.1800 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.5dB 100dBm  -0.036  -89.000  155.64  40.72  69.43
20200305.1801 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.5dB 100dBm  -0.036  -89.000  155.64  40.72  65.96
20200305.1802 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.4dB 100dBm  -0.036  -89.000  155.64  40.72  25.12
20200305.1803 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.3dB 100dBm  -0.037  -89.000  155.64  40.72  68.15
20200305.1804 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.9dB 100dBm  -0.037  -89.000  155.64  40.72  69.33
20200305.1805 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.4dB 100dBm  -0.037  -88.999  155.64  40.72  34.60
20200305.1806 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
11.9dB 100dBm  -0.037  -88.999  155.64  40.72  63.74
20200305.1807 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
11.8dB 100dBm  -0.037  -88.999  155.64  40.72   8.31
20200305.1808 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.2dB 100dBm  -0.037  -88.999  155.64  40.72  24.48
20200305.1809 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
12.8dB 100dBm  -0.037  -88.999  155.64  40.72  66.37
20200305.1810 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.2dB 100dBm  -0.038  -88.999  155.64  40.72  68.62
20200305.1811 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.3dB 100dBm  -0.038  -88.999  155.64  40.72  48.20
20200305.1812 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.3dB 100dBm  -0.038  -88.999  155.64  40.72  50.68
20200305.1813 DVBS2 1110.0 30.000 2/3_16APSK 0 On 18 On On 0/s 0.0000e+00 
13.1dB 100dBm  -0.038  -88.998  155.64  40.72  56.62

As you can see, the UW/SSEC machine did not experience solar interference
in the same time frame

The NOAAPort derived content in IDD datastreams is a union of the ingest
from us (leno.unidata.ucar.edu, chico.unidata.ucar.edu), LSU/SRCC 
(mistral.srcc.lsu.edu) and UW/SSEC (np1.ssec.wisc.edu, np2.ssec.wisc.edu).
This union makes sure that sites get all products received by all ingest
machine that are unique.  Products with the same MD5 signatures as
products already received and still in the LDM queue should be rejected
by the LDMs running on the front ends of our IDD relay clusters.  Unfortunately,
products that are damaged during receipt will have different MD5 signatures,
and, therefore, be sent as additional data.

As to products received very late, this is typically due to a recirculation
of the products being moved in the IDD, but this typically only happens
when one of the machines participating in the shared ingest duty have
LDM queues that are too small.  I can say that the LDM queue on the LSU/SRCC
machine is much smaller than we want (7 GB), but we have no power to get
them to install more RAM so that their queue can be made larger.

I hope that the above makes sense!  If it doesn't, please let me know
and I will try to make another run at explaining!!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: RTY-317977
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.