[Support #HDQ-517625]: Assistance requested for "Gap in packet sequence" log entries from noaaportIngester

Hi Gregg,

re:
> The compressed tarred log files are 180+ MB, so Google is sending Google
> links, let me know if you can't access the google links (there is a file
> June30_log_files.tar.Z, and July1_log_files.tar.Z,  ignore the other link
> if it exists).  If you can't, is there a place I can "sftp" the files, as
> SPC doesn't have access to "ftp"?

We do not operate an FTP site to which data files can be uplinked, sorry.

re:
> In summary to your queries below:
> 
> the SBN2 is still in the data center using media-converters to receive the
> data,

OK.  Just so you know, NOAA/GSL uses the same A-to-D, fiber optic link,
then D-to-A conversion approach, and this is not causing any problems
in their setup.

re:
> from ~1430Z June 30 through 00Z July 1, there were:
> 52 Gap errors across ports 1201, 1203, 1209, and 1210
> 8,088 Gap errors on port 1208.
> 
> from 0000Z July 1 through ~1741Z July 1, there were:
> 421 Gap errors across ports 1201, 1203, 1204, 1209 and 1210
> 12,410 Gap errors on port 1208

Quick aside comment: we, UW/SSEC, LSU/SRCC and NOAA/GSL all rotate
our NOAAPort ingest log files at 0 UTC each day.  This allows for
easy comparison of log file content (including number of Gap
messages and associated number of missed frames) among all machines.

re:
> The entire period from ~1430Z June 30 - ~1741Z July 1, there were:
> 
> 85 Gap errors on port 1201
> 0 Gap errors on port 1202
> 330 Gap errors on port 1203
> 4 Gap errors on port 1204
> 0 Gap errors on port 1205
> 19,032 Gap errors on port 1208
> 22 Gap errors on port 1209
> 32 Gap errors on port 1210

So, the number of Gap messages being logged for all channels except
the experimental one is on the order of 440 per day.  While this is
not "good", it is still usable.  The predominance of Gaps in the
experimental channel log file is, of course, terrible and not usable.
Aside from there being some software that is running on the machine
that is interfering with traffic coming in on port 1208, we have no
idea of what might be wrong.  If I didn't emphasize this previously,
I will now: no other site that we know of is having a problem ingesting
data on a single port while data on other ports are coming in and being
processed.

re:
> I wasn't aware of the Missed Frames option and I'm suprised at the large
> numbers for ports 1201, 1203, 1209 and obviously 1208.

OK.  Included at the end of each logged Gap message is the number of
missed frames listed as, for example, [skipped 6].  The "skipped"
number is the number of frames missed.
re: 
> 
> *           Missed Frames      Missed Frames *
> *Log File      Since Previous     Since *
> *Name         * *Report        *     *Start         *
> 
> 1.nmc.log.2          21              1,191
> 1.nmc.log.1       4,060              5,251
> 2.goes.log.2          0                  0
> 2.goes.log.1          0                  0
> 3.nmc2.log.2          3                982
> 3.nmc2.log.1     11,742             12,724
> 4.nopt.log.2          0                  1
> 4.nopt.log.1        184                185
> 5.nmc3.log.2          0                  0
> 5.nmc3.log.1          0                  0
> 8.exp.log.2   3,134,696         32,025,463
> 8.exp.log.1   5,283,024         37,308,487
> 9.grw.log.2          58              2,136
> 9.grw.log.1          11               2147
> 10.gre.log.2         76                110
> 10.gre.log.1        328                438

The numbers of missed frames for all channels except the experimental
one are a lot more than we would like to see.

If you are interested, I can add you to the email list to which I send
daily summary "Gap statistics" for the UCAR/Unidata, UW/SSEC, LSU/SRCC
and NOAA/GSL machines.  The objective of adding you to the list (which
I can remove you from whenever you say) would be for you to see a
comparison of how other sites are doing on a daily basis.  Please let
me know if you want me to add you (address@hidden) to the
list.

re:
> July1_log_files.tar.Z
> <https://drive.google.com/a/noaa.gov/file/d/1y4BzSlR4RDQJGRTjLFFCBLZr0TcEAJ4K/view?usp=drive_web>
> June30_log_files.tar.Z
> <https://drive.google.com/a/noaa.gov/file/d/1KvHZfPg-GhqNna3Wu5JapQfOEZGZtnom/view?usp=drive_web>

I will download one of these files and take a look later today.  I am not
sure that they will shed much light on what is going on, but, they might
reveal a pattern that could be useful.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: HDQ-517625
Department: Support NOAAPORT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.