Re: 20011124: noaaport.unidata.ucar.edu not getting NOAAPORT data

Wow.  What happened?

In looking in the logs on noaaport, it looks like both pqings decided to
stop:

Nov 24 10:39:59 desi.unidata.ucar.edu pqing[10299]: Not a WMO GRIB
format message. 33611           833  YRWB99 KWBG 241000
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]: Exiting
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   Queue usage
(bytes):540000256
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:             (nregions):  
112058
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   Duplicates
rejected:  491106
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   WMO Messages
seen:   8995587
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   SOH/ETX missing 
:        0
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   parity/chksum
err:     0
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   WMO format errors:
4458
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10299]:   FILE Bytes read:   
2122805111
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]: Exiting
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   Queue usage
(bytes):540000256
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:             (nregions):  
112058
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   Duplicates
rejected:  302846
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   WMO Messages
seen:   3879161
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   SOH/ETX missing 
:        0
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   parity/chksum
err:     0
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   WMO format errors:
62
Nov 24 11:11:02 desi.unidata.ucar.edu pqing[10298]:   FILE Bytes read:   
3674824904
Nov 24 17:41:25 desi.unidata.ucar.edu motherlode[21637]: Connection from
motherlode.ucar.edu
Nov 24 17:41:25 desi.unidata.ucar.edu motherlode(noti)[21637]: Starting
Up: 20011124174124.813 TS_ENDT {{ANY,  ".*"}}
N

This would have been about 4 AM.  And someone connected from motherlode
around 10:40, well after the outage, but before Russ first noticed the
problem...  odd...

I'll poke around more later.

Anne

>From address@hidden Sun Nov 25 09:38:53 2001
>To: Anne Wilson <address@hidden>, address@hidden
>Subject: Re: 20011124: noaaport.unidata.ucar.edu not getting NOAAPORT data
>Cc: address@hidden, address@hidden, address@hidden,
>   address@hidden

I don't see a problem with the OS in the system logs.  desi was up a while,
over 120 days in all;

reboot    system boot                   Sat Nov 24 16:54
reboot    system boot                   Fri Jul 27 06:51

I scanned the system for core files, but found none.  It's too bad that
/tmp/inge.log was flushed on reboot -- we may have been able to look at the
status of the ingest card.  I've modified the startup of the SSEC code to
put future inge logs in /var/tmp where they will be backed-up and will
be rotated to .old on restart.

mike