Successfully updated the ticket!
Failed trying to update the ticket.
Successfully updated the ticket!
Failed trying to update the ticket.
Downloading...

Numerous Gaps in Packets

Support Portal  »  Tickets  »  Ticket #JSG-3137677

Nissenbaum, Mark

Hello,

 

We recently managed to get LDM/NOAAPORT running on our VMware virtual machine. After our systems administrator set up the VMware Precision Clock, we were able to successfully process the multicast NOAAPORT stream within the LDM using the noaaportIngester. However, we are encountering numerous dropped packets, which is preventing many products from being added to the queue. Below is a sample from our log file. I suspect that the presence of retrograde packet numbers indicates a time mismatch, which is causing the dropped packets. On our Novra receiver, we have a strong signal with no issues related to uncorrectable rates or packet error rates.

 

Could you offer any advice on resolving the dropped packets issue, or suggest steps we might take to investigate further? Your assistance would be greatly appreciated.

 

Thank you,

Mark

 

---
Mark Nissenbaum

Senior Meteorologist

South Florida Water Management District

weather@sfwmd.gov | (561) 682-6113

 

 

 

20240918T010428.004533Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISNA62 BIRK 180100 inserted HDS [cat 5 type 4 ccb 2/0 seq 1583549 size 260]

20240918T010428.004563Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISNA62 BIRK 180100 inserted HDS [cat 5 type 4 ccb 2/0 seq 1583550 size 260]

20240918T010428.004588Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISNA62 BIRK 180100 inserted HDS [cat 5 type 4 ccb 2/0 seq 1583551 size 260]

20240918T010428.007712Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISNA62 BIRK 180100 inserted HDS [cat 5 type 4 ccb 2/0 seq 1583552 size 260]

20240918T010428.007748Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISNA62 BIRK 180100 inserted HDS [cat 5 type 4 ccb 2/0 seq 1583553 size 260]

20240918T010428.008328Z noaaportIngester[502806]    productMaker.c:pmStart:678          ERROR OOPS, start of new product [1583555 ] with unfinished product 1583554

20240918T010428.008352Z noaaportIngester[502806]    productMaker.c:pmStart:803          ERROR Product definition header version 1 pdhlen 16

20240918T010428.008359Z noaaportIngester[502806]    productMaker.c:pmStart:804          ERROR PDH transfer type 71

20240918T010428.008367Z noaaportIngester[502806]    productMaker.c:pmStart:812          ERROR header length 52 [pshlen = 36]

20240918T010428.008375Z noaaportIngester[502806]    productMaker.c:pmStart:814          ERROR blocks per record 1 records per block 1

20240918T010428.008382Z noaaportIngester[502806]    productMaker.c:pmStart:816          ERROR product seqnumber 1583555 block number 0 data block size 277

20240918T010428.008390Z noaaportIngester[502806]    productMaker.c:pmStart:817          ERROR product header flag 1

20240918T010428.008397Z noaaportIngester[502806]    productMaker.c:pmStart:818          ERROR prodspecific data length 0

20240918T010428.008404Z noaaportIngester[502806]    productMaker.c:pmStart:819          ERROR bytes per record 0

20240918T010428.008411Z noaaportIngester[502806]    productMaker.c:pmStart:820          ERROR Fragments = 0 category 5

20240918T010428.008428Z noaaportIngester[502806]    productMaker.c:pmStart:827          ERROR next header offset 0

20240918T010428.008435Z noaaportIngester[502806]    productMaker.c:pmStart:828          ERROR original seq number 0

20240918T010428.008442Z noaaportIngester[502806]    productMaker.c:pmStart:829          ERROR receive time 1726621464

20240918T010428.008449Z noaaportIngester[502806]    productMaker.c:pmStart:830          ERROR transmit time 1726621467

20240918T010428.008461Z noaaportIngester[502806]    productMaker.c:pmStart:831          ERROR run ID 0

20240918T010428.008468Z noaaportIngester[502806]    productMaker.c:pmStart:832          ERROR original run id 0

20240918T010428.008501Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISNA62 BIRK 180100 inserted HDS [cat 5 type 4 ccb 2/0 seq 1583555 size 260]

20240918T010428.008510Z noaaportIngester[502806]    productMaker.c:pmStart:522          WARN  Retrograde packet number: previous=14078371, latest=14078370, difference=4294967295

20240918T010428.008520Z noaaportIngester[502806]    productMaker.c:pmStart:533          WARN  Gap in packet sequence: 14078370 to 14078372 [skipped 1]

20240918T010428.008527Z noaaportIngester[502806]    productMaker.c:pmStart:535          WARN  prevProdSeqNum=1583554, pdh->seqno=1583556, prevBlockNum=1, pdh->dbno=0

20240918T010428.008551Z noaaportIngester[502806]    process_prod.c:process_prod:229     NOTE  ISXD03 EDZW 180000 CCA inserted HDS [cat 5 type 4 ccb 2/0 seq 1583556 size 453]

Stonie Cooper

Mark - From the snippet you provided, you are suffering from an issue that rest of us are.  NWS performed a channel data realignment on the 16th, and then moved the uplink from the primary to the backup yesterday morning.  The incoming data on some of the channels is now out of sequence - specifically NOPT, EXP, GRE, and if you have a separate modem configured for NWWS, it as well.

 

The retrograded packets are an artifact of an error on NWS's side, and it has been reported.  I will reach out to them again and remind them that this is not a non-trial issue.

Stonie Cooper, PhD

Software Engineer III

NSF Unidata Program Center

University Corporation for Atmospheric Research

I acknowledge that the land I live and work on is the traditional home of The Chahiksichahiks (Pawnee), The Umoⁿhoⁿ (Omaha), and The Jiwere (Otoe).

CC
archive-ldm@unidata.ucar.edu, archive-noaaport@unidata.ucar.edu
  •  
Drag and drop your attachments here.

Ticket Details

Ticket Details

Numerous Gaps in Packets
JSG-3137677
Local Data Manager (LDM)
Closed
Normal