[LDM #OWC-960244]: pqing stripping byte

Carissa,

I have an hypothesis as to why pqing(1) is deleting ESC characters if they 
follow the sequence CR CR LF in a WMO bulletin. That sequence is a control 
sequence: it introduces the bulletin header; new reports, and the end of the 
WMO message. If the WMO bulletin is binary rather than textual, there's a small 
but non-zero probability that it will contain that control sequence in its 
data. Such a non-control sequence can be delimited and vitiated by appending it 
with an ESC character, with the understanding that the ESC character will be 
removed by the downstream decoder.

Interestingly, the WMO No. 386 manual doesn't mention this. I can only assume 
that this was an ad-hoc hack necessitated by the increasing use of binary WMO 
bulletins (it does date from 1994, after all).

As I see it, you have the following options: 1) use pqinsert(1) rather than 
pqing(1); 2) append an ESC character to the sequence CR CR LF wherever it 
occurs in the data and is not a control sequence; or 3) have pqing(1) interpret 
the data as textual rather than as binary via use of the "-f feedtype" option, 
in which case the feedtype must have no bits in common with the feed pattern 
"IDS|DDPLUS|HDS|NMC2|NMC3|AFOS|FAA604" (see 
<https://www.unidata.ucar.edu/software/ldm/ldm-current/basics/feedtypes/>).

I don't know if the last option will work. It definitely won't if the data is 
binary.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: OWC-960244
Department: Support LDM
Priority: Normal
Status: Closed