Re: [ldm-users] Feedtype for /pFTM messages

On Tue, 2007-09-25 at 17:07 -0500, Daryl Herzmann wrote:
> Hi Steve,
> 
> I left out ldm-users to spare them the boredom. :)  Thanks for the 
> response.  What is a non-printable character?
> 
Daryl,

I put the following comments in my noaaport ingest code:

/* within product, allow RS, CR, NL, \0 */
/* add a little leeway....accept ETX in last 8 bytes since some products do */
/* this is only to keep HDS FOS category. Would rather use NOAAPORT types (SRC) 
*/

The RS character is historically used as the separator in MOS products
and a few
others, but newer bulletins mostly use "$$" between records.

> When I first turned on FTM (IDS|DDSPLUS) to my ingestor, it exposed 
> problems in my code due do nasty characters appearing there.  For example, 
> \000
> 
> Is the Null character printable? :)

Generally not, but I allow it in products since I typically found it as
a record value,
and some SACN AUTO8 products tended to insertit in otherwise all text
products . In fact, in C, if you try to printf() an array of characters,
and null is one of the values, the string will be terminated at that
point. 

Recall that the FOS  feedtypes were a legacy of our old days of
inserting data
into the IDD, and when I added NOAAport as a redundant ingest, the
primary goal
was to maintain backward compatibility and eliminate duplicates. Now, we
only
have NOAAport ingest, so the FOS conventions are archaic. The data that
wasn't part of FOS is put into NIMAGE, NNEXRAD, NGRID. I have reserved
NTEXT, NPOINT, NGRAPH as well. My preference would be to deliver
NOAAport
in the original format with CCB headers and not the FOS wrapped products
as we
have for backward compatibility. I would prefer to let the user FILE or
PIPE the products with the CCB, or have the pqact process strip the CCB
and wrap with FOS on the fly so that we could be compatible with AWIPS
software that uses he CCB headers and not the FOS- but the LDM is data
agnostic on the otherhand  and
isn't supposed to be overloaded with NOAAport specific processing
internals.
But that is still there or evolution- and to see where NWS takes
AWIPS2's data stream requirements.

If you are ever available to go out for a beer, I can give you a lot
more of my NOAAport ingest development headaches! But suffice it to say
that the CCB value
of "text" does not mean ascii readable, but instead free form.

Steve Chiswell
Unidata User Support


> 
> daryl
> 
> On Tue, 25 Sep 2007, Steve Chiswell wrote:
> 
> > Daryl,
> >
> > The CCB block of the NOAAport products isn't a 1to 1 match up with
> > our legacy FOS feedtypes. The NOAAport ingestion software here puts the
> > products with non-printable characters in the HDS data stream so that
> > you
> > don't inadvertantly see something totally unexpected in DDPLUS.
> >
> > You can use the "WMO" aggregate feedtype to insure that pattens
> > will match any and all of HDS, IDS and DDPLUS (aka DDS and PPS).
> >
> > Steve Chiswell
> > Unidata User Support
> >
> > On Tue, 2007-09-25 at 16:23 -0500, Daryl Herzmann wrote:
> >> Greetings,
> >>
> >> Today's wild goose chase involves tracking down the feedtype for the FTM
> >> products.  I notice some in the HDS feedtype and others in the IDS|DDPLUS,
> >> I know that the FTM often contains nasty characters
> >>
> >> Sep 25 21:17:52 pqcat INFO:      166 20070925202311.601     HDS 6636721
> >> NOUS64 KEPZ 252023 /pFTMEPZ
> >> Sep 25 21:17:52 pqcat INFO:      326 20070925202318.892 IDS|DDPLUS 6636837
> >> NOUS62 KMFL 252023 /pFTMBYX
> >> Sep 25 21:17:52 pqcat INFO:      326 20070925202320.391 IDS|DDPLUS 6636874
> >> NOUS62 KKEY 252023 /pFTMBYX
> >> Sep 25 21:17:59 pqcat INFO:      166 20070925203037.959     HDS 6641684
> >> NOUS64 KEPZ 252030 /pFTMHDX
> >>
> >> http://www.unidata.ucar.edu/software/ldm/ldm-current/basics/feedtypes/hds.html
> >>
> >> links to a non existant NWS page.
> >>
> >> Does the difference involve if the product came from the RPG or from
> >> AWIPS?
> >>
> >> Are there other 'text' products hiding from me in the HDS feed? :)
> >>
> >> thanks!
> >>    daryl
> >> _______________________________________________
> >> ldm-users mailing list
> >> address@hidden
> >> For list information or to unsubscribe,  visit: 
> >> http://www.unidata.ucar.edu/mailing_lists/
> > --
> > Steve Chiswell <address@hidden>
> > Unidata
> >
> 
-- 
Steve Chiswell <address@hidden>
Unidata