Re[6]: NOAAPORT and LDM (fwd)

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================

---------- Forwarded message ----------
Date: Tue, 2 Mar 1999 14:01:01 -0600
From: Gregory Grosshans <address@hidden>
To: Robb Kambic <address@hidden>
Subject: Re[6]: NOAAPORT and LDM 

     Robb,
     
     After some email with FSL I've been able to get LDM/pqact to work on 
     specified products.  Here is some information that I've found out.  I 
     do have some questions though.
     
     An entry like:
        FSL5    ^NOAAPORT       FILE    data/np/npdata
     
     Does work.  In fact if it looks like this:
        FSL5    (^NOAAPORT*)    FILE    data/np/npdata-fsl
     
     Both data files look exactly the same.  
     
     However, the following two entries don't produce the same results:
     
     FSL5       ^NOAAPORT.NWSTG.TEXT.SA FILE    data/np/sadata-normal
     FSL5       (^NOAAPORT.NWSTG.TEXT.SA*)      FILE    
     data/np/sadata-fslway
     
     The second entry/listing results in a data file that is much larger 
     than the data file generated by the first entry.  I don't understand 
     why this is happening.
     
     Here is some additional information and a question:
     
     The NOAAPORT header for each product is a lot different than the 
     typical Family Of Services headers.  For example, on NOAAPORT the 
     following metar ob was received:
     
     @^L^ARUKWBC^Bc^C^B^P^_^AKDENSPCN31 CWAO 021627^M^M
     SPECI CYFC 021627Z 19014KT 6SM -SHRA BR OVC018 RMK SC8=^M^M ^M^M
     ^C
     
     And on the FOS, Domestic Data Services feed the same product looks 
     like:
     
     ^A^M^M
     
     570 ^M^M
     SPCN31 CWAO 021627^M^M
     SPECI CYFC 021627Z 19014KT 6SM -SHRA BR OVC018 RMK SC8=^M^M ^M^M
     ^C
     
     What I don't understand is that to capture this product in pqact on 
     the NOAAPORT feed the following must be used:
     
     FSL5    (^NOAAPORT.NWSTG.TEXT.SP*)      FILE    data/np/metar-data
     
     From the DDS feed the following can be used:
     
     DDS        ^S[AP].... .... ([0-3][0-9])    FILE    data/dds/metar-data
     
     
     
     Sometimes it seems that the location of the parenthesis for the 
     NOAAPORT feed is important.  I've tried having them in different 
     locations or not at all and then LDM doesn't act on the product.  
     Perhaps it depends on if a '*' is being used or not.
     
     Since the actual product doesn't even have 'NOAAPORT.NWSTG.TEXT' 
     within it, it makes me suspect that the LDM queue has a product ID for 
     each product within the queue.  Then if the product ID matches 
     something in the pqact.conf file, then pqact acts on the data/product 
     in the queue.  Is this correct??
     
     On the IDD feed containing the NOAAPORT data do the product headers 
     look like they do on the FOS, or do they look similar to what I'm 
     seeing on NOAAPORT?
     
     
     Thanks,
     Gregg
     
     
     
     ______________________________ Reply Separator 
     _________________________________ Subject: Re: Re[6]: acqserver 
     question
     
     ------------- Begin Forwarded Message -------------
     
     Gregg,
     I suggest that you change the "pattern" part of your pqact.conf line 
     for FSL5 feeds.
     
     I made a simple pqact.conf file, that copies metars (and more) to a 
     file.
     
     #
     FSL5    (^NOAAPORT.NWSTG.TEXT.SA*)        FILE    -strip  
     /tmp/metar-data #
     And this appears to be working.
     
     The key that we use in acqserver in the pq_insert, is probably 
     different than that used in the other data feeds. Within our Nimbus 
     system the keys we look for metars are:
     
     NOAAPORT.NWSTG.TEXT.SABA31*
     NOAAPORT.NWSTG.TEXT.SACN31* 
     NOAAPORT.NWSTG.TEXT.SACN4*  
     NOAAPORT.NWSTG.TEXT.SACN51* 
     NOAAPORT.NWSTG.TEXT.SACU31* 
     NOAAPORT.NWSTG.TEXT.SAGC31* 
     NOAAPORT.NWSTG.TEXT.SAHA31* 
     NOAAPORT.NWSTG.TEXT.SAUC82*
     NOAAPORT.NWSTG.TEXT.SAUE80*
     NOAAPORT.NWSTG.TEXT.SAUS52*
     NOAAPORT.NWSTG.TEXT.SAUS70*
     NOAAPORT.NWSTG.TEXT.SAUS80* 
     NOAAPORT.NWSTG.TEXT.SAUS90* 
     NOAAPORT.NWSTG.TEXT.SAUW83* 
     NOAAPORT.NWSTG.TEXT.SPUS70* 
     NOAAPORT.NWSTG.TEXT.SPUS80* 
     NOAAPORT.NWSTG.TEXT.SPUS90* 
     
     All the keys that I create in the acqserver source code, 
     (storeProduct.c) are in the form:
     
     NOAAPORT.<type>.<cat>.<wmoHdr>.[BBB].<timestamp>
     
     Do your pattern matching based on this.
     
     The small file that I have put together that appears to work is: 
     ######################################################################
     ########## #
     # I am writing metars to /tmp
     FSL5    (^NOAAPORT.NWSTG.TEXT.SA*)      FILE    -strip  
     /tmp/metar-data #
     # mail for alert
     FSL5    (^NOAAPORT.NWSTG.TEXT.ACUS01*)  PIPE    -close  -strip
     mail    address@hidden
     # mail for sigmets
     FSL5    (^NOAAPORT.NWSTG.TEXT.WS*)      PIPE    -close  -strip
     mail    address@hidden
     #mail for a couple of metars
     FSL5    (^NOAAPORT.NWSTG.TEXT.SPUS90*)  PIPE    -close  -strip
     mail    address@hidden
     FSL5    (^NOAAPORT.NWSTG.TEXT.SPUS80*)  PIPE    -close  -strip
     mail    address@hidden
     
     Hope this helps.
     
     Leslie


______________________________ Reply Separator 
_________________________________
Subject: Re: Re[4]: NOAAPORT and LDM 
Author:  Robb Kambic <address@hidden> at 
NSSLGATE
Date:    2/26/1999 1:35 PM


On Thu, 25 Feb 1999, Gregory Grosshans wrote:

>      Yes, the data is getting in the queue.  I see it 
with ldmadmin watch >      and its in the ldmd.log 
file. Outside of spacing here is what one
>      entry looks like from ldmadmin watch:

Greg,

Good, Sometime questions may trivial to you, but I need 
to have a base to compose a solution.  If I can solve 
the problem.

>      
> Feb 25 23:25:23 pqutil:   906773 19990225232522.849   
FSL5 000  > 
NOAAPORT.GOES_EAST.IMAGE.TIGQ01.KNES.252315.19990562324
14504.* > 
So an entry like;

FSL5    ^NOAAPORT
FILE    data/noaaport/test.np


Doesn't produce any output?  I'm trying to create a 
generic entry to see if any data can be captured by 
pqact. An observation, the date looks future wrong, ie 
1999056232  If this is an date?

> The above two lines are actually on one line.  > 
> In the pqact file I've tried using the WMO header 
'TIGQ01' and also the > entire string from 
NOAAPORT...TIGQ01.  
> 
> Is the way the data being injected into the queue 
incorrect for naming > purposes?

That's a good question.  Does FLS have any sample pqact 
entries that they could share with you. It might help 
our guessing here. 
> 
> Should I have just the WMO header?  So the SSEC 
software does place the control 
> characters in the product prior to ingesting the 
product into the queue, > correct?
> 
Yep, They create a complete WMO type of product from 
the NOAAport stream to be handed off to the LDM.

Robb...
> Thanks,
> Gregg
> 
> 
> ______________________________ Reply Separator 
_________________________________
> Subject: Re: Re[2]: NOAAPORT and LDM 
> Author:  Robb Kambic <address@hidden> at 
NSSLGATE > Date:    2/25/1999 4:09 PM
> 
> 
> On Thu, 25 Feb 1999, Gregory Grosshans wrote: >      
> >      Robb,
> >      
> >      I've recently obtained code, after much paper 
work, from FSL that 
> >      injects the NOAAPORT data feed into the LDM 
queue as a specified feed > >      type (e.g. FSL5).  
However, pqact never sees or acts on any items that > > 
     i've configured in the pqact.conf file to work on 
the FSL5 data 
> >      stream.  
>      
> Greg,
>      
> Are you sure the data is getting into the LDM queue?  
What's the output > of:
>      
> % ldmadmin watch
>      
> If it isn't showing FSL5 product headers then no data 
is entering the 
> queue.The product headers will show the feed type and 
pattern.  Are there any > error log messages?  If no 
product headers then the process of entering
> the data to the queue is the culprit, at that point I 
would contact FSL. > I don't know anything about the 
FSL code. 
>      
>      
>      
> >      
> >      From what I've been told LDM waits until it 
sees a control-A and then > >      goes into a special 
state and processes data until it receives a 
> >      control-C.  From what I can tell the data 
coming off of NOAAPORT > >      doesn't contain 
control-As or Cs.  
>      
> Again this is FSL's implementation of reading 
NOAAport and making > products.
>      
> >      
> >      If NOAAPORT is missing these control 
characters, is that why pqact 
> >      isn't doing anything with the data in the LDM 
queue?  Did UNIDATA or > >      SSEC have to modify the 
SSEC/UNITDATA hardware/software to place 
> >      control characters at the beginning and end of 
each file for LDM and > >      the IDD to work 
correctly?
>      
> The SSEC processes the NOAAport products by eating 
the Np header and then > making a WMO header for the 
product with the control chars.
> >      
> >      As an example, I modified pqact.conf to send 
email of the ACUS1, > >      ACUS2, and ACUS3 bulletins 
when received from the FOS and 
> >      NOAAPORT/FSL5.  I only receive the bulletins 
from the FOS feed.  
> >      Similarly, I configured pqact.conf to decode 
metar obs from the FSL5 > >      feed but it doesn't 
work, yet I already decode metar obs from the FOS > >   
   and AFOS feeds.
> >      
> It appears that no NOAAport data is entering the LDM. 
>      
>      
> Robb...
>      
> >      Thanks,
> >      Gregg
> >      ---------------------------------------------------------------------- 
> >      Gregory Grosshans
> >      NWS/NCEP/Storm Prediction Center 
> >      Norman, OK        
> >      405-579-0720     fax : 405-579-0700 
> >      address@hidden
> >      address@hidden         
> >      /* standard disclaimer */
> >      ---------------------------------------------------------------------- 
> >       
> >      
> > 
>      
> ==============================================================================
= 
> Robb Kambic                                Unidata Program Center
> Software Engineer III                      Univ. Corp for Atmospheric Research
     
> address@hidden                   WWW: http://www.unidata.ucar.edu/ 
> ==============================================================================
=
>      
>      
> 
     
=============================================================================== 
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research 
address@hidden                   WWW: http://www.unidata.ucar.edu/ 
===============================================================================