[NOAAPORT #BGK-918302]: NOAAPort ingest not working

Hi,

re:
> Thanks Tom!!

No worries.

re: what did you do for "I added the routes"

> For the routes, I added them one by one as the following.
> ip route add 224.0.1.1/255.255.255.255 dev em4
> ip route add 224.0.1.2/255.255.255.255 dev em4
> ip route add 224.0.1.3/255.255.255.255 dev em4
> ip route add 224.0.1.4/255.255.255.255 dev em4

OK, the hard way ;-)

re: "I added the necessary ports"

> Both the firewall and iptables were running in the new servers. If I left the 
> firewall
> on I have to create the zone and add multicast rules as in the link you sent 
> on your
> previous email. I didn’t want to get into that complexity because I didn’t 
> see the use
> of the local firewall as we are not receiving other traffic on the em4 
> interface. 

OK.

re:
> The
> other interfaces are protected the global firewall.  Now that it is working I 
> could
> also sometime try to turn on the firewall and add the multicast rules to it 
> and see
> if it works.

This would be a good thing to test when you get time.

re:
> For the iptables, first I dumped the current iptables information using 
> iptables-save > iptables06222019
> 
> Then edited iptables06222019 file and added the following lines to match the 
> legacy
> server confiscations. The only changes I noticed due to OS change is in the 
> older OS
> it was done using -A RH-Firewall-1-INPUT not it is just -A IN_public_allow
> 
> -A IN_public_allow -d 224.0.0.251 -p udp -m udp --dport 5353 -j ACCEPT
> -A IN_public_allow -d 224.0.1.0/255.255.255.0 -i em4 -p udp -m state --state 
> NEW -m udp --dport 1201:1205 -j ACCEPT
> -A IN_public_allow -p igmp -m conntrack --ctstate NEW -j ACCEPT
> 
> Then I run the following command to restore the iptables
> 
> Iptables-restore < iptables06222019

Again, the hard way from my perspective.

re:
> And after noaaport started working, For the GOES-R, I added the 5, 8,9,& 10  
> as follows.
> 
> ip route add 224.0.1.5/255.255.255.255 dev em4
> ip route add 224.0.1.8/255.255.255.255 dev em4
> ip route add 224.0.1.9/255.255.255.255 dev em4
> ip route add 224.0.1.10/255.255.255.255 dev em4

Needing to do this is one of the reasons that I say that this is the hard way.

re:
> I edited the iptables file again include GOES-R rule. The reason I didn’t 
> combine it
> with the previouse line is because there is a jump on the port (1201-1205 
> then no
> 1206,1207 and we have 1208:1210)

OK.  My question remains: if the _only_ traffic on 'em4' is from the Novra 
S300N,
why go to so much bother to lock down access?

re:
> -A IN_public_allow -d 224.0.1.0/255.255.255.0 -p udp -m udp --dport 1208:1210 
> -m conntrack --ctstate NEW -i em4 -j ACCEPT
> 
> I also stopped the firewall (using systemctl stop firewalld) because there 
> wasn't one
> in the legacy.
> 
> The other thing I also did was set ip_forward to 1 using the following.
> echo 1 > /proc/sys/net/ipv4/ip_forward

Interesting.  Was this needed?  The reason I ask is we have IP forwarding
turned off in our /etc/sysctl.conf file.

re: Are you working with a university that is connected to the IDD to get feeds?

> As far as I know we are not working with a university. 

Then you will need to get the GOES-16/17 data from your NOAAPort reception
system and then stitch the tiles together to make full scenes.  We can not
feed a non-university site ** unless ** they have contracted with us to
provide the feed(s) that they want, and this is not cheap (since it costs
so much to setup via the contracting process).  If you are interested
in receiving not-for-free feeds from us, I can put you in touch with the
appropriate person to further discuss the matter.

re:
> If I have to get all GOES-R data
> via noaaport, what configuration changes do I need on LDM?

Nothing on the ingest side needs changing (assuming you have configured
your Novra S300N to process the needed PIDs in NOAAPort).  Typically,
sites feed the output from a NOAAPort ingest machine to one or more
downstream LDMs where data processing is done.  We do not recommend
trying to double duty an ingest machine with processing the data since
the output from the Novra S300N is a UDP stream, and UDP streams have
no retransmit capability like TCP does - if you miss processing packets,
you miss products.

re:
> Besides the noaaport ingest lines I added in ldmd.conf.

Again, we recommend that all processing be done on a machine which is
being fed from a NOAAPort ingest machine.

On the downstream machine, you would need to setup an LDM pattern-action
file action that stitches together the ABI L2 tiles being sent in
NOAAPort into full scenes.  If you are also interested in the other
L2 products in NOAAPort, you will need to process them appropiately
also (e.g., strip off NOAAPort broadcast headers and footers to leave
the underlying netCDF4 files).  One of the developers here created a
Python 3 script that handles the stitching together of tiles into
full scenes.  That code can be found in the ldm-alchemy package in
the Unidata section of GitHub:

https://github.com/Unidata/ldm-alchemy

re: NB: One GEMPAK user reported a problem with using the new images in GEMPAK

> I think I should invest some time to install and configure AWIPS in the 
> future.

OK, this is a big job.

re:
> However,
> my goal right now is to get the GOES-R data through noaaport using gempak 
> (because
> gempak is what we have right now).

Hopefully, there will be soon a new release of GEMPAK that fixes the problem 
that
one user has reported.  There is no estimate for when this may be at the moment.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: BGK-918302
Department: Support NOAAPORT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.