[NOAAPORT #ZNX-891732]: 20181004: hardware recommendation for ingesting NOAAPort data?

Hi Amenda,

I drove your message into our inquiry tracking system so that others
could chime in.  I hope you don't mind.

re:
> Do you have hardware recommendations for running the Unidata
> LDM/noaaportIngester to acquire all data on NOAAPort?
> 
> We're getting ready to replace some old hardware that is dropping packets
> and want to make sure the replacement doesn't!

Our only guidelines are that the machine running the NOAAPort ingest
have:

- at least 2 Ethernet adapters

  One is for the direct connection of the UDP output from the DVBS2
  receiver being used (e.g., Novra S300N).

  The other is to be used for connecting the machine to the net.

- sufficient RAM

  We like to make our LDM queues large enough to hold at least 1 hour
  of data received, and more is better.  Having a large store of received
  products will help the LDM detect AND reject duplicate products.  It
  is our observation that there are a LOT of duplicated products sent in
  the NOAAPort SBN, and we prefer to send as few of the duplicated products
  to those receiving LDM/IDD feeds populated by NOAAPort products as
  possible.

- be fast enough to keep up with the processing of the NOAAPort stream
  of products

  This is the least crucial part of our "requirements" since the burden
  of ingesting, LDM/IDD product creation and IDD relay is quite modest.
  In fact, for years we would use old equipment that was headed for
  the scrap heap so that we would be assured that our ingest code would
  run at any/all end-user sites.

Perhaps giving you an overview of the two Linux based NOAAPort ingest
machines we run here would be helpful:

chico.unidata.ucar.edu:

CenOS 6.10
8 processors    - Intel(R) Xeon(R) CPU E5430  @ 2.66GHz\
32 GB RAM
200 GB disk     - 7200 RPM HDD
4 Gbps Ethernet

leno.unidata.ucar.edu - Intel NUC
CenOS 6.10
4 processors    - Intel(R) Core(TM) i5-4250U CPU @ 1.30GHz
16 GB RAM
450 GB disk     - SSD + HDD
2 Gbps Ethernet

Some questions for you:

- I'm curious about your comment that your machine is dropping packets

  Are you sure that this the machine itself, or could missed/dropped
  packets be a result of how the NOAAPort stream is being sent to
  your machine?

  Are you seeing dropped packets in /sbin/ifconfig listings?

- what are the specs for the machine(s) you are currently using?

- what sort of Carrier to Noise (C/N also referred to as EsNo) values
  are you seeing on your Novra S300Ns?

  A couple of years ago when I was trying to tweak our NOAAPort ingestion,
  I had the opportunity to visit the Skaggs building and become informed about
  the NOAAPort ingest setup there. It was my observation that our ingest
  quality (as measured by (C/N)) was much better than yours, and we had
  correspondingly less numbers of Gaps (missing packets in a sequence).  At
  that time, our C/N values were typically in the mid to high 15s with 
occasional
  values as high as the low to mid 16s.  After the switch to the new satellite,
  our C/N improved to the very high 15s to as high as 18 - 18.3.  With the
  higher C/Ns, we saw a decrease in Gaps and corresponding improvement in
  ingest data quality.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: ZNX-891732
Department: Support NOAAPORT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.