[LDM #CTE-620214]: Satellite images slow or halt each day

Hi,

re:
> I'm monitoring our /data/ldm/gempak/images/sat/GOES16/ directory this morning.
> 
> I was deleting data older than 2 hours (via cron) in that directory and I 
> changed it
> this morning to 5 hours.  After I changed that, I saw the directory grow 
> steadily from
> 6.4GB to 11GB over about an hour from 9am-10:15am CDT and then the growth of 
> the folder
> ground to a near halt since ~10:15am only increasing by around 20KB per min 
> after that.

Which images are you writing to the /data/ldm/gempak/images/sat/GOES16/ 
directory,
the ones from the SATELLITE feed (aka DIFAX), or the ones from the NIMAGE feed?

NB: the most recent version of GEMPAK has support for the GOES-16/17 images
that we are distributing in the NIMAGE feed (these are ABI L2 images that
are created by stitching together the tiles that are sent in NOAAPort).
GEMPAK can _not_ handle the images that are in the SATELLITE feed.

re:
> Coinciding with that, our satellite images have not updated since.

Again, which satellite images.  You are REQUESTing both the GRB images that
are sent in the SATELLITE feed and the NOAAPort images that are being sent
in the NIMAGE feed.

re:
> Sure looks like we aren't getting the data we would normally get, but our 
> radar data
> has continued to be updated normally during this time.  Maybe this helps?  
> Very
> confusing.

Turning on reporting of real-time stats yesterday has really helped get
an insight into the data deliver to your machine, mammatus.ttu.edu.
In a nutshell, the latencies in the feeds being REQUESTed from 
idd.unidata.ucar.edu
are _terrible_, while the one feed (NGRID) being REQUESTed from 
iddb.unidata.ucar.edu
are good.

Given the results of the NGRID test yesterday, we are advising you to do the
following:

Change your feed REQUESTs from what you have now to the following:

# LIGHTNING from UAlbany
REQUEST LIGHTNING ".*" striker.atmos.albany.edu

# WMO, NEXRAD3 and UNIWISC from idd.unidata.ucar.edu
REQUEST WMO ".*" idd.unidata.ucar.edu
REQUEST NEXRAD3 ".*" idd.unidata.ucar.edu
REQUEST UNIWISC ".*" idd.unidata.ucar.edu

# Various feeds from iddb.unidata.ucar.edu
REQUEST CONDUIT "[05]$" iddb.unidata.ucar.edu
REQUEST CONDUIT "[16]$" iddb.unidata.ucar.edu
REQUEST CONDUIT "[27]$" iddb.unidata.ucar.edu
REQUEST CONDUIT "[38]$" iddb.unidata.ucar.edu
REQUEST CONDUIT "[49]$" iddb.unidata.ucar.edu

REQUEST NIMAGE "GOES16" iddb.unidata.ucar.edu
REQUEST NIMAGE "GOES17" iddb.unidata.ucar.edu

REQUEST DIFAX "GRB-R" iddb.unidata.ucar.edu
REQUEST DIFAX "WEST" iddb.unidata.ucar.edu

REQUEST NEXRAD2 "[02468]/[IES]" iddb.unidata.ucar.edu
REQUEST NEXRAD2 "[13579]/[IES]" iddb.unidata.ucar.edu

REQUEST NGRID ".*" iddb.unidata.ucar.edu
REQUEST NOTHER ".*" iddb.unidata.ucar.edu

However, before you settle on how to change your IDD feed REQUESTs,
you need to decide exactly which feeds you really need to
REQUEST.  Here are some things to think about:

1) the SATELLITE (aka DIFAX) feed contains all of the ABI L1b imagery,
   products from other instruments, and GLM L2 products from the
   GRB

   NIMAGE contains all of the ABI L2 imagery and all other L2 products
   that are being sent in NOAAPort, and it contains the value added
   GLM L2 products that Eric Bruning is creating there at TTU.

   GEMPAK can _only_ use the ABI L2 imagery that we are sending in
   the NIMAGE feed.  It can NOT use any of the images/products
   that are being distributed in the SATELLITE feed.

2) the volume of data in the SATELLITE feed averages around 13.5 GB/hr
   with maximums of around 19.5 GB/hr

   The NIMAGE feed has volumes significantly less than SATELLITE, but
   they are still substantial:

   Average volume around 6 GB/hr with maximums of just over 9 GB/hr.

3) the NOTHER feed contains the ABI L2 tiles and some of the L2
   products from NOAAPort

   We stitch together the tiles from NOTHER into full scenes (images),
   add all of the L2 products from NOTHER and HDS and send then im
   the NOTHER feed.  So, REQUESTing NOTHER is for the most part
   redundant.  Plus, do you even have any pattern-action file actions
   that do anything with data from the NOTHER feed?

4) the volume of data delivered in the CONDUIT feed ranges from
   around 12 GB/hr to maximums of just over 42 GB/hr

   If you are not actively using the model output in CONDUIT,
   you could stop REQUESTing it altogether.  If you are using
   the model output in CONDUIT, you will want to REQUEST the full
   volume using the split feed REQUESTs that I included above.

5) Do you have enough disk space to hold all of the data you want?

   I only ask because of your comment about scouring data older than
   2 hours.  If you are really tight on disk space, your system may
   get swamped when all of the data you are REQUESTing is received
   and processed.


Recommendation:

If you are simply getting GOES-16/17 imagery for use in GEMPAK,
our recommendation is:

- stop REQUESTing the SATELLITE (aka DIFAX) feed altogether
- stop REQUESTing the NOTHER feed altogether

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: CTE-620214
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.