[AWIPS #XCU-969788]: EDEX grib processing optimization question

I'd recommend doing a stop and start of edex_camel:

service edex_camel stop
service edex_camel start

address@hidden> wrote:

> New Client Reply: EDEX grib processing optimization question
>
> Thank you for this tidbit.  Do we just restart edex camel ingest grib or
> edex entirely to invoke the changes?
>
> Pete
>
> address@hidden> wrote:
> >
> > ******
> >
> > This message is from an external source.  Use caution when opening
> links/attachments.  Use the 'Report Message' tool in Outlook if suspicious.
> >
> > *******
> >
> > Hi Pete,
> >
> > Yes, this sounds normal. How many cpu's do you have on the system that
> you are running EDEX on?
> >
> > The script used to help calculate these numbers is in
> /awips2/edex/etc/ingestGrib.sh but it's dependent on the number of cpu's.
> >
> > The line you would want to look at is the "GRIB_MAX_GRID_POINTS" and
> compare how many grid points are set there and what your logs are
> complaining about. But with all of those models you still may continue to
> see a backup/latency of ingest around the major model times (0, 6, 12, 18Z)
> >
> > You could also try increasing the GRIB_DECODE_THREADS.
> >
> >
> >
> >> Greetings.
> >>
> >> We are running RHEL-7 with the latest version of AWIPS2 and our system
> has 256 GB mem.
> >>
> >> As grib files arrive we submit them to EDEX via the
> /awips2/ldm/dev/notifyAWIPS2-unidata.py script.
> >>
> >> The suite of models we pull in are many including global GFS, Canadian,
> UK, EC, ensemble means of EC, NAEFS, convective allowing models (conusarw,
> conusmem2, wrfnssl, hwfv3, hrrr, nam3km) and two regional models (NAM and
> Canadian).
> >>
> >> We've noticed more than occasionally during high volume times (during
> on-hour cycles like 12z or 00z data arrives vs off-hour 06 and 18z cycles)
> messages in edex-ingestGrib-YYYYMMDD.log such as "Max Grids in memory for
> Gridpersister exceeded. Waiting for grids to process".    And a significant
> delay of an hour can develop as gribs arrive and get processed (and they
> all do eventually get processed).
> >>
> >> The "Grids in memory" message makes us think that there may be an
> adjustable default setting that can be increased - is there such a setting
> you could point us to adjust?
> >>
> >
> >
> > Thanks,
> >
> > Tiffany Meyer
> > AWIPS Lead Software Engineer
> > UCAR-Unidata
> >
> > If you're interested, please feel free to fill out a survey about the
> support you receive:
> https://docs.google.com/forms/d/e/1FAIpQLSeDIkdk8qUMgq8ZdM4jhP-ubJPUOr-mJMQgxInwoAWoV5QcOw/viewform
> >
> > Ticket Details
> > ===================
> > Ticket ID: XCU-969788
> > Department: Support AWIPS
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the
> Unidata inquiry tracking system and then made publicly available through
> the web.  If you do not want to have your interactions made available in
> this way, you must let us know in each email you send to us.
> >
> >
>
> ------------------------------------------------------------------------------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are hereby
> notified that you have received this document in error and that any review,
> dissemination, distribution, or copying of this message is strictly
> prohibited. If you have received this communication in error, please notify
> us immediately, and delete the original message.
>
>
> Ticket Details
> ===================
> Ticket ID: XCU-969788
> Department: Support AWIPS
> Priority: Normal
> Status: Open
> Link:
> https://andy.unidata.ucar.edu/staff/index.php?_m=tickets&_a=viewticket&ticketid=33640
>
>

-- 

Tiffany Meyer
AWIPS Lead Software Engineer IV
UCAR - Unidata



Ticket Details
===================
Ticket ID: XCU-969788
Department: Support AWIPS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.