[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #GLG-935498]: CONDUIT GFS data

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #GLG-935498]: CONDUIT GFS data
  • Date: Sat, 04 Jun 2016 10:21:31 -0600
Hi Hsie,

I apologize for not being able to login to rainbow before this morning...

Comments:

- I changed the EXEC of 'xcd_run' in ~ldm/etc/ldmd.conf to use
  its fully qualified pathname

  The McIDAS-XCD decoders are now running.

- when I logged on, the first thing I did was check the LDM log
  file, ~ldm/logs/ldmd.log

  I saw LOTS of 'processing oldest product' messages.  These are
  indicating that rainbow is not able to process all of the data
  out of the LDM queue before the products get replaced by new
  ones being received.  I suspected that this might have been
  the case yesterday, but I was hoping that by increasing your
  LDM queue size, we could eliminate or, at least, minimize this
  "failure".  The numerous reports in the LDM log file indicate
  that a larger queue was/is not enough to prevent this, and this
  is undoubtedly the reason that not all of the CONDUIT products
  being received are being processed out of the queue.

  What to do?

  The simplest thing to do is split the data ingest and processing
  activities among two or more machines.  I know that this is not
  what you wanted to hear, but the alternative is to investigate
  in detail why rainbow is not able to keep up, and then take
  remedial action to try and correct the situation.  This could 
  doing things like upgrading the disk/raid controller, changing
  to use of the ZFS file system, etc.  The problem is that there
  is no guarantee that these actions will actually solve the
  problem you are experiencing.  Sharing the ingest and processing
  among two or more machines would likely solve the problem with
  the least amount of pain.  Decreasing what data is being REQUESTed
  and processed is another approach, but I figured you would not
  be interested in this route.

I'm sorry that I don't have better news...

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: GLG-935498
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.