[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[LDM #JOQ-172871]: LDM queue size
- Subject: [LDM #JOQ-172871]: LDM queue size
- Date: Fri, 31 May 2019 14:40:39 -0600
Hi Carl,
> I am with the University of Alaska Fairbanks/GINA (Geographic Information
> Network of Alaska) and we are using LDM to provide direct broadcast
> satellite products to National Weather Service offices in here Alaska.
> Everything has been working fine until recently they developed a slow
> network issue that is not resulting in considerable data loss between our
> sites. Although the root problem is the network slowdown, analysis of our
> LDM setup suggests that our queue is should be larger. It is currently set
> to 10 GB. Below is a sample of output from the "pqmon -i 5" command. From
> the output the age of the oldest product in the queue is only around 1100
> seconds which is another indication that our queue is too small.
The LDM works best if the minimum residence time of the product queue is
greater than the maximum latency parameter in the LDM registry (which is 3600
seconds by default); otherwise, the product-queue can't guarantee duplicate
product rejection.
The best way to discover the minimum residence time of the product queue is to
plot its time-series using the commands "ldmadmin addmetrics" and "ldmadmin
plotmetrics". See
<https://www.unidata.ucar.edu/software/ldm/ldm-current/basics/configuring.html#cron>
and
<https://www.unidata.ucar.edu/software/ldm/ldm-current/basics/monitoring.html#metrics>
for more information.
> In your support documents, I've read that the LDM works best when the queue
> can be kept in physical memory and we should try not to exceed 80% of
> that.
Correct -- assuming the computer is only used to relay data and not process any.
> So if we double our queue size to 20 GB, we would need at least that
> much RAM on that server as well. Does this all seem correct?
If you increase the queue to 20 GB, then you should have at least 20 GB/.8 = 25
GB of memory.
> $pqmon -i 5
> May 31 19:44:27 pqmon NOTE: Starting Up (20806)
> May 31 19:44:27 pqmon NOTE: nprods nfree nempty nbytes maxprods
> maxfree minempty maxext age
> May 31 19:44:27 pqmon NOTE: 16317 7 179754 8633409296 196077 313
> 0 409778760 1107
> May 31 19:44:32 pqmon NOTE: 16390 7 179681 8633475968 196077 313
> 0 409778760 1112
> May 31 19:44:37 pqmon NOTE: 16423 7 179648 8633489672 196077 313
> 0 409778760 1117
> May 31 19:44:42 pqmon NOTE: 16432 7 179639 8633571944 196077 313
> 0 409778760 1122
> May 31 19:44:47 pqmon NOTE: 16523 7 179548 8633602632 196077 313
> 0 409778760 1127
> May 31 19:44:55 pqmon NOTE: 16528 7 179543 8712167320 196077 313
> 0 409778760 1134
> May 31 19:45:04 pqmon NOTE: 16625 7 179446 8731669232 196077 313
> 0 409778760 1143
> May 31 19:45:09 pqmon NOTE: 16685 6 179387 8763574352 196077 313
> 0 409778760 1148
> May 31 19:45:14 pqmon NOTE: 16708 5 179365 8779823088 196077 313
> 0 409778760 1153
> May 31 19:45:19 pqmon NOTE: 16726 5 179347 8785962104 196077 313
> 0 409778760 1158
Regards,
Steve Emmerson
Ticket Details
===================
Ticket ID: JOQ-172871
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.