[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[LDM #GLG-935498]: CONDUIT GFS data
- Subject: [LDM #GLG-935498]: CONDUIT GFS data
- Date: Fri, 03 Jun 2016 15:45:08 -0600
Hi Hsie
re: what is the size of your LDM queue
> rainbow:[42]% ldmadmin config
>
> hostname: rainbow.al.noaa.gov
> os: Linux
> release: 2.6.18-410.el5
> ldmhome: /home/ldm
> LDM version: 6.13.0
> PATH:
> /home/ldm/ldm-6.13.0/bin:/usr/kerberos/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/opt/pgi/linux86-64/8.0-3/bin:/opt/intel/fce/10.1.008/bin:/opt/sun/sunstudio12/bin:/usr/local/ncarg/bin:/home/ldm/bin:/home/ldm/decoders/bin:/home/ldm/util:/home/mcidas/linux/bin:/home/mcidas/util:.
> LDM conf file: /home/ldm/etc/ldmd.conf
> pqact(1) conf file: /home/ldm/etc/pqact.conf
> scour(1) conf file: /home/ldm/etc/scour.conf
> product queue: /home/ldm/queues/ldm.pq
> queue size: 4G bytes
> queue slots: default
> reconciliation mode: increase queue
> pqsurf(1) path: /home/ldm/queues/pqsurf.pq
> pqsurf(1) size: 2M
> IP address: 140.172.240.190
> port: 388
> PID file: /home/ldm/ldmd.pid
> Lock file: /home/ldm/.ldmadmin.lck
> maximum clients: 256
> maximum latency: 3600
> time offset: 3600
> log file: /home/ldm/logs/ldmd.log
> numlogs: 7
> log_rotate: 1
> netstat: netstat -A inet -t -n
> top: /usr/bin/top -b -n 1
> metrics file: /home/ldm/logs/metrics.txt
> metrics files: /home/ldm/logs/metrics.txt*
> num_metrics: 4
> check time: 1
> delete info files: 0
> ntpdate(1): /usr/sbin/ntpdate
> ntpdate(1) timeout: 20
> time servers: cirrus.al.noaa.gov tick.boulder.noaa.gov
> tock.boulder.noaa.gov ntp.ucsd.edu ntp1.cs.wisc.edu ntppub.tamu.edu
> otc1.psu.edu timeserver.unidata.ucar.edu
> time-offset limit: 10
OK, your LDM queue is 4 GB in size. At peak times, rainbow is receiving
over 38 GB of data from all feeds:
Data Volume Summary for rainbow.al.noaa.gov
Maximum hourly volume 38478.037 M bytes/hour
Average hourly volume 17131.927 M bytes/hour
Average products per hour 307307 prods/hour
Feed Average Maximum Products
(M byte/hour) (M byte/hour) number/hour
CONDUIT 7909.353 [ 46.167%] 23417.657 91054.277
NGRID 6002.365 [ 35.036%] 11090.312 37421.340
NEXRAD3 2475.231 [ 14.448%] 3269.320 113673.830
HDS 285.235 [ 1.665%] 589.473 19145.447
NIMAGE 217.073 [ 1.267%] 363.443 208.872
FNEXRAD 114.977 [ 0.671%] 139.929 105.915
IDS|DDPLUS 65.866 [ 0.384%] 80.986 45648.191
UNIWISC 61.828 [ 0.361%] 118.010 49.383
The peak time volume is heavily dominated by CONDUIT data. So, at peak
reception times, the products being received have to be processed out
of the LDM queue within about 6 minutes. Given this, I recommend that
you increase your LDM queue size to at last 8 GB or more if you have
enough RAM installed on rainbow. After you increase your queue size,
we need to see if you are correctly processing all of the CONDUIT data.
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: GLG-935498
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.