This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
=============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research address@hidden WWW: http://www.unidata.ucar.edu/ =============================================================================== ---------- Forwarded message ---------- Date: Sun, 24 Sep 2000 20:53:37 -0600 From: Russ Rew <address@hidden> To: Tom Yoksas <address@hidden> Subject: Re: 20000924: statistics for data being handled by motherlode IDD? Tom, > Has there been any workup of the amount of data that motherlode is > handling currently? Also, I am under the impression that motherlode is I have some statistics on data from motherlode from running "pqutil -w" last week on motherlode after Chiz included a few more feeds that shemp had been getting. The resulting 124 Mbyte log covers 15 hours of time, product size, feed type, and product ID. It's on buddy in /buddy/russ/mother-prods.log. An earlier 12-hour notifyme log is also available as motherlode.log, but we decided that notifyme is not as accurate for product arrival times as pqutil. Here's some of what I got out of the log, covering 15 feeds going into motherlodes product queue, including some that are not redistributed (NLDN, SPARE, ACARS, ...): - For 15 hours, there were 14.47 Gbytes of data in 1138993 products. The min and max product sizes were 34 bytes (not counting a 12-byte spurious "product") and 14931306 bytes. The average product size was 12708.5 bytes, but the product-size distribution is lumpy. - Extrapolating: 23.16 Gbytes/day in 1822390 products/day Averaging: 965 Mbytes/hour in 75933 products/hour, or 268054 bytes/sec in 21 products/second - Maximum products per second: 164. The products/second distribution is pretty smooth, with a peak at about 18 products per second. There were only 449 seconds in 15 hours when no products were received. The longest interval during which no products were received was 6 seconds. These might be affected by occasional garbage-collection pauses in the LDM, so may not reflect what's happening on the data feeds. - Number of products per feedtype (extrapolated from 15 hours to 24 hours): 662890 WSI 324736 NTEXT 247482 NMC2 210518 IDS|DDPLUS 172115 NNEXRAD 125970 HDS 70346 NGRID 3371 NIMAGE 2557 NOGAPS 790 SPARE 640 NGRAPH 326 MCIDAS 264 FSL2 240 NLDN 144 PCWS - Number of Mbytes per feedtype (extrapolated from 15 hours to 24 hours): 7336 WSI 6443 NMC2 3231 NNEXRAD 2977 NIMAGE 1319 HDS 938 NGRID 262 NOGAPS 210 NTEXT 167 IDS|DDPLUS 95 MCIDAS 67 SPARE 65 PCWS 42 FSL2 4 NGRAPH 3 NLDN --Russ