[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

data file compression / cache / bz2?

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: data file compression / cache / bz2?
  • Date: Fri, 11 Aug 2006 12:05:58 -0800 (AKDT)
Hi,

Is there a way to have THREDDS systematically clean up uncompressed
data files?   We noticed that THREDDS can support compressed (gzip)
NetCDF files, but the uncompressed files linger.

A periodic program can be run to scan through and remove them, but
this would assume that the periodic cleaning would keep up with the
rate of uncompression as data is accessed.

Maybe something similar to OPeNDAP Server 3 (dods_cache)?  Even this
has its limits.  If the cache size is not high enough, with enough
traffic, requests will get stomped.   Or just a table in memory
tracking uncompressed files.  If the server/system ever crashed,
then this would leave stuff about.

We were wondering if the same compression support can be extended to
include bzip2 compression (bz2)?

Thanks!
Rob
-- 
Alaska Ocean Observing System
Database Manager
907-474-7948

===============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
===============================================================================