[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #OIV-218948]: caching netcdf variables in thredds

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [THREDDS #OIV-218948]: caching netcdf variables in thredds
  • Date: Wed, 02 Aug 2017 00:34:00 -0600
Hi Ghansham,

In NetCDF-Java, data actually IS sometimes cached per-variable [1]. However, we 
have no way for end users (i.e. people running THREDDS) to configure that.

> Will the netcdf object based caching help or disk cache?

It ought to, yes. Are you experiencing poor performance?

> When we are using netcdf object based cache, do the variables once
> uncompessed will remain like that or will be uncompressed everytime on read?

They'll need to be decompressed each time. However, decompression time is 
likely to be negligible compared to the time it takes to read data from disk.

> And how to check is netcdf object based caching working or not?
> Are there any traces in thredds logs?

There are log messages about object caching, but you have to enable them, which 
is a little tricky. You'll need to edit:
<TOMCAT_HOME>/webapps/thredds/WEB-INF/classes/log4j2.xml

and add the following to <loggers>:

    <logger name="cacheLogger" level="debug" additivity="false">
      <appender-ref ref="threddsServlet"/>
    </logger>

That SHOULD cause cache messages to be written to threddsServlet.log. I haven't 
actually tested it though.

> I am using thredds 4.3.22

Any reason why? There may be performance improvements in later versions.
Furthermore, we generally lack the resources to support any version of THREDDS 
apart from the latest.

Cheers,
Christian

[1] 
https://github.com/Unidata/thredds/blob/5.0.0/cdm/src/main/java/ucar/nc2/Variable.java#L1523


Ticket Details
===================
Ticket ID: OIV-218948
Department: Support THREDDS
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.