Managed Caching Layer Architecture

Managed Caching Layer Architecture

Since Couchbase built Couchbase Server on a memory-first architecture, achieving high performance and scalability requires effective memory management.

Caching Layer

Each service in Couchbase Server tunes its caching based on its needs.
  • Data service tunes its managed cache to enable fast key based read and write operations with low latency under high concurrency.
  • Index and Search services manage the cache to ensure index maintenance and fast scans for the most popular indexes in the system.
  • Query service manages memory to calculate query responses by processing streams effectively for the execution plans generated by the optimizer.

The Couchbase SDK never access the persistence layer directly, but communicate through the caching layer. Couchbase Server moves the data to and from the disk internally as needed, thereby acting as both a read-through and a write-through cache. This facilitates extremely high read-write rates and eliminates the need for an external caching tier. Unlike many other database systems, Couchbase Server does not depend on external caching systems. This simplifies development since developers do not need to deal with complex cache coherency issues or varied performance capabilities across technologies.

In order to provide the best possible performance for the most-frequently accessed items, Couchbase Server manages a subset of the users' entire dataset in the caching layer. The Server ensures performance remains acceptable by coordinating between the caching layer and persistent storage. For example, when an item enters the caching layer Couchbase Server also inserts the item into a disk-writing queue. As a result, if the software's algorithm later determines the item is accessed too infrequently to justify its continued use of cache resources, the server can safely remove the item from RAM to free up space for other incoming operations. Similarly, if a user requests an infrequently accessed item, the server retrieves the item from disk and stores it in the caching layer in case the user needs ongoing access to the item.

By default, the Couchbase Server automatically keeps frequently used data in memory and less-frequently used data on disk. Couchbase Server moves data from the managed cache to disk asynchronously, in the background, to ensure there is enough memory that can be freed up for incoming operations. The server constantly monitors the information accessed by clients and decides how to keep the active data within the caching layer. Items may be ejected from memory when additional memory is needed to perform incoming operations. These items have already been persisted to disk and require no additional I/O. The managed cache ensures that reads and writes are handled at a very fast rate, while removing the typical load and performance spikes that would otherwise cause a traditional RDBMS to produce erratic performance.

RAM quotas

RAM quota allocation is governed through individual services. Each service in Couchbase Server tunes its caching based on its needs.
  • The Data service uses a managed cache based on memcached that is tuned to enable fast key based read and write operations with low latency under high concurrency.
  • The Index and Search services manage cache to ensure index maintenance and scans can be serviced fast for the most popular indexes in the system.
  • Query service manages its memory to calculate query responses by processing streams effectively for the execution plans generated by the optimizer and caches certain parts of those query plans.

Allocation of memory to services is governed through RAM quota allocations. Data, Index and Search services both configure RAM quotas per node in the cluster. Query service automatically manages its memory without a defined quota.

Each node in the cluster running the relevant services inherits the value and may allocate up to the specified amount.
  • Index RAM Quota governs the index service RAM quota allocation per node. Each node running the index service inherits the value of Index RAM Quota for caching Global Secondary Indexes (GSI).
  • Search RAM Quota governs the search service RAM quota allocation per node. Each node running the search service inherits the value of search RAM Quota for caching Full Text Indexes.
  • Data RAM Quota governs the data service RAM quota allocation per node. Each node running the data service inherits the value set for Data RAM Quota for caching bucket data.

Bucket RAM Quotas are allocated out of the Cluster Data RAM quota. As an administrator, you can control the total RAM quota allocated to each bucket through Bucket RAM Quota under bucket settings. The total RAM configured across all buckets cannot exceed the total Data RAM Quota allocated for the data service at the cluster level.