Data Management

Data Management

Couchbase Server provides data management services through the data manager.

Atomicity Properties

Couchbase server provides atomicity at the document level. A document write will either clearly succeed or clearly fail. It is impossible to end up with a document write that partially succeeds, for example, recording only some changed fields but not others. Couchbase is strongly consistent at the document level. Cross document transactions typically can be avoided by consolidating often-accessed information into a single document, or addressed using other techniques.

Strong Consistency and durability

When talking about the CAP Theorem, distributed data management systems are generally considered to be either CP or AP, that is, evaluators focus on a tradeoff in either consistency (C) or availability (A). It's important to understand that the CAP Theorem is only relevant in so far as it describes how a system behaves during a network partition and how it recovers afterwards; when a system is working normally, no sacrifices need to be made.

Couchbase Server acts like a CP system in its default configuration and running as a single cluster. This is because any access to a given key (read, write, update, delete) is always directed to the node that hosts that active data at that point in time. The application client library transparently distributes requests to the appropriate nodes and therefore every application server/thread will immediately read the writes of any other application server/thread. Any write is also replicated within the cluster, but these replicas are primarily for the purpose of high availability and by default do not service any traffic until made active.

According to the CAP Theorem, a network partition cannot be distinguished from a failure of a part of the system. With Couchbase, in the event of a failure of one node, some data will be temporarily unavailable for writes until it is made active elsewhere in the cluster, either manually or automatically. Reads, however, can continue to be served by the replica copies of data elsewhere in the cluster.

If a single node fails, the data on a node that failed will not accept writes until the node is failed over, although reads can be serviced from replicas if desired.

Couchbase Server enables users to increase availability by replicating data between multiple Couchbase Server clusters running in the same or separate data centers using a capability called cross datacenter replication (XDCR, described later). With XDCR the state of information on both clusters will eventually be made consistent, and in the meantime Couchbase remains available to take read and write traffic.

Couchbase Lite further increases availability in mobile and IOT scenarios. It does this by preserving local changes on device as a user interacts with the application as normal and then managing sync to Couchbase Server when network connectivity is restored.

Although the CAP Theorem is a useful high-level formulation of principles, a full discussion of how Couchbase Server behaves and recovers from various distributed failure conditions is beyond the scope of this paper, especially given the various configuration and programming options available. For a more thorough discussion of the CAP Theorem as it applies to Couchbase Server, see http://blog.couchbase.com/cap-theorem-and-couchbase-server-time-xdcr.

Consistency of Indexes and Replicas

By default, indexes and replicas are updated asynchronously for best performance. On a per-write or per-query basis, an application can choose to relax some of the performance to favor stronger reliability (for replicas) or consistency (for queries). Traditional RDBMs and NoSQL databases slow down writes in order to keep indexes and replicas up-to-date. Couchbase by design allows the application developer to make informed tradeoffs depending on what is most important for that part of the application.

Couchbase indexes are built incrementally, so after the initial build Couchbase Server can immediately respond to a request, or it can be made to catch up to the point in time when the query was issued. A developer can choose how fresh the index data is at the time of query by setting the "stale" property. This offers a high degree of configurability, from requiring that Couchbase Server process all updates before issuing a response to allowing Couchbase Server to immediately respond to a request using the current state of the index at the time of the query, or somewhere in between. The range of options allows applications to be as fast as possible unless there is a hard requirement to have the freshest information in the index.

Tunable Durability Requirements

By default, writes are asynchronous and the data manager sends an acknowledge message (ACK) to a client as soon as an update is in RAM. Once a write is in memory, the data manager immediately adds it to replication, disk, and indexing queues. Replication is RAM-based, so it is extremely fast. To increase tolerance to failures at the cost of increased latency, Couchbase can acknowledge the change to the application only after the update is also replicated, persisted to disk or both. Data is replicated to 1, 2, or 3 nodes and saved to disk for a total of up to four copies, regardless of whether the ACK message is held up for those operations.

Concurrency

In a busy distributed data management system, conflicts might arise with multiple clients attempting to write the same document simultaneously. All Couchbase Server operations are atomic at the document level. Couchbase Server offers both optimistic and pessimistic locking to prevent concurrency errors.

Optimistic locking is based on the Compare-and-Swap (CAS) Value, a unique and atomically incrementing identifier that is part of every document’s metadata. Several update operations require a successful CAS value check in order to succeed. The application passes the CAS value as a parameter. Couchbase Server then verifies the CAS Value has not changed before a document is deleted or modified to effectively prevent conflicts without having to lock records. This is the preferred, best practice, method of handling concurrency with Couchbase.

Couchbase Server also supports pessimistic locking (similar to semaphores in an application), which is less commonly used. Pessimistic locks are automatically released to prevent unrecoverable deadlocks.

Document Expiration

Programmers can use Time to Live (TTL) to set an expiration time for a document. This is most commonly used with ephemeral data such as stored user sessions. When this optional value is set, the Couchbase Server will delete values during regular maintenance if the TTL for an item has expired. Documents can also be ‘touched’, updating their expiration time without modifying their contents. By default documents do not have TTLs and do not expire. TTL is one mechanism that enables Couchbase Server to manage its own capacity.