Rebalancing operation

Rebalancing operation

Rebalancing operation is performed to expand or reduce the size of a cluster, to react to fail-overs, or to temporary remove nodes for various upgrades.

During the rebalance operation:
  • Using the new Couchbase Server cluster structure, data is moved between the vBuckets on each node from the old structure. This process works by exchanging the data held in vBuckets on each node across the cluster. This has two effects:

    • Removes the data from the machines removed from the cluster. By totally removing the storage of data on these machines, it permits for each removed node to be taken out of the cluster without affecting the cluster operation.

    • Adds data and enables new nodes so that they can serve information to clients. By moving active data to the new nodes, these nodes become responsible for the moved vBuckets and for servicing client requests.

  • Rebalancing moves both the data stored in RAM, and the data stored on disk. This applies to each bucket and to each node within the cluster. The time taken for the move depends on the level of cluster activity and the amount of stored information.

  • During rebalancing the cluster remains up and continues to service and handle client requests. Updates and changes to the stored data during the migration process are tracked and will be updated and migrated with the data that existed when the rebalance was requested.

  • The current vBucket map, used to identify which nodes in the cluster are responsible for handling client requests, is updated incrementally as each vBucket is moved. The updated vBucket map is communicated to the Couchbase client libraries and to the enabled smart clients (such as Moxi), and allows clients to use the updated structure as the rebalance completes. This ensures that the new structure is used as soon as possible to help spread and even out the load during the rebalance operation.

Because the cluster stays up and active throughout the entire rebalancing process, clients can continue to store and retrieve information and do not need to be aware that a rebalance operation is taking place.

There are four primary reasons to perform a rebalance operation:

  • Adding nodes to expand the size of the cluster.

  • Removing nodes to reduce the size of the cluster.

  • Reacting to a failover situation, where you need to bring the cluster back to a healthy state.

  • Temporary removing one or more nodes to perform a software, operating system, or hardware upgrade.

Regardless of the immediate reason, the purpose of any rebalance is to bring up a cluster to a healthy state where the configured nodes, buckets, and replicas match the current state of the cluster.