Data Structures
Edit this article in GitHub
Version 2.4

Data Structures

You can use complex data structures such as dictionaries and lists in Couchbase. These data structures may be manipulated with basic operations without retrieving and storing the entire document.

Data structures in Couchbase are similar in concept to data structures in the Java Collections Framework:
  • Map is like Java Map<String, Object> and is a key-value structure, where a value is accessed by using a key string.
  • List is like a Java List<Object> and is a sequential data structure. Values can be placed in the beginning or end of a list, and can be accessed using numeric indexes.
  • Queue is a wrapper over a list which offers FIFO (first-in-first-out) semantics, allowing it to be used as a lightweight job queue.
  • Set is a wrapper over a list which ensures each value in the list is unique.

In fact, the Java SDK provides implementations of the Map, List, Set, and Queue interfaces which are described in the Collections Framework Integration section.

These data structures are stored as JSON documents in Couchbase, and can therefore be accessed using N1QL, Full Text Search, and normal key-value operations. Data structures can also be manipulated using the traditional sub-document and full-document KV APIs.

Using the data structures API may help your application in two ways:
  • Simplicity: Data structures provide high level operations by which you can deal with documents as if they were container data structures. Adding an item to a dictionary is expressed as mapAdd, rather than retrieving the entire document, modifying it locally, and then saving it back to the server.
  • Efficiency: Data structure operations do not transfer the entire document across the network. Only the relevant data is exchanged between client and server, allowing for less network overhead and lower latency.

Creating a Data Structure

When adding an element to a data structure, you can use a MutationOptionBuilder to request the document be created if it does not already exist. For example, to add an item to a map that may or may not already exist, specify the document ID of the map itself (i.e. the ID which uniquely identifies the map document in the server), the key within the map, the value to associate with the key, and finally the mutation options:
bucket.mapAdd("map_id", "name", "Mark Nunberg",
        MutationOptionBuilder.builder().createDocument(true));
Likewise, to create a list, specify the document ID and the value to add, along with the mutation options:
bucket.listAppend("list_id", "hello",
        MutationOptionBuilder.builder().createDocument(true));
Note that if the document already exists it will not be overwritten; the specified element will be added to the existing data structure document.
Data structures are stored on the Couchbase server as ordinary JSON documents. It is therefore possible to create and reset them using full-document methods like insert or upsert. To create a new empty list, set, or queue, use:
bucket.insert(JsonArrayDocument.create("list_id", JsonArray.empty()));
To create an empty map, use:
bucket.insert(JsonDocument.create("map_id", JsonObject.empty()));
Warning: The methods described below which modify the contents of a data structure have overloaded versions that accept a MutationOptionBuilder argument. If this argument is not provided, or if the option builder is not configured with createDocument(true), the default behavior is to throw DocumentDoesNotExistException instead of automatically creating the data structure.

Accessing Data Structures

The Bucket interface provides several methods for interacting with each kind of data structure. Most data access methods will return a generic type V which is provided as a generic parameter such as Class<V>.
bucket.listGet("list_id", 0, String.class); // "hello"
bucket.mapGet("map_id", "name", String.class);  // "mark nunberg"
The same subdocument exceptions are forwarded on the datastructures API, as occurs when the map key or list index is not found within the document. If the document itself does not exist, a com.couchbase.client.java.error.DocumentDoesNotExistException will be raised instead.
Here is a list of common operations:
Table 1. Data Structure Operations
   
mapAdd Add a key to the map.
bucket.mapAdd("map_id", "some_key", "value")
mapRemove Remove a key from a map.
bucket.mapRemove("map_id", "some_key")
mapGet Get an item from a map.
bucket.mapGet("map_id", "some_key", String.class) //=> value
If the key is not found, a PathNotFoundException is raised.
listAppend Add an item to the end of a list.
bucket.listAppend("list_id", 1234)
listPrepend Add an item to the beginning of a list.
bucket.listPrepend("list_id", "hello world")
listRemove Remove a value from a list.
bucket.listRemove("list_id", 2)
listSet Set an element at a specific index in the list.
bucket.listSet("list_id", 0, "first value")
listGet Get an item from a list by its index.
 bucket.listGet("list_id", 0, String.class)
If the index is out of range, a PathNotFoundException will be thrown. Note that you can get the last array element by specifying -1 as the index.
setAdd Add an item to a set, if the item does not yet exist in the set.
bucket.setAdd("set_id", "some_value")
Note that a set is just a list. You can retrieve the entire set by simply using a full-document get operation:
set = bucket.get("set_id").content()
Note: Currently, you can only store primitive values in sets, such as strings, ints, and booleans.
setContains Check if a value exists in the set.
bucket.setContains("set_id", "value")
setRemove Remove an item from a set, if it exists. An exception is not thrown if the item does not exist. You can determine if an item existed or not by the return value. If the item did not exist beforehand, null is returned.
bucket.setRemove("set_id", "some_value")
queuePush Add an item to the beginning of the queue.
bucket.queuePush("a_queue", "job123")
Note that a queue is just a list. You can retrieve items from the middle of the queue by using listGet
queuePop Remove an item from the end of the queue and return it.
item = bucket.queuePop("a_queue") //=> "job123"
If the queue is empty, then null is returned.
mapSize, listSize, setSize, queueSize These methods get the length of the data structure. For maps, this is the number of key-value pairs inside the map. For lists, queues, and sets, this is the number of elements in the structure.
len = bucket.listSize('a_list') //=> 42

Note that there are only two basic types: map and list. Types such as queue and set are merely derivatives of list.

Data Structures and Key-Value APIs

Data structures can be accessed using key-value APIs as well. In fact, the data structure API is actually a client-side wrapper around the key-value and sub-document API. Most of the data structure APIs wrap the sub-document API directly.
Note: Because the data structure API is just a wrapper around the various key-value APIs, you are free to switch between them in your code.

Collections Framework Integration

In addition to the Bucket level methods for working with data structures, the Java SDK provides implementations of the Map, List, Set, and Queue interfaces from the Java Collections Framework. Instead of maintaining in-memory storage, these implementations are backed by JSON documents stored in Couchbase Server. The implementations are thread-safe and suitable for concurrent use. The Map, List, and Queue implementations may contain values of the following types:
  • String
  • Integer
  • Long
  • Double
  • Boolean
  • BigInteger
  • BigDecimal
  • JsonObject
  • JsonArray
The Set implementation may contain values of all of the above types except JsonObject and JsonArray.

CouchbaseMap

The CouchbaseMap<V> class implements Map<String, V>. It allows null values, but does not allow null keys.

Example usage:
Map<String, String> favorites = new CouchbaseMap<String>("mapDocId", bucket);
favorites.put("color", "Blue");
favorites.put("flavor", "Chocolate");

System.out.println(favorites); //=> {flavor=Chocolate, color=Blue}

// What does the JSON document look like?
System.out.println(bucket.get("mapDocId").content());
        //=> {"flavor":"Chocolate","color":"Blue"}

CouchbaseArrayList

The CouchbaseArrayList<V> class implements List<V>. It allows null values.

Example usage:
List<String> names = new CouchbaseArrayList<String>("listDocId", bucket);
names.add("Alice");
names.add("Bob");
names.add("Alice");

System.out.println(names); //=> [Alice, Bob, Alice]

// What does the JSON document look like?
System.out.println(bucket.get(JsonArrayDocument.create("listDocId")).content());
        //=> ["Alice","Bob","Alice"]

CouchbaseArraySet

The CouchbaseArraySet<V> class implements Set<V>. It allows null values.

Example usage:
Set<String> uniqueNames = new CouchbaseArraySet<String>("setDocId", bucket);
uniqueNames.add("Alice");
uniqueNames.add("Bob");
uniqueNames.add("Alice");

System.out.println(uniqueNames); //=> [Alice, Bob]

// What does the JSON document look like?
System.out.println(bucket.get(JsonArrayDocument.create("setDocId")).content());
        //=> ["Alice","Bob"]

CouchbaseQueue

The CouchbaseQueue<V> class implements Queue<V>. It does not allow null values.

Example usage:
Queue<String> shoppingList = new CouchbaseQueue<String>("queueDocId", bucket);
shoppingList.add("loaf of bread");
shoppingList.add("container of milk");
shoppingList.add("stick of butter");

// What does the JSON document look like?
System.out.println(bucket.get(JsonArrayDocument.create("queueDocId")).content());
        //=> ["stick of butter","container of milk","loaf of bread"]

String item;
while ((item = shoppingList.poll()) != null) {
    System.out.println(item);
    // => loaf of bread
    // => container of milk
    // => stick of butter
}

// What does the JSON document look like after draining the queue?
System.out.println(bucket.get(JsonArrayDocument.create("queueDocId")).content());
        //=> []