Java SDK tutorial

Java SDK tutorial

The Java SDK tutorial bridges the gap between simple and advanced concepts by walking through a complete web application.

The full source code for the tutorial is available on GitHub couchbaselabs/try-cb-Java. The primary focus of the tutorial is to explain the function and theory behind the Couchbase Java client and how it works together with Couchbase Server, and especially new features in version 4.0 like N1QL. It makes use of the travel-sample data set. The code that generates the web application is provided with the source code but is not discussed in this tutorial.

Note: For reference, you can find the previous tutorial that focuses more on key/value operations and views (since N1QL was introduced in CB 4.0): Java SDK tutorial for CB 3.x.

Prerequisites and set up

You'll need:
  • Your favorite IDE with a JDK 1.6+ installed (this tutorial assumes IntelliJ with JDK 1.8)
  • Maven 3
  • The sample app source code from GitHub (we'll only cover the Java part of the app)
  • A local Couchbase 4.0 installation (make sure that the travel-sample bucket has been loaded and that there is, at least, one node with data, query, and index services in the cluster
  • That's it!
Tip: Installing Couchbase Server

Download Couchbase Server and install it. As you follow the download instructions and setup wizard, make sure you keep all the services (data, query, and index) selected. Make sure also to install the sample bucket named travel-sample (introduced in CB 4.0) because it contains the data used in this tutorial.

If you already have Couchbase Server installed but did not install the travel-sample bucket, open the Couchbase Web Console and select Settings > Sample Buckets. Select the travel-sample checkbox, and then click Create. A notification box in the upper-right corner disappears when the bucket is ready to use.

To get set up for the tutorial proper, follow these steps:
  • git clone https://github.com/couchbaselabs/try-cb-java.git or download the source
  • If you don't want to connect to localhost, change the configuration in src/main/resources/application.properties (e.g., hostname).
  • Open the project in your IDE, import as needed and let it build
  • Alternatively, go straight to running the project by issuing the following in command line inside the project's directory: mvn spring-boot:run
Note: If you want to code it yourself, the real work is done in the following classes:
  • trycb.config.Database
  • trycb.service.Airport
  • trycb.service.FlightPath
  • trycb.service.User

There's currently no "fill-in-the-blanks" branch so you'll have to delete method bodies and try to take it from there.

Tip: This tutorial focuses on querying through N1QL rather than views for now. If you want information about using views, see the following resources:

Configure and Bootstrap the SDK

Where: trycb.config.Database

Goals: Connecting to the Cluster and getting a reference to a Bucket, learn to reuse it.

Relevant Documentation Topics: Managing connections

The first step is to let the application connect to your cluster and obtain a reference to a Bucket (the Bucket is your entry point for the whole storage API). Spring Boot will inject values from your configuration file in the hostname, bucket and password attributes.

Note that both Cluster and Bucket are thread-safe and must be reused across your application (if you don't, you'll see a warning in the logs). Here we'll let Spring inject those as @Bean, so they'll be singletons in the application.

Connecting to the Cluster
public @Bean Cluster cluster() {
	return CouchbaseCluster.create(hostname);
} 

The cluster() method creates the bean for the cluster reference. Here we use one of the simpler factory methods, with just a hostname. Without a hostname, the default is to connect to localhost. Note that you can tune the SDK through a CouchbaseEnvironment instance that you could pass in as an additional first argument (that is particularly recommended if you need to connect to multiple clusters in the same application).

Tip: You could make the bootstrap process safer by providing a list of hostnames/IPs from which to tbootstrap in case the one node you provided for bootstrap is unfortunately down when you are creating the Cluster reference. In production, the best practice is to provide 3.
Getting a Bucket
public @Bean Bucket bucket() {
	return cluster().openBucket(bucket, password);
}

The second step is to connect to the Couchbase bucket you'll be using. Here we want to use sample data from travel-sample (and the application.properties configuration reflects that). To obtain the corresponding Bucket from the Cluster and register it as a @Bean in bucket() method, simply open it using the configured bucket name and password.

Both bucket and cluster can also be managed through the SDK (e.g., add views or create new buckets). See Managing clusters and Managing views for more information.

With these steps, the application is ready to use the API. In the next step, we'll cover the Key/Value (kv) store part of the API.

Manage Users using Key/Value API

Where: trycb.service.User

Goals: Use Bucket's Key/Value operations and discover the Document API.

Relevant Documentation Topics: Document basics, Creating documents, Retrieving documents, Updating documents, ... Mastering Observables

Couchbase is a document-oriented database that provides access to your data both through its document ID (for high-performance access), as well as through views and N1QL (as powerful query languages).

This is noticeable in the API, where the methods reflect Key/Value operations ( get, create, and so on) and work with a Document interface that has an id() and content. The default Document implementation, JsonDocument, accepts a simple representation of JSON as a content: the JsonObject.
Tip: If you already have mechanisms in place that deal with marshaling/unmarshaling of your domain objects to/from JSON, skip the extra step of converting them to JsonObject and use a RawJsonDocument instead.
Creating new users

Since this is a @Service, the createLogin method is part of the REST API and returns a ResponseEntity<String> (a Spring representation of a http response with code and all around the JSON string). Spring injects the Bucket reference for us, along with the request parameters username and password:

/**
 * Create a user.
 */
public static ResponseEntity<String> createLogin(final Bucket bucket, final String username, final String password) {

Next we'll prepare the content for our new user (as a JsonObject) and the associated document (in order to give it an ID):

JsonObject data = JsonObject.create()
    .put("type", "user")
    .put("name", username)
    .put("password", BCrypt.hashpw(password, BCrypt.gensalt()));
JsonDocument doc = JsonDocument.create("user::" + username, data);
Tip: The "user::" prefix is arbitrary to this application, this is just a convention that the app uses to obtain unique keys and have additional information in it, but the key could have been anything else (even sequence numbers or UUIDs) really.

Here comes the part where we use the Couchbase API to store the document, it's rather simple:

bucket.insert(doc);

We want to send a response with the content and a success flag to the HTTP client. We also want to indicate failure if the SDK throws an exception, so let's wrap that in a try-catch block:

try {
	bucket.insert(doc);
	JsonObject responseData = JsonObject.create()
        .put("success", true)
        .put("data", data);
    return new ResponseEntity<String>(responseData.toString(), HttpStatus.OK);
} catch (Exception e) {
    JsonObject responseData = JsonObject.empty()
        .put("success", false)
        .put("failure", "There was an error creating account")
        .put("exception", e.getMessage());
    return new ResponseEntity<String>(responseData.toString(), HttpStatus.OK);
}
Tip: When it comes to storing a document, you have broadly three method choices:
  • insert will only work if no document currently exists for the given ID. Otherwise, a DocumentAlreadyExistsException will be thrown.
  • replace on the contrary will only work if the document does already exist. Otherwise, a DocumentDoesNotExistException is thrown.
  • upsert will always work, replacing or creating the document as needed.
Checking login by getting the User's document

In the login method, we check a User's credential and for that we need to retrieve the corresponding document of course! Since user documents are identified by prefixing their username with user::.

JsonDocument doc = bucket.get("user::" + username);

If that particular key doesn't exist, the get method returns null. That's useful to check if the user exists at all:

if (doc == null) {
	responseContent = JsonObject.create().put("success", false).put("failure", "Bad Username or Password");
}

Otherwise it's just a matter of checking the hashed password with the one provided by the user, and responding accordingly. Notice how we get the hash by calling content().getString("password"):

//...continued
else if(BCrypt.checkpw(password, doc.content().getString("password"))) {
    responseContent = JsonObject.create().put("success", true).put("data", doc.content());
} else {
    responseContent = JsonObject.empty().put("success", false).put("failure", "Bad Username or Password");
}
A super quick glance at the async API with RxJava

The 2.x Java SDK relies on RxJava for its asynchronous API. It offers a powerful way of composing asynchronous streams for your processing. The getFlightsForUser() method can serve as a quick example of such an asynchronous call, we'll return the result of a chain started with the async SDK call:

bucket.async().get("user::" + username)

RxJava's Observable is a push model where you describe your stream (by composing and chaining rx operators) then subscribe to it (to consume the end data). You can also manage what to do with error notifications in the subscription.

The async() method on Bucket will switch to the async API. There, get will return an Observable in which the requested Document is emitted.

Note: If the requested key doesn't exist, the async API will instead result in an empty Observable, nothing gets emitted. See below for an example of how to deal with that particular case.

The next step in our chain is to extract the flight information that we need and return it as a ResponseEntity using the transforming operator map. We pass a function that will transform each emitted JsonDocument into a ResponseEntity<String>:

.map(new Func1<JsonDocument, ResponseEntity<String>>() {
	@Override
 	public ResponseEntity<String> call(JsonDocument doc) {
		return new ResponseEntity<String>(doc.content().getArray("flights").toString(), HttpStatus.OK);
 	}
})

To prove that the document doesn't exist, we have to do things a bit differently since the map function won't receive a null (it's the enclosing Observable stream that is empty). Fortunately, RxJava provides a method to emit a single default value if an upstream Observable is empty:

.defaultIfEmpty(new ResponseEntity<String>("{failure: 'No flights found'}", HttpStatus.OK))

In this example, we still must exit the method by returning a value in a synchronous manner. Therefore, we can revert to blocking behavior and say "we only expect one single() value to be emitted, wait for it and return it":

                     .toBlocking()
.single();
Note: To learn more about Observables, see the Mastering Observablessection.

A First N1QL Query: Finding Airports

Where: trycb.service.Airport

Goals: Use N1QL and the DSL to perform your first SELECT on Couchbase.

Relevant Documentation Topics: Working with N1QL.

In the SDK, we have a query method that accepts all variants of querying with Couchbase (views, spatial/geo views, and N1QL). For N1QL, the N1qlQuery is expected. This allows to wrap a N1QL Statement, provide query tuning through a N1qlParams and, if necessary, provide values for placeholders in the statement as JsonObject or JsonArray.

Tip: N1QL is a super-set of SQL, so if you're familiar with SQL, you feel at ease.

Statements can be provided either in String form or using the DSL. So let's issue our first SELECT using the DSL!

The findAll method is expected to return a List (several matching rows) of Maps representing the JSON value. Spring will inject the Bucket into it, and the params attribute from the HTTP request. From that we'll start building a Statement:

/**
 * Find all airports.
 */
public static List<Map<String, Object>> findAll(final Bucket bucket, final String params) {
    Statement query;
	//continued...

We'll want to select just the airport name from relevant documents in our bucket. Since we want to filter relevant document on criteria that depends on the input length, let's just do the SELECT and FROM clauses first:

AsPath prefix = select("airportname").from(i(bucket.name()));

Then we can choose the correct fields to look into depending on the length of the input. Notice the x method that produces a token/expression out of a string. From there you can apply operators like eq (equals).

if (params.length() == 3) {
	query = prefix.where(x("faa").eq(s(params.toUpperCase())));
} else if (params.length() == 4 && (params.equals(params.toUpperCase()) || params.equals(params.toLowerCase()))) {
	query = prefix.where(x("icao").eq(s(params.toUpperCase())));
} else {
    query = prefix.where(i("airportname").like(s(params + "%")));
}
Tip: Use static imports on these methods of the Expression class:
  • x to create an Expression representing a plain token, like a field.
  • s to create a string literal (with adequate quotes).
  • i to escape a token with backticks (for instance when referring to the travel-sample bucket, you need to escape it because otherwise N1QL will interpret the dash as a subtraction operator).

The statement is ready! You can view (and log it) via its toString() method:

logQuery(query.toString());
//query.toString() example: SELECT airportname FROM `travel-sample` WHERE faa = "LAX"

Then we need to actually execute this statement by wrapping it in a N1qlQuery and invoking bucket.query(). Here it is very simple, no placeholders and no particular tuning of the query is necessary, so we'll use the N1qlQuery.simple() factory method:

N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
return extractResultOrThrow(result);

Let's have a look at extractResultOrThrow to understand the structure of the N1QL response (as represented by N1qlQueryResult):

/**
 * Extract a N1Ql result or throw if there is an issue.
 */
private static List<Map<String, Object>> extractResultOrThrow(N1qlQueryResult result) {
    if (!result.finalSuccess()) {
        LOGGER.warn("Query returned with errors: " + result.errors());
        throw new DataRetrievalFailureException("Query error: " + result.errors());
    }

    List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
    for (N1qlQueryRow row : result) {
        content.add(row.value().toMap());
    }
    return content;
}

The N1qlQueryResult has two status flags: one intermediary parseSuccess() that indicates immediately if there is a syntax error (parseSuccess() == false) or not, and one that indicates the definite result of the query (finalSuccess()).

If the query is successful, it will offer a list of N1qlQueryRow through allRows(). Otherwise it will have JsonObject errors in errors(). That's what we inspect to respectively build a list of results or throw a DataRetrievalFailureException containing all the errors.

More Complex Queries: Finding Routes

Where: trycb.service.FlightPath

Goals: Let the DSL guide you into making more complex N1QL queries.

Relevant Documentation Topics: Working with N1QL.

In this service, we have two more complex queries. The first aims at going from human-readable airport name for the departure and arrival airports to FAA codes:

SELECT faa AS fromAirport FROM `travel-sample` WHERE airportname = "Los Angeles Intl"
UNION SELECT faa AS toAirport FROM `travel-sample` WHERE airportname = "San Francisco Intl"

The second aims at constructing the result set of available flight paths that connect the two airports:

SELECT a.name, s.flight, s.utc, r.sourceairport, r.destinationairport, r.equipment
FROM `travel-sample` AS r
UNNEST r.schedule AS s
JOIN `travel-sample` AS a ON KEYS r.airlineid
WHERE r.sourceairport = "LAX" AND r.destinationairport = "SFO" AND s.day = 6
ORDER BY a.name ASC
Tip: Yes, you read that right, N1QL can do joins (on a single bucket or several). It works as long as the "foreign key" described by ON KEYS clause can be mapped to a document's Key in the joined bucket.

A specificity of N1QL that we see in the second statement is UNNEST. It extracts a sub-JSON and puts it at the same root level as the bucket (so its possible to do joins on each element in this sub-json as if they were entries in a left-hand side bucket).

For this final step, try to obtain the equivalent of these statements via the DSL and see how it guides you through the possibilities of the query language.

Indexing the Data: N1QL & GSI

Where: trycb.utils.StartupPreparations

Goals: Use the Index DSL to make sure data is indexed for N1QL to query it.

Index management is a bit more advanced (and is already done when loading the sample), so now that you've learned about N1QL, you can have a look at it. For N1QL to work, you must first ensure that at least a Primary Index has been created. For that you can use the DSL from the Index class:

Index.createPrimaryIndex().on(bucket.name())

The fluent API will guide you with the available options; you just have to declare that you want to createPrimaryIndex() and specify on(...) which Bucket.

You can also create secondary indexes on specific fields of the JSON, for better performance:

Index.createIndex(name).on(bucket.name(), x(name.replace("def_", "")))

In this case, give a name to your index, specify the target bucket AND the field(s) in the JSON to index.