Troubleshooting

Troubleshooting

The most common problems that might arise when using Couchbase Server with Elasticsearch are data transfer failure and data not getting indexed.

The following sections describe some of the causes of these issues and provides steps you can follow to resolve these problems. There are a few tasks you can perform to monitor the progress of Couchbase Plug-in for Elasticsearch. These tasks will help you determine if data is successfully transferred and indexed. It will help insure you resolve the most common problems that occur using the plug-in:

Monitoring Operations

Here are some things you can check to verify whether the data is successfully transferred and indexed:

  • Check outbound XDCR operations

    In Couchbase Web Console under Data Buckets, Click | Bucket-name | Outbound XDCR to view information about data replication via XDCR to Elasticsearch. The statistics in this section will indicate the rate of data transfer between Couchbase Server and Elasticsearch. .

  • Check the log files

    The plug-in logs to the same directory and file that Elasticsearch uses at the root of an Elasticsearch node. For production systems the log is at /var/log/elasticsearch.

  • Compare document counts

    You can compare the number of documents in Couchbase Server with the number of documents in your Elasticsearch cluster. Be aware that this assumes your Couchbase Server has a static number of items, for instance your cluster is in a test environment where the number of documents is set and you do not add more during replication.

    To find the number of items in a bucket, in the Couchbase web console click Data Buckets. The number of documents for a named bucket appears under Item Count. This example shows 7303 items in the beer-sample bucket:

    To get the number of documents indexed by Elasticsearch, send an HTTP request (be sure to substitute the correct index-name):

    curl http://[elasticsearch_host]:9200/[index-name]/couchbaseDocument/_count

    Upon success, you get the following response:

    {"count":7303,"_shards":
        {
        "total":5,"successful":5,"failed":0
        }
    }

    The count returned by Elasticsearch is the same value as the number of items shown in the Couchbase web console. The matching values provide assurance that all items from Couchbase have been transferred and indexed by Elasticsearch.

Troubleshooting Data Transfer Issues

Follow these steps to troubleshoot data transfer issues:

  1. Check your Elasticsearch version.

    The most common problem you can encounter with Couchbase-Elasticsearch integration is that data fails to transfer due to an incompatible Elasticsearch version. Elasticsearch has evolved between versions and Couchbase Elasticsearch plug-in has been specifically designed and tested for a particular version. If you use an earlier or later version, it will result in failure to transfer data. If you have an incompatible version of the plug-in, you might see the following message:

    Attention - Failed to grab remote bucket info from any of known nodes

    If you check the Elasticsearch head console, a stack trace similar to the following displays:

    [2012-12-19 05:50:41,758][WARN ][org.eclipse.jetty.servlet.ServletHandler]
    Error for /pools/default/buckets java.lang.NoSuchMethodError:
    ....

    If you get this error, make sure you are using the plug-in with the correct version of Elasticsearch.

  2. Check destination cluster references.

    Another common error occurs when you create a cluster reference in Couchbase Web Console and then at a later time create and start the replication. After you create a reference to your Elasticsearch cluster, the IP address might change, especially if you are using Elasticsearch on a laptop. In this case you will get this error under XDCR | Ongoing Replications | Status:

    To resolve this error, check your remote Elasticsearch reference in XDCR and make sure the IP address is correct.

Troubleshooting Indexing Issues

If you encounter issues with indexing, such as failure to index items from Couchbase or unexpected items in your search results, try checking the following items and performing the described fixes:

  • Change settings for initial indexing

    If you have an existing Couchbase data bucket with a large number of documents already in production, these documents will be transferred to Elasticsearch in bulk. Typically this works with Elasticsearch default settings, however there are some Elasticsearch settings you can change so that indexing quickly completes.

    You use the Elasticsearch refresh_interval setting to indicate how frequently the engine provides newly indexed items. During an initial bulk load of documents from Couchbase, you can reduce access to newly indexed items in exchange for overall faster indexing time. For more information about enabling and disabling this setting, see Update Indices Settings in the Elasticsearch Reference.

  • Check Elasticsearch mappings

    When you send documents to Elasticsearch it will automatically generate a mapping that contains rules for indexing fields. You can also provide your own mapping or update this mapping. Be aware that this default mapping from Elasticsearch includes assumptions about data types and data structures in your documents. Based on these assumptions, Elasticsearch may omit your document from the index. For instance, objects within an array may not be indexed as you expect.

    For general information about expected data structures see Mapping in the Elasticsearch Reference.

  • Check your documents

    Ensure your documents contain well-formed JSON. The Couchbase Plug-in for Elasticsearch will take any items that are binary data and will log an error message. Elasticsearch cannot index documents that are not valid JSON. JPEG files and other forms of binary data cannot be indexed by Elasticsearch.

    If you change a field type for your documents after Elasticsearch has indexed, it may omit your document from the index.

Enabling Logging

You can configure the Couchbase Transport logging by editing the file ./config/logging.yml and adding the following lines to the logger section:
transport.couchbase: TRACE com.couchbase: TRACE