cbexport

cbexport

cbexport is a utility to export data from a Couchbase cluster.

Synopsis

cbexport [--version] [--help] <command> [<args>]

Description

cbexport is used to export data from Couchbase in various formats. For more information on how specific commands work, run cbexport <command> --help.

Options

Table 1. cbexport options
Option Description

--version

Prints the cbexport suite version that the cbexport program came from.
--help Prints the synopsis and a list of commands. If a cbexport command is named, this option will bring up the manual page for that command.

Commands

cbexport json: Exports JSON data from Couchbase to a file.

Discussion

The cbexport command is used to export JSON data from Couchbase to various formats. Each supported export format is a sub-command of the cbexport utility.

cbexport json

cbexport json exports JSON data from Couchbase to a file.

Synopsis

cbexport json [--cluster <url>] [--bucket <bucket_name>][--format <data_format>]
              [--username <username>][--password <password>]
              [--include-key <key>][--cacert <path>][--no-ssl-verify]
              [--threads <num>][--log-file <path>]

Description

Exports JSON data from Couchbase. The cbexport json command supports exporting JSON documents to a file with a document on each line or a file that contains a JSON list where each element is a document. The file format to export to can be specified with the --format flag. See Dataset Formats for more details on the supported file formats.

Options

The following tables list the required and optional parameters for the cbexport json command.

Table 2. Required options for cbexport json
Option Description
-c,--cluster <url> The host name of a node in the cluster to export data from. See Host Formats for details about host name specification formats.
-u,--username <username> The user name for cluster authentication. The user must have the appropriate privileges to read the bucket from which data will be exported.
-p,--password <password> The password for cluster authentication. The user must have the appropriate privileges to read from the bucket from which data will be exported. Specifying this option without a value will allow the user to type a non-echoed password to stdin.
-b,--bucket <bucket_name> The name of the bucket to export data from.
-f,--format <format> The format of the dataset specified (lines or list). See Dataset Formats for more details on the formats supported by cbexport.
-o,--output <path> The path to the location of the file that JSON documents from Couchbase should be exported to. This may be an absolute or relative path, but must point to a file. The file does not have to exist when the command is invoked.
Table 3. Optional options for cbexport json
Option Description
--include-key <key> Couchbase stores data as key value pairs where the value is a JSON document and the key is an identifier for retrieving that document. By default cbexport will only export the value portion of the document. If you wish to include the key in the exported document, then this option should be specified. The value passed to this option should be the field name that the key is stored under.
--no-ssl-verify Skips the SSL verification phase. Specifying this flag will allow a connection using SSL encryption, but will not verify the identity of the server you connect to. You are vulnerable to a man-in-the-middle attack if you use this flag. Either this flag or the --cacert flag must be specified when using an SSL encrypted connection.
--cacert Specifies a CA certificate that will be used to verify the identity of the server being connecting to. Either this flag or the --no- ssl-verify flag must be specified when using an SSL encrypted connection.
-t,--threads <num> Specifies the number of concurrent clients to use when exporting data. Fewer clients means exports will take longer, but there will be less cluster resources used to complete the export. More clients means faster exports, but at the cost of more cluster resource usage. This parameter defaults to 1 if it is not specified and it is recommended that this parameter is not set to be higher than the number of CPUs on the machine where the export is taking place.
-l,--log-file Specifies a log file for writing debugging information about cbexport execution.

Host Formats

When specifying a host for the cbexport command the following formats are expected:
  • couchbase://<addr>
  • <addr>:<port>
  • http://<addr>:<port>

We recommend using the couchbase://<addr> format for standard installations. The other two formats allow an option to take a port number which is needed for non-default installations where the admin port has been set up on a port other that 8091.

Dataset Formats

The cbexport command supports the following formats:
  • Lines
    The lines format specifies a file that contains one JSON document on every line in the file. This format is specified by setting the --format option to "lines". Here's an example of a file in lines format:
    {"key": "mykey1", "value": "myvalue1"}
    {"key": "mykey2", "value": "myvalue2"}
    {"key": "mykey3", "value": "myvalue3"}
    {"key": "mykey4", "value": "myvalue4"}
  • List
    The list format specifies a file which contains a JSON list where each element in the list is a JSON document. The file may only contain a single list, but the list may be specified over multiple lines. This format is specified by setting the --format option to "list". Here's an example of a file in list format:
    [
      {
        "key": "mykey1",
        "value": "myvalue1"
      },
      {"key": "mykey2", "value": "myvalue2"},
      {"key": "mykey3", "value": "myvalue3"},
      {"key": "mykey4", "value": "myvalue4"}
    ]

Examples

To export data to /data/lines.json using the lines format and running with 4 threads the following command can be run.
$ cbexport json -c couchbase://127.0.0.1 -u Administrator -p password \
          -b default -o /data/lines.json -f lines -t 4
To export data from /data/list.json using the list format the following command can be run.
$ cbimport json -c couchbase://127.0.0.1 -u Administrator -p password \
           -b default -o /data/list.json -f list