Examples

Examples

Examples of how to use the Eventing Service.

Preparations

This page contains examples of how to use the Eventing Service, by means of the Couchbase Web Console. The examples require the creation of three buckets.(Note that detailed information on creating buckets, is provided in Create a Bucket.)

Proceed as follows:

  1. Create a Metadata bucket: Each Eventing function requires a metadata bucket to be defined on the cluster. This bucket is used for the storage of artifacts and checkpoint information, generated during the course of execution. You must create or reference such a bucket explicitly, for each Eventing function that you define. Ideally, no metadata bucket should be used for storing information from any other application, though a single metadata bucket may be used by multiple Eventing functions.

    The examples on this page refer to a metadata bucket named metadata. Therefore, to create such a bucket:

    1. Access the Buckets screen, by left-clicking on the Buckets tab, in the left-hand navigation panel.

    2. On the Buckets screen, left-click on the Add Bucket tab, at the upper right.

    3. When the Add Data Bucket dialog appears, specify metdata as the bucket-name, and specify an appropriate Memory Quota for the bucket. All other settings can be left with their default value. Then, left-click on the Add Bucket button.

    The metadata bucket now appears on the Buckets screen.

  2. Create a Source bucket: This is a bucket that will contain a source document, updates to which cause execution of the function you define. You must create both the bucket and the source document. (For a detailed explanation of the user-interface components on the Buckets and Documents screens, see the page Look at the Results, within the Couchbase Server Getting Started sequence.)

    Proceed as follows:

    1. Using the procedure described immediately above for the metadata bucket, create a bucket named source.

    2. When the source bucket appears on the Buckets screen, left-click on the Documents tab, towards the right-hand side of the source bucket's row. This brings up the Documents screen.

    3. When the Add Document dialog appears, specify the name SampleDocument for the new document. Then, left-click on the Save button. This brings up the Edit Document dialog.

    4. Within the edit panel of the Edit Document dialog, paste the following:

      {
        "country": "AD",
        "ip_end": "5.62.60.9",
        "ip_start": "5.62.60.1"
      }
      Left-click on the Save button.

  3. Create a Target bucket: This is a bucket that will contain a new document, created by a function you will add to the Eventing Service. Using Couchbase Web Console, create a bucket named target, using the procedure described above for the metadata bucket. No document need explicitly be added to this bucket.

This concludes all required preparations.

Example 1

Goal: A document contains attributes whose form makes them difficult to search on. Therefore, on the document's creation or modification, a new attribute should be created to accompany each original attribute; this new attribute being instantiated with a value that directly corresponds to that of its associated, original attribute; but takes a different form, thereby becoming more supportive of search. Original attributes are subsequently retrievable based on successful searches performed on new.

Implementation: Create a JavaScript function that contains an OnUpdate handler. The handler listens for data-changes within a specified, source bucket. When any document within the bucket is created or mutated, the handler executes a user-defined routine. In this example, if the created or mutated document contains two specifically named fields containing IP addresses (these respectively corresponding to the beginning and ending of an address-range), the handler-routine converts each IP address to an integer. A new document is created in a specified, target bucket: this new document is identical to the old, except that it has two additional fields, which contain integers that correspond to the IP addresses. The original document, in the source bucket, is not changed.

Proceed as follows:

  1. Access the Eventing screen, by left-clicking on the Eventing tab, in the navigation panel at the left-hand side of the Couchbase Web Console:

    The Eventing screen now appears. This shows all currently defined functions, and so is initially empty.

  2. Elect to add a function, by left-clicking on the Add Function button, at the upper right:

    This brings up the Add Function dialog:

    The elements within the dialog are as follows:

    • Source Bucket: The name of a bucket currently defined on the cluster. The function you define listens to changes made to the contents of this bucket — such as the creation or mutation of a document. The bucket must be of type Couchbase or Ephemeral: buckets of type Memcached are not supported. Use the arrows control at the right-hand side of the field to select from currently defined buckets:

      For the current example, select the bucket you already created for this purpose, source.

    • Metadata Bucket: The name of a bucket currently defined on the cluster. This bucket will be used to store artifacts and checkpoint information. The bucket must be of type Couchbase or Ephemeral: buckets of type Memcached are not supported. Use the arrows control at the right-hand side of the field to select from currently defined buckets.

      For this example, select the bucket you already created for this purpose, metadata.

    • Function Name: A name, for the function you are creating. For the current example, use enrich_ip_nums.

    • Description: A description of the function you are creating. This is optional. For the current example, specify Enrich a document, converts IP Strings to Integers that are stored in new attributes.

    • Settings: Left-click on the arrow control, to display the subpanel:

      The available settings are as follows. (For the current example, use the default values.)

      • Log Level: The granularity at which messages are logged. The options (available from the arrows control at the right of the field) are Info, Error, Warning, Debug, and (the default) Trace.

      • Workers: The number of worker threads to be allocated to the function. The default is 3.

      • Script Timeout: The number of seconds that should elapse before the script times out. The default is 1.

    • Bindings: One or more strings, each of which will be used as a reference to an existing bucket, from within the JavaScript code of your function. One binding-type is currently available, which is Alias. This associates the name of a defined bucket (selected from the menu accessed from the arrow controls) with the string you enter into the value field. This string can be used as an alias for the named bucket, in your function.

      To add multiple bindings, left-click on the + control, at the upper right of the panel. To remove a binding, left-click on the - control.

      For the current example, specify target as the name of the bucket; and specify tgt as the associated value.

  3. When all the appropriate fields of the Add Function dialog have been completed, left-click on the Next: Add Code button, at the lower-right:

    This brings up the enrich_ip_nums screen, which appears as follows:

    The main panel of this page initially contains place-holder code. You will substitute your actual enrich_ip_nums code for this.

  4. Copy the following function, and paste it over the place-holder function in the main panel of the enrich_ip_nums screen:

    function OnUpdate(doc, meta) {
      log('document', doc);
      doc["ip_num_start"] = get_numip_first_3_octets(doc["ip_start"]);
      doc["ip_num_end"]   = get_numip_first_3_octets(doc["ip_end"]);
      tgt[meta.id]=doc;
    }
    
    function get_numip_first_3_octets(ip)
    {
      var return_val = 0;
      if (ip)
      {
        var parts = ip.split('.');
    
        //IP Number = A x (256*256*256) + B x (256*256) + C x 256 + D 
        return_val = (parts[0]*(256*256*256)) + (parts[1]*(256*256)) + (parts[2]*256) + parseInt(parts[3]);
        return return_val;
      }
    }

    After the paste, the screen appears approximately as follows:

    The OnUpdate routine specifies that when a change occurs to data within the bucket, the routine get_numip_first_3_octets is run on each document that contains ip_start and ip_end. A new document is created whose data and metadata are based on those of the document on which get_numip_first_3_octets is run; but with the addition of ip_num_start and ip_num_end data-fields, which contain the numeric values returned by get_numip_first_3_octets.

    The get_numip_first_3_octets routine splits the IP address it finds, converts each fragment to a numeral, and adds the numerals together, to form a single value; which it returns.

    Left-click on the Save button, at the lower right, to save.

  5. Return to the Eventing screen, by left-clicking on the Eventing tab, in the navigation bar at the left. The main panel now appears as follows:

    The function enrich_ip_nums is now listed as a defined function. As yet, it is listed as undeployed and paused.

  6. Left-click on the row for enrich_ip_nums. Additional controls are now displayed:

    The controls are:

    • Delete: Deletes the function from the system.

    • Export: Exports the function as a JSON document. For sample output based on the enrich_ip_nums function, see Eventing Function Export.

    • Deploy: Deploys the function, making it active across the cluster.

    • Edit JavaScript: Allows edits to be made on the function, in an edit dialog.

  7. Left-click on the Deploy button. This displays the Confirm Deploy Function dialog:

    The Feed Boundary determines whether documents previously in existence should be included in the function's activities: the options are Everything and From now. For the current example, select Everything. A cleanup operation on existing artifacts can be executed by checking the Cleanup artifacts? checkbox. For the current example, leave this unchecked.

  8. Left-click on Deploy Function. This deploys the function, and returns you to the main Eventing screen, which now appears as follows:

    The display indicates that the function is now deployed and running. Note also that the Deploy button has been changed to Undeploy.

    From this point, your defined function will run, first, on all existing documents; and subsequently, whenever a creation or mutation occurs.

  9. To check the effects of deploying the function, access the Buckets screen, by left-clicking on the Buckets tab, in the navigation panel at the left. Then, left-click on the Documents tab on the row for the target bucket. This displays the following:

    As this shows, a version of SampleDocument has been added to the target bucket. It contains all the attributes of the original document, with the addition of ip_num_start and ip_num_end; which contain the numeric values that correspond to ip_start and ip_end, respectively.

    Additional documents added to the source bucket, which share the ip_start and ip_end attributes, will be similarly handled by the defined function: creating such a document, and changing any attribute in such a document both cause the function's execution.

Example 2

Goal: When a document in an existing bucket is about to expire, ideally, one or more of its attributes become archived, in a newly created bucket.

(For information on bucket expiration, see the Item Deletion section of Bucket Disk Storage.)

Implementation: Write an OnUpdate handler, which runs whenever a document is created or mutated. The handler calls a docTimer routine, which itself executes a callback function, two minutes prior to any document's established expiration: this function retrieves a specified value from the document, and stores it in a document of the same name, in a specified target bucket. The original document in the source bucket is not changed.

Important:

The Timers construct is still in development and may have some rough edges and bugs, and may change significantly before the final GA release.

For this example, Example 2, the buckets used in the previous example, Example 1, are reused: source, target, and metadata. A new document is created within the source bucket: this document has its expiration — or Time To Live (TTL) — set to occur ten minutes after the document's creation. Note that a document's expiration cannot be established by means of the Couchbase Web Console: the standard procedure for establishing or modifying document-expiration is provided by the Couchbase SDK. The following Python script, based on the Couchbase SDK Python client, illustrates this; and will be used in this example:

from couchbase.cluster import Cluster
from couchbase.cluster import PasswordAuthenticator
import time
cluster = Cluster('couchbase://localhost:8091')
authenticator = PasswordAuthenticator('Administrator', 'password')
cluster.authenticate(authenticator)

cb = cluster.open_bucket('source')
cb.upsert('SampleDocument2', {'a_key': 'a_value'})
cb.touch('SampleDocument2', ttl=10*60)

This script imports a Couchbase cluster object, and authenticates against it, using (for demonstration purposes) the Full Administrator username and password (the cluster is assumed to be accessible on localhost). The script then opens the existing source bucket, and inserts a new document, named SampleDocument2, whose body is {'a_key': 'a_value'}.

Information on installing the Couchbase Python SDK can be found in Start Using the SDK. Information on using the Couchbase Python SDK to establish bucket-expiration can be found in Document Operations.

Proceed as follows:

  1. Install the Couchbase SDK Python client, and, from the appropriate folder, start Python.

    ./python

  2. At the Python prompt, enter the code given above. The session should appear as follows:

    >>> from couchbase.cluster import Cluster
    >>> from couchbase.cluster import PasswordAuthenticator
    >>> import time
    >>> cluster = Cluster('couchbase://localhost:8091')
    >>> authenticator = PasswordAuthenticator('Administrator', 'password')
    >>> cluster.authenticate(authenticator)
    >>> cb = cluster.open_bucket('source')
    >>> cb.upsert('SampleDocument2', {'a_key': 'a_value'})
    OperationResult<rc=0x0, key='SampleDocument2', cas=0x1519ec8cdee90000>
    >>> cb.touch('SampleDocument2', ttl=10*60)
    OperationResult<rc=0x0, key='SampleDocument2', cas=0x1519ec8e686c0000>
    >>>

  3. As a further check that the document has been created, access the Buckets screen of Couchbase Web Console, and left-click on the Document tab of the source bucket. This should appear as follows, with the new document displayed:

    Note that by left-clicking on a document's id, its metadata is displayed:

    This pop-up confirms the established document expiration-time.

  4. Access the Eventing screen, by left-clicking on the Eventing tab, in the left-hand navigation panel. Then, when the Eventing screen is displayed, click on the Add Function tab, at the upper right.

    When the Add Function dialog appears, specify source as the Source Bucket, and metadata as the Metadata Bucket. Then, specify add_timer_before_expiry as the function-name. Optionally, add a function-description. Leave the Settings at default, for all values. In Bindings, specify tgt as the value for the name target, and specify src as the value for the name source. The completed dialog should appear approximately as follows:

  5. Left-click on the Next: Add Code button, at the lower right of the Add Function dialog. The dialog disappears, and the add_timer_before_expiry screen is displayed. Copy the following JavaScript routine, and paste it over the place-holder code that appears in the main, interactive panel:

    function OnUpdate(doc, meta) {
      if (meta.expiration > 0 ) //do only for those documents that have a non-zero TTL
      {
        //have to x by 1000, as timestamp in secs; and for Date operations need in milli-secs
        var expiry = new Date(meta.expiration*1000); 
        // Compute 2 minutes from the TTL timestamp        
        var twoMinsPrior =  Math.round(expiry.setMinutes(expiry.getMinutes()-2)/1000); 
        docTimer(DocTimerCallback, meta.id, twoMinsPrior);  //create the docTimer
        log('Added Doc Timer to DocId:', meta.id);
      }
    }
    
    function DocTimerCallback(docid, expiry) {
      log('DocTimerCallback Executed for DocId:', String(docid));
      tgt[docid] = "To Be Expired Key's Value is:" + JSON.stringify(src[docid]);
      log('Doc Timer Executed for DocId', String(docid));
    }

    The interactive panel now appears as follows:

    The OnUpdate routine, for each document with a non-zero expiration-time, computes a docTimer execution-time that is two minutes prior to the document expiration-time. The callback for the docTimer creates a new document in the tgt bucket, with the same docid as the original document; and makes the value of the new document a modified version of the original.

  6. Left-click on the Save button, at the lower right. Then, return to the Eventing screen: when it reappears, left-click on the Deploy button for the add_timer_before_expiry function. When the Confirm Deploy Function dialog appears, select Everything from the Feed boundary menu, and leave Cleanup artifacts? unchecked. Then, left-click on the Deploy Function button. The function should be confirmed as deployed and running within a few seconds.

  7. The new document will be created two minutes before the expiration-time of the original. Therefore, when sufficient time has elapsed, access the Buckets screen, and examine the documents currently within the target bucket. Eventually, the display will be as follows:

    This indicates that the new document — like the original, named SourceDocument2 — has been created, with a value based on that of the original. After ten minutes have elapsed, check the documents within the source bucket: the original SourceDocument2 is no longer visible, having been removed at its defined expiration-time.