Release Notes

Release Notes

Release notes for the 1.2 version of the Kafka connector.

Couchbase Kafka Connector 1.2.1 GA (23 December 2015)

Version 1.2.1 is the general availability (GA) release of the Kafka connector. It fixes issues several important issues like KAFKA-20, KAFKA-21, improves performance of default Zookeeper initializer and makes state management more precise.

New features and Behavioral Changes

This release contains the following features:

  • KAFKAC-20: By default the state for stream has been stored incorrectly, which leads to rollbacks and ERR_RANGE errors. The issue has been fixed in the core java library, and the connector code has been adopted to it. The API of building state has been changed and now it initializes vbucketUUID property of the stream state. For example:
    CouchbaseKafkaConnector connector = CouchbaseKafkaConnector.create(builder.build());
    BucketStreamAggregatorState state = connector.buildState(Direction.TO_CURRENT);
    connector.run(state, RunMode.RESUME);
    
    As important API change CouchbaseKafkaConnector.buildState() does not accept the name for the state. And in general name of the state has been removed from the public API.
  • KAFKAC-21: Improve performance of Zookeeper by fixing the bug, when it triggered iteration over all streams to persist the state when just one was changed. Also an option to throttle write rate of the serializer introduced: couchbaseStateSerializationThreshold() which controls minimum period in milliseconds (by default 2) which have to pass between to state serialization operation. So under high load this option could be larger.
  • KAFKAC-23: This improvement exposes sequence number for each mutation and remove message going in DCP stream, so that it could be used to inspect the current state.

Couchbase Kafka Connector 1.2.0 GA (13 October 2015)

Version 1.2.0 is the general availability (GA) release of the Kafka connector. It fixes issues with missing events in the streams, improves state initialization, and introduces flow control.

New Features and Behavioral Changes

This release contains the following features:

  • KAFKAC-16: Helper to gather current state of the streams, to simplify initialization of the streams. For example the following code will fetch all changes up to current state. See more info on documentation for class Direction.
    CouchbaseKafkaConnector connector = CouchbaseKafkaConnector.create(builder.build());
    BucketStreamAggregatorState state = connector.buildState("hello", Direction.TO_CURRENT);
    connector.run(state, RunMode.RESUME);
    
  • KAFKAC-11: Update Zookeeper state serializer to work with recent core library.
  • Allow to override Couchbase and Kafka credentials during connector initialization from the code.
    CouchbaseKafkaConnector connector = CouchbaseKafkaConnector.create(
            Collections.singletonList("couchbase1.vagrant"), "default", "",
            "kafka1.vagrant", "default",
            environmtnt);
    
  • JVMCBC-217: In the core library flow control for streaming connection has been implemented. The connector can specify the buffer limit, which represents capacity of the events consumer. When the producer (Couchbase Server) will reach this value, it will stop emitting the events until the connector will notify about free space. The API of this feature exposed as two settings on environment: dcpConnectionBufferSize(), which represents the size of the window in bytes (default value is 20MB), and dcpConnectionBufferAckThreshold() which controls when the connector will emit buffer acknoledgement message (default is 4MB, which means that the acknowledgements will be sent while the current buffer level is greater than this value).