The command set flush_param tunes the dynamic shared thread pool performance.
The basic syntax is:
cbepctl [host]:11210 -b [bucket-name] set flush_param [parameter] [value]
The syntax to configure the access log settings is:
cbepctl [hostname]:11210 -b [bucket-name] -p [bucket-password] set flush_param alog_sleep_time [value] cbepctl [hostname]:11210 -b [bucket-name] -p [bucket-password] set flush_param alog_task_time [value]
The syntax for disk cleanup is:
cbepctl [host]:11210 -b [bucket-name] -p [bucket-password] set flush_param exp_pager_stime [value]
The syntax for ejection is:
cbepctl [host]:11210 -b [bucket-name] -p [bucket-password] set flush_param [parameter] [value]
Parameters used for changing ejection thresholds:
The syntax to set the out of memory threshold is:
cbepctl [host]:11210 -b [bucket-name] -p [bucket-password] set flush_param mutation_mem_threshold [value]
Tune the dynamic shared thread pool performance by changing the thread types inside the ep-engine and memcached at run time. The command set flush_param adjusts the number of threads that prioritize read, write, non-i/o and auxiliary-i/o operations. These settings take effect immediately and do not require that the bucket be restarted.
- alog_sleep_time, alog_task_time
Couchbase Server has an optimized
disk warmup. Couchbase Server pre-fetches a list of most-frequently accessed keys and fetches these documents first. The server runs a periodic scanner process which determines which keys are most frequently used. The
cbepctl flush_param command is used to change the initial time and the interval for the process. For example, the initial time and interval might be changed to accommodate a peak time when an application needs these keys to be quickly available.
By default, the scanner process runs once every 24 hours at 10:00 AM GMT. To reduce the cluster-wide impact of running this task, stagger the start time to a different value on each node in the cluster.
cbepctl flush_param exp_pager_stime command sets the time interval for disk cleanup. Couchbase Server does lazy
expiration, that is, expired items are flagged as deleted rather than being immediately erased. Couchbase Server has a maintenance process that periodically looks through all information and erases expired items. By default, this maintenance process runs every 60 minutes, but it can be configured to run at a different interval.
Note: The compaction process will also remove expired items.
- mem_low_wat, mem_high_wat, pager_active_vb_pcnt
Ejection means that documents are removed from RAM but the key and metadata remain. If the amount of RAM used by items reaches the high water mark (upper threshold), both active and replica data are ejected until the memory usage (amount of RAM consumed) reaches the low water mark (lower threshold). The server determines that items are not recently used based on a not-recently-used (NRU) value.
Use the mem_low_wat, mem_high_wat, and pager_active_vb_pcnt settings to change the server thresholds for ejection.Warning: Do not change the ejection defaults unless required by Couchbase Support.
By default, Couchbase Server sends clients a temporary out-of-memory error message if RAM is 95% consumed and only 5% RAM remains for overhead. Use the
cbepctl set flush_param mutation_mem-threshold command parameter to change this threshold value.
Note: Do not change this default to a higher value. However, this value might be reduced if you need more RAM for system overhead such as disk queue or for server data structures.
The following are the command options:
|alog_sleep_time||Access scanner interval (minute)|
|alog_task_time||Access scanner next task time (UTC)|
|backfile_mem_threshold||Memory threshold (%) on the current bucket quota before backfill task is made to back off.|
|bg_fetch_delay||Delay before executing a bg fetch (test feature).|
|couch_response_timeout||timeout in receiving a response from CouchDB.|
|exp_pager_stime||Expiry Pager interval. Time interval that Couchbase Server waits before it performs cleanup and removal of expired items from disk. Setting this value to 0 will disable the Expiry Pager from running.|
|flushall_enabled||Deprecated. Enable flush operation.|
|pager_active_vb_pcnt||Percentage of active vBuckets items among all ejected items by item pager.|
|max_size||Maximum memory used by the server.|
|mem_high_wat||High water mark in bytes.|
|mem_low_wat||Low water mark in bytes.|
|mutation_mem_threshold||Amount of RAM that can be consumed in that caching layer before clients start receiving temporary out of memory messages.|
|timing_log||Path to log detailed timing stats.|
|warmup_min_memory_threshold||Memory threshold (%) during warmup to enable traffic.|
|warmup_min_items_threshold||Item number threshold (%) during warmup to enable traffic.|
|klog_compactor_queue_cap||Queue cap to throttle the log compactor.|
|klog_max_log_size||Maximum size of a mutation log file allowed.|
|klog_max_entry_ratio||Max ratio of # of items logged to # of unique items.|
|pager_unbiased_period||Period after last access scanner run during which item pager preserve working set.|
|queue_age_cap||Maximum queue age before flushing data.|
|max_txn_size||Maximum number of items in a flusher transaction.|
|min_data_age||Minimum data age before flushing data.|
Performance tuning options
The following are the command options used for performance tuning.
All these options can be used only to tune down the number of threads: if the value was set as N at initial startup, then only a value < N will work.
|max_num_readers||Overrides the default number of global threads that prioritize read operations.|
|max_num_writers||Overrides the default number of global threads that prioritize write operations.|
|max_num_auxio||Overrides the default number of global threads that prioritize auxio operations.|
|max_num_nonio||Overrides the default number of global threads that prioritize nonio operations.|
Examples for setting the maximum number of writers:
Number of writers
To set the maximum number of writers, use max_num_writers parameter.
In the following example, max_num_writers sets the number of writer threads to four (4). The command can both increase and decrease the writer threads. However, the increase is capped to the limit set when the bucket warms up. If you start the bucket with four writer threads, you can dynamically reduce to a lower value such as two and then later dynamically increase back to four, but increasing above four will have no effect.
cbepctl 10.5.2.117:11210 -b foo_bucket set flush_param max_num_writers 4
The following is an example response when setting the maximum number of writer threads.
setting param: max_num_writers 4 set max_num_writers to 4
Number of threads
Check how many threads of various types are currently running by using cbstats workload. For example:
cbstats [host]:11210 -b [bucket-name] workload
For example, the following shows an example request and response for cbstats workload, in this case, for the default bucket. The setting change, ep_workload:max_writers: 5, is displayed via the cbstats workload response:
cbstats 10.5.2.117:11210 workload ep_workload:LowPrioQ_AuxIO:InQsize: 2 ep_workload:LowPrioQ_AuxIO:OutQsize: 0 ep_workload:LowPrioQ_NonIO:InQsize: 18 ep_workload:LowPrioQ_NonIO:OutQsize: 0 ep_workload:LowPrioQ_Reader:InQsize: 8 ep_workload:LowPrioQ_Reader:OutQsize: 0 ep_workload:LowPrioQ_Writer:InQsize: 12 ep_workload:LowPrioQ_Writer:OutQsize: 0 ep_workload:max_auxio: 1 ep_workload:max_nonio: 1 ep_workload:max_readers: 4 ep_workload:max_writers: 5 ep_workload:num_auxio: 1 ep_workload:num_nonio: 1 ep_workload:num_readers: 4 ep_workload:num_shards: 4 ep_workload:num_sleepers: 10 ep_workload:num_writers: 5 ep_workload:ready_tasks: 0
Examples for setting the access scanner process
To change the time interval when the access scanner process runs to every 20 minutes.
cbepctl 10.5.2.117:11210 -b foo-bucket -p foo-password \ set flush_param alog_sleep_time 20
To change the initial time that the access scanner process runs from the 2:00 AM UTC default to 11:00 PM UTC.
This response shows the time interval changed to 20 minutes.
cbepctl 10.5.2.117:11210 foo-bucket -p foo-password \ set flush_param alog_task_time 23
setting param: alog_sleep_time 20 set alog_sleep_time to 20
This response shows the initial access scanner run time changed to 11:00 PM UTC.
setting param: alog_task_time 23 set alog_task_time to 23
Examples for setting the disk cleanup
The following example sets the cleanup process to run every 600 seconds (10 minutes). This is the interval that Couchbase Server waits before it tries to remove expired items from disk.
cbepctl 10.5.2.117:11210 -b mybucket -p password \ set flush_param exp_pager_stime 600
The following example response shows the cleanup process set to 600 seconds.
setting param: exp_pager_stime 600 set exp_pager_stime to 600
Examples for setting the out-of-memory error message
In this example, the threshold is reduced to 65% of RAM.
cbepctl 10.5.2.117:11210 -b foo-bucket -p foo-password \ set flush_param mutation_mem_threshold 65%
The following example response shows the RAM threshold set to 65%.
setting param: mutation_mem_threshold 65 set mutation_mem_threshold to 65
Example for setting the low water mark
The low water mark sets the lower threshold of RAM for a specific bucket on a node. The item pager stops ejecting items once the low water mark is reached.
The following example sets the low water mark percentage to 70% of RAM.
cbepctl 10.5.2.117:11210 -b foo-bucket -p foo-password \ set flush_param mem_low_wat 70
Example for setting the high water mark
The high water mark set the amount of RAM consumed by items that must be breached before infrequently used active and replica items are ejected.
The following example sets the high water mark percentage to 80% of RAM for a specific bucket on a node. This means that items in RAM on this node can consume up to 80% of RAM before the item pager begins ejecting items.
cbepctl 10.5.2.117:11210 -b foo-bucket -p foo-password \ set flush_param mem_high_wat 80
Examples for setting percentage of ejected items
Based on the NRU algorithm, the server ejects active and replica data from a node. By default, the server is configured to 60% active items and 40% replica data from a node.
The following example increases the percentage of active items that can be ejected from a node to 50%.
Be aware of potential performance implications when changing the percentage of ejected items. It may be more desirable to eject as many replica items as possible and limit the amount of active data that can be ejected. By doing so, active data from a source node is maximized while maintaining incoming requests to that node. However, if the server is ejecting a very large percentage of replica data and a node fails, the replica data is not immediately available. In this case, the items are retrieved from disk and put back into RAM before the request is fulfilled.
cbepctl 10.5.2.117:11210 -b foo-bucket -p foo-password \ set flush_param pager_active_vb_pcnt 50
The following example response shows the low water mark, high water mark, and percentage of ejected items being set.
setting param: mem_low_wat 70 set mem_low_wat to 70 setting param: mem_high_wat 80 set mem_high_wat to 80 setting param: pager_active_vb_pcnt 50 set pager_active_vb_pcnt to 50