The Rebalancer is managed primarily through a set of global variables, and can be monitored through several system tables (or vrels). As described in the Rebalancer section, the rebalancer applies a number of actions such as copying replicas, moving replicas, and splitting slices in order to maintain an optimal distribution of data on the cluster. It is designed to perform these operations in a manner that minimizes impact to user queries, and requires little administrative action. However, there may be circumstances where you wish to either increase or decrease the aggressiveness of the rebalancer, such as quickly rebalancing the cluster after node addition or eliminating any possible interference with user queries during periods of heavy load.

...

Anchor
Rebalancer_Monitoring Rebalancer_Monitoring

Monitoring the Rebalancer

...

The table rebalancer_activity_log maintains a record of current and past rebalancer work. To see recent activity, order by started, as shown below. You can also filter for currently executing rebalancer actions with WHERE finished IS NULL.

...

For details such as target/destination for in-progress rebalancer actions, JOIN (using id) to rebalancer_activity_targets, rebalancer_copy_activity, or rebalancer_splits.  These are vrels (virtual relations, as opposed to actual tables), and so are only populated for the duration of the activity. 

...

Configuring the Rebalancer

The aggressiveness of the rebalancer is controlled by several global variables. 

• The number of concurrent rebalancer actions, (rebalancer_global_task_limit), applies : specifies the number of concurrent rebalancer actions, applicable to all rebalancer actions.
• The frequency of tasks for a given rebalancer action is controlled by globals like task_rebalancer_%_interval_msrebalancer_: defines the interval time of when a particular rebalancer task is initiated.
• rebalancer_rebalance_task_limit: controls the number of concurrent rebalancing tasks permitted.
• rebalancer_vdev_task_limit limits : limits the number of concurrent rebalancer actions that touch a single device  .

The frequency of the tasks determine how often operations, such as rebalancer moves, may be enqueued. When there are many small containers, the copies and moves take only a few seconds. As such, a default frequency of 30 seconds may mean that the rebalancer queues operations less frequently than it could. Most rebalancer tasks enqueue a limited number of operations at a time, as the required operations to achieve ideal balance change over time. The notable exception is SOFTFAIL, which enqueues all work to be performed once a node or disk has been softfailed.

...

If these settings cause too great a load, reduce the rebalancer_rebalance_task_limit or rebalancer_vdev_task_limit.

Once the rebalancer has finished, reset these globals back to ClustrixDB's default:

...

If these settings cause too great a load, reduce the rebalancer_global_task_limit or rebalancer_vdev_task_limit.

Once the rebalancer has finished, reset these globals back to ClustrixDB's default:

sql> SET GLOBAL variable_name = DEFAULT; 

Disabling

...

the Rebalancer 

To disable the rebalancer, set each of the rebalancer task intervals to 0:

sql> set global task_rebalancer_rebalance_distribution_interval_ms = 0;
sql> set global task_rebalancer_rebalance_interval_ms = 0;
sql> set global task_rebalancer_split_interval_ms = 0; 

It is not necessary to reset the global related to Rebalancer:

sql> set global rebalancer_optional_tasks_enabled = false;

This disables the rerank, split, redistribute and rebalance tasks. The value for rebalancer_optional_tasks_enabled supersedes the values in the global variables used to configure the individual rebalancer tasks. 

The Rebalancer tasks for reprotecting data (task_rebalancer_reprotect_interval_ms) as it is only active in the event of a disk or node failure. Inadvertently leaving it disabled increases the chance of data loss, should a double failure occur.should never be disabled. 

Global Variables

The following global variables impact Rebalancer activity. Note that these variables do not apply to an individual sessions.

...