The device1 file in Xpand is used for all database data, undo logs, temporary tables, Binlogs, and Xpand system objects. Xpand ensures that the device1 file is the same size on all nodes of the cluster. Once the optimum size of the device1 file is derived, it rarely requires change.

To increase the size of the device1 file, see ALTER CLUSTER RESIZE DEVICES.

To reduce the size of the device1 file cluster-wide, follow these instructions. This may be necessary when upgrading to Xpand 7.5 or greater on a cluster that is already at or near capacity or for workloads that utilize large sorts or aggregates.

These instructions are meant to be completed for one node at a time, but if you have space within your cluster, you may complete the cycle using more than one node at a time.

Step 1: Verify, Set Initial Global  

The global device_auto_resize_to_largest controls how device1 files are consistently sized throughout your cluster. Ensure that this is initially set to false so it does not interfere with this process.

Verify that the initial setting of device_auto_resize_to_largest is false
sql> SHOW GLOBAL VARIABLES LIKE 'device_auto_resize_to_largest';

Set the global to false, if necessary.

Run this only if needed
sql> SET GLOBAL device_auto_resize_to_largest = 'false';

Step 2: Softfail a Node

Softfail a node or nodes, if your cluster has enough free space.

Step 3: Remove Old device1 File

After the node(s) have been softfailed and removed from the cluster, run the following from a command line prompt. This will result in a group change.

These commands will format the node to its default state and remove all the data.

Run these commands on each node that was softfailed
shell> rm /data/clustrix/pnid
shell> systemctl stop clustrix
shell> systemctl start clustrix

Step 4: Re-add the Node to the Cluster

Use this syntax from a MySQL prompt to create a default device1 file. This will result in a group change.

Run this from a node other than the node(s) just softfailed            
ALTER CLUSTER ADD 'ip' [, 'ip'] ... 

For example:


Step 5: Resize device1

Resize the device1 file on each node added above by using the following control port command:                                          

Run this command on each node whose IP was added to the cluster            
shell> nc localhost 2048 
$ device resize deviceid size 

For example, this sets the size for the first (and only) device of the system, (device1), to 800 GiB on the node to which you are connected. Do this on each node added above. 

$ device resize 1 800g 

Xpand recommends leaving 50 - 100 GiB of free or work space available on each drive. This is site-specific and Xpand Support can provide guidelines for your specific workload.

Step 6: Rebalance Data

The Rebalancer will automatically distribute data to the reconfigured node(s). If your cluster has enough free space, you do not need to wait for this operation to finish before continuing. See Managing the Rebalancer for additional information.

Step 7: Repeat for All Nodes

Repeat steps 2-6 until all nodes of the cluster have had their device1 file decreased.

Step 8: Set Global for Future Use

Once the procedure is complete on all nodes and the device1 files have been reduced in size, set the global variable device_auto_resize_to_largest  to true. This ensures that all future nodes added to the cluster will have their device1 files sized the same.

 Do not set this global to true until all nodes in the cluster have completed this process.

Run this on one node after all device1 files have been decreased            
sql> SET GLOBAL device_auto_resize_to_largest = 'true';
  • No labels