Skip to end of metadata
Go to start of metadata

As part of the Xpand Installation process you made some configuration selections which are written to clxnode.conf. Post-installation, you can modify the configuration by editing the file on each node while ensuring that the options specified are the same (other than node specific items like BACKEND_ADDR). 

The clxnode.conf is stored on each node in /etc/clustrix/ and options specified during installation are written to this file. The following table summarizes these options and provides information on how they can be modified post-installation.

The steps outlined in this document apply to most installations of Xpand, but do not apply to deployments based on the Xpand AWS AMI.

Xpand requires that all nodes have identical configurations (other than node specific items like BACKEND_ADDR). Modifications to any of the options described below must be made on every node in the cluster. Values commented out with a # will have no effect on the database. Changes to the configuration requires a restart of Xpand services (see below) to take effect.

Default Options for clxnode.conf

Use standard bash conventions for clxnode.conf. The file should contain variable definitions, comments, and blank lines only. Option and value settings should not contain any extraneous spaces. For example CPU_CORES=4

OptionDescriptionDefault ValueTo Modify
NODE_MEMORYMemory to use for Xpand, in MiBauto detected
  1. On every node in your cluster edit the clxnode.conf file and uncomment (if commented out) the line #NODE_MEMORY and change the value to the desired amount.

  2. Stop Xpand, restart hugetlb, and then Start Xpand on all nodes simultaneously by running the following command from a single node:
    clx -P cmd 'systemctl stop clustrix; systemctl restart hugetlb; systemctl start clustrix'

MAX_REDOMaximum Xpand Redo Space, in MiB1024Only modify this variable with guidance from Xpand Support. 
CPU_CORESNumber of CPU cores Xpand will useAll
  1. Edit the clxnode.conf file for every node in your cluster to the desired value.
  2. Stop/Start all Xpand services.

For additional info, please see section below on setting CPU_CORES.

DATA_PATHXpand Database Storage Path/data/clustrix
  1. Stop all Xpand services.
  2. On each node:
    1. Move the directory to the desired location.
    2. Modify the value of DATA_PATH in clxnode.conf to reflect the new location.
  3. Start all Xpand services.
DATA_ALLOCATEStorage space to pre-allocate to Xpand80% of free space on $DATA_PATH volume
 This value here is used for initialization only. To allocate more space on a node that already has Xpand installed, see ALTER CLUSTER RESIZE DEVICES
LOG_PATHXpand Logs Path/data/clustrix/log
  1. Stop all Xpand services.
  2. On each node:
    1. Move the directory to the desired location.
    2. Create a simlink in /data/clustrix/ pointing to the new log folder.
      Example: ln -s /log /data/clustrix/log/
  3. Start all Xpand services.
UI_LOGDIRXpand WebUI Logs Path/data/clustrix/log/clustrix_ui
  1. Stop all Xpand services.
  2. On each node:
    1. Move the directory to the desired location.
    2. Create a simlink in /data/clustrix/log pointing to the new log folder.
      Example: ln -s /log/clustrix_ui /data/clustrix/log/clustrix_ui
  3. Start all Xpand services.
UI_CACHEDIRXpand WebUI Cache Path/var/cache/clustrix/djangoOnly modify this variable with guidance from Xpand Support.
UNIX_SOCKET_PATHXpand MySQL Protocol Unix Socket/var/lib/mysql/mysql.sockOnly modify this variable with guidance from Xpand Support.
LISTEN_ADDRXpand Database Listen Address (Front-End IP) Interface0.0.0.0
MYSQL_PORTXpand Database MySQL Port3306
BACKEND_ADDRXpand Private Back-End IP Interfaceauto detectedSee section below on changing BACKEND_ADDR.
BACKEND_PORTXpand Back End Network Port24378It is not recommended to modify the default value for the Xpand Back End Network port.
HEALTHMON_PORTThe port used for the Xpand health monitor.3581
  1. Edit value in clxnode.conf
  2. Stop/Start all Xpand services.
HTTP_PORTXpand WebUI HTTP Port80
  1. Edit value in clxnode.conf
  2. Stop/Start all Xpand services.
NANNY_PORTXpand Nanny Port2424It is not recommended to modify the default value for the Xpand nanny port.
CONTROL_PORTXpand Control Port2048It is not recommended to modify the default value for the Xpand control port.
CLX_USERXpand OS UserrootDo not modify this value after installation.
MANAGE_USERXpand Manager UserrootDo not modify this value after installation.
WRITE_HOSTSAllow Xpand to modify sshd_config and /etc/hosts.

This used by root installations only, for internode communication for administrative tasks, including upgrades.

#Commented out
  1. On each node: 
    1. Remove comment and set variable to any string.
    2. Stop/Start all Xpand services.
HUGE_TLB_ENABLE

Xpand enables HugeTLB memory allocation for faster startup. NOTE: This causes instability on some systems (particularly certain virtual machines).

Required for non-root installations that have more than 20GiB of $NODE_MEMORY

Contact Xpand Support before changing from default.

True
  1. On each node, comment out to disable this option. (i.e. #HUGE_TLB_ENABLE)
  2. Stop/Start all Xpand services.

Additional options for clxnode.conf

The following options are NOT included by default in clxnode.conf, but can be added to the end of that file to modify behavior from the default. Xpand must be restarted for changes to take effect. 

OptionDescriptionDefault ValueTo Modify 
DISABLE_MULTIPORTWhen this option is present, the multiport feature is disabled.This option is not included in clxnode.conf, resulting in multiport being enabled by default.

To disable multiport, add an entry for DISABLE_MULTIPORT = Y.


To enable multiport, remove or comment out this entry from clxnode.conf on all nodes.

MULTIPAGE_ALLOCBytes reserved for multi-page allocators.The option is not included clxnode.conf, resulting in a value of 1GiBAdd this entry and define the number of bytes that Xpand Support recommends for your workload.
USE_DNS_HOSTNAMESEnables node cluster membership tracking by hostname, instead of IPThe option is is not included in clxnode.conf, resulting in using IPs to track cluster membership. Add this entry and set it to True

Details of Modifying Startup Configuration Options

Xpand requires that all nodes must have the same values in clxnode.conf

Stop/Start all Xpand Services

To apply the changes that have been made to clxnode.conf, the Xpand services on all the nodes will need to be restarted:

shell> /opt/clustrix/bin/clx dbrestart
            

CPU_CORES

The default setting of CPU_CORES=All will make use of all available cores in the system. If you would like to limit the number of cores utilized by Xpand you must modify the value for CPU_CORES to specify the number of cores that should be used. Xpand licensing will base its enforcement on the number of cores being utilized in the system as specified by CPU_CORES

BACKEND_ADDR

  1. Modify the BACKEND_ADDR entry in the /etc/clustrix/clxnode.conf file with the new IP.
  2. Modify /data/clustrix/broadcast_nodes file with the new IP addresses on all nodes.
    1. Note: Specifying port 24378 is not necessary unless you're using a non-default configuration.
  3. Run the clx command to ensure it has cached the IP address of each node.

    shell> /opt/clustrix/bin/clx cmd 'clx cache'      
  4. Stop all Xpand services (per above).
  5. Change the backend interface IP address on each node.
  6. Modify iptables as needed.
  7. Start all Xpand services (per above).
  • No labels