ClustrixDB constantly self-monitors to ensure your cluster is healthy and operating optimally. When it detects conditions that require attention, ClustrixDB will send alerts via email using its Alerter. Alerts are of different severities (INFO, WARNING, ERROR, and CRITICAL) and ClustrixDB is preconfigured with default thresholds for each.

The contacts and communication details that control how alerts are sent must be configured for your cluster.

Configuring Alerts

Use the following steps to configure the alerts for your system:

Step 1. Set Identifying Global Variables

Set these identifying global variables for your database. These are especially important to aid Clustrix Support in troubleshooting.

sql> SET GLOBAL customer_name = 'customer name';
sql> SET GLOBAL cluster_name = 'cluster identifier';

Step 2. Configure alerts_parameters for SMTP Server 

The parameters defined in the system.alerts_parameters table control how alerts are formatted and sent. 

Set the following SMTP parameters as they apply to your cluster.

Follow this syntax to update the parameters shown:  

sql> UPDATE  system.alerts_parameters
     SET     value =  'your smpt-specific value'
     WHERE   name=  'parameter name';

Step 3. Configure alerts_subscriptions

Add email addresses of the individual(s) or group(s) who are to receive the alerts to the system.alerts_subscriptions table. You can insert, update, and delete from this table using standard SQL.

To see current list of alert subscriptions: 

sql> SELECT * FROM system.alerts_subscriptions;

To add a new email address:

sql> INSERT INTO system.alerts_subscriptions VALUES ('[email protected]_name.com');

Step 4. RESET Alerter

To reset the Alerter:

sql> ALTER CLUSTER RESET ALERTER;

This will not cause a group change on your cluster.

If invalid information is provided, you may encounter the following error:

sql> ALTER CLUSTER RESET ALERTER; 
ERROR 1 (HY000): [64512] Bad configuration for alerts:

Check clustrix.log for more information. Here is an example where the smtp_server parameter was not specified:

2018-10-11 21:07:51.068524 UTC karma068.colo.sproutsys.com clxnode: ERROR cluster/alerter.ct:219 prepare_write(): Couldn't write alerter config: Bad configuration for alerts: No smtp_server specified

Step 5. Request Alert

To verify that the configuration works properly, execute this SQL to send a test alert:

sql> SELECT alert(severity, 'alert text');

If you do not receive the expected email alert, please re-review your configuration. 

Sample Emailed Alerts

Here are some sample emailed alert messages that may be similar to some you could encounter on your cluster. These alerts will also appear in the query.log.

Sample 1: Database Space WARNING

This alert is a WARNING for a cluster with a device1 file that is at least 80% full. If you receive a similar warning, see “Issue Resolution” in Managing File Space and Database Capacity.

Sample 2: Backup INFO

This INFO alert shows that the backup has failed. If you receive similar errors during backup processing, please see List of Errors for Backup and Restore. This particular sample shows additional information that is available from clusters deployed in AWS.

Sample 3: Read ERROR

This ERROR alert indicates that your system’s disk is experiencing hardware failures. Contact Clustrix Support for suggestions.

Additional Information

Alert Severity Codes 

Alerting Conditions

These are the conditions that ClustrixDB monitors and for which alerts are issued. These alerts are predefined within the database (system.alerts_messages) and may not be changed. The severity of these alerts range from critical to simply informational.

Preconfigured alerts_parameters

These additional entries from the system.alerts_parameters table are pre-configured and shown here for information only.

Some of these parameters include “meta tags” to denote that metadata contents will be substituted in the alert content when that parameter is used. The meta tags are explained in the next section.

Metadata used in alerts_parameters  

The alert parameters sometimes contain metadata that is identified by “meta tags”. These meta tags cause real-time information to be substituted within a generated alert.

The following chart shows how each meta tag will be resolved whenever it is used.