Cluster Recovery Add-On

Database Cluster Recovery Add-On

The Database Cluster Recovery add-on provides free diagnostic and automated recovery tools for supported MariaDB, MySQL, and Percona database clusters. It helps identify replication, cluster-health, and service issues and can attempt to restore cluster operability automatically.

In this article

Supported Database Clusters

  • Primary-Secondary Cluster based on MySQL, MariaDB, or Percona.
  • Primary-Primary Cluster based on MySQL, MariaDB, or Percona.
  • Galera Cluster based on MariaDB.
  • XtraDB Cluster based on Percona.

Add-On Specifics

The add-on operates in two modes:

Diagnostic modeScans the database cluster and identifies potential problems.
Recovery modeUses the diagnostic information to try to resolve the detected issue.

During diagnostics, the add-on checks:

  • Replication for Primary-Secondary and Primary-Primary topologies.
  • Cluster size for Galera and XtraDB clusters.
  • Galera and XtraDB cluster status.
  • Database cluster services status.

During installation, the add-on applies several changes required for validation and recovery operations.

Replica User

A dedicated replica user is added for cluster validation and is also used in the ProxySQL configuration. Its credentials are available through environment variables:

REPLICA_USER: repl-xxxxxx
REPLICA_PSWD: xxxxxxxxxxxx
Replica user credentials
The add-on creates a dedicated replica user and exposes its credentials through environment variables.

SSH Access Between Database Nodes

The add-on also configures SSH access between database nodes using a dedicated set of SSH keys. This connection is used to copy data with rsync during the restoration procedure.

Add-On Installation

The add-on is available through the platform Marketplace. The corresponding Database Cluster Recovery package can also be imported from its GitHub repository.

Database Cluster Recovery add-on in Marketplace
Install the Database Cluster Recovery add-on from Marketplace.
1

Open the add-on installation

Launch the Database Cluster Recovery package from Marketplace.

2

Provide installation parameters

In the confirmation window, provide:

  • User and Password: database administrator credentials.
  • Environment name: select the environment containing the required database cluster.
  • Nodes: choose the layer that contains the database cluster.
Database Cluster Recovery add-on installation
Provide database administrator credentials, environment name, and database layer.
3

Install the add-on

Click Install and wait a few minutes. After installation, the recovery package appears in the add-ons list for the selected database layer.

Database Cluster Recovery add-on actions
The installed add-on provides Cluster Diagnostic and Cluster Recovery actions.

Add-On Usage

The installed add-on provides two main actions:

Cluster DiagnosticDetects database-cluster problems by checking node accessibility and database consistency.
Cluster RecoveryAttempts to resolve common problems and restore cluster operability.

Run Cluster Diagnostic

Click Cluster Diagnostic. The dashboard returns one of two results:

  • Cluster is OK
  • Errors discovered
Database cluster diagnostic errors
The Cluster Diagnostic action reports whether the cluster is healthy or errors were discovered.

Review Recovery Logs

For detailed diagnostic information, click Show Logs. The add-on opens:

/var/log/db_recovery.log
Database recovery log
Use Show Logs to review the db_recovery.log file.

Run Cluster Recovery

If diagnostic errors are detected, click Cluster Recovery. The add-on attempts to resolve common cluster issues and restore operability automatically.

!

Create a backup first

The source documentation recommends creating a database backup before running the recovery operation.

Successful database cluster recovery
A successful recovery restores the cluster to an operable state.

If recovery fails or more detail is required, review db_recovery.log. The source documentation also points to the Manual Recovery guide for cases that are not resolved automatically.

Expected Result

The add-on diagnoses the supported database cluster, reports whether the cluster is healthy or contains errors, and can attempt an automated repair. Successful recovery returns the cluster to an operable state, while detailed actions and failures remain available in the recovery log.

Important Notes

  • The add-on supports Primary-Secondary, Primary-Primary, MariaDB Galera, and Percona XtraDB clusters.
  • Diagnostic checks differ according to cluster topology.
  • A dedicated replica user is created during installation.
  • SSH keys are configured between database nodes for rsync-based restoration operations.
  • Database administrator credentials are required during add-on installation.
  • Back up the database before starting Cluster Recovery.
  • Use /var/log/db_recovery.log for detailed diagnostics and recovery information.