Job for mariadb.service failed because the control process exited with error code

Read the MariaDB error log, not the systemd message. Run sudo tail -40 /var/log/mysql/error.log on Debian and Ubuntu, or /var/log/mariadb/mariadb.log on RHEL, Rocky and Alma. MariaDB writes the real reason there and exits, and systemd only sees the exit code.

sudo tail -40 /var/log/mysql/error.log
sudo journalctl -xeu mariadb.service --no-pager | tail -40

Note the log path does not match the service name on Debian family systems. The package is mariadb, the log directory is still mysql, and people waste time looking for a mariadb directory that was never created.

Validate the configuration first

MariaDB can parse its options without starting a server, which catches the single most common cause before you restart anything:

sudo mariadbd --help --verbose > /dev/null

An unknown option aborts with unknown variable 'name=value' naming the offender. This matters more on MariaDB than on most servers because the MySQL and MariaDB option sets have diverged. Configuration copied from a MySQL 8 host frequently contains variables MariaDB never had, and configuration written for MariaDB often uses options MySQL removed. Either direction produces an abort on startup, and the file is usually one of the drop-ins under /etc/mysql/mariadb.conf.d/ or /etc/my.cnf.d/ rather than the main file people check first.

The causes worth checking, in order

  1. Data directory ownership. The daemon runs as the mysql user and refuses to start if it cannot write. This is the most common cause after a restore, a disk move, or any operation performed as root:
    sudo chown -R mysql:mysql /var/lib/mysql
    sudo chmod 750 /var/lib/mysql

    The log usually says Can't create/write to file or names a denied temporary file, with InnoDB failing immediately afterwards.

  2. The system database was never initialised. If the data directory is empty, MariaDB has nothing to start. This happens when the directory was recreated manually or pointed somewhere new. Initialise it before starting:
    sudo mariadb-install-db --user=mysql --datadir=/var/lib/mysql
  3. An unknown option in a drop-in file. Covered above. Check every file under the conf.d directories, not just the main configuration.
  4. The disk is full. Check df -h and df -i both. Exhausted inodes present exactly like exhausted bytes and df -h alone will look healthy.
  5. Port 3306 already bound. Nearly always a leftover MySQL install on a host that was migrated to MariaDB, with both packages present:
    sudo ss -tlnp | grep ':3306 '

Reading the exit code

status=1/FAILURE means the daemon started and rejected something, so go to the error log. status=137 is SIGKILL and for a database almost always the OOM killer: confirm with dmesg -T | grep -i oom, and expect it to recur until innodb_buffer_pool_size comes down or the host gets more memory. The full table, which applies to any systemd unit, is in the nginx version of this error.

How to stop it happening again

Validate before restarting. The check exits non-zero on a bad option, so chaining on && means a broken configuration never takes the server down:

sudo mariadbd --help --verbose > /dev/null && sudo systemctl restart mariadb

That catches unknown and malformed options, which is the one cause here you can detect in advance. It will not catch permissions, disk space or an uninitialised data directory, so keep the error log open in a second terminal whenever you restart after a change.

Related

Why is the log in /var/log/mysql when the service is mariadb?

Debian and Ubuntu package MariaDB as a drop-in replacement for MySQL and kept the original paths so existing tooling keeps working. The unit is mariadb.service, the binary is mariadbd, but the log directory, the data directory and the system user are all still called mysql. On RHEL family systems the log is at /var/log/mariadb/mariadb.log instead.

I migrated from MySQL and now it will not start

Almost certainly an option MariaDB does not recognise, left behind in a configuration drop-in. Run the validation command above: it names the variable. The usual casualties are MySQL 8 options with no MariaDB equivalent. Comment the line out rather than deleting it, so you can see what was removed if something later depends on it.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *