Read the MariaDB error log, not the systemd message. Run sudo tail -40 /var/log/mysql/error.log on Debian and Ubuntu, or /var/log/mariadb/mariadb.log on RHEL, Rocky and Alma. MariaDB writes the real reason there and exits, and systemd only sees the exit code.
sudo tail -40 /var/log/mysql/error.log sudo journalctl -xeu mariadb.service --no-pager | tail -40
Note the log path does not match the service name on Debian family systems. The package is mariadb, the log directory is still mysql, and people waste time looking for a mariadb directory that was never created.
Validate the configuration first
MariaDB can parse its options without starting a server, which catches the single most common cause before you restart anything:
sudo mariadbd --help --verbose > /dev/null
An unknown option aborts with unknown variable 'name=value' naming the offender. This matters more on MariaDB than on most servers because the MySQL and MariaDB option sets have diverged. Configuration copied from a MySQL 8 host frequently contains variables MariaDB never had, and configuration written for MariaDB often uses options MySQL removed. Either direction produces an abort on startup, and the file is usually one of the drop-ins under /etc/mysql/mariadb.conf.d/ or /etc/my.cnf.d/ rather than the main file people check first.
The causes worth checking, in order
- Data directory ownership. The daemon runs as the mysql user and refuses to start if it cannot write. This is the most common cause after a restore, a disk move, or any operation performed as root:
sudo chown -R mysql:mysql /var/lib/mysql sudo chmod 750 /var/lib/mysql
The log usually says
Can't create/write to fileor names a denied temporary file, with InnoDB failing immediately afterwards. - The system database was never initialised. If the data directory is empty, MariaDB has nothing to start. This happens when the directory was recreated manually or pointed somewhere new. Initialise it before starting:
sudo mariadb-install-db --user=mysql --datadir=/var/lib/mysql
- An unknown option in a drop-in file. Covered above. Check every file under the
conf.ddirectories, not just the main configuration. - The disk is full. Check
df -handdf -iboth. Exhausted inodes present exactly like exhausted bytes anddf -halone will look healthy. - Port 3306 already bound. Nearly always a leftover MySQL install on a host that was migrated to MariaDB, with both packages present:
sudo ss -tlnp | grep ':3306 '
Reading the exit code
status=1/FAILURE means the daemon started and rejected something, so go to the error log. status=137 is SIGKILL and for a database almost always the OOM killer: confirm with dmesg -T | grep -i oom, and expect it to recur until innodb_buffer_pool_size comes down or the host gets more memory. The full table, which applies to any systemd unit, is in the nginx version of this error.
How to stop it happening again
Validate before restarting. The check exits non-zero on a bad option, so chaining on && means a broken configuration never takes the server down:
sudo mariadbd --help --verbose > /dev/null && sudo systemctl restart mariadb
That catches unknown and malformed options, which is the one cause here you can detect in advance. It will not catch permissions, disk space or an uninitialised data directory, so keep the error log open in a second terminal whenever you restart after a change.
Related
- Job for mysql.service failed: the same failure on MySQL, where the option set differs.
- Job for nginx.service failed: the same systemd error, plus the full exit-code table.
- logrotate config generator: keep the error log from filling the disk and causing cause 4 above.
Why is the log in /var/log/mysql when the service is mariadb?
Debian and Ubuntu package MariaDB as a drop-in replacement for MySQL and kept the original paths so existing tooling keeps working. The unit is mariadb.service, the binary is mariadbd, but the log directory, the data directory and the system user are all still called mysql. On RHEL family systems the log is at /var/log/mariadb/mariadb.log instead.
I migrated from MySQL and now it will not start
Almost certainly an option MariaDB does not recognise, left behind in a configuration drop-in. Run the validation command above: it names the variable. The usual casualties are MySQL 8 options with no MariaDB equivalent. Comment the line out rather than deleting it, so you can see what was removed if something later depends on it.
Leave a Reply