Job for mongod.service failed because the control process exited with error code

Read /var/log/mongodb/mongod.log, not the systemd message. mongod writes the real reason there and exits, and systemd only ever sees the exit code. The last twenty lines almost always name the cause outright.

sudo tail -40 /var/log/mongodb/mongod.log
sudo journalctl -xeu mongod.service --no-pager | tail -40

Check this first if the server is older or virtualised

MongoDB 5.0 and later require a CPU with AVX support. On an older physical server, a budget VPS, or a VM whose host masks CPU features, mongod dies instantly with Illegal instruction or a core dump, and frequently writes nothing useful to its log at all. The silence is the clue.

grep -o avx /proc/cpuinfo | head -1

No output means no AVX, and MongoDB 5.0 or newer will never start on that machine. The only fixes are installing MongoDB 4.4, moving to a host with a newer CPU, or enabling CPU passthrough on the hypervisor. This wastes entire afternoons because nothing in the error suggests hardware.

The other causes, in order

  1. A tab character in mongod.conf. The configuration file is YAML, and YAML forbids tabs for indentation. An editor that inserts a tab produces a parse error on startup, and the file looks perfectly aligned on screen. Find them with:
    grep -Pn '\t' /etc/mongod.conf
  2. Data directory ownership. mongod runs as the mongodb user and refuses to start if it cannot write. The usual trigger is a restore or a disk move performed as root:
    sudo chown -R mongodb:mongodb /var/lib/mongodb /var/log/mongodb

    On RHEL family systems the user and paths are mongod and /var/lib/mongo instead.

  3. An unclean shutdown left a lock file. The log names mongod.lock and refuses to proceed. If the host lost power or the process was killed, remove the lock only after confirming no mongod is running, then start again. If WiredTiger reports corruption rather than just a lock, take a filesystem copy before attempting any repair.
  4. The disk is full. MongoDB stops rather than risk the data files. Check df -h and df -i both, since exhausted inodes present exactly like exhausted bytes.
  5. Port 27017 already bound. Usually a second instance or a stale process: sudo ss -tlnp | grep ':27017 '.
  6. SELinux on RHEL family systems. Moving the data directory off the packaged path fails until the context is set. sudo ausearch -m avc -ts recent shows the denial if this is your cause.

Reading the exit code

status=14 is mongod’s own general failure code and means it started and rejected something, so go to the log. status=1/FAILURE covers configuration parse errors. A SIGILL or core dump with an empty log points at the AVX problem above rather than at anything you configured. status=137 is SIGKILL and normally the OOM killer. The full table is in the nginx version of this error.

How to stop it happening again

mongod can validate its configuration without starting a server, which catches the YAML problems before they cost you a restart:

sudo -u mongodb mongod --config /etc/mongod.conf --outputConfig

That parses the file and prints the resulting configuration without binding anything. A tab, a bad key or broken indentation fails here instead of taking the database down. It will not catch permissions, disk space or missing AVX, so keep the log open in a second terminal when restarting after a change.

Related

The log file is completely empty

An empty log is itself a diagnosis. It means mongod died before it could open the file, which narrows things to three causes: the log path in mongod.conf points somewhere unwritable, the mongodb user does not own the log directory, or the process was killed by the kernel before it ran, which on MongoDB 5.0 and later usually means the missing AVX instruction set described above.

Can I just delete mongod.lock?

Only after confirming no mongod process is running, with ps aux | grep mongod. If one is genuinely alive and you remove its lock, a second instance can start against the same data files and corrupt them. If no process exists, the file is a leftover from an unclean shutdown and removing it is correct. Take a filesystem copy of the data directory first either way.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *