Job for postgresql.service failed because the control process exited with error code

Read the cluster log, not the systemd message. On Debian and Ubuntu that is /var/log/postgresql/postgresql-16-main.log, on RHEL it is /var/lib/pgsql/data/log/. PostgreSQL writes the real reason there and exits, and systemd only ever sees the exit code.

sudo tail -40 /var/log/postgresql/postgresql-16-main.log
sudo journalctl -xeu postgresql@16-main.service --no-pager | tail -40

Check the unit name first

This is the step that wastes the most time on Debian and Ubuntu, and it is specific to PostgreSQL. There are two units. postgresql.service is an empty wrapper that starts nothing itself; the real work happens in a per-cluster unit named postgresql@16-main.service. Running systemctl status postgresql frequently shows a green, happy wrapper while the actual cluster is dead.

# list every cluster and its real state
pg_lsclusters

# then target the cluster unit, not the wrapper
sudo systemctl status postgresql@16-main.service

pg_lsclusters is the fastest diagnostic on this platform: it shows version, port, status and log path for every cluster in one line each. RHEL family systems use a single postgresql-16.service and do not have this split.

The causes, in order of how often they appear

  1. Data directory permissions. PostgreSQL refuses to start if the data directory is group or world readable. This catches people constantly after a restore or a manual copy:
    sudo chown -R postgres:postgres /var/lib/postgresql/16/main
    sudo chmod 700 /var/lib/postgresql/16/main

    The log says data directory has invalid permissions and permissions should be u=rwx.

  2. A stale postmaster.pid. After an unclean shutdown or a host that lost power, the lock file survives and PostgreSQL will not start while it believes another instance is running. The log line is lock file "postmaster.pid" already exists. Confirm no postgres process is actually running with ps aux | grep postgres, then remove the file.
  3. A syntax error in postgresql.conf or pg_hba.conf. The log names the file and line. Note that a bad pg_hba.conf blocks startup entirely, not just individual connections, which surprises people who expect it to only affect authentication.
  4. Port 5432 already bound. Nearly always a second cluster, very often an older major version left behind by an upgrade. pg_lsclusters shows both and their ports. Check with sudo ss -tlnp | grep ':5432 '.
  5. The disk is full. PostgreSQL stops rather than risk corruption. Check df -h and df -i both, since exhausted inodes present exactly like exhausted bytes. A runaway WAL directory is the usual reason, often because a replication slot was left behind and is pinning WAL that can never be recycled.

Reading the exit code

status=1/FAILURE means the postmaster started and rejected something, so go to the cluster log. status=137 means SIGKILL and normally the OOM killer, which for PostgreSQL usually means shared_buffers or work_mem multiplied by concurrent connections exceeded available memory. Confirm with dmesg -T | grep -i oom. The full table, which applies to any systemd unit, is in the nginx version of this error.

How to stop it happening again

PostgreSQL can check its own configuration without starting, which catches cause 3 before it costs you a restart:

sudo -u postgres /usr/lib/postgresql/16/bin/postgres -C shared_buffers -D /var/lib/postgresql/16/main

If the configuration is invalid that command fails and prints the reason; if it is valid it prints the setting and exits without starting a server. For pg_hba.conf changes specifically, prefer sudo systemctl reload postgresql@16-main over a restart: a reload re-reads authentication rules without dropping connections, and a bad file leaves the running server untouched instead of refusing to come back.

Related

systemctl status postgresql says active but I cannot connect

That is the wrapper unit reporting success while the cluster behind it is down. On Debian and Ubuntu, postgresql.service starts nothing by itself. Run pg_lsclusters to see the true state, then act on postgresql@16-main.service instead. This is the single most misleading thing about PostgreSQL on these distributions.

Can I delete postmaster.pid safely?

Only after confirming no postgres process is running. Check with ps aux | grep postgres. If a postmaster is genuinely alive and you remove its lock file, you risk a second instance starting against the same data directory, which can corrupt the cluster. If no process exists, the file is a leftover from an unclean shutdown and removing it is the correct fix.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *