Job for nginx.service failed because the control process exited with error code

Run sudo nginx -t. It names the exact file and line that broke, which the systemd message never does. Fix that line, then sudo systemctl reload nginx. In roughly nine cases out of ten nginx -t is the whole job.

$ sudo nginx -t
nginx: [emerg] unknown directive "proxy_pas" in /etc/nginx/sites-enabled/app:14
nginx: configuration file /etc/nginx/nginx.conf test failed

File, line number, and the offending token. That is the information systemd withheld from you.

Why systemd refuses to tell you anything

This message is not an nginx error at all. It is systemd reporting that the process it started returned a non-zero exit code. systemd has no idea what nginx objected to, and no interest in finding out.

nginx parses its entire configuration before it binds a socket or forks a single worker. When the parse fails it writes the reason to stderr and exits with status 1, all of it before systemd would consider the unit started. systemd sees one integer. It prints its generic line about a control process and stops there. The reason went to the journal instead.

So the error message is technically accurate and practically useless, and the fix is always to go ask nginx directly.

The 60-second triage

Three commands, in this order. Most people run them in exactly the wrong order and start with the least informative one.

# 1. the reason, in one line
sudo nginx -t

# 2. the reason, with context, if step 1 was clean
sudo journalctl -xeu nginx.service --no-pager | tail -40

# 3. only for the exit code itself
systemctl status nginx

systemctl status is last for a reason: it truncates long lines, and the nginx error you need is usually the part it cut off.

What nginx -t will tell you, and what each one means

These are the messages that account for most of this failure. Match the text, apply the fix.

  1. unknown directive "...": a typo, or a directive belonging to a module that is not compiled in. The line number is exact. Check spelling first; if the spelling is right, the module is missing.
  2. open() "/var/log/nginx/app/access.log" failed (2: No such file or directory): nginx creates log files but never the directories above them. Create the directory and retry:
    sudo mkdir -p /var/log/nginx/app
    sudo chown www-data:adm /var/log/nginx/app
  3. cannot load certificate "/etc/letsencrypt/live/site/fullchain.pem": almost always a Certbot renewal that removed a lineage while a vhost still referenced it. Check whether the symlink target still exists with ls -l on the path in the error. A dangling live/ symlink is the classic version of this.
  4. host not found in upstream "api": nginx resolves every proxy_pass hostname at startup, not per request. If the upstream is a container or a service that boots after nginx, nginx dies. This is the one that makes nginx fail on reboot but start fine by hand ten seconds later. Fix by pointing at an IP, or by setting a resolver and using a variable in proxy_pass so resolution is deferred to request time.
  5. duplicate default server for 0.0.0.0:80: default_server appears in two enabled vhosts. Usually the stock default site plus your own. Remove the flag from one, or unlink the stock site.
  6. dlopen() ".../ngx_http_image_filter_module.so" failed: a load_module line surviving a package upgrade that moved or dropped the module. Extremely common after a distro release upgrade.

When nginx -t passes but the service still will not start

A clean config test narrows things considerably. nginx -t parses the config, opens log files, loads certificates and resolves upstreams: so if it passed, none of those are your problem. What it does not do is bind a socket, and it does not run under the unit’s sandboxing. That leaves a short list:

  1. Something already holds port 80 or 443. The single most common cause in this category, because nginx -t never attempts the bind:
    sudo ss -tlnp | grep -E ':(80|443) '

    Apache on a repurposed server, or a stale nginx master that survived a bad shutdown, are the two usual answers. The journal shows bind() to 0.0.0.0:80 failed (98: Address already in use).

  2. SELinux is refusing a non-standard port. On RHEL, Rocky and Alma, listening outside the default HTTP port set fails until SELinux is told:
    sudo semanage port -a -t http_port_t -p tcp 8085
  3. A unit drop-in is sandboxing a path nginx needs. ProtectSystem=strict, ReadOnlyPaths or PrivateTmp in a drop-in will block writes that succeed when you run nginx by hand. Inspect the effective unit with systemctl cat nginx: it shows the unit plus every drop-in applied to it.
  4. You ran nginx -t as root but the unit runs as someone else. Permission errors invisible to the test then appear at start. The journal names the path.

Reading the exit code

systemctl status prints a status number that narrows the cause before you read a single log line. This table applies to any systemd unit, not just nginx.

  • status=1/FAILURE: the process ran and rejected its own configuration. Nearly all nginx cases. Go to nginx -t.
  • status=127: command not found. The ExecStart path does not exist; usually a package removed or a path changed.
  • status=203/EXEC: the binary exists but could not be executed: wrong permissions, missing execute bit, or a bad interpreter line.
  • status=200/CHDIR: the unit’s WorkingDirectory does not exist.
  • status=217/USER: the User= in the unit does not exist on the system.
  • status=226/NAMESPACE: sandboxing directives failed to set up. Points straight at a drop-in.
  • status=137: killed by SIGKILL (128 + 9). Not a config problem at all; this is normally the OOM killer. Check dmesg -T | grep -i oom.

How to stop it happening again

Test before you apply, and reload instead of restart. nginx -t exits non-zero on a bad config, so chaining on && means a broken config never reaches the running server:

sudo nginx -t && sudo systemctl reload nginx

reload keeps existing workers serving traffic while the new config comes up, so a mistake costs nothing. restart tears the server down first, which is how a one-character typo becomes real downtime. Make the chained form your habit and this error stops being something you debug under pressure.

Related

Does this apply to services other than nginx?

Yes. The wording belongs to systemd, not nginx, so the identical line appears for Apache, MySQL, PostgreSQL, Docker and every other unit. Only the diagnostic command changes: apache2ctl -t for Apache, the error log for MySQL. The exit-code table above applies unchanged. The method is always the same: ignore the systemd line and go find what the service itself said.

nginx -t says the config is fine, but the site is still down

Then nginx is probably running and the fault is downstream. Confirm with systemctl is-active nginx. If it reports active, this error is not your problem any more: look at the upstream application, the firewall, or DNS. A 502 with nginx running is a different failure from nginx refusing to start.

Why does it fail at boot but start fine manually?

Ordering. At boot, nginx may start before the network is fully configured or before a container it proxies to exists, so listen on a specific IP fails or an upstream hostname will not resolve. By the time you log in and start it by hand, everything it needed is up. Add After=network-online.target plus Wants=network-online.target via a drop-in, or remove the startup-time dependency using the resolver approach in cause 4.


Comments

9 responses to “Job for nginx.service failed because the control process exited with error code”

  1. […] systemctl status apache2 prints a status number that narrows things before you read a single log line. status=1/FAILURE means Apache ran and rejected its own configuration, which covers nearly every case here. status=137 means it was killed by SIGKILL and is usually the OOM killer rather than a config problem. The full table, which applies to any systemd unit, is in the nginx version of this error. […]

  2. […] status=1/FAILURE means the postmaster started and rejected something, so go to the cluster log. status=137 means SIGKILL and normally the OOM killer, which for PostgreSQL usually means shared_buffers or work_mem multiplied by concurrent connections exceeded available memory. Confirm with dmesg -T | grep -i oom. The full table, which applies to any systemd unit, is in the nginx version of this error. […]

  3. […] status=1/FAILURE means dockerd started and rejected its own configuration, which covers causes 1, 3, 4 and 6. status=137 is SIGKILL and usually the OOM killer. A code of 203/EXEC is worth knowing here specifically: it means the dockerd binary named in the unit does not exist or is not executable, which happens after a partial package upgrade or when a manually installed binary was removed. The full table is in the nginx version of this error. […]

  4. […] status=1/FAILURE means the daemon started and rejected something, so go to the error log. status=137 is SIGKILL and for a database almost always the OOM killer: confirm with dmesg -T | grep -i oom, and expect it to recur until innodb_buffer_pool_size comes down or the host gets more memory. The full table, which applies to any systemd unit, is in the nginx version of this error. […]

  5. […] status=255 is sshd’s own general failure code and normally means a configuration problem, so run sshd -t. status=1/FAILURE covers most other startup rejections. status=226/NAMESPACE points at a unit drop-in with sandboxing directives rather than at sshd_config at all: inspect it with systemctl cat ssh. The full table is in the nginx version of this error. […]

  6. […] status=14 is mongod’s own general failure code and means it started and rejected something, so go to the log. status=1/FAILURE covers configuration parse errors. A SIGILL or core dump with an empty log points at the AVX problem above rather than at anything you configured. status=137 is SIGKILL and normally the OOM killer. The full table is in the nginx version of this error. […]

  7. […] status=1/FAILURE means the daemon started and rejected something, so read the journal. status=203/EXEC means the binary named in the unit does not exist or is not executable, which happens after a partial upgrade. For NetworkManager-wait-online specifically, a timeout result rather than an exit code confirms it is the harmless case described above. The full table is in the nginx version of this error. […]

  8. […] status=1/FAILURE means the node started and stopped, so read the node log above rather than the journal, which usually only says the process exited. An activating (auto-restart) state that never settles means systemd keeps retrying a node that keeps dying, and the cause is the same. The full exit-code table is in the nginx version of this error. […]

  9. […] status=1/FAILURE means Grafana started and rejected something, so read its log. status=217/USER means the user in the unit does not exist, common when configuration management installs Grafana from a tarball rather than the package. status=203/EXEC means the binary path in the unit is wrong. The full table is in the nginx version of this error. […]

Leave a Reply

Your email address will not be published. Required fields are marked *