Run sudo nginx -t. It names the exact file and line that broke, which the systemd message never does. Fix that line, then sudo systemctl reload nginx. In roughly nine cases out of ten nginx -t is the whole job.
$ sudo nginx -t nginx: [emerg] unknown directive "proxy_pas" in /etc/nginx/sites-enabled/app:14 nginx: configuration file /etc/nginx/nginx.conf test failed
File, line number, and the offending token. That is the information systemd withheld from you.
Why systemd refuses to tell you anything
This message is not an nginx error at all. It is systemd reporting that the process it started returned a non-zero exit code. systemd has no idea what nginx objected to, and no interest in finding out.
nginx parses its entire configuration before it binds a socket or forks a single worker. When the parse fails it writes the reason to stderr and exits with status 1, all of it before systemd would consider the unit started. systemd sees one integer. It prints its generic line about a control process and stops there. The reason went to the journal instead.
So the error message is technically accurate and practically useless, and the fix is always to go ask nginx directly.
The 60-second triage
Three commands, in this order. Most people run them in exactly the wrong order and start with the least informative one.
# 1. the reason, in one line sudo nginx -t # 2. the reason, with context, if step 1 was clean sudo journalctl -xeu nginx.service --no-pager | tail -40 # 3. only for the exit code itself systemctl status nginx
systemctl status is last for a reason: it truncates long lines, and the nginx error you need is usually the part it cut off.
What nginx -t will tell you, and what each one means
These are the messages that account for most of this failure. Match the text, apply the fix.
unknown directive "...": a typo, or a directive belonging to a module that is not compiled in. The line number is exact. Check spelling first; if the spelling is right, the module is missing.open() "/var/log/nginx/app/access.log" failed (2: No such file or directory): nginx creates log files but never the directories above them. Create the directory and retry:sudo mkdir -p /var/log/nginx/app sudo chown www-data:adm /var/log/nginx/app
cannot load certificate "/etc/letsencrypt/live/site/fullchain.pem": almost always a Certbot renewal that removed a lineage while a vhost still referenced it. Check whether the symlink target still exists withls -lon the path in the error. A danglinglive/symlink is the classic version of this.host not found in upstream "api": nginx resolves everyproxy_passhostname at startup, not per request. If the upstream is a container or a service that boots after nginx, nginx dies. This is the one that makes nginx fail on reboot but start fine by hand ten seconds later. Fix by pointing at an IP, or by setting aresolverand using a variable inproxy_passso resolution is deferred to request time.duplicate default server for 0.0.0.0:80:default_serverappears in two enabled vhosts. Usually the stockdefaultsite plus your own. Remove the flag from one, or unlink the stock site.dlopen() ".../ngx_http_image_filter_module.so" failed: aload_moduleline surviving a package upgrade that moved or dropped the module. Extremely common after a distro release upgrade.
When nginx -t passes but the service still will not start
A clean config test narrows things considerably. nginx -t parses the config, opens log files, loads certificates and resolves upstreams: so if it passed, none of those are your problem. What it does not do is bind a socket, and it does not run under the unit’s sandboxing. That leaves a short list:
- Something already holds port 80 or 443. The single most common cause in this category, because
nginx -tnever attempts the bind:sudo ss -tlnp | grep -E ':(80|443) '
Apache on a repurposed server, or a stale nginx master that survived a bad shutdown, are the two usual answers. The journal shows
bind() to 0.0.0.0:80 failed (98: Address already in use). - SELinux is refusing a non-standard port. On RHEL, Rocky and Alma, listening outside the default HTTP port set fails until SELinux is told:
sudo semanage port -a -t http_port_t -p tcp 8085
- A unit drop-in is sandboxing a path nginx needs.
ProtectSystem=strict,ReadOnlyPathsorPrivateTmpin a drop-in will block writes that succeed when you run nginx by hand. Inspect the effective unit withsystemctl cat nginx: it shows the unit plus every drop-in applied to it. - You ran
nginx -tas root but the unit runs as someone else. Permission errors invisible to the test then appear at start. The journal names the path.
Reading the exit code
systemctl status prints a status number that narrows the cause before you read a single log line. This table applies to any systemd unit, not just nginx.
status=1/FAILURE: the process ran and rejected its own configuration. Nearly all nginx cases. Go tonginx -t.status=127: command not found. TheExecStartpath does not exist; usually a package removed or a path changed.status=203/EXEC: the binary exists but could not be executed: wrong permissions, missing execute bit, or a bad interpreter line.status=200/CHDIR: the unit’sWorkingDirectorydoes not exist.status=217/USER: theUser=in the unit does not exist on the system.status=226/NAMESPACE: sandboxing directives failed to set up. Points straight at a drop-in.status=137: killed by SIGKILL (128 + 9). Not a config problem at all; this is normally the OOM killer. Checkdmesg -T | grep -i oom.
How to stop it happening again
Test before you apply, and reload instead of restart. nginx -t exits non-zero on a bad config, so chaining on && means a broken config never reaches the running server:
sudo nginx -t && sudo systemctl reload nginx
reload keeps existing workers serving traffic while the new config comes up, so a mistake costs nothing. restart tears the server down first, which is how a one-character typo becomes real downtime. Make the chained form your habit and this error stops being something you debug under pressure.
Related
- nginx rewrite rule generator: builds redirect and rewrite blocks that pass
nginx -tfirst time. - logrotate config generator: for the log directories in cause 2, and for keeping them from filling the disk.
- systemd timer and crontab converter: for turning cron entries into systemd units without meeting this same error on the timer.
Does this apply to services other than nginx?
Yes. The wording belongs to systemd, not nginx, so the identical line appears for Apache, MySQL, PostgreSQL, Docker and every other unit. Only the diagnostic command changes: apache2ctl -t for Apache, the error log for MySQL. The exit-code table above applies unchanged. The method is always the same: ignore the systemd line and go find what the service itself said.
nginx -t says the config is fine, but the site is still down
Then nginx is probably running and the fault is downstream. Confirm with systemctl is-active nginx. If it reports active, this error is not your problem any more: look at the upstream application, the firewall, or DNS. A 502 with nginx running is a different failure from nginx refusing to start.
Why does it fail at boot but start fine manually?
Ordering. At boot, nginx may start before the network is fully configured or before a container it proxies to exists, so listen on a specific IP fails or an upstream hostname will not resolve. By the time you log in and start it by hand, everything it needed is up. Add After=network-online.target plus Wants=network-online.target via a drop-in, or remove the startup-time dependency using the resolver approach in cause 4.
Leave a Reply