PHP-FPM and Nginx logging

In this lesson you will configure Nginx and PHP-FPM to write per-site logs under your /sites layout, add a JSON access log for analysis, and set up Logrotate to rotate and reopen your new log files.

We've configured PHP and Nginx to serve our data from the per-site directory layout, however we haven't done anything with logging yet. This means that logs will be written to the standard /var/log locations on the system.

While this is not problematic, especially if you run just one or a couple of sites on your server, it may soon become a mess when you run more. Per-site logging will also allow you to enable and customize certain logs for specific sites, for example the PHP slow log.

Nginx

Let's start with Nginx. We'll leave our main access log format unchanged for now, but send it to a different location. We'll also set our error log to write to our per-site location and change its severity.

We can do this using the access_log and error_log directives in our server block for our specific site. Mine is in /config/nginx/uncached.org.conf:

server {
    # listen, server_name, etc.

    access_log /sites/uncached.org/logs/access.log;
    error_log /sites/uncached.org/logs/error.log warn;

    # rest of server block
}

The available error_log severity settings are:

debug: very verbose information about the request, including module-specific output. Requires compiling Nginx with --with-debug.
info: informational messages about normal events, like worker processes starting, reloading, reopening logs.
notice: significant but expected conditions, such as configuration reloads, signals and graceful shutdowns.
warn: non-critical warnings, such as when an upstream server is temporarily unavailable.
error: a recoverable error, such as "connection refused" from upstream, "file not found" and others.
crit: a critical error, such as disk IO problems, internal consistency and data corruption errors.
alert: errors that require immediate attention, such as memory allocation failures.
emerg: an emergency which causes Nginx to exit, crash or not start, for example inability to bind to a configured port.

The default severity is set to error but I like to use warn with my WordPress sites for more information about upstreams (such as PHP-FPM).

Log format

Nginx has the ability to write logs in different formats, provided by the log_format directive. This allows us to customize the log output and add things like the upstream_response_time to see how long PHP took to process the request.

It also allows us to log various information coming from Cloudflare too, such as the request ID, or as they call it, Ray ID. It can help us tag a request all the way through, which may be helpful when diagnosing performance issues.

While it is possible to alter the access.log format, I do like to retain the original format for that file. It allows me to use various existing scripts and tools to analyze Apache and Nginx log files. It also allows me to use JSON for my custom logs.

The log_format must be defined in the http context, so we're going to use our /config/nginx/nginx.conf file for that:

http {
    log_format json escape=json
        '{'
            '"time_local":"$time_local",'
            '"remote_addr":"$remote_addr",'
            '"cloudflare_ray_id":"$http_cf_ray",'
            '"cloudflare_country":"$http_cf_ipcountry",'
            '"upstream_response_time":"$upstream_response_time",'
            '"request":"$request",'
            '"status":"$status"'
        '}';

    # ...
}

We're building a JSON object here and should be very careful about single quotes, double quotes, commas, and especially that last ;, which if omitted may still result in a valid configuration, but add part of your config file (like "gzipon" for example) to every log line.

There are a lot more variables which you can add to Nginx logs, request headers coming from Cloudflare, and response headers coming from PHP-FPM or any other upstream.

After the format has been defined, we can use it in our server context for our future WordPress site:

server {
    access_log /sites/uncached.org/logs/access.log;
    access_log /sites/uncached.org/logs/access.json.log json;
    error_log /sites/uncached.org/logs/error.log warn;

    # ...
}

Reload Nginx using:

sudo systemctl reload nginx.service

Then try visiting your test.php page via Cloudflare while watching your access logs, using tail (use Ctrl+C to stop watching):

cd /sites/uncached.org
tail -f logs/access.json.log

Given that this log file is in JSON format, we can also use one of my favorite utilities jq. For example, here's how to fetch the status, request ID and URL of non-200 requests:

jq 'select(.status != "200") | { status, request, cloudflare_ray_id }' access.json.log

Access log jq

We'll do a lot more of this in the Monitoring and Maintenance modules of this course.

PHP

The default PHP logging configuration on Ubuntu and other Debian-based distributions is quite minimal. We only have a global master process log at /var/log/php8.3-fpm.log. The default logging location is not set, and will default to writing to stderr, which will end up in our Nginx error log.

We can change this per site in our PHP-FPM pool configuration. I'll add the following directives to my configuration in /config/php/uncached.org.conf:

php_admin_value[error_reporting] = E_ALL
php_admin_value[error_log] = /sites/uncached.org/logs/php-errors.log
php_admin_flag[log_errors] = on

Reload the PHP-FPM service:

sudo systemctl reload php8.3-fpm.service

Now try calling an undefined function in your test.php file. The fatal error will end up in your logs/php-errors.log file along with the stack trace.

Runtime configuration

PHP allows us to set certain configuration at runtime, and logging is one of them. This allows us and applications like WordPress to alter the behaviour of what is logged and where, for example using the WP_DEBUG constant.

While this is quite flexible, it can easily create situations where we lose our logging altogether. For example, I've recently seen a WordPress plugin which pipes error messages to a Slack channel or direct message. By default, this plugin changes the log_errors and error_reporting settings, causing the messages to only be written to that Slack channel or private message.

To avoid this, we use php_admin_value and php_admin_flag to set our settings. This tells PHP that these settings have been set by an admin (systems admin, not WordPress admin) and may not be changed using php_ini() or other runtime functions.

Even if we explicitly set WP_DEBUG_LOG to false in our WordPress configuration, that setting will not override our admin settings. However, we can still use display_errors if we want error messages to appear in the browser for debugging.

PHP slow log

PHP-FPM has a slowlog feature, which allows us to separately log requests that take longer than a certain threshold. We can enable this in our pool configuration:

slowlog = /sites/uncached.org/logs/php-slow.log
request_slowlog_timeout = 5s

Reload the PHP-FPM service, add something slow to your test.php file, and run the request. You should see a new entry in the php-slow.log file with the script filename and a little stack trace too, which may help identify slow themes and plugins in a WordPress install.

The request_slowlog_timeout directive sets the threshold at which things end up in this log. My typical baseline is 5 seconds, but could be slightly higher for heavier WooCommerce and LearnDash sites, especially around the dashboard.

Logrotate

Now that we have a handful of new log files, we need to make sure they don't outgrow our filesystem over time. I covered logrotate in a previous module.

Let's add a new configuration for our Nginx and PHP logs in /config/misc/logrotate.sites:

/sites/*/logs/*.log
{
    rotate 14
    daily
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
        invoke-rc.d nginx rotate
        invoke-rc.d php8.3-fpm reopen-logs
    endscript
}

The rotate and reopen-logs helpers are available for Nginx and PHP-FPM respectively in the Ubuntu 24.04 official packages. If you're working with a different system, you may lack these helpers and might need to signal Nginx and PHP-FPM (search for signals) directly.

Symlink the new configuration to /etc/logrotate.d/sites:

sudo ln -sfn /config/misc/logrotate.sites /etc/logrotate.d/sites

Don't forget to add this symlink to your bin/symlinks.sh script for future use. To make sure rotation works as expected, we can force it with logrotate (you can use -d for a dry-run):

sudo logrotate -f /etc/logrotate.conf

Don't forget to add the new file, along with all other configuration changes in this lesson to your configs Git repository.

Enroll

Enjoying the course content? Enroll today to keep track of your progress, access premium lessons and more.