CodeWalkers is in beta

Server Statistics and Observability

Published Updated

Server statistics are the instrument panel for a web service. One gauge can show that the machine is busy, but a useful panel combines request traffic, errors, latency, resource pressure, and dependency behavior so an operator can decide what to check next.

The old pattern of running uptime from a public PHP page exposes too little context and creates a risky shell boundary. Current observability collects metrics, logs, traces, and browser timing through narrow instrumentation, then connects each signal to a question or response.

What Observability Measures

Metrics record numerical measurements that change over time. Request count, response duration, error rate, CPU use, memory pressure, disk space, active database connections, cache hit ratio, and queue depth all fit this signal. Prometheus stores metrics as time series with names, timestamps, and optional labels.

Logs record individual events together with operational context. A useful application log can include a timestamp, severity, service name, request ID, route, status code, deploy version, and a bounded error message. Keep credentials, session tokens, payment data, and unnecessary personal data out of the event.

{
  "level": "error",
  "service": "checkout",
  "request_id": "req_8414",
  "route": "/api/orders",
  "status": 503,
  "deploy": "2026.07.29.2"
}

Traces connect the work performed for one request across boundaries. A trace can show the application waiting on SQL, an external provider, or a queue. OpenTelemetry defines traces, metrics, and logs as telemetry signals and provides shared conventions for carrying context between instrumented components.

Each signal answers a different operational question during diagnosis. Metrics show when a pattern changed, logs explain individual events, and traces show where a request spent time. Shared request or trace identifiers let an operator move between them without guessing which events belong together.

Build a Useful Server Baseline

Begin with request rate, error rate, and latency for the service's important routes. Add deploy markers so a change in the panel can be compared with the code and configuration version that produced it. Then measure the dependencies that can slow or stop those routes, such as database connections, query duration, cache results, and queue backlog.

Prometheus counters and histograms support this baseline. A counter can track completed requests by route and status class, while a histogram records observations in buckets that can be used to estimate latency percentiles.

100 * (
  sum by (route) (
    rate(http_requests_total{status=~"5.."}[5m])
  )
  /
  sum by (route) (
    rate(http_requests_total[5m])
  )
)

histogram_quantile(
  0.95,
  sum by (le, route) (rate(http_request_duration_seconds_bucket[5m]))
)

The first query divides server errors by all requests and multiplies the result by 100 to show the recent error percentage for each route. The second estimates p95 latency, the boundary below which roughly 95 percent of observations fall. Keep route labels bounded to templates such as /orders/{id}; a raw customer or order ID creates a new time series for every value.

Interpret percentile measurements alongside the matching request counts. A low-traffic route can show a high p95 after one slow request, while a busy route can hide many affected users behind a small percentage. The panel should show enough volume to interpret the timing.

Measure Dependencies That Control Throughput

Application latency often begins in a dependency rather than the web process. A database panel should pair query duration with active and waiting connections, lock waits, transaction failures, and replication lag when replicas are part of the read path. Compare those measurements with the same route and deploy window shown in the request panel.

Caches need hit, miss, eviction, error, and origin-request counts. A high hit ratio can still hide stale or incorrectly shared responses, so track cache bypass reasons and purge events too. Queues need depth, oldest-job age, processing duration, retries, and permanently failed jobs. Queue depth alone cannot distinguish healthy traffic growth from workers that stopped processing.

app_database_connections_active
/
app_database_connections_limit

max(app_queue_oldest_job_age_seconds)
sum(app_queue_failed_jobs_total)

These example metric names compare database use with its limit and expose queue age and failures. Place those results beside request signals on the instrument panel so an operator can identify dependency pressure before the host-level gauges reach their limits.

Measure Host and Disk Pressure

Host measurements explain whether the runtime has enough capacity to do its work. Track CPU saturation, available memory, swapping, disk bytes and inodes, network errors, open file descriptors, and process restarts. Database and container limits may fail before the whole host reaches a visible ceiling, so include their configured limits and current use.

A private PHP health tool can inspect disk space without executing df. PHP provides disk_free_space() and disk_total_space() for a filesystem or disk partition.

<?php

function diskUsageForPath(string $path): array
{
    $free = disk_free_space($path);
    $total = disk_total_space($path);

    if ($free === false || $total === false || $total <= 0) {
        throw new RuntimeException("Disk space is unavailable.");
    }

    return [
        "free_bytes" => $free,
        "used_percent" => round((($total - $free) / $total) * 100, 1),
    ];
}

Keep this output behind authentication and authorization, and treat it as one current reading. Host monitoring should retain the historical series and send the alert. Watch both available bytes and inodes because a filesystem can reject new files after either resource runs out.

Set thresholds from observed growth and recovery time. An alert at 90 percent disk use may arrive too late if logs grow quickly and the person responding needs an hour. The instrument panel should warn while a safe action is still available.

Connect Server and Browser Timing

Server measurements stop before rendering, device work, and much of the network path. The Server-Timing response header can expose selected server durations to browser developer tools, which helps connect backend work with the request visible in the browser.

<?php

$started = hrtime(true);
$article = loadArticle($pdo, $slug);
$databaseMs = (hrtime(true) - $started) / 1_000_000;

header(sprintf("Server-Timing: db;dur=%.2f", $databaseMs));
echo renderArticle($article);

This example measures one database operation and reports its duration with a short db label. Keep SQL text, table names, internal hostnames, and user-specific values out of the header because the response recipient can see it.

Compare server timing with browser navigation and rendering data. If the server reports 80 milliseconds but the page becomes usable after two seconds, investigate transfer size, client-side work, fonts, images, and device constraints instead of tuning the database first.

Turn Signals into Decisions

Every alert needs a condition, duration, severity, owner, and first action. A single failed request should not page an operator, while a sustained error rate on checkout may need an immediate incident. Record the threshold in the same units shown on the panel.

alert: CheckoutHighErrorRate
condition: 5xx rate above 5 percent for 10 minutes
severity: critical
owner: application-on-call
first_action: compare errors with the latest deploy
recovery_proof: checkout succeeds from two locations

The alert becomes useful because it tells the operator where to begin and how to confirm recovery. Link it to the server contingency plan when rollback, restore, cache bypass, credential rotation, or user communication may be required.

Review alerts after incidents and noisy weeks. Remove conditions that never lead to action, adjust thresholds that trigger too late, and add missing context to the runbook. Keep the panel small enough that each gauge still has a decision attached to it.

Common Pitfalls & Debugging

High Cardinality Exhausts Storage

Symptom: the metrics system creates an unexpected number of time series and consumes excessive memory or disk. Cause: labels contain unbounded values such as user IDs, request IDs, or raw URLs. Fix: keep those values in logs and use bounded route templates and status classes as metric labels.

Averages Hide Slow Requests

Symptom: average latency looks healthy while some users wait much longer. Cause: fast requests pull the mean down and hide the slow tail. Fix: record a latency distribution and inspect p50, p95, and p99 beside request counts.

Alerts Have No Runbook

Symptom: an alert fires, but the recipient cannot identify the affected route, deploy, or first check. Cause: the alert reports a threshold without operational context. Fix: include labels and annotations that point to the relevant panel, owner, and tested first action.

A Shell-command Endpoint Leaks Host Details

Symptom: a diagnostics route exposes command output, hostnames, process data, or environment details. Cause: a web request executes a broad shell command and returns raw output. Fix: remove the public endpoint and collect narrow measurements through instrumented code or the host monitoring system.

Conclusion

Useful server statistics explain a decision instead of filling a dashboard. Build the instrument panel from bounded metrics, structured events, connected request context, and browser timing, then attach every important alert to an owner and tested response.

Frequently Asked Questions

What is the difference between monitoring and observability?

Monitoring checks known conditions through selected metrics, logs, and alerts. Observability describes how well the system's outputs let you investigate both known and unexpected behavior. Monitoring is an activity within the wider goal of making a system understandable.

Does a small site need distributed tracing?

A single-process site can usually begin with request metrics and structured logs that share a request ID. Add tracing when one request crosses services, queues, databases, or providers and the existing signals cannot show where time or failure occurs.

How often should server metrics be collected?

Prometheus defaults its global scrape interval to one minute, and OpenTelemetry's periodic exporting MetricReader defaults to 60 seconds. Treat those defaults as starting points, then choose an interval that detects important changes without excessive cost or noise and supports the alert's required response time.

Can Server-Timing replace real user monitoring?

Server-Timing cannot replace real user monitoring by itself. It exposes selected server measurements to browser tools without measuring rendering, input delay, device limits, or the complete network path. Use it alongside browser performance data to explain the server's part of the request.

Sources

  1. [1]
    OpenTelemetry Signals
    (opentelemetry.io)
  2. [2]
    OpenTelemetry Metrics SDK
    (opentelemetry.io)
  3. [3]
    Prometheus Overview
    (prometheus.io)
  4. [4]
  5. [5]
  6. [6]
  7. [7]
    Server-Timing
    (developer.mozilla.org)