Error Monitoring and Logging: A Practical Setup Guide

Error monitoring and logging is the infrastructure that lets you see what your production software is doing, and where it breaks, before a customer tells you. It has three parts: structured logs that record every request and important job, an error tracking tool that collects failures with their stack trace and context, and metrics with alerts that show system health in numbers. If all three are not set up together, the team only learns about a problem when a user writes "it does not work", and then has to guess what broke, when and for whom.

Logs, error tracking and metrics are not the same thing

These three are often confused, and once one is in place people assume the others are too. In reality each answers a different question, and none of them covers for the absence of another.

• Logs answer "what happened?". Every request, every payment attempt and every background job is written as one line with a timestamp and context. Looking back at an incident is done from the logs.
• Error tracking answers "what broke and how many people did it affect?". A thousand repeats of the same error are grouped into one issue, with the stack trace, release version and number of affected users.
• Metrics answer "is the system healthy right now?". Numbers such as response time, error rate, queue length and disk usage are tracked over time, and alerts are attached to those numbers.

The most common picture in practice: there are plain-text log files on the server that nobody reads and that get deleted when the disk fills up, there is no error tracking, and the only metric is the CPU graph in the hosting panel. That setup looks fine as long as nothing goes wrong and is useless during the first serious incident.

Structured logging: a log you cannot search is not a log

The first rule of logging is that logs are written for queries, not for humans. Free-text lines like "user 42 placed an order" look fine in a file but cannot answer "which orders failed at the payment step yesterday between 14:00 and 15:00?". In a structured log every line is a record with fields, usually JSON: level, message, time, request ID, user ID, operation name and duration each sit in their own field.

The second rule is the correlation ID. When a request enters the API it is given a single ID, and every log line written during that request, every internal service it calls and every job it puts on a queue carries that ID. A single request from a customer complaint then shows up as a forty-line trace across five services with one query. In a microservice architecture debugging without this ID is close to impossible, and even in a modular monolith queues and scheduled jobs create the same need.

import { randomUUID } from 'node:crypto';
import pino from 'pino';

export const logger = pino({
  level: process.env.LOG_LEVEL ?? 'info',
  redact: ['req.headers.authorization', '*.password', '*.cardNumber', '*.nationalId'],
});

// Correlation ID per request: reuse the incoming header, otherwise generate one
export function requestLogger(req, res, next) {
  const requestId = req.headers['x-request-id'] ?? randomUUID();
  req.log = logger.child({ requestId, path: req.path, method: req.method });
  res.setHeader('x-request-id', requestId);

  const started = Date.now();
  res.on('finish', () => {
    req.log.info({ status: res.statusCode, durationMs: Date.now() - started }, 'request completed');
  });
  next();
}

Two details in the example matter. The ID is also written to the response header, so if support gets that ID from a customer they go straight to the right trace. Sensitive fields are redacted at the logger level; instead of relying on every developer being careful on every line, the rule lives in one place.

What must never be logged

In most companies the log store is protected less than the database: more people can read it, it is shipped to third-party tools and it stays in backups for years. Personal data written to logs is therefore as serious an obligation under data protection rules as data in the database. Passwords, card numbers, session and access tokens, verification codes, national ID numbers and health data are never logged at any level. If email or phone is needed it is masked, and a user ID is logged instead of the person. The retention period is also decided up front: logs kept forever are both a cost and a legal risk. These are general principles, not legal advice; settle your own retention periods with your legal adviser.

Monitoring succeeds not by how many log lines it collects, but by the share of incidents you notice before a customer does.

Alert rules: not every error is an alert

The most common mistake after monitoring is set up is sending a notification for every error. In the first week the channel receives hundreds of messages, in the second week it gets muted, and in the third week a real outage arrives as a message nobody sees. An alert should be tied not to a single event but to a condition that affects users.

• Error-rate threshold: alert when more than a set percentage of requests in the last five minutes return 5xx; a single 500 is not an alert.
• New error type: notify when a never-seen error appears right after a release; this is the input to the rollback decision in your CI/CD pipeline.
• Critical flow: alert when failures at payment, checkout or sign-in rise above normal, even if the number is small.
• Silence: alert when a backup or sync job that should run every night has not logged "completed" by a set time. A job that does not run produces no error, so it can only be caught by watching for its absence.
• Resources: warn on disk usage, database connection pool and queue length based on the trend, before they run out.

The silence alert matters most: a data backup job that has not run for months is usually discovered only when a restore is needed. Every alert needs an owner and a short written step for "what to do when this fires"; an alert without an owner is noise.

Hosted tool or your own setup?

Error tracking has hosted SaaS tools as well as open-source alternatives you run on your own servers, and the same split applies to logs and metrics. The decision depends on three questions: where can the data live (if personal data is going to a tool abroad, the legal side needs separate review), how large is the volume (hosted tools usually charge per event or per GB, and bills surprise you as volume grows), and who will operate the server (your own setup lowers licence cost but someone has to maintain it). A common balance for small and mid-sized projects is error tracking in a hosted tool and logs somewhere with predictable cost. If the application is packaged with Docker, writing logs to standard output and letting a collector pick them up from outside the container is the simplest setup.

What to ask your software vendor for at handover

Monitoring is usually missing from the quote, and at the end of the project it becomes "we will add it later". A team that inherits a system without monitoring flies blind during the first incident. The handover list should state clearly: where logs go and how many days they are kept, that the error tracking account is opened in your company’s name, the defined alert rules and who receives each one, a short note on how to find a correlation ID, and the list of fields that are never logged. Written into the software development contract as acceptance criteria, these items prevent later disputes.

Conclusion

Error monitoring and logging is the infrastructure that runs your software’s life after release: structured logs and correlation IDs make the past readable, error tracking shows what broke and for how many people, and well-designed alerts tell you about a problem before your customer does. If you would like to plan monitoring as part of the first release of a new project, or add it to an existing system, take a look at our custom software service or request a quote.