Most of your logs were written to help you debug, and they answer the question "what happened?" Security logs answer different questions: who did it, on whose behalf, and was it malicious? The same login event that's a one-line success message to your app is, to an investigator, a record that needs an actor, a source, and an outcome attached. The gap between those two kinds of logging is what turns a three-hour investigation into a three-week one.
What makes a log useful for security
Six things, and a log missing any of them tends to be the one you wish you had:
- Who. The user, API key, or IP.
- What. The action: a login, a download, a permission change.
- Where. The service and endpoint.
- When. In UTC, to the millisecond.
- Outcome. Success or failure.
- Context. A request ID, a tenant ID.
Structured, that looks like:
{
"timestamp": "2025-03-15T14:23:01.042Z",
"event": "auth.login.success",
"actor": { "userId": "u_8f3k2", "ip": "203.0.113.45" },
"service": "auth-api",
"traceId": "abc-123-def"
}
Name events consistently and hierarchically, so you can query a whole category at once: auth.login.failure, authz.permission.changed, resource.file.uploaded.
What to log, and what never to
Always log authentication events, authorisation decisions, data access, configuration changes, and rate-limit triggers. These are the events an investigation is built from.
Never log passwords, tokens, API keys, full card numbers, or session token values. A log that leaks credentials is its own breach, and logs tend to be widely readable.
// Pino - redact sensitive fields at the logger
const logger = pino({
redact: ['req.headers.authorization']
});
logger.info({ event: 'auth.login.success', actor: { userId, ip } });
From logs to detections
Logs that nobody queries are just storage. Detection engineering is writing the rules that turn them into signal:
name: Brute force
query: auth.login.failure | count by actor.ip | where count > 10
timeframe: 5m
There are four shapes most useful detections take:
- Threshold. More than five failed logins in two minutes.
- Anomaly. A login from a country this account has never used.
- Sequence. A failed login, then a success, then a permission change.
- Absence. No backup completed in 24 hours.
Map your rules to MITRE ATT&CK so they speak a shared language: T1110 (Brute Force), T1078 (Valid Accounts), T1098 (Account Manipulation). And write detections vendor-agnostically with Sigma rules where you can, so they move with you.
The pipeline behind it
The logs travel a path: the app emits them, a shipper forwards them, an aggregator collects them, the detection layer evaluates the rules, and alerts come out the far end. The thing worth internalising is the last point: treat your detection rules like code. Version them, review them, and test that they actually fire, because an untested rule is a rule you're only assuming works.