Structured logging in Python that survives production

The problem with string logs

2025-12-01 09:14:32 INFO Processed order 12345 for user 67890 in 142ms

That is fine to read. It is awful to query. You cannot ask "how many orders did user 67890 process this week" without parsing the message with a regex.

Structured logs fix this.

What structured logs look like

{
  "ts": "2025-12-01T09:14:32Z",
  "level": "info",
  "event": "order_processed",
  "order_id": 12345,
  "user_id": 67890,
  "duration_ms": 142
}

Same information. Now you can grep, filter, aggregate, and pipe into any log platform without parsing.

Setup in Python

We use structlog. It plays well with the stdlib logging module and is easy to read.

import structlog

structlog.configure(
    processors=[
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
)

log = structlog.get_logger()
log.info("order_processed", order_id=12345, user_id=67890, duration_ms=142)

In development, swap the JSON renderer for a pretty console renderer. The fields stay the same.

What to log

• One event per business action — order_processed, payment_failed, sync_completed
• Always include: request ID, user/tenant, the relevant object ID
• Numbers, not just strings: duration_ms, retry_count, item_count

Avoid logging full payloads unless you really need them. PII in logs is a compliance footgun.

What not to log

• Passwords, tokens, full card numbers, SSNs — ever
• Successful internal heartbeats — they are noise
• The same event twice from different layers

Where the logs go

For SMBs:

• Local dev: console
• Production small: stdout, scraped by the host (Cloud Run logs, journald, etc.)
• Production with multiple services: ship to a SaaS (Better Stack, Axiom, Datadog) or self-hosted (Grafana Loki)

Do not build a log platform. Buy or self-host an existing one.

The payoff

The first incident where you can answer "show me every order this user placed in the last hour and the latency of each" in 20 seconds — that is when structured logging pays for itself. It happens every time.