Python logging setup: dictConfig, JSON log lines and per-library levels

Python’s standard logging module is configurable enough for most applications, but its defaults are unhelpful: nothing below WARNING is shown, the format has no timestamp, and every library’s debug chatter goes through the same pipe as yours. A dozen lines of configuration fix all of that.
This post sets up logging for a small application with logging.config.dictConfig: human-readable lines on the console, one JSON object per line in a file, and separate levels for noisy libraries. It also covers logging.exception and what libraries should and should not do. Examples were run on Python 3.12 with only the standard library.
The logger hierarchy in two minutes
Section titled “The logger hierarchy in two minutes”Loggers are named with dots, and the dots form a tree. logging.getLogger("myapp.billing") is a child of myapp, which is a child of the root logger. When you log a message:
- The logger checks its level (or, if it has none set, the nearest ancestor’s level). Below that level, the record is dropped.
- The record is passed to the logger’s own handlers, then to its parent’s handlers, and so on up to the root. This is called propagation.
- Each handler has its own level and a formatter that turns the record into text.
The practical consequence: put handlers on the root logger only, and use levels on named loggers to control what gets through. Every module in your code does the same thing:
import logging
log = logging.getLogger(__name__)__name__ is the module’s dotted path, so the logger tree mirrors your package layout, and one setting on myapp controls all of myapp.*.
A small application
Section titled “A small application”Here is the layout for the example:
main.pymyapp/ __init__.py billing.py jsonlog.py logging_config.pymyapp/billing.py logs like any module would. The extra= dict attaches structured fields to the record:
import logging
log = logging.getLogger(__name__)
def charge(order_id: int, cents: int) -> None: log.info("charging order %s", order_id, extra={"order_id": order_id, "cents": cents}) if cents <= 0: raise ValueError(f"bad amount {cents}") log.debug("charge accepted")Note "charging order %s", order_id instead of an f-string. The formatting is deferred until a handler actually emits the record, so a debug call that is filtered out costs almost nothing. It also keeps the message template constant, which helps when you group log lines by message in a log search tool.
A JSON formatter
Section titled “A JSON formatter”There are good third-party JSON formatters, but the core is short enough to own:
import jsonimport loggingfrom datetime import datetime, timezone
# Attributes every LogRecord has; anything else came from `extra=`._STANDARD = set(vars(logging.makeLogRecord({}))) | {"message", "asctime", "taskName"}
class JSONFormatter(logging.Formatter): def format(self, record: logging.LogRecord) -> str: entry = { "ts": datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(timespec="milliseconds"), "level": record.levelname, "logger": record.name, "msg": record.getMessage(), } for key, value in vars(record).items(): if key not in _STANDARD and not key.startswith("_"): entry[key] = value if record.exc_info: entry["exc"] = self.formatException(record.exc_info) return json.dumps(entry, default=str)record.getMessage() applies the % arguments. The loop copies any attribute that is not part of a standard LogRecord, which is exactly what extra= added. default=str makes sure an odd value (a Decimal, a UUID) does not crash the logging call.
The dictConfig
Section titled “The dictConfig”LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "console": {"format": "%(asctime)s %(levelname)-8s %(name)s: %(message)s", "datefmt": "%H:%M:%S"}, "json": {"()": "myapp.jsonlog.JSONFormatter"}, }, "handlers": { "console": {"class": "logging.StreamHandler", "formatter": "console", "level": "INFO"}, "file": { "class": "logging.handlers.RotatingFileHandler", "formatter": "json", "filename": "app.jsonl", "maxBytes": 10_000_000, "backupCount": 3, "encoding": "utf-8", }, }, "loggers": { "myapp": {"level": "DEBUG"}, "urllib3": {"level": "WARNING"}, "httpx": {"level": "WARNING"}, }, "root": {"level": "INFO", "handlers": ["console", "file"]},}Reading it from the bottom up:
roothas both handlers and levelINFO. Anything that propagates to the root and passes the logger’s level is written to both.loggerssets levels only.myappgoes down toDEBUG, so the application’s debug lines reach the handlers.urllib3andhttpxgo up toWARNING, so their per-requestDEBUGandINFOlines are dropped at the source. Add a line here for any library that is too chatty.handlershave their own levels. The console handler isINFO, so debug lines reach the file but not the terminal."()"in a formatter tellsdictConfigto call that factory instead of building a plainlogging.Formatter. The same key works for custom handlers and filters.disable_existing_loggers: Falseis the line people forget. The default isTrue, which disables every logger that already exists whendictConfigruns, unless it or an ancestor is named in the config. Loggers are created at import time, so with the default, a library you imported before configuring logging goes silent:
import logging, logging.config
lib = logging.getLogger("stripe_client") # created at import timelogging.config.dictConfig({"version": 1, "root": {"level": "INFO", "handlers": ["h"]}, "handlers": {"h": {"class": "logging.StreamHandler"}}})lib.warning("card declined") # prints nothingprint("disabled:", lib.disabled) # disabled: TrueWiring it up
Section titled “Wiring it up”Configure logging once, as early as possible, in the entry point and nowhere else:
import loggingimport logging.config
from myapp.billing import chargefrom myapp.logging_config import LOGGING
logging.config.dictConfig(LOGGING)log = logging.getLogger("myapp")
log.info("starting")logging.getLogger("urllib3.connectionpool").debug("Starting new HTTPS connection") # droppedlogging.getLogger("urllib3.connectionpool").warning("Retrying after connection broken")
for order_id, cents in [(1001, 2500), (1002, 0)]: try: charge(order_id, cents) except ValueError: log.exception("charge failed for order %s", order_id)The two urllib3 lines stand in for what the real library would log. Running python main.py prints this to the console (file paths in the traceback shortened):
09:30:00 INFO myapp: starting09:30:00 WARNING urllib3.connectionpool: Retrying after connection broken09:30:00 INFO myapp.billing: charging order 100109:30:00 INFO myapp.billing: charging order 100209:30:00 ERROR myapp: charge failed for order 1002Traceback (most recent call last): File "main.py", line 16, in <module> charge(order_id, cents) File ".../myapp/billing.py", line 9, in charge raise ValueError(f"bad amount {cents}")ValueError: bad amount 0And app.jsonl gets one object per line, including the debug line the console skipped and the extra fields:
{"ts": "2026-07-03T09:30:00.002+00:00", "level": "INFO", "logger": "myapp", "msg": "starting"}{"ts": "2026-07-03T09:30:00.003+00:00", "level": "WARNING", "logger": "urllib3.connectionpool", "msg": "Retrying after connection broken"}{"ts": "2026-07-03T09:30:00.004+00:00", "level": "INFO", "logger": "myapp.billing", "msg": "charging order 1001", "order_id": 1001, "cents": 2500}{"ts": "2026-07-03T09:30:00.005+00:00", "level": "DEBUG", "logger": "myapp.billing", "msg": "charge accepted"}{"ts": "2026-07-03T09:30:00.006+00:00", "level": "INFO", "logger": "myapp.billing", "msg": "charging order 1002", "order_id": 1002, "cents": 0}{"ts": "2026-07-03T09:30:00.007+00:00", "level": "ERROR", "logger": "myapp", "msg": "charge failed for order 1002", "exc": "Traceback (most recent call last):\n File \"main.py\", line 16, ..."}The traceback in the last line is shortened here; in the file it is the full text with \n escapes. JSON lines like these can be shipped as-is to most log collectors, and jq handles them locally: jq 'select(.level == "ERROR")' app.jsonl.
logging.exception
Section titled “logging.exception”log.exception(msg) logs at ERROR level and attaches the current exception’s traceback. It is the same as log.error(msg, exc_info=True), and it only makes sense inside an except block, where there is a current exception.
Two habits that keep error logs useful:
- Log or raise, not both. If you log an exception and then re-raise it, the caller probably logs it again, and you get the same traceback twice. Log at the level that handles the error.
- Put the identifying values in the message or in
extra. “charge failed” is useless at 3 a.m.; “charge failed for order 1002” is not.
For a warning that should still show the traceback, pass exc_info=True to log.warning. To show where a log call came from without an exception, use stack_info=True.
Changing levels without editing code
Section titled “Changing levels without editing code”A common need is turning on debug output for one component in production. Since the config is a dict, read overrides from the environment before applying it:
import os
level = os.environ.get("MYAPP_LOG_LEVEL")if level: LOGGING["loggers"]["myapp"]["level"] = level.upper()logging.config.dictConfig(LOGGING)At runtime you can also call logging.getLogger("myapp.billing").setLevel(logging.DEBUG) from a debug endpoint or a signal handler.
Rules for libraries
Section titled “Rules for libraries”If you are writing a package other people import, the rules are different:
- Never call
logging.basicConfig()ordictConfig()in library code. They configure the root logger, which belongs to the application.basicConfigalso does nothing if the root logger already has handlers, so a library that calls it at import time can make the application’s ownbasicConfigcall silently do nothing. - Don’t add handlers other than
NullHandler. Get a logger withlogging.getLogger(__name__)and log to it. The application decides where the output goes. - Optionally add a
NullHandlerto your top-level logger (logging.getLogger("mylib").addHandler(logging.NullHandler())). Without any handler configured, Python falls back to a last-resort handler that printsWARNINGand above to stderr. TheNullHandlerstops that, so your library is silent unless the application configures logging. - Use
DEBUGfor per-call detail and reserveWARNINGfor things the application developer should act on. That way the"urllib3": {"level": "WARNING"}style of override in the application config does the right thing.
Gotchas
Section titled “Gotchas”basicConfigafter something else configured the root does nothing. Passforce=Trueif you really need to replace the existing handlers, for example in a notebook.- Handler levels and logger levels both apply. If debug lines are missing, check the logger level, every ancestor’s level if it is unset, and the handler level.
RotatingFileHandleris not safe with several processes writing the same file. With Gunicorn workers or multiprocessing, log to stdout and let the process manager or container runtime collect it, or use aQueueHandlerthat sends records to one writer.- Don’t log secrets through
extra. The JSON formatter writes every extra field. Filter or redact tokens and personal data before they reach the record.
Further reading
Section titled “Further reading”© 2026 David Li · Home · Blog · RSS ·Classic site · Built with Astro, Starlight and three.js · Logos:Font Awesome Free (CC BY 4.0)