Tracepanel

Integrations

What the Python SDK records in each web framework and in each library it finds installed, and how to turn an integration off.

The Python SDK hooks into your web framework to record each request. It also traces the libraries your app uses, such as task queues, database drivers, HTTP clients, Redis and MongoDB. You don't configure a library integration. It turns on when your app first imports the library, if its setting is on. Tracepanel never loads a library your app doesn't import, so it adds little to startup.

Web frameworks

Each framework module turns Tracepanel on only in development. Django follows DEBUG. FastAPI, Starlette and Flask follow the app's debug mode. Falcon has no debug mode, so you turn it on yourself. tracepanel run turns Tracepanel on for any of them. See Installation for the setup of each.

In every framework, Tracepanel records the request's method, route, status, duration and errors. It adds the panel to HTML pages and serves the console at /__tracepanel__/.

Django

Tracepanel supports Django 5.2, 6.0 and 6.1. Django 6.x needs Python 3.12 or newer, and Python 3.15 works with Django 6.1 only.

Add tracepanel.django to INSTALLED_APPS. Tracepanel puts its middleware first, so it measures the whole request. It records these parts of Django with no further setup:

  • SQL on every database connection, including the async ORM, with the number of rows each query returned. Tracepanel keeps the place in your code across sync_to_async(), so it finds N+1 queries in async views too.
  • Each atomic() block as a transaction or a savepoint, and each on_commit() callback.
  • The view, named after its function or class, and each template render.
  • Cache calls on every backend you create through caches.
  • Signals that have receivers in your project, with a span for each receiver. Tracepanel leaves out pre_init and post_init, which fire once per model instance.
  • Each email sent with EmailMessage.send(), send_mail() or EmailMultiAlternatives, with its text and HTML.
  • Management commands as traces of their own. The ignore_commands setting leaves out the dev server, shells, the test runner and task workers by default.
  • The logged-in user, if the view loaded one, and logouts.

The Django guide shows what each part gives you and the issues found in it.

FastAPI and Starlette

from fastapi import FastAPI

import tracepanel.fastapi

app = FastAPI(debug=True)
tracepanel.fastapi.install(app)

For Starlette, call tracepanel.asgi.install(app) on a Starlette(debug=True) app. tracepanel run uvicorn app.main:app needs no code for either.

Tracepanel names each request after its route template, such as GET /items/{item_id}, and runs the endpoint in a view span.

Flask

from flask import Flask

import tracepanel.flask

app = Flask(__name__)
tracepanel.flask.install(app)

Tracepanel is on when the app runs in debug mode, as with flask run --debug or app.run(debug=True). It names each request after its URL rule and runs the view in a view span. Each Jinja render is a template span. Tracepanel also records the errors Flask turns into a 500, and adds the trace's headers to Werkzeug's debugger page. It reads settings from app.config["TRACEPANEL"].

Falcon

import falcon

import tracepanel.falcon

app = falcon.App()
tracepanel.falcon.install(app, enabled=True)

Falcon has no debug mode, so pass enabled=True or set TRACEPANEL_ENABLED=1. Tracepanel names each request after the route's URI template, such as GET /posts/{post_id}. It runs the responder in a view span named after it, such as app.resources.PostResource.on_get, and records the errors Falcon turns into a 500. Tracepanel doesn't cover Falcon's ASGI app, falcon.asgi.App, yet.

Any ASGI or WSGI app

tracepanel.asgi.install() and tracepanel.wsgi.install() work with any app, such as Litestar, Quart, Bottle, Pyramid or a plain callable. They wrap the app and return the wrapper, so serve what they return:

import tracepanel.asgi


async def app(scope, receive, send): ...


app = tracepanel.asgi.install(app)
import tracepanel.wsgi


def app(environ, start_response): ...


app = tracepanel.wsgi.install(app)

Tracepanel is on when the app has debug = True or when TRACEPANEL_ENABLED=1 is set. It names each request after its path, because it doesn't know the framework's routes. You pass settings as keyword arguments, with the same names as in Configuration.

Libraries in each framework

Django gets every library integration below. FastAPI, Starlette, Flask, Falcon, plain ASGI or WSGI apps, and scripts and workers under tracepanel run get those that don't need Django: logging, SQLAlchemy, the database drivers, the HTTP clients, Redis, MongoDB, Celery, Dramatiq, RQ, Strawberry and OpenTelemetry. All the frameworks record htmx requests.

Libraries

Tracepanel traces these libraries once your app imports them:

LibrarySettingWhat Tracepanel records
logginglogging_enabledEach log record of a request, task or command, with the fields you pass in extra=, attached to the span it was logged in.
Database driversdbapi_enabledEach statement of a driver you use without an ORM, as SQL with the same N+1, duplicate and slow checks as Django's ORM, and EXPLAIN in the console. It covers sqlite3, aiosqlite, psycopg 3, psycopg2, asyncpg, PyMySQL, mysqlclient, MySQL Connector/Python, aiomysql and python-oracledb. A statement that an ORM runs through the driver is recorded once, by the ORM. pyodbc, pymssql and asyncmy aren't covered.
HTTP clientshttp_clients_enabledEach outgoing call of urllib3 and what is built on it (requests, boto3), httpx and aiohttp, with its method, URL and status. Credentials and sensitive query values stay out of the URL. A status of 400 or more marks the call as failed. A call slower than slow_http_ms is slow. Each call carries a traceparent header. See Calls between apps.
Redisredis_enabledEach command of redis-py and valkey-py, sync and async, with its key. A pipeline is one span. The commands that Django's cache or a task queue sends sit under that library's span.
Celerycelery_enabledEach task run as a trace, linked to the request, task or command that queued it. The time in the queue, retries with their reason, failures, arguments and the result. Tasks that Celery beat sends count as scheduled.
Dramatiqdramatiq_enabledThe same for each message an actor processes.
RQrq_enabledThe same for each job, through RQ's forking worker and for Queue(is_async=False).
Django Tasksdjango_tasks_enabledThe same for django.tasks on Django 6.0 and later, and for the django-tasks backport on Django 5.2.
MongoDBmongodb_enabledEach command of pymongo's MongoClient and AsyncMongoClient, and of Motor, with its shape ({"find": "users", "filter": {"email": "?"}}), its values kept apart as SQL's, the database, the server and the line of code. Tracepanel keeps the values only with capture_sql_params. A find per item is an N+1, the same find twice a duplicate. A command slower than slow_sql_ms is a slow call, and its hint names the index it needs. Nothing to install beyond Tracepanel: it uses pymongo's own command monitoring, for the clients created after the app imports pymongo.
SQLAlchemysqlalchemy_enabledEvery statement of every engine as SQL, sync or async, with the same N+1 and duplicate checks as Django's ORM, and the number of rows it returned. Each transaction and savepoint. This covers SQLModel and Flask-SQLAlchemy too.
Strawberrystrawberry_enabledGraphQL requests named after the operation, such as GraphQL query PostList, and your resolvers as spans. Errors that Strawberry returns with HTTP 200.
Django REST frameworkdrf_enabledThe view named after the ViewSet action, such as PostViewSet.list. Authentication, permissions with the class that refused, throttles, validation with the 400's errors, and serialization as steps.
Django Ninjaninja_enabledThe view named after the operation's function. Authentication, input validation with the 422's errors, and response serialization as steps.
Channelschannels_enabledEach WebSocket event as a trace. Group messages linked to the span that sent them.
htmxnoneWhat sent each htmx request and the HX-* headers of the answer. A redirect or a whole page that htmx would swap into the page.
OpenTelemetryotel_enabledSpans from the OpenTelemetry instrumentations your app enables for systems Tracepanel has no integration for, such as gRPC, Elasticsearch or Kafka, in the call tree.

The optional extras pin the oldest version Tracepanel supports. The database drivers, the HTTP clients and Redis have no extra.

LibraryExtraMinimum version
Djangodjango5.2
FastAPIfastapi0.142
Starlettestarlette0.40
Flaskflask3.0
Falconfalcon4.0
Celerycelery5.3
Dramatiqdramatiq1.17
RQrq2.0
SQLAlchemysqlalchemy2.0
pymongomongodb4.13
Strawberrystrawberry0.250
Django REST frameworkdrf3.15
Django Ninjaninja1.4
Channelschannels4.2
OpenTelemetry SDKotel1.45
pip install "tracepanel[fastapi,celery]"

Logging

Tracepanel reads log records from Python's standard logging module. It adds no handler to any logger and leaves your logging configuration as it is. basicConfig(), Django's LOGGING and a worker that takes over the root logger all work as they did before.

Tracepanel sees each record when a logger hands it to its handlers. It keeps a record once, however many loggers and handlers the record passes through. A logger's level still applies. If a logger's level is INFO, its DEBUG records don't reach Tracepanel.

Tracepanel keeps up to 200 records per trace. It counts the records past that limit in the trace's tracepanel.dropped_logs attribute. A record that carries an exception, as logger.exception() writes, keeps its traceback.

Pass fields in extra= as usual:

import logging

logger = logging.getLogger(__name__)

logger.info("Order paid", extra={"order_id": 42, "amount": 990})

Tracepanel stores each field under extra. with the name in lower case, such as extra.order_id. Characters other than letters, digits and underscores become an underscore. A string, number, boolean or None keeps its value. Any other value is stored as a short repr().

Tracepanel masks an extra= field whose name contains one of these words, in any case:

pass, passwd, password, secret, token, api_key, apikey, auth, cookie, session, csrf, signature, private

Tracepanel skips the records of these loggers and their children:

LoggerWhy
tracepanelTracepanel's own records.
django.db.backendsThe SQL echo. The SQL tab already shows every query.
celery.app.trace, celery.workerCelery's bookkeeping for each task, such as "Task x succeeded in 0.1s". The task's trace already shows its state and duration.
dramatiq.workerDramatiq's worker bookkeeping.
rq.worker, rq.jobRQ's bookkeeping, such as "Job OK".
django.channels.server, daphne, uvicorn.access, gunicorn.access, hypercorn.accessServer access logs, such as "GET /items/1 200". The trace itself has the request line and the status.

To skip more loggers, list them in the ignore_loggers setting. Tracepanel skips each one with its children:

[tool.tracepanel]
ignore_loggers = ["botocore", "urllib3"]

See Logs for the Logs page and its search.

Tasks and the request that queued them

When a request queues a task, the request's call tree shows the enqueue as a step. Each run of the task is a trace of its own that links back to that step. The same holds for a task queued by another task or by a command.

Each queue carries the link in its own way:

  • Celery puts the current trace and span in the message headers.
  • Dramatiq puts them in the message options.
  • RQ puts them in job.meta, which it saves with the job.
  • Django Tasks carry no headers. The agent links the run to its enqueue by the task id when it has both.

A task that runs inside the caller, such as an eager Celery task or a task on Django's ImmediateBackend, links to the caller's span directly.

Each run records how long the task waited in its queue before a worker took it. A wait over 1 s is a queue-wait issue: all the workers were busy. This holds for Celery, Dramatiq, RQ and Django Tasks.

Tracepanel keeps task arguments and results for local debugging. They never leave your machine. The capture_task_args setting turns this off. The ignore_tasks setting leaves out tasks by name, such as celery.backend_cleanup.

Start a worker with tracepanel run, or in a Django project with tracepanel.django installed. The worker prints a line after each task with a link to its trace:

tracepanel run celery -A app worker

Channels

A WebSocket connection can stay open for hours, so Tracepanel doesn't make it one trace. The connect, each frame received, each message from the channel layer and the disconnect are traces of their own. Each links to the connect trace. Frames sent to the client and messages sent to the channel layer are spans. A group message links each handler it reaches to the span that sent it.

htmx

Tracepanel reads the HX-* headers of every request that htmx sends, and of the answer. Two answers only show as a broken page. htmx follows a redirect and swaps in whatever the new URL returns. And a whole page can land where htmx expected a fragment. Tracepanel records both, and the agent reports them as issues.

Rows a query returns

Each SQL query records how many rows it returned. Django and SQLAlchemy count the rows as they fetch them, sync or async, on every database. A driver you use directly gives its row count where it knows it, as psycopg and the MySQL drivers do. asyncpg gives its results, and sqlite3 and aiosqlite count the rows your code fetches.

A SELECT with no LIMIT that returned 1000 rows or more is a large-result issue. Its hint says how to page or stream the rows.

Blocking calls in async code

A sync call on an event loop's thread makes every other request on that loop wait. Tracepanel marks SQL, an HTTP call, a Redis command or a Django cache call that runs synchronously on the loop's thread, such as a sync driver in an async view or endpoint. The agent reports the calls from one place in your code as a blocking-call issue once they take 5 ms together. Its hint names the async client or driver to use instead.

Async clients aren't marked. Neither are sync calls in a thread of their own, such as sync_to_async(), asyncio.to_thread() or a sync view.

Calls between apps

Tracepanel links the traces of two apps when one calls the other over HTTP. It uses the W3C trace context:

  • Each outgoing HTTP call from urllib3, requests, httpx or aiohttp carries a traceparent header. A request that sets its own traceparent keeps it.
  • A request that comes in with a traceparent continues that trace context. This works in Django, in ASGI apps such as FastAPI and Starlette, and in WSGI apps such as Flask and Falcon.

The agent then shows the called app's trace under the call in the call tree, and draws an arrow from app to app on the Services page. The other app can use this SDK or OpenTelemetry. You don't configure the header. The agent links only the traces it holds, so send both apps to one agent, each under its own service name. See the agent and service settings in Configuration.

OpenTelemetry

A system that Tracepanel has no integration for may have an OpenTelemetry instrumentation. If your app already uses OpenTelemetry, enable the instrumentation as usual. Its spans show in the request, task or job they ran in, under the span that was current:

from opentelemetry import trace
from opentelemetry.instrumentation.grpc import GrpcInstrumentorClient
from opentelemetry.sdk.trace import TracerProvider

trace.set_tracer_provider(TracerProvider())
GrpcInstrumentorClient().instrument()

Each instrumentation is a package of its own, such as opentelemetry-instrumentation-grpc. You need it only for a system that Tracepanel doesn't record itself. Everything in the table above needs only pip install tracepanel.

Tracepanel joins the TracerProvider that your app sets, before or after Tracepanel starts. It never sets a provider itself, so your exporters keep working. Tracepanel drops OpenTelemetry spans for what it records itself, so nothing shows twice. That covers the frameworks, SQL and the database drivers, SQLAlchemy, the HTTP clients, Redis, pymongo, Celery and logging. The integration needs opentelemetry-sdk.

Turn an integration off

Each library integration except htmx has a <name>_enabled setting. Set it to false and Tracepanel leaves the library alone:

# pyproject.toml
[tool.tracepanel]
drf_enabled = false
celery_enabled = false
TRACEPANEL_DRF_ENABLED=false

In Django, the same names go in the TRACEPANEL dict. tracepanel config prints each setting from the files and variables, and where its value comes from. It doesn't read Django's settings. See Configuration.

Tracepanel reads the *_enabled settings when the app starts, and patches each library when the app first imports it. Restart the app after you change an *_enabled setting.

On this page