# How it works (/docs/how-it-works)

Tracepanel has two parts. An SDK runs inside your app and records traces. The
agent is a separate program that analyzes and stores them, and serves the
console.

## The SDK [#the-sdk]

The SDK is a library you add to your app. When your app starts, the SDK reads
its settings and decides whether to turn on. If it is off, it stops there and
changes nothing in your app.

When it is on, the SDK hooks into your web framework and into the libraries it
supports. Each request opens a trace, and everything the request runs records
into it. The trace collects these parts:

- SQL queries and transactions.
- The view, templates and cache calls.
- Calls to Redis, HTTP APIs, MongoDB and other systems.
- Sent emails.
- Log records.
- Errors, with their stack and the source of your frames.

When the response is ready, the SDK adds the route, the view and the status to
the trace. It adds the user only if the view already loaded one, and it never
queries for the user. Then a background thread sends the trace to the agent.

The SDK also adds headers to the response: `Tracepanel-Id`, `Tracepanel-Url`
and `Server-Timing`. The badge reads them. For an HTML page, the SDK also adds
the panel's script.

A background task run and a command each become a trace of their own. A task
queued during a request shows in the request's call tree, and its run links
back to that request.

<Callout title="In the Python SDK">
The SDK is the `tracepanel` package, and it installs the agent with it. Where
a framework has a hook, the SDK uses it. Where there is none, it wraps a few
methods, such as the view call and template rendering. The trace lives in a
context variable, so async code records into it too. Tracepanel skips
servers, shells and workers among management commands. See
[Integrations](/docs/python/integrations) and `ignore_commands` in
[Configuration](/docs/python/configuration).
</Callout>

<Callout title="In the JavaScript SDK">
The SDK is the `@tracepanel/js` package, and it installs the agent with it.
It loads before your app and records through OpenTelemetry's
instrumentations, which come with it. See
[Integrations](/docs/javascript/integrations).
</Callout>

## The browser [#the-browser]

A page can record itself in the browser too, with the JavaScript SDK. Each
view of the page and each action on it, such as a click that called the API,
becomes a trace with its calls, errors and console output. The page sends it
to the console on its own origin, at `/__tracepanel__/api/ingest`, and the
console passes it on to the agent. The console takes these traces as JSON
from this machine only, so a page on another site can't send to it. See
[Pages in the browser](/docs/javascript/browser).

## The agent [#the-agent]

`tracepanel-agent` is a separate program. Each project has one agent. The
first trace starts it, and it exits after 30 minutes unused. The agent keeps
running across the reloads of your dev server and across workers, so every
process of the project sends to the same agent. Several apps can share one
agent, each under its own `service` name.

The agent does the work your app doesn't:

- It finds the [issues](/docs/issues) in each trace.
- It stores the traces.
- It serves the console, its API and the panel's script.

The badge shows a request's status and time at once. It shows the request's
issues a moment later, once the agent has found them.

To run the agent in the foreground, with the console on
http://127.0.0.1:8765/, start it from your project's directory:

```bash
tracepanel-agent --serve 127.0.0.1:8765
```

`tracepanel dev` does the same, from the Python SDK or the JavaScript SDK.
See the commands of [Python](/docs/python/cli) and
[JavaScript](/docs/javascript/cli).

Use it to watch the agent's output, or to read the traces after your app has
stopped. If the agent can't run, the SDK prints the reason in the terminal.
Requests still get their headers, but Tracepanel stores nothing.

## OpenTelemetry [#opentelemetry]

The agent also takes OpenTelemetry spans over OTLP/HTTP, on `127.0.0.1:4318`.
That is where OpenTelemetry's exporters send spans unless told otherwise. A
service that exports its traces with any OpenTelemetry SDK shows in the
console with no Tracepanel SDK in it. The agent reads the spans by
OpenTelemetry's semantic conventions and finds the same issues in them. It
names the service's language and framework from the spans, such as Java and
Spring Boot, .NET and ASP.NET Core, Ruby and Rails, or Go and Gin.

`--otlp HOST:PORT` moves the address, and `--otlp off` turns it off. If the
port is in use, the agent says so and runs without it.

## Traces across apps [#traces-across-apps]

A call from one app to another carries the W3C trace context. The agent links
the caller's trace to the trace of the app it reached, whether each app uses a
Tracepanel SDK or OpenTelemetry. The call tree shows the reached app's trace
under the call, and the Services page draws an arrow from app to app.

## The store [#the-store]

The agent keeps the traces in a SQLite file under `.tracepanel/` in your
project, `traces.v21.sqlite3`. The number in the name is the store's format.
When a new version of the agent changes the format, it starts a new, empty
file, and the traces in the old one aren't carried over.

The agent keeps the newest 2000 traces and deletes older ones. One trace
keeps at most 2000 spans and 1000 log records: its earliest spans, each with
its parent, and its first records. The Request tab shows how many it didn't
keep, under **Not recorded**. An issue keeps its count and the time it was first seen after its traces
are deleted. The SDK's
settings change that number and move the store. The trash button in the
console deletes everything in it.

## What runs in your app's process [#what-runs-in-your-apps-process]

Your app's process records the trace and sends it. It does no analysis and
keeps no store. Collecting a trace adds little to a request, and a background
thread sends the trace after the response is made. Tracepanel collects nothing
when it is off.

Requests under `/__tracepanel__/` go to the agent. Your app handles two of
them itself, because it has the database connections and the sessions:

- `EXPLAIN` of a query from the console. In the agent's own console, the
  agent sends it to the app over the connection the app sends traces on.
- Switching the user from the panel, in a Django project.

## Several projects on one machine [#several-projects-on-one-machine]

Each agent lists itself in a file in your user's cache directory, readable by
you only. When your app calls another app on the same machine, and that app's
project runs its own agent, the trace's Request tab links to the call's trace
in the other project's console. The agents match the call by the W3C trace
context the SDKs send with each outgoing HTTP call.

## Clocks [#clocks]

The agent notes when each trace arrives. When the app's clock is more than 10
seconds ahead of the agent's, or a minute behind, the Request tab shows a
**Clock** note, since the trace's times may not line up with other apps or
with `since:`. A note for a clock behind can also mean the app sent the trace
late.

## Privacy and safety [#privacy-and-safety]

Tracepanel is a development tool, and its defaults keep it local.

**It stays on your machine.** The agent stores traces in `.tracepanel/` in your
project and sends nothing anywhere. By default the console and the OTLP
intake listen on `127.0.0.1`, and the agent takes traces from this machine
only.

**Add the store to `.gitignore`.** The traces hold development data, so keep
them out of your repository:

```bash
echo ".tracepanel/" >> .gitignore
```

**It masks secret-looking values.** To help you debug, traces keep SQL
parameters for `EXPLAIN`, local variables in errors, email bodies and task
arguments. Tracepanel masks values in local variables, task arguments and log
records when their names look secret. That covers names with `password`,
`secret`, `token`, `api_key`, `auth`, `cookie`, `session`, `csrf`,
`signature` or `private` in them. Each kind of data has a setting that turns
it off.

**It is off outside development.** The SDK turns on only in development, such
as in your framework's debug mode. Otherwise it installs nothing.

<Callout title="In the Python SDK">
The settings `capture_sql_params`, `capture_locals`, `capture_mail_body` and
`capture_task_args` turn off each kind of data. Tracepanel turns on only with
`DEBUG` or the framework's debug mode, under `tracepanel run`, or with
`TRACEPANEL_ENABLED=1`. See [Configuration](/docs/python/configuration).

In Django, switching the user needs `DEBUG = True`, even when you turned
Tracepanel on another way. It also needs `django.contrib.auth` and sessions.
The request must come from this machine or from an address in `INTERNAL_IPS`,
and from the page itself, as a same-origin call with Tracepanel's header. The
`user_switch` setting turns it off.
</Callout>