# Integrations (/docs/go/integrations) Tracepanel for Go records through the standard library: `net/http`, `database/sql` and `log/slog`. A router, driver or library built on them shows in the trace with no setup of its own. Other libraries have an integration module each: | Library | Module | What it records | | --- | --- | --- | | gin | `github.com/tracepanel/tracepanel-go/gintrace` | requests, named by route | | Echo v4 | `github.com/tracepanel/tracepanel-go/echotrace` | requests, named by route | | Echo v5 | `github.com/tracepanel/tracepanel-go/echo5trace` | requests, named by route | | pgx v5 | `github.com/tracepanel/tracepanel-go/pgxtrace` | SQL and transactions | | GORM v2 | `github.com/tracepanel/tracepanel-go/gormtrace` | SQL and transactions | | go-redis v9 | `github.com/tracepanel/tracepanel-go/redistrace` | Redis commands | | zap | `github.com/tracepanel/tracepanel-go/zaptrace` | log records | | zerolog | `github.com/tracepanel/tracepanel-go/zerologtrace` | log records | | gRPC | `github.com/tracepanel/tracepanel-go/grpctrace` | calls served and made | | kafka-go | `github.com/tracepanel/tracepanel-go/kafkatrace` | messages sent and handled | Add a module with `go get`, at the same version as the core module: ```bash go get github.com/tracepanel/tracepanel-go/gintrace@v0.10.0 ``` Everything joins a trace through its context. Pass the request's context down to the database, the logger and the HTTP client. Work done without it isn't recorded. ## Requests [#requests] `tracepanel.WrapHandler` traces each request. It names the request by its route, such as `GET /books/{id}`, the pattern `ServeMux` matched. On a `ServeMux`, the handler that served the request is a `view` span, named after its function, such as `(*App).showBook`. Every response carries `Tracepanel-Id`, `Tracepanel-Url` and `Server-Timing`, and an HTML page gets the badge. The page is held until the handler returns, unless the handler flushes it. Any other response, such as JSON, a file or a stream, goes out as it is written. A response the handler takes over, such as a WebSocket, is its own. `http.ResponseController` reaches the connection. A panic in a handler is an error of the trace, with your frames and the source around them. The request answers 500. A request that carries a W3C `traceparent` continues the caller's trace, and the agent links the two. See [Traces across apps](/docs/how-it-works#traces-across-apps). ### Other routers [#other-routers] Another router on `net/http` takes `tracepanel.Middleware`. It can't tell Tracepanel the route, so name it with `Config.Route`, as here for chi: ```go tracepanel.Setup(tracepanel.Config{ Route: func(r *http.Request) string { return chi.RouteContext(r.Context()).RoutePattern() }, }) router.Use(tracepanel.Middleware) ``` Without it, a request is named by its path, such as `GET /books/42`. Fiber runs on fasthttp, not `net/http`, and isn't supported yet. ### gin [#gin] ```go router := gin.Default() router.Use(gintrace.Middleware()) ``` Each request is named by its route, such as `GET /books/:id`, with its handler as a `view` span. Pass `c.Request.Context()` down, or `c` itself with the engine's `ContextWithFallback` on. ### Echo [#echo] ```go e := echo.New() e.Use(echotrace.Middleware()) // echo5trace.Middleware() for Echo v5 ``` Each request is named by its route, such as `GET /books/:id`. On Echo v4, its handler is a `view` span. An error a handler returns is answered by Echo's error handler inside the trace. Pass `c.Request().Context()` down. The middleware of gin and Echo sets Tracepanel up when it records, and the badge names the router, such as `Go 1.27 · gin`. To give it a `Config`, call `tracepanel.Setup` before it. ## SQL [#sql] Your app opens its database as before. `tracepanel.Driver` gives `sql.Open` the name of the same driver with its statements recorded: ```go db, err := sql.Open(tracepanel.Driver("pgx"), dsn) ``` It works with any driver of `database/sql`, such as pgx's `stdlib`, lib/pq, go-sql-driver/mysql or SQLite. Whatever runs on the `*sql.DB` is recorded: your own queries, sqlx, sqlc, GORM or Bun. Wherever a driver's name goes, the name it returns goes too: ```go db, err := sqlx.Open(tracepanel.Driver("pgx"), dsn) db, err := gorm.Open(postgres.New(postgres.Config{ DriverName: tracepanel.Driver("pgx"), DSN: dsn, })) ``` For GORM, use this or the [GORM plugin](#gorm), not both. `tracepanel.WrapConnector` wraps a driver's connector, for `sql.OpenDB`: ```go db := sql.OpenDB(tracepanel.WrapConnector(stdlib.GetConnector(*config))) ``` Each statement keeps its parameters, the rows it returned and the line of your code that ran it. The agent finds N+1 queries, duplicates and slow SQL from them. The console runs `EXPLAIN` of a recorded `SELECT` through your app, on the database the query came from, in a transaction it rolls back. With a second database, give it a name, so `EXPLAIN` runs on the right one: ```go replicaDriver := tracepanel.Driver("pgx", tracepanel.DBAlias("replica")) replica, err := sql.Open(replicaDriver, replicaDSN) ``` Two more options describe a database. `tracepanel.DBSystem` names its kind, such as `"postgresql"`, when the driver's name doesn't tell it. `tracepanel.DBName` is the name shown with its queries. ### Transactions [#transactions] Each transaction is a span around its statements. The rest of its work, such as an HTTP call, a log record or a span, goes under it too when it runs with the transaction's context: ```go tx, err := db.BeginTx(ctx, nil) ctx = tracepanel.TxContext(ctx, tx) ``` The agent then finds an HTTP call made inside a transaction. Once the transaction ends, work done with that context goes under the request again. ### pgx [#pgx] pgx's own interface, a `pgx.Conn` or a `pgxpool.Pool`, takes a tracer from `pgxtrace`: ```go config, err := pgxpool.ParseConfig(dsn) config.ConnConfig.Tracer = pgxtrace.Tracer() pool, err := pgxpool.NewWithConfig(ctx, config) ``` Each statement is recorded as `tracepanel.Driver` records it. A batch's queries are recorded as they are read, and a `CopyFrom` as one statement. `pgxtrace.Tracer` takes the options `tracepanel.Driver` takes, such as `tracepanel.DBAlias` for a second database, and the console runs `EXPLAIN` on the database a query came from. A transaction's statements go under it. For the rest of its work, `pgxtrace.TxContext` puts it there too: ```go tx, err := pool.Begin(ctx) ctx = pgxtrace.TxContext(ctx, tx) ``` ### GORM [#gorm] GORM v2 takes a plugin from `gormtrace`: ```bash go get github.com/tracepanel/tracepanel-go/gormtrace@v0.10.0 ``` ```go db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{}) err = db.Use(gormtrace.Plugin()) ``` The plugin records each statement as `tracepanel.Driver` records it, with the line of your code that ran it, and each transaction, GORM's own around a create, update or delete too. Run the queries with the request's context, as in `db.WithContext(ctx).Find(&books)`. A query without it isn't recorded. The rows a query returned are there for `Find`, `First`, `Pluck`, `Count` and `Scan`, as in `db.Raw(...).Scan(&rows)`. GORM tells its logger how many rows `Scan` read, so the plugin wraps `gorm.Config.Logger` to read it. A `Session` given a logger of its own after `db.Use` goes without. `Rows()` and `Row()` have no count: your code reads those rows from the `*sql.Rows` that `database/sql` hands over. With `tracepanel.Driver` as the dialector's `DriverName`, in place of the plugin, the driver counts them as it hands each row over. The database's kind comes from the dialector, such as `postgres`, `mysql`, `sqlite`, `sqlserver` or `clickhouse`. `gormtrace.Plugin` takes the options `tracepanel.Driver` takes, such as `tracepanel.DBAlias` for a second database, and the console runs `EXPLAIN` on the database a query came from. A transaction's statements go under it. For the rest of its work, `gormtrace.TxContext` puts it there too: ```go err = db.WithContext(ctx).Transaction(func(tx *gorm.DB) error { ctx := gormtrace.TxContext(ctx, tx) // ... }) ``` Use the plugin or `tracepanel.Driver` as the dialector's `DriverName`, not both. A database opened through `tracepanel.Driver` or `tracepanel.WrapConnector` is recorded already, and the plugin leaves it to it. ## Redis [#redis] ```go rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"}) redistrace.Instrument(rdb) ``` `redistrace.Instrument` records the commands of go-redis v9. Each command is a span with its key, the database and the server, so the agent finds a command repeated for each item (N+1) and duplicates. A pipeline or a `MULTI`/`EXEC` is one span. Cluster and ring clients work too, without the server. A missing key is an answer, not a failure. Run each command with the request's context, as in `rdb.Get(ctx, key)`. ## Logs [#logs] A value you print to look at goes through `tracepanel.Dump`, which the call tree always shows. See [Dump](/docs/go/tracing-your-code#dump). ### slog [#slog] `slog` records logged with the request's context join its trace, with the line that logged them: ```go slog.InfoContext(ctx, "order paid", "order_id", order.ID) ``` Tracepanel puts itself in front of the default logger's handler, so your output stays as it was. Each attribute is a field under `extra.`, such as `extra.order_id`, and a group's attributes are named by its path, such as `extra.user.id`. A value under a secret-looking name is masked. A logger of your own takes `tracepanel.WrapLogHandler` in front of its handler: ```go logger := slog.New(tracepanel.WrapLogHandler(slog.NewJSONHandler(os.Stdout, nil))) ``` A record logged without the context, as with `slog.Info`, isn't recorded. ### zap [#zap] zap's loggers carry no context. `zaptrace.Context(ctx)` is a field that names it, on a record or on a logger, and writes nothing. Wrap the logger's core with `zaptrace.WrapCore`: ```go logger = logger.WithOptions(zap.WrapCore(zaptrace.WrapCore)) logger.Info("order placed", zaptrace.Context(ctx), zap.Int("order", id)) ``` ### zerolog [#zerolog] zerolog takes a writer and a hook, and needs both. The hook sees the record's context, and the writer its fields: ```go logger := zerolog.New(zerologtrace.Writer(os.Stdout)).Hook(zerologtrace.Hook{}) logger.Info().Ctx(ctx).Int("order", id).Msg("order placed") ``` A record of zap or zerolog keeps its level, its message, its fields with secret-looking ones masked, and the line that logged it when the logger adds the caller (`zap.AddCaller()`, zerolog's `Caller()`). Your output stays as it was. ## Outgoing HTTP calls [#outgoing-http-calls] Calls through `http.DefaultClient` are recorded with their method, address and status. For a client of your own, wrap its transport: ```go client := &http.Client{Transport: tracepanel.WrapTransport(nil)} ``` `WrapTransport(nil)` wraps `http.DefaultTransport`. Pass another `http.RoundTripper` to wrap that one. Make each call with the request's context, through `http.NewRequestWithContext`. A call without it, such as `http.Get(url)`, isn't recorded. Each call carries a `traceparent`, so the app it reaches links its trace to the call. When the panel's Profile asked for the request, its calls carry `X-Tracepanel-Profile`, so the apps down the chain profile their part. ## gRPC [#grpc] ```go server := grpc.NewServer(grpctrace.ServerOptions()...) options := append(grpctrace.DialOptions(), grpc.WithTransportCredentials(creds)) conn, err := grpc.NewClient(target, options...) ``` Each call a server serves is a trace of its own, named `Service/Method`. A server's fault, such as `Internal`, `Unavailable` or a panic, fails it. A client's error, such as `NotFound`, doesn't. Each call a client makes is a span of the trace in its context, and carries a `traceparent` in its metadata, so the server links its trace to it. A call with the `x-tracepanel-profile` metadata key, `1` as the HTTP header has it, asks the server to profile its trace. A call made in a profiled trace asks the same, so the servers down the chain profile their part. The interceptors are there one by one too, for a chain: `UnaryServerInterceptor`, `StreamServerInterceptor`, `UnaryClientInterceptor` and `StreamClientInterceptor`. A recovery interceptor goes before them. ## Kafka [#kafka] `kafkatrace` works with segmentio/kafka-go. Wrap the writer to record what it sends. A consumer hands each message to `kafkatrace.Process`: ```go writer := kafkatrace.WrapWriter(&kafka.Writer{Addr: kafka.TCP(broker), Topic: "orders"}) err := writer.WriteMessages(ctx, kafka.Message{Value: order}) msg, err := reader.FetchMessage(ctx) err = kafkatrace.Process(ctx, msg, handleOrder) ``` The messages a call sends to a topic are one span, such as `publish orders`, and each message carries a `traceparent`. Each message handled is a task trace of its own, named after its handler function, or the topic for a function literal. It links to the trace that sent it, even in another app with Tracepanel. An error the handler returns, or a panic, fails the trace. A message sent in a profiled trace carries a `tracepanel-profile` header, as the other SDKs' messages do, and the trace that handles it is profiled too. ## What a request carried [#what-a-request-carried] The panel's [Request tab](/docs/panel#request) shows what each request carried: its headers, query and cookies, the body as far as your handler read it, and the response's headers. The SDK never reads a body your handler didn't. It keeps up to 16 KB of it: JSON and forms with their secrets masked, a multipart form's fields and its files by name and size, text as it is, and binary left out. A value under a name that looks secret, such as `Authorization`, `password`, `sessionid` or `api_key`, is masked before it is recorded, and none of it leaves your machine. A name is read word by word, split at camelCase, `snake_case`, `kebab-case` and dots, as every Tracepanel SDK reads it: `authToken`, `AUTH_HEADER` and `X-Api-Key` are masked, `Author`, `tokenizer`, `token_type` and `hasPassword` aren't, and `tokens` holding numbers, a model's counts, is shown. The same rule masks a log record's fields and a dump's values. `TRACEPANEL_CAPTURE_REQUEST=0` keeps none of it. ## Profiles [#profiles] The panel's Profile button, or a call carrying `X-Tracepanel-Profile: 1`, records the next requests with a profile, which the [Profile tab](/docs/panel#profile) shows: a flame graph of where their CPU time went, from Go's own profiler, `runtime/pprof`. The request's goroutine carries a profiler label, and so do the goroutines it starts, so a profile holds only that request's samples, however many requests run at once. A task it starts with `tracepanel.Go` is profiled too. `TRACEPANEL_PROFILE=1`, or `Config.Profile`, profiles every trace. Your code can profile the rest of the trace it runs in: ```go ctx = tracepanel.Profile(ctx) ``` The calls it makes and the tasks it starts with that context from then on ask the same. A trace profiled already stays as it is. A profiled trace says why on its root span, in `profile.reason`: `header` for an `X-Tracepanel-Profile` header, `cookie` for the panel's button, `parent` when a profiled caller asked through a traced call or message, or started the task, `setting` for `TRACEPANEL_PROFILE`, and `code` for `tracepanel.Profile`. Go's profiler samples 100 times a second of CPU time, so a request that takes less than 10 ms of CPU may have no sample. Time spent waiting, such as for the database or a lock, isn't in the profile, and the trace's spans show it. An app that runs a CPU profile of its own, such as `/debug/pprof/profile`, gets no Tracepanel profiles meanwhile. Go's profiler hands its samples over only once it stops, and on macOS stopping it takes up to 100 ms, when nothing is sampled. So one profile runs on from one profiled trace to the next, and is cut once none has run for 200 ms, or every 2 s under steady load. Back-to-back traces lose no samples, and a profile reaches the console a moment after its trace. A trace that runs while a cut stops the profiler says for how long, in `tracepanel.profile_gap_ms`. Samples the profiler lost are counted as `dropped`. Every profile has the garbage collector's runs while the trace ran, and how long they stopped the program. ### Memory [#memory] The panel's Profile → Memory, `X-Tracepanel-Profile: memory`, or `TRACEPANEL_PROFILE_MEMORY=1` for every profiled trace, adds what the heap did meanwhile, from `runtime/metrics`: the bytes it allocated, its highest level above the start and its level at the end. Go's heap is the whole process's, and nothing the runtime tells of it is per goroutine: its heap profile's samples carry no labels. So these are the heap's numbers while the trace ran, the trace's own when it ran alone, and marked shared when another trace ran meanwhile. Garbage counts until the collector frees it, and there are no allocation sites. ## The App tab [#the-app-tab] The console's App tab shows Go, the router, your module and the modules it depends on, and the build's settings that hold no secret. `-ldflags` is never among them. ## Not there yet [#not-there-yet] Some things other SDKs record aren't in the Go SDK yet: - Tasks in flight. `tracepanel.Go` and `kafkatrace.Process` record each run once it ends, and the Tasks page has no Now for them. - Tests. There is no test helper to fail a test on an N+1. - Fiber, and other servers that don't run on `net/http`.