# OpenTelemetry for Go: Traces, Metrics & Logs with OpenObserve

> Instrument Go apps with the OpenTelemetry SDK - traces, metrics, and logs via OTLP to OpenObserve. Complete setup, code examples, and troubleshooting.

Source: https://openobserve.ai/opentelemetry/go/
Published: 2026-07-08
Language: Go

---

Go is a first-class citizen in the OpenTelemetry project: the tracing and metrics SDKs are stable, the instrumentation ecosystem covers the standard library and most popular packages, and the explicit, code-first setup style fits how Go programs are usually built. This guide walks through instrumenting a Go service for traces, metrics, and logs, and shipping all three to [OpenObserve](https://openobserve.ai/) over OTLP.

If you are new to OpenTelemetry itself - the signals, the collector, OTLP - start with [What is OpenTelemetry? A Complete Guide](/blog/what-is-opentelemetry/) and come back here for the Go specifics.

## Why OpenTelemetry for Go

Go services tend to sit in the hot path - API gateways, backends, workers - where you need request-level visibility without a heavyweight agent. OpenTelemetry gives you:

- **Vendor-neutral instrumentation.** Instrument once, export anywhere OTLP is accepted. No proprietary SDK lock-in.
- **A stable, idiomatic API.** Context propagation rides on `context.Context`, which Go code already threads everywhere.
- **A mature library ecosystem.** `otelhttp`, `otelgrpc`, `otelsql`, and dozens of other instrumentation libraries in the `opentelemetry-go-contrib` repository wrap common dependencies with a line or two of code.

### Signal stability in Go

Be precise about what "supported" means. As of mid-2026, the status of the Go implementation is:

| Signal  | Status | Notes |
|---------|--------|-------|
| Traces  | Stable | API and SDK stable since 2021; safe for production |
| Metrics | Stable | API and SDK stable; OTLP metric exporters are GA |
| Logs    | Beta   | `otel/log` API, `sdk/log`, and bridges such as `otelslog` work but may see breaking changes between minor releases |

Traces and metrics are covered by the project's stability guarantees. The logs signal is usable in production - many teams run it - but pin your versions and read the changelog when you upgrade, because the API surface is not yet frozen.

## Prerequisites

- **Go 1.25 or later.** The OpenTelemetry Go SDK supports the two most recent Go releases; older toolchains fall out of support quickly.
- **An OpenObserve instance.** Either an [OpenObserve Cloud](https://cloud.openobserve.ai) account or a self-hosted instance. For local testing, a single container is enough:

```bash
docker run -d --name openobserve \
  -p 5080:5080 -p 5081:5081 \
  -e ZO_ROOT_USER_EMAIL="root@example.com" \
  -e ZO_ROOT_USER_PASSWORD="Complexpass#123" \
  public.ecr.aws/zinclabs/openobserve:latest
```

- **Your OpenObserve credentials.** OTLP ingestion uses HTTP Basic auth. Generate the token once:

```bash
echo -n 'root@example.com:Complexpass#123' | base64
```

This prints `cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=` - the value you will pass in the `Authorization: Basic` header.

In OpenObserve Cloud, the ingestion page under **Data Sources** shows your organization-specific endpoint and a ready-made Basic auth token, so you rarely need to construct it by hand.

## Zero-code instrumentation: the short answer for Go

Go has **no runtime agent**. A Go program is a statically compiled binary - there is no JVM-style bytecode manipulation or Python-style monkey patching to hook into. If you are coming from Java or .NET, this is the biggest mental shift: in Go, *you* wire up the SDK in `main()`.

Two things exist that are adjacent to zero-code, and it is worth knowing their real status:

- **OpenTelemetry eBPF Instrumentation (OBI)** - the successor to the earlier `opentelemetry-go-instrumentation` eBPF project - can capture HTTP and gRPC spans from unmodified binaries at the kernel level. It is Linux-only (kernel 5.8+, amd64/arm64) and still pre-1.0, with breaking changes expected between minor releases. Useful for coarse visibility into services you cannot modify; not a replacement for SDK instrumentation.
- **Instrumentation libraries** such as `otelhttp` are as close to "automatic" as idiomatic Go gets: wrap your handler or client once and every request gets a span, without touching handler logic.

The rest of this guide covers the recommended, production-proven path: explicit SDK setup plus instrumentation libraries. It works with any framework - Gin, Echo, Chi, Fiber - because they all sit on `net/http` or expose middleware hooks, but the examples below use the plain standard library so nothing is framework-specific.

## Manual instrumentation

Install the core modules:

```bash
go get go.opentelemetry.io/otel \
  go.opentelemetry.io/otel/sdk \
  go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp \
  go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp \
  go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp \
  go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp \
  go.opentelemetry.io/contrib/bridges/otelslog
```

### Traces

The pattern is always the same: create an exporter, attach it to a `TracerProvider` with a batch span processor and a `Resource`, register the provider globally, and shut it down cleanly on exit. Here is a complete, runnable HTTP service:

```go
package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"
	"time"

	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
	"go.opentelemetry.io/otel/propagation"
	"go.opentelemetry.io/otel/sdk/resource"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	semconv "go.opentelemetry.io/otel/semconv/v1.37.0"
)

var tracer = otel.Tracer("checkout-service")

func initTracerProvider(ctx context.Context) (*sdktrace.TracerProvider, error) {
	// Reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS
	// from the environment — see the OpenObserve section below.
	exporter, err := otlptracehttp.New(ctx)
	if err != nil {
		return nil, fmt.Errorf("create trace exporter: %w", err)
	}

	// Merge with resource.Default() so OTEL_SERVICE_NAME and
	// OTEL_RESOURCE_ATTRIBUTES are honored. service.name and
	// deployment.environment come from the environment; service.version
	// is fixed in code here.
	res, err := resource.Merge(
		resource.Default(),
		resource.NewWithAttributes(
			semconv.SchemaURL,
			semconv.ServiceVersion("1.4.2"),
		),
	)
	if err != nil {
		return nil, fmt.Errorf("create resource: %w", err)
	}

	tp := sdktrace.NewTracerProvider(
		sdktrace.WithBatcher(exporter),
		sdktrace.WithResource(res),
	)
	otel.SetTracerProvider(tp)

	// W3C Trace Context + Baggage propagation for distributed traces.
	otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
		propagation.TraceContext{},
		propagation.Baggage{},
	))
	return tp, nil
}

func checkoutHandler(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context() // otelhttp already started a server span here

	// Child span for a unit of work inside the request.
	ctx, span := tracer.Start(ctx, "process-payment")
	defer span.End()

	span.SetAttributes(
		attribute.String("payment.provider", "stripe"),
		attribute.Int("cart.items", 3),
	)

	time.Sleep(40 * time.Millisecond) // simulate work
	fmt.Fprintln(w, "order confirmed")
	_ = ctx
}

func main() {
	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
	defer stop()

	tp, err := initTracerProvider(ctx)
	if err != nil {
		log.Fatal(err)
	}
	// Flush remaining spans before exit — skipping this drops the last batch.
	defer func() {
		shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
		defer cancel()
		if err := tp.Shutdown(shutdownCtx); err != nil {
			log.Printf("tracer shutdown: %v", err)
		}
	}()

	// otelhttp wraps the handler: one server span per request, automatic
	// http.* semantic-convention attributes, and context extraction from
	// incoming traceparent headers.
	handler := otelhttp.NewHandler(http.HandlerFunc(checkoutHandler), "POST /checkout")
	http.Handle("/checkout", handler)

	srv := &http.Server{Addr: ":8080"}
	go func() { log.Println(srv.ListenAndServe()) }()

	<-ctx.Done()
	srv.Shutdown(context.Background())
}
```

Two details matter more than anything else in this file:

1. **Context flows through `context.Context`.** `tracer.Start(ctx, ...)` parents the new span to whatever span lives in `ctx`. If you drop the context - or start goroutines with `context.Background()` - you break the trace.
2. **`tp.Shutdown` is not optional.** The batch processor holds spans in memory; a process that exits without shutting down the provider silently loses its final batch.

For outgoing calls, wrap your HTTP client the same way so the `traceparent` header propagates downstream:

```go
client := http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, "http://inventory:8081/stock", nil)
resp, err := client.Do(req)
```

For a deeper walk-through of tracing a multi-service Go system - including gRPC propagation - see [Distributed Tracing in Go with OpenTelemetry](/blog/monitoring-go-with-opentelemetry/).

### Metrics

Metrics use the same provider pattern, with a periodic reader instead of a batch processor. The reader collects and exports on an interval (60 s by default; 15–30 s is a common production choice):

```go
package main

import (
	"context"
	"fmt"
	"time"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp"
	"go.opentelemetry.io/otel/metric"
	sdkmetric "go.opentelemetry.io/otel/sdk/metric"
	"go.opentelemetry.io/otel/sdk/resource"
	semconv "go.opentelemetry.io/otel/semconv/v1.37.0"
)

func initMeterProvider(ctx context.Context) (*sdkmetric.MeterProvider, error) {
	exporter, err := otlpmetrichttp.New(ctx)
	if err != nil {
		return nil, fmt.Errorf("create metric exporter: %w", err)
	}

	res, err := resource.Merge(
		resource.Default(),
		resource.NewWithAttributes(
			semconv.SchemaURL,
			semconv.ServiceVersion("1.4.2"),
		),
	)
	if err != nil {
		return nil, fmt.Errorf("create resource: %w", err)
	}

	mp := sdkmetric.NewMeterProvider(
		sdkmetric.WithResource(res),
		sdkmetric.WithReader(sdkmetric.NewPeriodicReader(
			exporter,
			sdkmetric.WithInterval(15*time.Second),
		)),
	)
	otel.SetMeterProvider(mp)
	return mp, nil
}

func main() {
	ctx := context.Background()
	mp, err := initMeterProvider(ctx)
	if err != nil {
		panic(err)
	}
	defer mp.Shutdown(context.Background())

	meter := otel.Meter("checkout-service")

	orders, _ := meter.Int64Counter("orders.processed",
		metric.WithDescription("Number of orders processed"),
		metric.WithUnit("{order}"),
	)
	latency, _ := meter.Float64Histogram("payment.duration",
		metric.WithDescription("Payment processing duration"),
		metric.WithUnit("ms"),
	)

	// Record from anywhere in your request path:
	start := time.Now()
	// ... process a payment ...
	orders.Add(ctx, 1, metric.WithAttributes(attribute.String("payment.provider", "stripe")))
	latency.Record(ctx, float64(time.Since(start).Milliseconds()))

	time.Sleep(20 * time.Second) // let one export cycle run in this demo
}
```

Instruments are cheap to create but should be created once (package-level or in a struct), not per request. If you use `otelhttp` from the traces section, you also get standard HTTP server metrics - request duration and counts - without writing any of this by hand: it records them through the global `MeterProvider` automatically once one is registered.

### Logs

The logs signal in Go is **beta**. It is designed as a *bridge*: you keep logging with a normal Go logging API - ideally `log/slog` from the standard library - and the `otelslog` bridge forwards records to the OpenTelemetry logs pipeline, which exports them over OTLP. The payoff is that every log written with a request context automatically carries the active `trace_id` and `span_id`, so you can pivot from a slow trace to its exact log lines in OpenObserve.

```go
package main

import (
	"context"
	"fmt"

	"go.opentelemetry.io/contrib/bridges/otelslog"
	"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
	"go.opentelemetry.io/otel/log/global"
	sdklog "go.opentelemetry.io/otel/sdk/log"
	"go.opentelemetry.io/otel/sdk/resource"
	semconv "go.opentelemetry.io/otel/semconv/v1.37.0"
)

func initLoggerProvider(ctx context.Context) (*sdklog.LoggerProvider, error) {
	exporter, err := otlploghttp.New(ctx)
	if err != nil {
		return nil, fmt.Errorf("create log exporter: %w", err)
	}

	res, err := resource.Merge(
		resource.Default(),
		resource.NewWithAttributes(
			semconv.SchemaURL,
			semconv.ServiceVersion("1.4.2"),
		),
	)
	if err != nil {
		return nil, fmt.Errorf("create resource: %w", err)
	}

	lp := sdklog.NewLoggerProvider(
		sdklog.WithResource(res),
		sdklog.WithProcessor(sdklog.NewBatchProcessor(exporter)),
	)
	global.SetLoggerProvider(lp)
	return lp, nil
}

func main() {
	ctx := context.Background()
	lp, err := initLoggerProvider(ctx)
	if err != nil {
		panic(err)
	}
	defer lp.Shutdown(context.Background())

	// A drop-in *slog.Logger backed by the OTel pipeline.
	logger := otelslog.NewLogger("checkout-service")

	// Pass the request context so trace/span IDs are attached automatically.
	logger.InfoContext(ctx, "order processed",
		"order_id", "ord_8412",
		"amount_cents", 4599,
	)
	logger.ErrorContext(ctx, "payment declined", "reason", "insufficient_funds")
}
```

Always use the `...Context` variants (`InfoContext`, `ErrorContext`) inside request handlers - that is how the bridge correlates a log record with the span active in that context. Because the signal is beta, pin `go.opentelemetry.io/otel/sdk/log` and `go.opentelemetry.io/contrib/bridges/otelslog` in `go.mod` and check the changelog before bumping versions. If you prefer to defer adoption, a pragmatic interim setup is: traces and metrics via the SDK, plus JSON logs to stdout collected by an OpenTelemetry Collector or FluentBit and forwarded to OpenObserve.

## Sending data to OpenObserve

OpenObserve ingests OTLP natively - no collector required in between, although running one is a fine architecture too. Two transports are supported:

- **OTLP/HTTP** at `<host>/api/<org>/v1/traces`, `/v1/metrics`, and `/v1/logs` (port `5080` self-hosted). This is what the `otlptracehttp` / `otlpmetrichttp` / `otlploghttp` exporters above speak.
- **OTLP/gRPC** on port `5081` (self-hosted), which additionally requires an `organization` header alongside `Authorization`.

Stick with HTTP unless you have a reason not to; it needs the least configuration.

The exporters in the code above take no hardcoded endpoint on purpose - configuration comes from standard environment variables, so the same binary runs against local, staging, and cloud backends.

**OpenObserve Cloud:**

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.openobserve.ai/api/<your-org-name>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic%20<your-base64-token>"
export OTEL_SERVICE_NAME="checkout-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production"

go run .
```

**Self-hosted OpenObserve:**

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:5080/api/default"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic%20cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM="
export OTEL_SERVICE_NAME="checkout-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=dev"

go run .
```

Three rules that prevent 90% of setup failures:

1. **No trailing slash on the endpoint.** The exporter appends `/v1/traces` (or `/v1/metrics`, `/v1/logs`) itself. `.../api/default/` produces `.../api/default//v1/traces` and a 404.
2. **Percent-encode the space in the header value.** `OTEL_EXPORTER_OTLP_HEADERS` uses a `key=value,key=value` format in which the value must be URL-encoded - hence `Basic%20<token>`, not `Basic <token>`.
3. **The organization is part of the URL path.** `/api/default` targets the `default` organization; replace it with your org name in OpenObserve Cloud (visible on the ingestion/Data Sources page, which also shows the exact endpoint and token to copy).

If you would rather configure in code than via the environment - common for CLIs or when headers come from a secret manager - every exporter accepts options:

```go
exporter, err := otlptracehttp.New(ctx,
	otlptracehttp.WithEndpointURL("http://localhost:5080/api/default/v1/traces"),
	otlptracehttp.WithHeaders(map[string]string{
		"Authorization": "Basic cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=",
	}),
)
```

Note that `WithEndpointURL` takes the *full* signal path, whereas the environment variable takes the base URL. Environment variables win in fewer surprises; prefer them. For a broader tour of exporter configuration, see [A Beginner's Guide to OTLP Exporters](/blog/otel-exporters-introduction/).

## Verifying data arrival in OpenObserve

Run the service, send a few requests (`curl http://localhost:8080/checkout`), then open the OpenObserve UI (`http://localhost:5080` self-hosted, or your cloud URL):

1. **Streams** - you should see a `default` traces stream plus streams for your metrics and logs. If a stream exists, ingestion is working end to end.
2. **Traces** - pick your service (`checkout-service`) in the Traces page, and open a trace. You should see the `POST /checkout` server span from `otelhttp` with your `process-payment` child span nested under it, along with the attributes you set.
3. **Logs** - query the log stream and confirm records include `trace_id` and `span_id` fields; that confirms the `otelslog` bridge is correlating correctly.
4. **Metrics** - your `orders.processed` and `payment.duration` instruments appear as metric streams you can chart or attach to dashboards and alerts.

If nothing appears within ~30 seconds, jump to Troubleshooting below - and check your process's stderr first, because the Go SDK reports export failures there.

## Production tips

- **Keep the batch processor (never `SimpleSpanProcessor` in production).** `WithBatcher` batches and compresses exports off the hot path. The defaults - 5 s schedule delay, 512-span batches, 2048-span queue - are sensible; tune `OTEL_BSP_*` variables only if you see queue-full drops.
- **Sample at the head, keep parents authoritative.** For high-traffic services, ratio sampling with parent-based override is the standard starting point. Unlike some other language SDKs, the Go SDK does **not** read `OTEL_TRACES_SAMPLER` / `OTEL_TRACES_SAMPLER_ARG` - configure the sampler in code when you build the `TracerProvider`:

  ```go
  tp := sdktrace.NewTracerProvider(
      sdktrace.WithSampler(sdktrace.ParentBased(sdktrace.TraceIDRatioBased(0.1))),
      sdktrace.WithBatcher(exporter),
      sdktrace.WithResource(res),
  )
  ```

  `ParentBased(...)` matters: it ensures a downstream Go service honors the sampling decision already made upstream, so you get complete traces instead of fragments. Tune the ratio (`0.1` keeps 10% of root traces) to match your traffic.
- **Invest in resource attributes.** `service.name`, `service.version`, and `deployment.environment` are the dimensions you will filter by in every OpenObserve query. Because the resource is built with `resource.Merge(resource.Default(), ...)`, `service.name` and `deployment.environment` are picked up from `OTEL_SERVICE_NAME` / `OTEL_RESOURCE_ATTRIBUTES` (so operations can override them per environment without a rebuild), while `service.version` is set in code.
- **Propagate context into goroutines deliberately.** `go func() { ... }()` with a fresh `context.Background()` orphans any spans started inside. Capture the request context (or `trace.SpanContext`) explicitly when fanning out work, and remember the request context is canceled when the handler returns - derive with `context.WithoutCancel(ctx)` (Go 1.21+) for fire-and-forget work that should still be trace-linked.
- **Shut down every provider on exit.** Tracer, meter, and logger providers each buffer data. Wire all three `Shutdown` calls into your signal-handling path with a timeout, as in the trace example.
- **Record errors on spans, not just in logs.** `span.RecordError(err)` plus `span.SetStatus(codes.Error, err.Error())` is what makes failed requests filterable in the Traces UI.
- **Pin and upgrade `otel` modules together.** The API, SDK, exporters, and contrib modules are versioned in lockstep; mixing releases is the most common cause of confusing compile errors after an upgrade. `go get go.opentelemetry.io/otel@latest` alone is not enough - update the exporters and contrib packages in the same commit.

## Troubleshooting

**Symptom: exporter logs `404 Not Found` and no data arrives.**
Fix: almost always a URL problem. Remove any trailing slash from `OTEL_EXPORTER_OTLP_ENDPOINT`, and confirm the path includes your organization (`/api/default`, not just the host). The final URL the exporter calls should look like `http://localhost:5080/api/default/v1/traces`.

**Symptom: `401 Unauthorized` on every export.**
Fix: regenerate the token with `echo -n 'email:password' | base64` (the `-n` matters - a trailing newline corrupts the token), and make sure the header value in `OTEL_EXPORTER_OTLP_HEADERS` is `Authorization=Basic%20<token>` with the space percent-encoded. If you set headers in code via `WithHeaders`, use a literal space there instead - the encoding rule applies only to the environment variable.

**Symptom: traces appear but each service shows disconnected fragments instead of one distributed trace.**
Fix: context propagation is broken. Confirm `otel.SetTextMapPropagator` is called with `propagation.TraceContext{}` in *every* service, outgoing clients use `otelhttp.NewTransport`, and requests are created with `http.NewRequestWithContext(ctx, ...)` - a request built without the context sends no `traceparent` header.

**Symptom: the service exits cleanly but the last few spans/logs never arrive.**
Fix: the batch processors were never flushed. Call `Shutdown` (or `ForceFlush`) on the tracer, meter, and logger providers before exit, with a context timeout so a dead backend cannot hang your shutdown. This bites short-lived jobs and CLIs hardest - for those, consider `ForceFlush` after each unit of work.

**Symptom: spans created inside goroutines are missing or parented to the wrong trace.**
Fix: the goroutine started from `context.Background()`. Pass the request-scoped `ctx` into the goroutine; if the work outlives the request, wrap it with `context.WithoutCancel(ctx)` so cancellation doesn't kill the span mid-flight.

**Symptom: compile errors like mismatched types across `otel` packages after `go get -u`.**
Fix: partially upgraded modules. Update `go.opentelemetry.io/otel`, the `sdk`, all `exporters/otlp/...` modules, and `go.opentelemetry.io/contrib/...` packages together, then run `go mod tidy`. Check each module's release notes - the log-signal packages in particular can change between minor versions while in beta.

With the SDK wired up once in `main()`, instrumentation libraries handling the boilerplate, and OTLP pointed at OpenObserve, a Go service gets correlated traces, metrics, and logs in a single backend - searchable side by side, with no vendor-specific code anywhere in your application.