# OpenTelemetry Collector: Setup, Config & Best Practices > Install and configure the OpenTelemetry Collector - receivers, processors, exporters, deployment patterns, and shipping OTLP data to OpenObserve. Source: https://openobserve.ai/opentelemetry/collector/ Published: 2026-07-08 Language: Collector --- The OpenTelemetry Collector is the workhorse of most production telemetry pipelines: a standalone, vendor-neutral service that receives traces, metrics, and logs in a dozen formats, processes them (batching, filtering, enrichment, sampling), and exports them to one or more backends such as [OpenObserve](https://openobserve.ai/). This guide covers what the Collector is, when you actually need one, how to install and configure it, the deployment patterns that matter, and a complete working configuration for shipping data to OpenObserve. If you are new to OpenTelemetry as a whole - the signals, OTLP, the SDKs - start with [What is OpenTelemetry? A Complete Guide](/blog/what-is-opentelemetry/) and come back here for the Collector specifics. ## What the Collector Is - and When You Need One An OpenTelemetry SDK inside your application can export OTLP directly to a backend. So why insert another process in the middle? The Collector is a pipeline engine. Data flows in through **receivers**, through a chain of **processors**, and out through **exporters**. Because it runs outside your application, it can do work you do not want in your request path: - **Offload quickly.** Applications hand telemetry to a nearby Collector in milliseconds and move on. Retries, backoff, TLS handshakes, and slow backends become the Collector's problem, not your service's. - **Centralize credentials and routing.** API tokens live in one place. Switching backends - or fanning out to two during a migration - is a config change, not a redeploy of every service. - **Process before you pay.** Drop health-check spans, debug logs, and unused metrics before they reach (and get billed by) your backend. - **Receive non-OTLP data.** Prometheus scrape targets, log files, syslog, Kafka topics, StatsD - the Collector translates them all into one pipeline. **When you can skip it:** honestly, for a single service, a local dev setup, or a small deployment sending modest volume straight to OpenObserve, direct SDK export works fine and is one less thing to operate. The OpenTelemetry project itself describes direct export as appropriate for development and small-scale environments. Adopt a Collector when you have multiple services, need to control cost or enforce sampling centrally, ingest non-OTLP sources, or want telemetry to survive backend outages via queuing and retries. ## Core vs. Contrib: Choosing a Distribution The project publishes several official distributions built from the same codebase: | Distribution | Binary | What's in it | |---|---|---| | **Core** | `otelcol` | A solid baseline - otlp, hostmetrics, jaeger, kafka, prometheus, zipkin receivers; batch, memory_limiter, attributes, resource, span, probabilistic_sampler, filter processors; debug, otlp, otlp_http, file, kafka, prometheus, prometheusremotewrite, zipkin exporters; zpages, health_check, pprof extensions; forward connector. Small surface, conservative stability. | | **Contrib** | `otelcol-contrib` | The full community catalog - hundreds of receivers, processors, exporters, and connectors (filelog, prometheus, kafka, filter, transform, tail_sampling, and many more). | | **K8s** | `otelcol-k8s` | A curated set for Kubernetes use cases, sized between core and contrib. | | **Custom (OCB)** | yours | Built with the OpenTelemetry Collector Builder (`ocb`): exactly the components you list, nothing else. Smallest binary and attack surface. | Practical guidance: start with **contrib**. Core is too minimal for most real pipelines - it lacks transform, filelog, tail_sampling, k8sattributes, and the long tail of contrib-only receivers/processors. Once your config stabilizes, consider building a custom distribution with OCB so production runs only the components you actually reference. To check whether a component you need exists and how mature it is, search the [OpenTelemetry Registry](https://opentelemetry.io/ecosystem/registry/) or the `opentelemetry-collector-contrib` repository README - each component documents its stability level (development, alpha, beta, stable) per signal. For a tour of the most useful contrib components and when to reach for them, see [OpenTelemetry Collector Contrib: A Comprehensive Guide](/blog/opentelemetry-collector-contrib/). ## Installing the Collector ### Docker The fastest way to get a Collector running locally: ```bash docker run --rm \ -v $(pwd)/otel-collector.yaml:/etc/otelcol-contrib/config.yaml \ -p 4317:4317 -p 4318:4318 \ otel/opentelemetry-collector-contrib:latest ``` Ports 4317 (OTLP/gRPC) and 4318 (OTLP/HTTP) are the standard receiver ports every OpenTelemetry SDK defaults to. Pin a specific version tag in anything beyond a quick test - the Collector releases frequently and `latest` will move under you. Images are published to Docker Hub (`otel/opentelemetry-collector-contrib`) and GitHub Container Registry (`ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib`). ### Binary and Linux packages Every release on the [opentelemetry-collector-releases](https://github.com/open-telemetry/opentelemetry-collector-releases/releases) page ships tarballs plus `.deb` and `.rpm` packages for each distribution and architecture. The packages install a `systemd` service that reads `/etc/otelcol-contrib/config.yaml`: ```bash # Example: contrib distribution on Debian/Ubuntu (amd64) # Replace with the latest release tag from the releases page, e.g. 0.156.0 # (asset filenames are versioned, so /latest/download/ returns 404) curl -sSLO https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v/otelcol-contrib__linux_amd64.deb sudo dpkg -i otelcol-contrib__linux_amd64.deb sudo systemctl status otelcol-contrib ``` Edit the config, then `sudo systemctl restart otelcol-contrib`. ### Kubernetes with Helm The official chart deploys the Collector as a DaemonSet (agent pattern) or Deployment (gateway pattern): ```bash helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts helm install otel-collector open-telemetry/opentelemetry-collector \ --set mode=daemonset \ --set image.repository="otel/opentelemetry-collector-contrib" ``` Set `mode=deployment` for a gateway tier. Your pipeline config goes under the chart's `config:` value, which is merged over sane defaults. Kubernetes also has the **OpenTelemetry Operator**, which manages Collectors via an `OpenTelemetryCollector` custom resource and adds auto-instrumentation injection for pods. It is the right tool once you run many Collector instances with different configs; for a first deployment, the Helm chart is simpler and this guide sticks with it. ## Configuration Anatomy A Collector config has up to five top-level component sections plus a `service` section that wires them together. Defining a component does nothing by itself - it must also be referenced in a pipeline (or in `service.extensions`) to be active. ```yaml # 1. Receivers: how data gets IN receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 # default is localhost-only; bind wider deliberately http: endpoint: 0.0.0.0:4318 # 2. Processors: what happens to data in flight processors: memory_limiter: check_interval: 1s limit_percentage: 80 # soft cap: % of total available memory spike_limit_percentage: 25 # headroom above the soft cap resource: attributes: - key: deployment.environment.name value: production action: upsert batch: send_batch_size: 8192 timeout: 5s # 3. Exporters: where data goes OUT exporters: otlp_http: endpoint: https://api.openobserve.ai/api/your-org headers: Authorization: Basic debug: # prints telemetry to stdout; dev only verbosity: basic # 4. Extensions: capabilities outside the data path extensions: health_check: endpoint: 0.0.0.0:13133 # GET / -> 200 when healthy zpages: {} # live debug pages on localhost:55679 # 5. Service: activate components and define pipelines service: extensions: [health_check, zpages] pipelines: traces: receivers: [otlp] processors: [memory_limiter, resource, batch] exporters: [otlp_http] metrics: receivers: [otlp] processors: [memory_limiter, resource, batch] exporters: [otlp_http] logs: receivers: [otlp] processors: [memory_limiter, resource, batch] exporters: [otlp_http] ``` Things worth internalizing: - **Pipelines are per-signal.** `traces`, `metrics`, and `logs` pipelines are independent; a component listed in a pipeline must support that signal. - **Processor order matters.** Data flows through processors in exactly the order listed. Receivers and exporters, by contrast, fan in/out - order in their lists is irrelevant. - **Components can be instantiated multiple times** with the `type/name` syntax, e.g. `otlp_http/openobserve` and `otlp_http/other-backend`, each with its own settings. - **Environment variable substitution** uses `${env:VAR_NAME}` (with defaults via `${env:VAR_NAME:-fallback}`). Use it to keep credentials out of config files. - **Connectors** are a fifth component type that acts as the exporter of one pipeline and the receiver of another - e.g. the `spanmetrics` connector turns spans into RED metrics. You will not need one on day one, but know they exist. Validate a config without starting the service: ```bash otelcol-contrib validate --config=otel-collector.yaml ``` ## Deployment Patterns: Agent vs. Gateway There are two fundamental shapes, and mature setups usually combine them. ### Agent A Collector instance runs on every host - a **DaemonSet** in Kubernetes, a system service on VMs, or occasionally a sidecar per pod. Applications export to `localhost` (or the node IP), which keeps the network hop short and reliable. The agent is the right place for anything host-local: tailing container log files with the `filelog` receiver, collecting host metrics, and enriching telemetry with Kubernetes metadata via the `k8sattributes` processor (which must run near the workload to resolve pod identity). ### Gateway One horizontally scaled pool of Collectors per cluster, datacenter, or region, behind a load balancer, exposing a single OTLP endpoint. The gateway is where centralized policy lives: credentials for the backend, tail-based sampling, uniform filtering and redaction, and rate control. Costs: one more tier to operate, and a bit of added latency. One subtlety: because a load balancer spreads requests across gateway instances, spans of one trace can land on different Collectors. Anything that needs a complete trace (tail sampling) or all data for one service (span-to-metrics aggregation) requires a two-tier gateway where the first tier runs the **load-balancing exporter** routing by `traceID` or `service` to the second tier. ### Combining them The common production topology is **agent → gateway → backend**: node agents do local collection and metadata enrichment, then forward OTLP to the gateway, which applies sampling and cost controls and exports to OpenObserve with a single set of credentials. Small clusters can run gateway-only (apps export straight to the gateway Deployment); single-node setups can run agent-only. ## Sending Data to OpenObserve OpenObserve accepts OTLP natively, so the standard `otlp_http` exporter is all you need. Two rules from the OpenObserve docs that account for most integration failures: 1. The endpoint is your OpenObserve base URL **plus the organization path**: `.../api/` - and it must **not** end with a trailing slash. The exporter appends `/v1/traces`, `/v1/metrics`, and `/v1/logs` itself. 2. Authentication is HTTP Basic: base64-encode `email:password` (use an ingestion token as the password - generate one under **Data sources** in the OpenObserve UI rather than using your login password). Generate the header value: ```bash echo -n "you@example.com:your-token" | base64 ``` ### Complete config: OpenObserve Cloud ```yaml receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: memory_limiter: check_interval: 1s limit_percentage: 80 spike_limit_percentage: 25 batch: send_batch_size: 8192 timeout: 5s exporters: otlp_http/openobserve: endpoint: https://api.openobserve.ai/api/your-org-name headers: Authorization: "Basic ${env:OPENOBSERVE_TOKEN}" service: pipelines: traces: receivers: [otlp] processors: [memory_limiter, batch] exporters: [otlp_http/openobserve] metrics: receivers: [otlp] processors: [memory_limiter, batch] exporters: [otlp_http/openobserve] logs: receivers: [otlp] processors: [memory_limiter, batch] exporters: [otlp_http/openobserve] ``` Here `OPENOBSERVE_TOKEN` holds the base64 string from the command above. Your organization name and a ready-made, copy-pasteable version of this exporter block (with the token filled in) are shown in the OpenObserve UI under **Data sources → Custom → OTEL Collector**. ### Self-hosted OpenObserve Only the endpoint changes - the default organization on a fresh install is `default`, and OTLP/HTTP shares the main port 5080: ```yaml exporters: otlp_http/openobserve: endpoint: http://localhost:5080/api/default headers: Authorization: "Basic ${env:OPENOBSERVE_TOKEN}" ``` Replace `localhost` with your service hostname as appropriate (e.g. `http://openobserve.observability.svc.cluster.local:5080/api/default` in Kubernetes). Two optional refinements: - **Stream name.** Logs land in the `default` stream unless you add a `stream-name: my-stream` header to the exporter to route them elsewhere. - **gRPC instead of HTTP.** OpenObserve also accepts OTLP/gRPC (port 5081 self-hosted) via the `otlp` exporter; it additionally requires an `organization: your-org` header, and for plain-HTTP local endpoints you must set `tls: { insecure: true }`. OTLP/HTTP is the simpler default - it traverses proxies and load balancers without ceremony. Start the Collector, generate some traffic, and data should appear in OpenObserve under Streams within seconds. ## Production Processors: Ordering, Batching, and Cost Control ### memory_limiter first, batch last The two processors every production pipeline needs, in a specific arrangement: ```yaml processors: memory_limiter: check_interval: 1s limit_percentage: 80 spike_limit_percentage: 25 batch: send_batch_size: 8192 send_batch_max_size: 16384 timeout: 5s service: pipelines: traces: receivers: [otlp] processors: [memory_limiter, batch] # this order, always exporters: [otlp_http/openobserve] ``` - **`memory_limiter` goes first** - immediately after the receivers. It periodically checks memory usage; above the soft limit it refuses new data (returning errors so receivers propagate backpressure to senders, which retry), and above the hard limit it forces garbage collection. If it sits later in the chain, upstream processors can balloon memory before it ever engages. Use `limit_mib`/`spike_limit_mib` instead of percentages if you prefer absolute values. - **`batch` goes last**, just before the exporters, so it groups fully processed data into efficient export requests. Batching dramatically reduces request count and compresses better. The batch processor defaults are `send_batch_size: 8192` and `timeout: 200ms` (whichever is reached first triggers a flush); the examples in this guide bump `timeout` to 5s as a common custom starting point that batches more aggressively while still bounding latency. The general ordering rule for anything in between: **drop early, enrich late**. Sampling and filtering processors go right after `memory_limiter` so you stop spending CPU on data you will discard; `resource`/`attributes`/`transform` enrichment follows; `batch` closes the chain. ### Filtering for cost control The `filter` processor (contrib) drops whole spans, metrics, or log records matching OTTL conditions. Classic wins - dropping health-check spans and debug-level logs: ```yaml processors: filter/noise: error_mode: ignore traces: span: - attributes["url.path"] == "/healthz" - attributes["url.path"] == "/readyz" logs: log_record: - severity_number < SEVERITY_NUMBER_INFO ``` Every record dropped here is ingestion, storage, and query load you never pay for. For a deeper treatment of log filtering strategies at the Collector, see [Filter Logs at the Source in the OTel Collector](/blog/filter-logs-at-source-in-otel-collector/). ### Transforming and redacting The `transform` processor applies OTTL statements to mutate data in flight - trim oversized attributes, normalize names, scrub PII: ```yaml processors: transform/scrub: error_mode: ignore log_statements: - context: log statements: - replace_pattern(body, "\\b\\d{16}\\b", "****REDACTED****") - delete_key(attributes, "http.request.header.authorization") ``` Related single-purpose processors: `resource` (add/modify resource attributes, like tagging environment or region), `attributes` (same for span/log/metric attributes), and `redaction` (allowlist-based attribute scrubbing). ## Scaling and Resilience ### Sending queue and retries Exporters like `otlp_http` ship with a sending queue and retry logic - enabled by default, but worth configuring deliberately: ```yaml exporters: otlp_http/openobserve: endpoint: https://api.openobserve.ai/api/your-org-name headers: Authorization: "Basic ${env:OPENOBSERVE_TOKEN}" sending_queue: enabled: true num_consumers: 10 queue_size: 5000 # batches held in memory during backend slowness retry_on_failure: enabled: true initial_interval: 5s max_interval: 30s max_elapsed_time: 300s # give up after 5 minutes ``` The queue absorbs backend blips; retries with exponential backoff handle transient failures. When the queue fills, new data is rejected and the failure is counted - watch for it (below). For durability across Collector restarts, add the `file_storage` extension and point the queue at it (`sending_queue.storage: file_storage`); queued data then survives a crash or redeploy. ### Scaling out Collectors are mostly stateless, so the gateway tier scales horizontally behind a load balancer - for OTLP/HTTP any L7 balancer works. The exceptions are stateful components: tail sampling and span-metrics aggregation need all spans of a trace (or service) on one instance, which is what the **load-balancing exporter** in a first-tier Collector solves by consistently routing on `traceID` or `service` to a second tier. Agents scale by definition - one per node. Watch each Collector's own telemetry: it exposes internal metrics (Prometheus format on `localhost:8888/metrics` by default) including `otelcol_receiver_accepted_*`, `otelcol_exporter_sent_*`, `otelcol_exporter_send_failed_*`, and queue size/capacity - the four numbers that tell you whether the pipeline is keeping up. ## Troubleshooting **No data arriving in OpenObserve.** Bisect the pipe. First confirm data reaches the Collector: temporarily add the `debug` exporter (`verbosity: detailed`) alongside the real one - `exporters: [otlp_http/openobserve, debug]` - and watch stdout. If nothing prints, the problem is upstream (SDK endpoint, ports 4317/4318 reachable, receiver bound to `0.0.0.0` rather than `localhost` when apps connect over the network). If data prints but never lands, read the Collector logs for export errors and check the exporter endpoint. **`401 Unauthorized` / `403` from the exporter.** The Authorization header is wrong. Rebuild it: `echo -n "email:token" | base64` - the `-n` matters, a trailing newline corrupts the encoding. Confirm the header reads `Basic ` with exactly one space, and that the token belongs to a user in the organization named in the URL. **`404 Not Found` on export.** Almost always a malformed endpoint: a trailing slash after the org (`/api/default/` - remove it), a missing `/api/` path segment, or `/v1/traces` appended manually (the exporter adds signal paths itself). **Collector OOM-killed or memory climbing.** Verify `memory_limiter` exists and is the *first* processor in every pipeline. Check whether the sending queue is growing because the backend is slow or down (`otelcol_exporter_queue_size` vs `..._queue_capacity`). Reduce `queue_size`, `send_batch_size`, or receiver concurrency, and set container memory limits so the Collector's soft limit is computed against reality. **Data silently dropped.** Look at internal metrics on `:8888`: nonzero `otelcol_exporter_send_failed_spans` (or `_log_records`/`_metric_points`) with exhausted retries means the backend rejected data; `otelcol_processor_refused_*` points at `memory_limiter` engaging; `enqueue_failed` counters mean the sending queue overflowed. Each counter names a different fix: backend health, memory headroom, or queue size. **Config loads but a component "does not exist."** You are running core but referencing a contrib-only component (e.g. `filter`, `transform`, `filelog`) - switch to the `otelcol-contrib` binary/image - or the component is defined in its section but the error is a typo in the `service.pipelines` reference. `otelcol-contrib validate --config=...` catches both before deploy. For live inspection, enable the `zpages` extension and open `http://localhost:55679/debug/pipelinez` to see exactly which components are wired into which pipelines, and `/debug/tracez` for recent span activity.