Project 04 · Go

OTLP Tap

A Go project for understanding OTLP below the dashboard: receive telemetry over gRPC or HTTP, decode it, process it, store it in Parquet, and query it back as OTLP JSON.

The pipeline

Receive
OTLP over gRPC and HTTP
Decode
Protobuf to an internal model
Process
Batching, tail sampling, backpressure
Store
Columnar Parquet files on disk
Query
OTLP-shaped JSON API → Grafana

Three signals, one path

Traces
Spans are persisted and queried back as OTLP-shaped trace data.
Metrics
Gauge, sum, histogram, exponential histogram, and summary points share one storage path.
Logs
Log records keep trace and span ids as top-level fields for trace-to-log lookup.

Why build a receiver from scratch

Most observability tools hide the protocol and storage details behind a dashboard. OTLP Tap keeps those details visible: how the OTLP request is decoded, how each signal becomes rows, how processors fit between a receiver and a store, and how query results are shaped back into OTLP.

The project implements milestones 1-8 from its roadmap: traces, metrics, logs, Parquet/Arrow storage, an OTLP-shaped query API, a Grafana datasource plugin, and optional batch, tail-sampling, and backpressure processors. A parity smoke test against the upstream OpenTelemetry Collector caught receiver behavior such as gzip-compressed OTLP, but the project does not try to be a replacement for the Collector.

Built

OTLP/gRPC and OTLP/HTTP receivers
Protobuf decode → internal model
Columnar storage on Parquet + Arrow
Query API with cross-signal joins
Grafana datasource integration
Tail sampling, backpressure, cardinality tracking

What’s real here

Nothing on this page runs OTLP Tap. It is a server-side Go pipeline, while this site is static. The page describes the real codebase and links to the repository for the receiver, storage, query API, and Grafana plugin.

Stack
Go · OTLP · protobuf
Status
Milestones 1-8 implemented: receive, store, query, Grafana
Why
Breaking into observability work
View on GitHub