Vector

Vector is an open-source observability data pipeline written in Rust. It collects logs and metrics from your infrastructure, transforms them, and routes them to any destination you choose, whether that's a SaaS monitoring vendor, a data lake, or another pipeline stage. It's built for teams running production infrastructure who need to control observability costs, avoid vendor lock-in, or consolidate multiple logging agents into one tool. Vector is maintained by Datadog's Community Open Source Engineering team and used in production by companies like Atlassian, Comcast, Discord, and Zendesk.

Key features

• End-to-end deployment: runs as either a lightweight agent on edge hosts or a centralized aggregator, so you can build a full pipeline with one tool.
• Unified data model: handles logs and metrics through the same pipeline, reducing the need for separate collectors per data type.
• Broad source and sink support: sources include files, HTTP, Kafka, journald, and Docker logs; sinks include S3, Elasticsearch, ClickHouse, CloudWatch, and cloud storage, among many others.
• Transform and enrichment: built-in transforms (VRL remap, filter, dedupe, log-to-metric, Lua) let you reshape, filter, and enrich data before it leaves your infrastructure.
• Rust-based reliability: memory-safe implementation with delivery guarantees and disk buffer persistence, avoiding common crash and data-loss issues in older agents.
• High throughput: benchmarks in the project's test harness show it outperforming Filebeat, FluentBit, FluentD, Logstash, and Splunk forwarders on several common protocols.
• Vendor neutrality: open-source and designed to let you switch or add downstream vendors without rewriting your collection layer.

Ideal use cases

Vector fits teams that want to reduce reliance on a single observability vendor's agent, especially if they're running several different collectors (Filebeat, FluentD, Telegraf) and want to consolidate them into one binary. It's a good match for cost-sensitive environments where you want to filter, sample, or aggregate data before sending it to a paid metrics or log platform, cutting ingestion volume and spend.

It also works well for organizations migrating between observability vendors, since Vector's routing and transform layer decouples your applications from a specific backend. Kubernetes environments benefit from Vector's agent/aggregator model for centralizing log and metric collection across a cluster.

Vector is not the right choice if you need distributed tracing today. Trace support is not part of the current feature set. It's also probably overkill for a single small service that just needs to ship a few log lines to one destination with minimal configuration; a lighter, purpose-built shipper might be simpler in that case.

Installation

The fastest way to try Vector is the quickstart guide at vector.dev, which walks through installing a binary and running a sample pipeline. Vector ships prebuilt binaries, container images, and packages for most operating systems.

Container images are published and can be pulled directly:

docker pull ghcr.io/vectordotdev/vector:latest

For manual installation, download a release binary for your platform from the releases page and follow the manual installation docs. Vector also has dedicated install paths for Kubernetes, where it typically runs as a DaemonSet (agent role) alongside optional aggregator deployments, and for common package managers depending on your OS.

After installing, configure Vector with a TOML, YAML, or JSON config file that defines sources, transforms, and sinks, then start it with:

vector --config /etc/vector/vector.yaml

Use vector validate against your config file before deploying to catch configuration errors early. Full reference docs for sources, transforms, sinks, and the VRL remap language are available at vector.dev/docs.