zstd

Zstandard, or zstd, is a fast lossless compression algorithm built for real-time scenarios. It matches or beats zlib on compression ratio while running significantly faster on both compression and decompression. It's aimed at developers and infrastructure teams who need to compress data streams, files, or small records without paying a heavy CPU tax.

The project is the reference C implementation of the Zstandard format, which is stabilized and documented in RFC8878. It ships as both a library (libzstd) and a command-line tool that can produce and decode .zst, .gz, .xz, and .lz4 files. Bindings for other languages exist as separate ports, listed on the Zstandard homepage.

Key features

• Configurable speed/ratio trade-off: compression levels, including negative --fast=# levels, let you trade ratio for raw throughput; decompression speed stays roughly constant regardless of level.
• High throughput: benchmarks on the Silesia corpus show zstd at level 1 compressing around 510 MB/s and decompressing around 1550 MB/s, ahead of zlib and comparable fast compressors on ratio.
• Dictionary training for small data: a --train mode builds a dictionary from sample files, which dramatically improves compression ratio and speed on small, similar records (e.g., JSON blobs, log lines).
• Multi-format CLI: the zstd binary can decode .gz, .xz, and .lz4 files in addition to its native .zst format.
• Multiple build systems: supports make (the reference build), CMake, Meson, vcpkg, Conan, Buck, Bazel, and Visual Studio project files.
• Production-hardened: deployed at Meta and other large infrastructures, and continuously fuzzed by Google's oss-fuzz project.

Ideal use cases

• Compressing logs, backups, or data pipelines where CPU time and I/O throughput matter more than squeezing out the last few percent of compression ratio.
• Storing or transmitting many small, structurally similar records (API responses, database rows, JSON documents) where dictionary training gives an outsized ratio improvement.
• Replacing zlib/gzip in systems that need faster compression/decompression at a similar or better ratio, without switching to a slower algorithm like brotli or lzma.
• Building or integrating a compression layer in a language other than C, by using one of the community bindings listed on the Zstandard homepage rather than this repo directly.

It's not the right choice when you need the absolute highest compression ratio regardless of speed (dedicated slow-mode algorithms outperform zstd's higher levels in that narrow case), or when your data set is so small and one-off that dictionary training isn't worth the setup.

Installation

The reference build uses make:

make

This generates the zstd CLI at the repo root and builds libzstd in lib/. Other useful targets:

make install   # install zstd cli, library, and man pages
make check     # run zstd and test essential behavior locally

CMake is also supported:

cmake -S . -B build-cmake
cmake --build build-cmake

For a Universal2 (Apple Silicon + Intel) build:

cmake -S . -B build-cmake-debug -G Ninja -DCMAKE_OSX_ARCHITECTURES="x86_64;x86_64h;arm64"
cd build-cmake-debug
ninja
sudo ninja install

Other supported options include Meson (see build/meson), vcpkg:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install zstd

Conan:

conan install --requires="zstd/[*]" --build=missing

Buck (buck build programs:zstd) and Bazel (via the Bazel Central Repository module) are also available, along with Visual Studio project files under build/ for Windows.

Once built, dictionary-based compression for small data works like this:

zstd --train FullPathToTrainingSet/* -o dictionaryName
zstd -D dictionaryName FILE
zstd -D dictionaryName --decompress FILE.zst