turbovec

turbovec is an in-process vector index for Python and Rust that stores embeddings at 2 or 4 bits per dimension using Google Research's TurboQuant method, then searches the compressed form directly with SIMD kernels. It targets developers building RAG pipelines, agent memory, or any nearest-neighbor search system who need to keep large embedding collections in memory without the cost of float32 storage or the overhead of a separate vector database service.

Key features

• Compression: stores vectors at 4-bit (7.8x smaller than float32) or 2-bit (15.5x smaller) while matching FAISS recall at the same bit rate.
• Speed: benchmarks show 4.4x faster search than FAISS FastScan at 4-bit and 2.2x faster at 2-bit, on both ARM and x86.
• Online updates: no training step or codebook fitting from your data. Vectors are searchable immediately after add(), and remove() runs in microseconds instead of FAISS's full-corpus repack.
• Incremental persistence: sync(path) writes only the delta since the last call, with a single fsync and crash safety at any byte.
• In-process, offline: it's a library, not a server. No network calls, no external dependencies at query time, pairs with any local embedding model for a fully offline stack.
• Filtered search: pass an allowlist of ids or a bitmask to restrict results to a subset without a separate post-filter pass.
• ID mapping: IdMapIndex lets you key vectors by your own uint64 ids that survive deletes, instead of relying on raw slot positions.
• Drop-in framework integrations: ships adapters for LangChain, LlamaIndex, Haystack, and Agno that match each framework's in-memory vector store interface.

Ideal use cases

turbovec fits when you're holding tens of thousands to low millions of embeddings in a single process and want brute-force exact-scan search over compressed vectors, with memory and speed advantages over storing float32 and using FAISS's PQ-based indexes. It works well for RAG systems, agent memory stores, and embedding-heavy applications where you control the process and don't want to run a separate vector database.

It's not the right choice if you need a graph-based index like HNSW for hundreds of millions of vectors under strict latency budgets, since turbovec scans every vector and cost grows linearly with corpus size. It's also not a fit if you need exact float results: quantization is lossy, and low-dimensional embeddings like GloVe lose more recall than high-dimensional ones. Finally, it has no network API, replication, or multi-tenant features, so if you need a server-based vector database, look elsewhere. Vector dimensions must be multiples of 8 and no wider than 16,384, with only 2-bit and 4-bit widths supported.

Installation

For Python:

pip install turbovec

Basic usage:

import numpy as np
from turbovec import TurboQuantIndex

vectors = np.load("embeddings.npy").astype(np.float32)

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)

scores, indices = index.search(vectors[:1], k=5)

index.sync("index.tv")
index = TurboQuantIndex.load("index.tv")

For Rust:

cargo add turbovec

use turbovec::{IdMapIndex, TurboQuantIndex};

let mut index = TurboQuantIndex::new(1536, 4)?;
index.add(&vectors);
let results = index.search(&queries, 10);
index.write("index.tv")?;

Framework integrations install as extras, for example pip install turbovec[langchain]. To build from source, use maturin build --release for the Python wheel or cargo build --release for the Rust crate; x86_64 builds target a baseline instruction set and select AVX-512/AVX2 kernels at runtime depending on the CPU.