Anubis

Anubis is a Web AI Firewall Utility written in Go. It sits in front of your web server and "weighs the soul" of each incoming connection with a challenge before letting the request reach your origin. It's built for site operators, hobbyists, and small teams running self-hosted services (forums, git hosts, wikis, blogs) who are getting hammered by scraper traffic from AI companies and don't want to route everything through a big CDN.

The project describes itself bluntly as a "nuclear response." It can block smaller, less aggressive scrapers along with some legitimate crawlers (like the Internet Archive) unless you explicitly allowlist them through bot policy definitions. If you're already comfortable putting your origin behind Cloudflare or a similar CDN, the README suggests you probably don't need Anubis. It exists for the cases where you can't or won't do that.

Key features

• Proof-of-work challenge: forces incoming clients to solve a computational challenge before their request reaches your backend, which raises the cost of mass scraping.
• Bot policy definitions: a configuration layer that lets you allowlist or block specific user agents, IP ranges, or request patterns instead of applying the challenge to everyone.
• Lightweight footprint: written in Go and designed to run with minimal resource overhead so small operators can afford to deploy it.
• Reverse proxy placement: drops in front of an existing web server or application without requiring changes to the upstream service itself.
• Self-hosted and open source: MIT licensed, no dependency on a third-party CDN or vendor.
• Packaged for common distros: available through several package repositories, tracked on Repology as "anubis-anti-crawler."

Ideal use cases

Anubis fits operators running self-hosted forges, wikis, blogs, or forums who are seeing their bandwidth and CPU consumed by AI crawler traffic and have no CDN in front of the site. It also suits people who run infrastructure on constrained or metered connections where every unnecessary scraper request has a real cost.

It is not a good fit if your priority is maximum discoverability by all crawlers, including smaller or niche bots and archival crawlers like the Internet Archive, unless you're willing to spend time tuning the allowlist policies. It's also probably overkill if a CDN like Cloudflare is already an acceptable option for your setup; the README itself says most people should just use that instead. Anubis is aimed at the specific gap where CDN-based protection isn't available or desired.

Installation

The README doesn't include full command-by-command install instructions inline; it points to the documentation site at anubis.techaro.lol for setup and configuration details, including how to write bot policy definitions. Anubis is also tracked on Repology under the name "anubis-anti-crawler," so check whether your distro's package manager already carries a build.

For a from-source setup, the standard Go workflow applies:

git clone https://github.com/TecharoHQ/anubis.git
cd anubis
go build ./cmd/anubis

After building, place the resulting binary in front of your existing web server as a reverse proxy, then define bot policies to control which clients get challenged, allowlisted, or blocked. Consult the docs site for policy syntax and deployment examples specific to your stack.