firecrawl/firecrawl

Firecrawl is an established project for scraping web data for AI agents. On 11 18, the official team released Firecrawl Local, with code located in this repository under apps/local. It is installed from crates.io; a single program can convert web pages into clean Markdown, crawl entire sites, or list all URLs. Pages requiring JavaScript are rendered using the local Chrome installation, and PDFs and Office documents are parsed locally. The serve subcommand provides the same v2 interface as the cloud version on local port 3002, and local routes do not require an API Key. Search, Alexandria data, structured extraction, and Agent features still require Firecrawl Cloud.

Firecrawl is an established project for scraping web data for AI agents. On 11 18, the official team released Firecrawl Local, with code located in this repository under apps/local. It is installed from crates.io; a single program can convert web pages into clean Markdown, crawl entire sites, or list all URLs. Pages requiring JavaScript are rendered using the local Chrome installation, and PDFs and Office documents are parsed locally. The serve subcommand provides the same v2 interface as the cloud version on local port 3002, and local routes do not require an API Key. Search, Alexandria data, structured extraction, and Agent features still require Firecrawl Cloud.

Why it matters Scraping pages, crawling entire sites, and listing URLs are all done locally, without needing databases, queues, or containers; the local interface is identical to the cloud version, so existing code only needs a URL change to work.

Stars 204248 (+825 / 24h)

Reasons

  • 825 new stars in 24 hours, total stars 204248 (indicating attention only); the main body is AGPL-3.0, while SDKs and some UI components are MIT licensed
  • The official changelog states that Firecrawl Local is based on Rust and fully open source, running 37 times faster than the self-hosted version with 230 times less memory usage

Usage

  • After installing Rust and a C compiler, run cargo install firecrawl-local, then try scraping a page with firecrawl-local scrape https://firecrawl.dev --markdown; running firecrawl-local init adds it as an MCP service to local tools like Claude Code, Cursor, Codex, and Gemini CLI

Risks

  • By default, single pages that cannot be scraped locally are handed off to Firecrawl Cloud for scraping (results marked firecrawl-cloud); to keep everything strictly local, set FIRECRAWL_CLOUD=off
  • The init command modifies configuration files for multiple agent tools (backups are created first), and the MCP service requires Node.js 22 or higher; if the AGPL-3.0 components are provided as a network service, modifications must be disclosed

Open

Trendshift(GitHub Trending 快照)

Explore the latest AI Radar updates