# crawler4seo

> crawler4seo is a free, open-source SEO and GEO (Generative Engine Optimization) audit tool that runs entirely on your local machine. It ships 20 analyzers covering technical SEO, content quality, structured data, hreflang, Core Web Vitals, and AI-crawler readiness — plus an MCP server so Claude Code and other AI agents can drive crawls and query results directly.

Everything runs locally against a SQLite database. No data leaves your machine. Available as a CLI, desktop app (macOS / Windows / Linux), and an MCP server that any LLM agent can call.

## What it does

- **Full technical SEO audit** — titles, meta descriptions, H1/H2/H3 hierarchy, canonicals, robots.txt compliance, redirect chains, noindex/nofollow, URL normalization, SSL, sitemaps.
- **GEO audit** — per-bot robots.txt rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended and 5 more; llms.txt presence; JSON-LD schema coverage; JS-dependency risk; 0–100 GEO score.
- **Internal-link graph with PageRank** — discover your highest-authority pages and orphaned content.
- **Schema.org structured data** — Rich Results validation, type coverage, nesting issues.
- **hreflang clusters** — detect broken reciprocal tags, missing x-default, conflicting language codes.
- **Core Web Vitals** — via PageSpeed Insights integration.
- **Duplicate content detection** — near-duplicate and exact-duplicate clustering by content shingle.
- **Crawl diffs** — compare two snapshots to see what changed between deploys.

## Key pages

- [Homepage](https://crawler4seo.com/): overview, features, screenshots, download links.
- [GEO audit](https://crawler4seo.com/audit-geo/): how crawler4seo scores a site for AI-engine readiness.
- [SEO audit](https://crawler4seo.com/audit-seo/): the 20-analyzer technical SEO audit breakdown.
- [Alternatives](https://crawler4seo.com/alternatives/): comparisons with Screaming Frog, Ahrefs, Sitebulb, Semrush, and other crawlers.
- [GitHub repository](https://github.com/simobain/crawler4seo): source code, issues, release notes.

## How to use it

- **CLI / desktop**: `npm install -g crawler4seo` or download the desktop app from the homepage.
- **MCP server**: point Claude Code or any MCP-compatible agent at the crawler4seo MCP server to run crawls, inspect issues, and query the PageRank graph through natural language.
- **Local-first**: crawls write to a local SQLite DB. Data never leaves the machine; there is no SaaS backend.

## License & contact

MIT-licensed. Source, issues, and releases on [GitHub](https://github.com/simobain/crawler4seo).