Open SEO Crawler

A self hosted, open source SEO crawler and website audit that doubles as an AI crawler checker. It crawls every page for technical issues, then tests whether Googlebot, Bingbot, Applebot and AI crawlers such as GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot and CCBot receive the same readable page, flagging pages that a crawler is blocked from, challenged on or redirected away from. It is CMS aware, exports to XLSX, includes a broken link checker and crawl budget analysis, and offers bulk reports. It positions itself as a drop in alternative to Screaming Frog, Sitebulb and Ahrefs Site Audit with no accounts, no API keys and no per URL limits. An MCP server and a command line tool let Claude Code, Claude Desktop and Cursor start crawls, and a free companion Chrome extension, SEO Side Panel, covers the single open page. Written in Python under the MIT license.