LibreCrawl

A free desktop crawler for SEO work, offered as an open source alternative to Screaming Frog. It crawls websites, analyzes internal and external links, extracts titles, meta descriptions and headings, and exports results without subscription fees. Other features include JavaScript rendering for React, Vue and Angular pages, n-gram phrase analysis and PageSpeed Insights integration, where a Google API key raises the daily rate limits. It also runs as a web based multi-tenant crawler for website auditing. A plugin system lets you build and share extensions, with an available data reference, and a full API reference is published on its own domain. Written in Python with Flask, MIT licensed, and customizable and extensible. It is intended for agencies and companies as well as individual users.