Python-Sitemap

A mini website crawler written in Python that builds a sitemap.xml from every public link it finds. It has a simple usage mode and an advanced mode, and it runs in Docker too. Video sitemap support reads each video tag's title, falling back to the alt text when no title exists. A long crawl can be resumed: running the same command again continues from the saved file instead of starting over. By default the crawler reads only the raw HTML the server returns, so links added purely by client-side scripts are not followed unless configured. GPL-3.0 licensed and free to use.