Heritrix

Heritrix is an open-source, extensible web crawler designed for Web archiving, originally developed at the Internet Archive. Archiving institutions use it for broad and focused crawls, writing output commonly to WARC files.

See also

• Wayback Machine
• WARC
• Web archiving

Sources

• Heritrix project documentation and IIPC tooling notes.