I remember there was one tool written using Python, if not mistaken, it's called Scrapy, which is used for the purpose of web scraper or web crawler.

I also remember HTTracks, by the way, the interface of WebHarvy is very user-friendly and effective.