Web Crawler

I created a web crawler that could handle malformed HTML, server errors, circular link paths, and links outgoing to different websites. For the design, I created one class to parse responses and one class to perform the business logic (eg. what pages to visit and what to do with the parsed information).

Tech: Python

Sorry, this project is in a private repo.

If you'd like to see the code, please feel free to email me.