I created a web crawler that could handle malformed HTML, server errors, circular link paths, and links outgoing to different websites. For the design, I created one class to parse responses and one class to perform the business logic (eg. what pages to visit and what to do with the parsed information).
Tech: Python