Anyone have some good examples for beautiful complex web scraper in bs4?
Currently I'm writing my own web scraper using beautiful soup 4. Mostly for learning purposes, but I would like to upload the code to github when I'm satisfied with it. After countless frustrating hours I finally was able to build a working prototype but the code is, well, not really nice to look at... So my current objective is to make my scraper more representable.
At the moment I'm trying to adapt PEP8 but I think what would help me the most would be some good examples of complex web scraper using bs4 and how they manage to still be readable and well organized. Using Google I only find easy examples and tutorials. A quick Github search wasn't really successful neither. :-(
So are there any well written bigger web scraper you guys could recommend so that I can see how a well organized scraper looks like?
Thanks in advance!
Posted Jun 26, 2017, 9:05 PM UTC