How To Crawl A Web Page with Scrapy

Photo by Arian Darvishi on Unsplash

Scrapy is one of the most popular and powerful Pythons scraping libraries; it takes a “batteries included” approach to scraping, meaning that it handles a lot of the common functionality that all scrapers need so developers don’t have to reinvent the wheel each time. It makes scraping a quick and fun process. Scrapy, like most Python packages, is on PyPI (also known as pip). PyPI, the Python Package Index, is a community-owned…