Finding all the URLs on a website is one of the most vital tasks in a web scraping workflow. In this tutorial, we’ll explain URL acquisition methods such as leveraging the power of Google search all the way to exploring SEO expert level software like ScreamingFrog, and even crafting our own Python script to grab URLs at scale from a sitemap. Don't worry, we've got you covered on formulating a list of URLs to scrape (and as a bonus we'll even show you how to scrape some data as well).
