Transcript for:
Guide to Web Scraping E-commerce Sites

hi in this video I'm going to show a quick demo how using the web scraper you can scrape a simple ecommerce site let's start by looking at the site the site has a two-level navigation with categories and subcategories each subcategory has a list of products that lead to product pages in this tutorial I will show how to make a scraper that scrapes these products by navigating through all of the categories and subcategories to open web scrapers toolbar open developer tools where you can find web scraper tap I'll start by creating a new site map let's call this site map ecommerce the start URL will be the landing page of this site web scraper will start scraping from this URL and then it will navigate the rest of the site now I am going to build multiple selectors that will be arranged in a tree like structure similarly how the site is built these selectors will define how the site will be navigated and how the data is going to be extracted to visit these categories I'm going to create a link selector the selector will be called category link and I will change its type to link now I can select the category link elements from the page by clicking select button I'm also checking the multiple check box because multiple links will be found in this page the selector is now ready to validate the selector I click data preview to see if it works now let's go to one of the category pages to create a subcategory link selector also I am clicking on the category selector so that the subcategory selector is executed in category page similarly like the category link selector I will create a subcategory selector you and I am also checking the selector with data preview I can move forward to the subcategory page here I will create a link selector that will navigate to product pages you at this point the selectors are configured to navigate from start URL to product pages in all categories under the product link selector I will add multiple data extraction selectors to extract data from a product page I'll start by creating a text selector that will extract the name of this product similarly I'll add text selectors to extract price and description you to extract the image URL I'll create an image selector as you can see there are different types of selectors to extract specific data types now I have all the product information selectors that I wanted I can quickly check them with data preview also you should go to other product pages to check whether the selectors work as expected there after creating selectors you can open selector graph to get an overview how the sitemap is built here you can see how the scraper will navigate to product page through all of the link selectors the site map is now complete and I can launch the scraper locally I'll open the scrape view and click start scraping button as you can see web scraper launched a pop-up window in which it is loading URLs to extract data while the data is being scraped you can click the refresh button to check that everything is working as expected I'll pause the video while the scraper is running the scraper window is now closed and here you can see all of the scraped data to export the data I'm going to open export data view where a download link will appear you and here you can see the scraped data if you want to learn more about web scraper go to web scraper io there you will find more video tutorials and documentation about all of the selector types