Goal
Each page has a few persons.
Create an agent and find a way to scrape every pages.
Documentation
Instructions
-
Create the agent
scraping-challenge-2
-
Access this page using
Puppeteer.newPage() and Puppeteer.Page.goto() to open the page.
-
Wait for an element to load before scraping with
Puppeteer.Page.waitForSelector().
-
Use the function
Puppeteer.Page.addScriptTag() to inject jQuery in the page.
-
Create a function to scrape the page with jQuery now that it is injected and execute it in the page using
Puppeteer.Page.evaluate().
-
Use the url pagination and a for loop and repeat the scraping.
-
Send the data using
buster.setResultObject().
Solution