Collectives™ on Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Teams
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
Learn more about Teams
Ask Question
Am new to using puppeteer and i want to scrape some data of of a certain website but i get these warnings and i get no data displayed on console.
Why do I get the following warnings, and how can I get rid of them?
Here is the code i used :
const puppeteer = require("puppeteer");
(async () => {
// prepare for headless chrome
const browser = await puppeteer.launch();
const page = await browser.newPage();
// set user agent (override the default headless User Agent)
await page.setUserAgent(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36"
// go to website home page
await page.goto("https://www.nytimes.com/");
await page.waitForSelector("body");
// get the User Agent on the context of Puppeteer
const userAgent = await page.evaluate(() => navigator.userAgent);
var rposts = await page.evaluate(() => {
postItems = [];
let posts = document.body.querySelectorAll(".assetWrapper");
posts.forEach((item) => {
try {
title = item.querySelector("h2").innerText;
link = item.querySelector("a").href;
summary = item.querySelector("p").innerText;
postItems.push({ title: title, link: link, summary: summary });
} catch (e) {}
var items = {
posts: postItems,
return items;
// If everything correct then no 'HeadlessChrome' sub string on userAgent
console.log(userAgent);
console.log(rposts);
await browser.close();
})();
here are the errors am getting :
(node:4072) UnhandledPromiseRejectionWarning: ReferenceError: browser is not defined
(node:15452) UnhandledPromiseRejectionWarning: ReferenceError: Cannot access 'page' before initialization
Create browser with headless, then you can work with those code.
const browser = await puppeteer.launch({
headless: false
Regards,
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.