puppeteer-consistency

Puppeteer — consistent, modern idioms

Puppeteer is stable, but generated code mixes eras: the removed page.waitForTimeout, headless: true from before the new-headless transition, page.$x XPath helpers, and click-then-waitForNavigation races. The serialization boundary of page.evaluate is the other constant source of silent bugs. This skill pins one canonical idiom set — Puppeteer 21+ semantics — so scripts are deterministic and leak-free.

Canonical idioms — always X, never Y

Always	Never	Why
await page.waitForSelector(sel) / page.locator(sel).wait()	await page.waitForTimeout(3000)	waitForTimeout was removed; time-based waits are flaky by construction.
Promise.all([page.waitForNavigation(), page.click(sel)])	await page.click(sel); await page.waitForNavigation()	Navigation can finish before the wait starts — a classic race/timeout.
pass data as args: page.evaluate((t) => ..., token)	closing over Node variables inside evaluate	The callback is serialized and runs in the browser; outer scope does not exist there.
try { ... } finally { await browser.close(); }	letting errors skip close()	Orphaned Chrome processes accumulate and exhaust memory/CI runners.
page.locator("::-p-text(Submit)").click() or waitForSelector + act	page.$(sel) then acting on a maybe-null handle	Locators wait and re-resolve; bare $ returns null silently on a miss.
headless: true (current default "new" headless)	headless: "new" string or old-headless assumptions	22+ made new headless the default; the "new" literal is obsolete.
page.$$eval(sel, els => els.map(e => e.textContent))	looping page.$$ handles for text extraction	One evaluate round-trip; element handles are for interaction, not bulk reads.
explicit waitUntil: choice on goto ("domcontentloaded" or "networkidle2")	assuming goto waits for your data	Default is load; SPA content often arrives later — wait for the selector you need.
const browser = await puppeteer.launch() once, pages per task	a new browser per page/screenshot	Launch is the expensive part; reuse the browser, isolate via contexts/pages.

House style for a scrape:

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();
  try {
    const page = await browser.newPage();
    await page.goto("https://example.com/products", { waitUntil: "domcontentloaded" });
    await page.waitForSelector("[data-product]");
    const products = await page.$$eval("[data-product]", (cards) =>
      cards.map((c) => ({
        name: c.querySelector(".name")?.textContent?.trim(),
        price: c.querySelector(".price")?.textContent?.trim(),
      })),
    );
    console.log(products);
  } finally {
    await browser.close();
  }
})();

Pitfalls that produce silently wrong results

• evaluate returns only serializable values. DOM nodes, functions, and class instances come back as {}/undefined — extract plain data inside the callback.
• ReferenceError: x is not defined inside evaluate means a closed-over Node variable; pass it as an argument after the callback.
• page.$eval throws on no match; page.$ returns null — know which you called when handling "element missing".
• textContent vs innerText: textContent includes hidden text and lacks layout whitespace; pick deliberately and .trim().
• Default 30s timeouts on goto/waitFor mask logic errors as timeouts; set page.setDefaultTimeout(...) consciously and treat timeouts as failures to investigate.
• Viewport defaults to 800×600 — responsive pages render the mobile/narrow branch; set defaultViewport explicitly for screenshots.
• networkidle0/2 never settles on pages with polling/websockets — prefer a concrete selector wait.
• page.click on a covered/animating element silently clicks the wrong thing pre-wait; wait for visibility first (waitForSelector(sel, { visible: true })).

Version notes

Target Puppeteer 21+ (22/23+ keep the idioms). Key breaking lines: 21 removed page.waitForTimeout; 22 switched the default to the new headless mode and page.$x/ waitForXPath went away (use ::-p-xpath selectors if XPath is truly needed). If you see headless: "new" warnings, the project predates 22 — write headless: true and note it.

Workflow

1. One browser, explicit lifecycle: launch with the options you mean (headless, defaultViewport, args), close in finally.
2. For every page action, decide the wait target first — a selector, a navigation, a response (page.waitForResponse) — never a duration.
3. Cross the evaluate boundary consciously: arguments in, plain data out.
4. Extract in bulk with $$eval/$eval; interact via locators/waited selectors.
5. When reviewing, hunt the classics: waitForTimeout, click/navigation races, closures into evaluate, missing browser.close(), bare page.$ null-chains.

For the fuller migration map, locator/selector reference, network interception, and screenshot/PDF recipes, read references/puppeteer-patterns.md.