01 · Web · Scale · SEO-driven
Next.js + Postgres + edge caching
Millions of indexable pages with incremental regeneration, a CDN in front, and a content pipeline separate from the app.
Agent fit4/5 Scale · Run it for many Reviewed
The stack
- Framework
- Next.js App Router with ISR
- Data
- Postgres (read replicas) via Prisma or Drizzle, plus a search index
- Auth
- Only on the parts that are not public
- Hosting
- Vercel, or self-hosted behind a CDN
- Also
-
- Headless CMS or a content pipeline in its own package
- Meilisearch or Typesense for on-site search
- Lighthouse CI and a crawl budget dashboard
Why agents do well here
4/5- Incremental regeneration is a per-page setting the agent can read, so caching behaviour is visible in the route file instead of in infrastructure.
- Separating the content pipeline into its own package lets the agent change how pages are built without touching how they are rendered, and vice versa.
- Next.js at this size has the most public examples of large-scale SEO sites, so the agent's patterns for sitemaps, canonicals and structured data are well rehearsed.
Avoid at this tier
- Rendering everything on demand; decide per route what is static, revalidated or dynamic.
- Letting search and rendering share a database connection pool.
- A rewrite to a newer framework for performance you have not measured.
How it fits together
At this scale the site is two systems: a content pipeline that produces rows, and a rendering app that turns rows into cached pages. Keep them in one monorepo with a shared schema package so the agent can still trace a field end to end.
Folder layout
apps/web/ Next.js: app/, features/, lib/seo/
packages/schema/ Prisma or Drizzle schema, generated types
packages/content/ ingestion, enrichment, search indexing jobs
packages/ui/ the component kit First five decisions
Lock these in before the agent writes the second feature. Each one removes a choice it would otherwise make differently every time.
- 01 ISR with explicit revalidation per route
- 02 Read replicas for rendering
- 03 A search index that is rebuilt not patched
- 04 A Lighthouse budget in CI
- 05 A shared schema package so both systems agree
When to move on
This is the top tier. From here the work is keeping the boundaries explicit, not changing the stack. The Web Scale page lists what stays the same.