MDX Table of Contents: From rehype Extraction to Sticky Gutter Layout

Adding a table of contents to article pages sounds simpler than page transitions. The layout alone went through four iterations — float-right invading the prose column, position: fixed scrolling away with the page, Nav rolling off-screen and setting top to -716px…

This post documents the final solution and the root cause of each pitfall.

#The Goal

• Extract h2 / h3 headings from MDX at build time
• Desktop: TOC in the viewport gutter to the right of the 720px article column
• At page top: TOC aligned with the first line of prose
• On scroll: pin only after the prose start reaches the pin line; stay visible through the rest of the article
• Mobile: collapsible <details> block above the prose
• Highlight the current section (IntersectionObserver)

#Stack Choice: Build-Time Extraction, Not Runtime DOM Scanning

Our stack is Next.js 16 + @next/mdx + RSC static rendering. The best community fit:

In next.config.ts, order matters — these must run before Expressive Code:

1
rehypePlugins: [
2
  'rehype-slug',
3
  '@stefanprobst/rehype-extract-toc',
4
  '@stefanprobst/rehype-extract-toc/mdx',
5
  // ...expressive-code
6
],

In the page:

1
const { default: MDXContent, tableOfContents } = await loadPostMDX(locale, slug)

h2 / h3 in mdx-components.tsx must accept id, or anchor links won't work:

1
h2: ({ children, id }) => (
2
  <h2 id={id} className="... scroll-mt-6">{children}</h2>
3
),

#Pitfall 1: Server Can't Call Functions from Client Files

Initially filterToc() lived in 'use client' TocNav.tsx. Calling it from Server Component Toc.tsx threw:

Attempted to call filterToc() from the server but filterToc is on the client

Fix: Move the pure function to src/lib/toc.ts for shared use.

#Pitfall 2: float-right Inside a 720px Column Looks Wrong

The design spec said float: right, but inside a max-width: 720px article container the TOC wraps into the prose flow and fights code blocks for space.

Fix: Keep the article at 720px; position the TOC with position: fixed in the viewport gutter:

1
.toc-gutter {
2
  position: fixed;
3
  width: 168px;
4
  left: calc(50vw + min(360px, 50vw - 32px) + 32px);
5
}

Similar to Josh W. Comeau's layout approach — the TOC lives outside the article structure, in the screen margin.

#Pitfall 3: fixed Scrolls Away (Portal)

With the TOC inside PageTransition's motion.div, position: fixed is relative to that layer, not the viewport. It scrolls with the page — no sticky behavior at all.

Fix: createPortal to document.body:

1
return createPortal(gutter, document.body)

#Pitfall 4: Nav Rolling Off-Screen Makes top Negative

For a while we used Nav's getBoundingClientRect().bottom + 16 as the pin line when scrollY > 0. But our Nav is not sticky — once it scrolls away, bottom becomes something like -716px, and the TOC flies off the top of the screen.

Fix:

1
function getPinTop(): number {
2
  const nav = document.querySelector('.site-nav')
3
  if (!nav) return 24
4
  return Math.max(24, nav.getBoundingClientRect().bottom + 16)
5
}

When Nav is visible, pin below it. After Nav leaves, fall back to 24px from the viewport top.

#Pitfall 5: When Exactly to Pin?

The requirement evolved three times:

The final scroll logic is one line:

1
const pinTop = getPinTop()
2
const proseTop = prose.getBoundingClientRect().top
3
setGutterTop(Math.max(pinTop, proseTop))

#article-prose marks the prose start; recalculate on scroll, resize, and ResizeObserver.

#Final Architecture

1
page.tsx
2
├── PostDetail (720px article + mobile collapsible TOC)
3
├── Toc variant="fixed" (Portal → body)
4
└── PostNav
5

6
MDX pipeline
7
rehype-slug → rehype-extract-toc → /mdx → Expressive Code
8

9
TocNav (Client)
10
├── IntersectionObserver → active section highlight
11
├── getPinTop() → pin line (with floor)
12
└── max(pin, proseTop) → align + pin

Fixed TOC shows at ≥ 1080px viewport width; narrower screens hide it and show the in-article <details> version instead.

#Summary

If Nav becomes position: sticky later, getPinTop() adapts automatically — no TOC changes needed.