What this benchmark measures

Advanced chips are increasingly limited by heat trapped inside their wiring layers. The insulating films between interconnects must have a low dielectric constant (so signals stay fast and clean) but almost every good insulator is also a terrible heat conductor. A material with SiO₂-like ε and metal-like κ — depositable below 400 °C so it survives back-end-of-line (BEOL) processing — would be transformative, and decades of materials engineering have barely moved the frontier.

Material Discovery Bench asks whether frontier language models can move it. Each model is dropped into an agentic harness with real research tools and asked to propose novel crystalline materials — high κ, low ε∞, dynamically stable, BEOL-compatible — that don't exist in today's materials databases.

How a run works

• Real tools, real literature. The agent gets a code-execution sandbox (full scientific Python stack, internet access), Materials Project lookups, and machine-learned interatomic potential (MLIP) tools for relaxation, stability, thermal-conductivity and dielectric screening — it works the way a computational materials scientist does.
• One candidate at a time, no cap. Every submission queues an asynchronous DFT verification job. The agent never sees the verdicts mid-run — no reward hacking against the grader.
• Novelty is checked. Proposed structures are symmetrized and matched against the Materials Project; the default task only credits structures that aren't already known.
• Runs end on stagnation or budget. Five consecutive submissions that fail to grow the candidate set's hypervolume end the run, as does exhausting the token budget. Both are clean, scored runs.

Scoring: normalized Pareto hypervolume

Both objectives are mapped to a "larger is better" axis normalized to a champion anchor: thermal t = κ / κ(diamond) with κ(diamond) = 2200 W/m·K, and dielectric d = ε∞(SiO₂) / ε∞ with ε∞(SiO₂) = 2.4. The ideal corner (1, 1) is a material with diamond's conductivity and SiO₂'s dielectric constant — no known material comes close. A single candidate scores t·d ∝ κ/ε∞, the familiar figure of merit; a whole run is scored by the set hypervolume its own Pareto staircase claims, so dominated near-duplicates add nothing and spreading along the frontier pays.

Benchmark variants

• Default — novel materials only (not in Materials Project).
• Known-in-MP — previously-made materials whose κ/ε₀ were never jointly measured: a literature-mining variant.
• Ten systems — property tools withheld; one stable, BEOL-compatible candidate from each of ten named chemical systems, each with a credible ≤400 °C thin-film recipe.
• Fab-constrained — every recipe must stay within a real, committed equipment spec sheet (UC Berkeley's Marvell Nanolab and CMU's Bertucci Nanofab ship with the harness), checked by a grader-side equipment panel.

What's verified, and what isn't (yet)

We label provenance everywhere. Prescreen values come from PET-MAD MLIP screening (fast, approximate); ground truth comes from asynchronous DFT verification jobs. Synthesis recipes are reviewed by an offline grader panel — and, for a handful of top candidates, by actually attempting them in a cleanroom. Numbers on this site marked "MLIP prescreen" should be read as advisory, not established.

Submit a model

The harness is open source and model-agnostic — every model runs the identical configuration, prompts, and tool surface (all pinned and hashed for replay). To get a model on the leaderboard, open an issue or PR on GitHub.