A chatbot walks into a laboratory

“Most AI has been trained on the final artifacts of science, like the final paper, the final recount in a textbook. That isn’t often how the scientific process unfolded – it’s sort of a retelling of the story.” –Liam Fedus, Co-Founder of Periodic Labs

Liam Fedus and Dogus Cubuk are the co-founders and co-CEOs of Periodic Labs, a startup building “synthesis superintelligence.” In practice, that means an AI system that closes the loop between hypothesis, experiment, and learning.

Liam previously led post-training at OpenAI, where he was one of the creators of ChatGPT. Dogus spent years at Google DeepMind leading a large team of chemists and materials scientists. Together, they’ve brought those two trajectories to bear on a problem that has stymied science for decades: not just discovering new materials, but making them reliably and at scale.

In this conversation, we explore the limits of language models, the commercialization bottleneck in materials science, why Periodic chose high-temperature superconductors as its first target, and what Liam learned from watching ChatGPT go from an internal product to one of the most successful products of all time.

From my notebook

We’re trying a new thing where I share a few of my thoughts from after the episodes. The moments that struck me, the tidbits that reframed my thinking, or something I just really enjoyed. Let me know if you like reading these!

1. To do science you have to do science. This is Periodic’s conviction in a nutshell. Science is the discovery of new things; you cannot prove it in a corpus of existing information. It has to involve trial and error in the real world.

2. Sci-fi remains an endless source of alpha. Why did Periodic set an initial goal to discover a new high-temperature superconductor? In part because ~every sci-fi future features it. (It also doesn’t hurt that it’s stayed top of mind in condensed matter physics.)

3. I underestimated the gap between discovery and utility. It can take 10-20 years for a new material to get to industrial scale. This feels like an equally important place for AI. We don’t want a century of discoveries stuck in twenty year manufacturing bottleneck.

4. Virality is ~impossible to predict. With the benefit of hindsight, it’s insane to hear Liam say that people inside OpenAI expected ChatGPT to attract tens of thousands of users. Perhaps the closer you are to something, the harder to know how useful it will be.

5. Capability ≠ virality. Another interesting wrinkle on the ChatGPT story. Liam said that they had a stronger internal model that hadn’t gone viral internally. ChatGPT used a weaker one!

6. Towards fuller observability of science. When humans conduct experiments, they write up the parts they believe matter. The ‘useless’ data you generate at every other step is discarded. This feels like one of the coolest parts of Periodic’s approach. In theory you can capture the full experimental trace.

7. Little things are big things. A bunch of samples get mislabeled in a lab. Usually, that would take a bunch of irritating human time to unwind. At Periodic’s lab, AI found it and fixed it.

8. “Contamination-free” RL is sort of an experimental veil of ignorance? Liam talked through one of Periodic’s advantages, which is constructing “snapshots” of experimental states. They can then ask: “What should we have done next?” This is possible because they have a corpus of fresh, untainted data. As a result, the model hasn’t inherited the right answer.

Tangents

What I’m into this week:

• Nuclear War: A Scenario by Annie Jacobsen. Reading for…oh, no reason.

• DJI Mic Mini plus WisprFlow is a pretty perfect combination for dictation.

• Instinct is the best AI proof of concept I’ve seen in ~6 months. Good reasons to tread lightly, but the utility is incredible

—Mario

Every timestamp

(00:00) Intro

(02:05) Periodic’s mission

(03:14) What it takes to build a synthetic superintelligence

(04:48) Why “thinkism” falls short

(05:46) Why Periodic is pursuing high-temperature superconductors

(07:21) How Heike Kamerlingh Onnes discovered superconductivity

(09:36) Why synthesizing new materials is so difficult

(16:44) Dogus’s path to AI

(20:18) Liam’s path to AI

(23:28) Lessons from launching ChatGPT

(25:24) Why Dogus and Liam chose each other

(27:01) How Periodic attracts top talent

(29:50) Inside an experimental loop at Periodic

(35:10) What AI learns across experiments

(40:10) What Periodic builds versus buys

(44:37) Balancing diversity and depth across experiments

(49:23) The difference in LLM performance on math vs. science

(51:26) Why Periodic doesn’t depend on better models

(53:20) Periodic’s short-term goals

(54:09) The competitive landscape

(55:39) Final meditations

Liam and Dogus’s complete reading list

• The Beginning of Infinity: Explanations That Transform the World (recommended by Liam): https://www.amazon.com/Beginning-Infinity-Explanations-Transform-World/dp/0143121359

• Subtle Is the Lord: The Science and the Life of Albert Einstein (recommended by Dogus): https://www.amazon.com/Subtle-Lord-Science-Albert-Einstein/dp/0192806726

Other resources

• Periodic Labs: https://periodic.com

• Bell Labs: https://www.nokia.com/bell-labs

• QBasic: https://en.wikipedia.org/wiki/QBasic

• Google DeepMind: https://deepmind.google

• Google Brain: https://en.wikipedia.org/wiki/Google_Brain

• OpenAI: https://openai.com

Where to find Liam Fedus

LinkedIn: https://www.linkedin.com/in/liam-fedus-26547811

X: https://x.com/LiamFedus

Where to find Dogus Cubuk

LinkedIn: https://www.linkedin.com/in/ekin-dogus-cubuk-9148b8114

X: https://x.com/ekindogus

Subscribe to the show

I’d love it if you’d subscribe and share the show. Your support makes all the difference as we try to bring more curious minds into the conversation.

YouTube

Spotify

Apple

Production and marketing by penname.co. For inquiries about sponsoring the podcast, email [email protected].