Listen now | Baseten CEO Tuhin Srivastava on inference as AI’s “final market.”
͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­͏     ­
Forwarded this email? Subscribe here for more
 
The Generalist
The Generalist
Why inference is harder than …
0:0044:16
 
Listen now
 

Why inference is harder than you think

Baseten CEO Tuhin Srivastava on inference as AI’s “final market.”

Mario Gabriele
Nov 17
 
READ IN APP
 

“If AI is going to eat the world, all that’s going to matter is how that AI is run. And even in a world where there is AGI, the only thing left to do will be inference: to run the intelligence itself.” – Tuhin Srivastava, Co-founder and CEO of Baseten


Tuhin Srivastava is the Co-founder and CEO of Baseten, a company building infrastructure for training and inference. When an AI model runs in production (responding to a user, powering an app, making a decision), Baseten is one of the companies that makes that happen reliably, quickly, and at scale. The company counts Abridge, Cursor, and OpenEvidence among its customers, spans 20 clouds and 90 regions, and was most recently valued at $13 billion following a $1.5 billion funding round.

Tuhin sees inference as AI’s “final market”: as AI spreads through the economy, the cost of running intelligence grows with it. Baseten’s breakout began when it discarded three years of other product work in roughly six weeks to focus on an inference engine it had been giving away.

In this conversation, we explore why inference has become one of the most consequential and least understood markets in AI, what it actually takes to run a model in production at scale, and why the compute squeeze may be the single largest constraint on AI growth right now. We also examine the customer journey from closed models to custom fine-tuned deployments, the concept of owned intelligence, and why Tuhin believes luck accounts for roughly 90% of startup success, while insisting that the 10% skill is still indispensable.

Listen on YouTube • Spotify • Apple Podcasts

From my notebook

  1. “Inference is the final market” is a great provocation. If AI takes over more of the economy, then the inference opportunity could grow to a truly ludicrous size. Tuhin’s back-of-the-napkin estimate: about $2 trillion of inference spend by 2030.

  2. Off by 4X. Baseten’s January 2025 forecast versus year-end actuals was off by four times, and its forecasts have been wrong every year since 2023. It’s fascinating to me how difficult it is for even extremely sharp, knowledgeable humans to adjust to exponentials.

  3. The hyperscalers’ blind spots are organizational. Inference bridges compute, networking, observability, and workflow primitives that live in four different internal teams. A customer using a hyperscaler is effectively dealing with four different organizations at once, not just one.

  4. “Owned intelligence” as a thesis (and possible customer moat). This is a theme I hear from an increasing number of companies. Deploy open-weight models, generate production data, capture feedback signals, post-train custom models, redeploy, repeat. The idea is that companies that close that loop accumulate compounding advantages.

  5. Lessons from hospitality. Tuhin referenced Danny Meyer’s full service philosophy as a hiring lens. Alongside things like intellectual curiosity, Baseten looks for a high-end hospitality mindset. Teammates should talk to customers every day and go the extra mile. That leads to deep relationships, with one Baseten employee recently attending a customer’s wedding!


Tuhin’s bookshelf

When I asked Tuhin what book he’d assign to everyone, he named Breakneck. As mentioned above, he also pointed to Danny Meyer’s book about Union Square Cafe, Setting the Table.


Tangents

What I’m reading, watching, and into this week:

  • Few people outside of the UK seem to have heard of Taskmaster. It is one of the most fun shows you could hope to watch, and its latest season is no exception

  • An idea I have been having that I think I will write a piece on soon is the trend towards intensive, personal surveillance. AI relies on context to do its best work and as it becomes more capable, the right amount of context for it may be: everything. That might lead to some really strange behaviors that effectively amount to state-style surveillance of yourself (e.g. tracking your movement through cameras, recording your every word, and so on.)

—Mario


Every timestamp

(00:00) Intro

(02:06) What Baseten is building and how it began

(04:47) Finding balance in a fast-growing startup

(06:45) Why inference is the “final market”

(08:34) Tuhin’s path to founding Baseten

(12:06) What Australia gets right

(13:43) The opportunity Baseten found in inference

(20:34) How customers use Baseten

(29:00) Managing compute across clouds

(30:40) The compute crunch

(32:11) Financing inference at scale

(35:40) Why inference is hard to build in-house

(36:41) Who Baseten wants to hire

(39:46) Hard work and skill vs. luck

(42:20) Final meditations


Resources

  • Baseten: https://www.baseten.co

  • Sierra: https://sierra.ai

  • Coinbase: https://www.coinbase.com

  • No Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava: https://www.youtube.com/watch?v=oe3OSlsf4YA

  • Etched: https://www.etched.com

  • Cerebras: https://www.cerebras.ai

  • NVIDIA Groq 3 LPX: https://www.nvidia.com/en-us/data-center/lpx

  • Unreasonable Hospitality: The Remarkable Power of Giving People More Than They Expect: https://www.amazon.com/Unreasonable-Hospitality-Remarkable-Giving-People/dp/0593418573

  • Blackbird VC: https://www.blackbird.vc

  • Greylock: https://greylock.com

  • Conviction: https://www.conviction.com


Where to find Tuhin

X: https://x.com/tuhinone

LinkedIn: https://www.linkedin.com/in/tuhin-srivastava


Subscribe to the show

I’d love it if you’d subscribe and share the show. Your support makes all the difference as we try to bring more curious minds into the conversation.

YouTube

Spotify

Apple


Production and marketing by penname.co. For inquiries about sponsoring the podcast, email [email protected].

You're currently a free subscriber to The Generalist. For the full experience, upgrade your subscription.

Upgrade to paid

 
Like
Comment
Restack
 

© 2026 The Generalist LLC
228 Park Ave S PMB 19543, New York, New York 10003-1502 US
Unsubscribe

Get the appStart your Substack