Benchmarking AI with Stanislav Khromov
In this episode, Stanislav Khromov joins the Svelte Radio team to discuss his work on Svelte Bench, a
benchmarking tool that scientifically measures how well different LLMs understand and write Svelte 5 code.
The conversation explores the challenges of AI-assisted coding with Svelte 5, the development of the official
Svelte MCP (Model Context Protocol) that provides LLMs with documentation and auto-fixing capabilities, and
the surprising performance differences between major AI providers. Stanislav also shares insights on working
with AI tools, the future of local models, and the economics of AI coding assistants.
Notes
Unpopular Opinions
- Kevin: Serverless is overrated
- Stanislav: OpenAI has the worst models of the big three
- Antony: Heat Pumps are bad
Picks
- Kevin: Kasai Hibachi Grill (now tested, and can confirm, it is PENG!)
Stanislav: Ball x Pit
Antony: Saunas By the Sea
In this episode, Stanislav Khromov joins the Svelte Radio team to discuss his work on Svelte Bench, a
benchmarking tool that scientifically measures how well different LLMs understand and write Svelte 5 code.
The conversation explores the challenges of AI-assisted coding with Svelte 5, the development of the official
Svelte MCP (Model Context Protocol) that provides LLMs with documentation and auto-fixing capabilities, and
the surprising performance differences between major AI providers. Stanislav also shares insights on working
with AI tools, the future of local models, and the economics of AI coding assistants.
Notes
Unpopular Opinions
- Kevin: Serverless is overrated
- Stanislav: OpenAI has the worst models of the big three
- Antony: Heat Pumps are bad
Picks
- Kevin: Kasai Hibachi Grill (now tested, and can confirm, it is PENG!)
Stanislav: Ball x Pit
Antony: Saunas By the Sea