Episode 15: Did You Just Screw Up the UI? with Ryan Burgess of Netflix

Web Rush

Ryan Burgess from Netflix talks with us about how to do better A/B testing so we don't screw up the UI - whether you've got Netflix sized data or doing smaller client work, A/B testing can help give you the data you need to make better decisions.

Recording date: 2018-11-29

John Papa @John_Papa

Ward Bell @WardBell


Someone to follow:


  • 00:56 Mailbag question
  • 06:34 Guest introduction
  • 08:32 Have you screwed up the UI?
  • 09:55 How do we A/B test UI changes well?
  • 13:14 How do you decide who to test?
  • 15:38 Defining light and dark vs a/b
  • 20:45 Sponsor: Progress
  • 21:21 Does Netflix A/B test during a show?
  • 23:23 How big should a feature be to A/B test?
  • 27:38 Are there seperate SPAs for signups, members, and streaming?
  • 29:34 How do you make A/B testing actionable?
  • 33:11 What tools do you recommend for A/B testing?
  • 35:00 Sponsor:
  • 36:05 How do I get the right metric?
  • 37:55 UI Don't's
  • 39:18 Someone to follow

Sponsored by:


Audio Player