A/B Testing ● Weyk Global Podcast Network · 2020-12-12 · 0 min
Key moments - from our scoring
Substance score
2 / 100
Five dimensions, 20 points each
The episode addresses a fundamental question in experimental design: determining the right sample size for reliable results. The guidance provided is straightforward - aim for 10% of your population, but cap it at 1000 samples maximum. This practical rule of thumb applies across vastly different scales; even when testing against a population of 200,000, sampling 1,000 users delivers statistically meaningful results without requiring the additional expense and time of larger samples. This approach balances statistical rigor with operational efficiency, making it valuable for product managers, data analysts, and growth teams running A/B tests who need to decide when they have enough data to make decisions.
Use 10% of your population as a maximum, but never exceed 1000 samples. This threshold balances accuracy with efficiency across population sizes.
No - cap your sample at 1000 even if 10% would be larger, as 1000 samples from a 200,000-person population will normally give fairly accurate results.
Our reviewer’s read on each dimension, with quotes from the episode.
The entire transcript is a single generic statistics rule-of-thumb repeated twice with no elaboration, context, or follow-on ideas. There is virtually zero insight for any B2B operator.
A good maximum sample size is usually 10% as long as it does not exceed 1000.
The content is a textbook statistics 101 heuristic with no novel framing, contrarian angle, or first-principles reasoning whatsoever.
A good maximum sample size is usually around 10% of the population.
No guest is identified or present; there is no speaker attribution, credentials, or evidence of any practitioner contributing.
Even in a population of 200,000, sampling 1000 will normally give a fairly accurate result.
The only number offered (10%, 1000, 200,000) is a bare generic heuristic with no named company, real study, context, or applied example to give it meaning.
Even in a population of 200,000, sampling 1000 will normally give a fairly accurate result.
There is no conversation, no host, no questions, and no follow-up of any kind - the transcript is a single undeveloped paragraph.
A good maximum sample size is usually 10% as long as it does not exceed 1000.
Computed from the transcript - who did the talking, and the words that came up most.
Hire pre-vetted freelancers within days, not months. Schedule a Call Hire Sign Up Pricing Case Studies Apply as a Freelancer Apply as an Agency How It Works Weyk Global Fixed-Rate Packages Hire Administrative Assistants Hire Community Managers Hire Content Writers Hire Copywriters Hire Designers & Developers Hire Email Marketers Hire Growth Marketers Hire Marketing Advisors Hire Paid Advertising Specialists Hire Public Relations Pros Hire SEO Specialists Hire Social Media Strategists Hire Strategy Consultants Divisions Commercial Political Non-Profit About Us Overview Skills Clients Bookstore Business & Operations Brand & Identity Marketing & Measurement Workshops Entry-Level Intermediate Advanced If you're reading this, you've probably heard about how the podcast got started. Thanks for listening and feel free to email me with questions at hey@weykglobal.com or call/text to let me know you just got your dream job. If you're still looking for a job, or are currently looking for talent, keep reading! Thanks, Zachary (402) 850-3076 If you're one of our loyal listeners, please consider contributing to keep us going.
Transcribed and scored by The B2B Podcast Index.
What's a good sample size? A good maximum sample size is usually 10% as long as it does not exceed 1000. A good maximum sample size is usually around 10% of the population. Even in a population of 200,000, sampling 1000 will normally give a fairly accurate result.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
The Future of A/B Testing: Insights from Spiralize CEO Sahil PatelSaaS Growth Podcast · on A/B testing95 / 100
What Vendors Get Wrong - A Fractional CMO’s Honest Take on MarTech SalesThe MarTech Matrix · on A/B testing86 / 100
Reclaim Your Brand Voice and Rise Above the AI Slop with Chris SilvestriHow I Grew This: Real Stories of Digital Growth · on A/B testing82 / 100
DOP 362: Feature Flags vs Canary DeploymentsDevOps Paradox · on A/B testing80 / 100
How Does Marketing Automation Fit into a Broader Strategy? | With Daria KravchenkoThe Strategic Marketing Show · on A/B testing80 / 100
How Senior Leaders Use Reverse Mentoring to Stay CurrentExecutive Careers with Fexingo · on A/B testing79 / 100What is email a B testing?Is AB testing dead?What are testing tools?What is AB testing in digital marketing?What is AB testing in machine learning?