The 11 Biggest A/B Testing Mistakes and How To Avoid Them

With a variety of easy-to-use tools available, A/B testing is something that every business can and should be doing. However, there’s more to A/B testing than just setting up the test. So many companies waste time and money by making simple and easy to avoid mistakes. In this presentation, conversion rate expert and guest presenter, Peep Laja, discusses 11 of the biggest testing mistakes he frequently sees companies make. Leave with actionable tips and strategies on how to avoid them. The webinar was moderated by Hanapin’s CRO Manager, Roopa Carpenter. You’ll get expert-level CRO tips like: *Why testing random ideas comes at a huge expense *How to avoid calling a test too early *Utilizing small wins to make a big impact
of 52
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  • 1. #thinkppc HOSTED BY: The 11 13 Biggest A/B Testing Mistakes and How To Avoid Them Featuring Peep Laja
  • 2. #thinkppc Presenters • Peep Laja – Guest Presenter – Founder of Conversion XL – @peeplaja • Roopa Carpenter – Moderator – CRO Manager at Hanapin Marketing – @roopacarpenter
  • 3. #thinkppc Join the conversation • Include the hashtag #thinkppc in your Twitter tweets. Or use the webinar question box to send us questions.
  • 4. #thinkppc Live Poll Question #1 How long have you been doing CRO? #thinkppc A. Less than 1 year B. 1-3 years C. 3-5 years D. 5+ years
  • 5. #thinkppc Live Poll Question #2 How do you manage your account(s)? #thinkppc a) I’m part of an in-house PPC team. b) I do all the marketing myself. c) I work with an agency. d) I’m a consultant.
  • 6. #thinkppc Peep Laja, ConversionXL @peeplaja Typical scenario: A company runs 100 tests / year. Lots of wins. But no difference on the bank account.
  • 7. #thinkppc Precious time wasted on stupid tests Testing mistake #1:
  • 8. #thinkppc You don’t have a data- driven, learning- oriented hypothesis Testing mistake #2:
  • 9. #thinkppc Learn more: winning-tests-using-data/
  • 10. #thinkppc You think you know what will work Testing mistake #3:
  • 11. #thinkppc Your sample size is way too small Testing mistake #4:
  • 12. #thinkppc
  • 13. #thinkppc Calculate needed sample size in advance:
  • 14. #thinkppc I won’t even look at results until there’s at least 250-350 conversions PER variation. Way more if you want to segment the data
  • 15. #thinkppc Your tests don’t run long enough Testing mistake #5:
  • 16. #thinkppc
  • 17. #thinkppc You don’t test full weeks at a time Testing mistake #6:
  • 18. #thinkppc
  • 19. #thinkppc You run tests when you have no traffic Testing mistake #7:
  • 20. #thinkppc •Learn more: optimization-with-very-little-traffic/
  • 21. #thinkppc Test data is not sent to third-party analytics Testing mistake #8:
  • 22. #thinkppc
  • 23. #thinkppc (function(){ var experimentID = 2809164105; window.optimizely = window.optimizely || []; if (typeof[experimentID] !== 'undefined') { var experimentName =[experimentID].name; var variationName =[experimentID]; = || function() {( = || []).push(arguments);}; =+ new Date();'send', 'event', 'Optimizely', experimentID + ' ' + experimentName, variationName, {'nonInteraction': 1}); } })();
  • 24. #thinkppc Peep Laja, ConversionXL @peeplaja
  • 25. #thinkppc You give up after your first test for a hypothesis fails. Testing mistake #9:
  • 26. #thinkppc
  • 27. #thinkppc Full case study: 79.3% uplift in conversions
  • 28. #thinkppc
  • 29. #thinkppc
  • 30. #thinkppc
  • 31. #thinkppc
  • 32. #thinkppc
  • 33. #thinkppc
  • 34. #thinkppc You’re not aware of validity threats Testing mistake #10:
  • 35. #thinkppc History effect Instrumentation effect Selection effect
  • 36. #thinkppc You’re ignoring small gains Testing mistake #11:
  • 37. #thinkppc 5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how math works.
  • 38. #thinkppc You copy competitors / leaders / other people’s tests Testing mistake #12:
  • 39. #thinkppc Which Test Won?
  • 40. #thinkppc WhichTestWon? WhichTestWasNot RunProperly?
  • 41. #thinkppc Somebody else’s test results are useless
  • 42. #thinkppc “Site shows stats from various A/B tests - Finally I've got evidence to show clients on a load of design decisions!”
  • 43. #thinkppc
  • 44. #thinkppc The process, the method, the analysis
  • 45. #thinkppc You’re not running tests at all times Testing mistake #13:
  • 46. #thinkppc PPC Retainers Need some assistance for your PPC? We offer on-going and one-time retainers. Learn More:
  • 47. #thinkppc Live Q&A Time!
  • 48. #thinkppc Have more questions? Thank you for attending our webinar! #thinkppc • Contact us Directly: » Hanapin Feedback:
  • Search
    Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks