Home » UI/ UX Design » A/B Testing Examples: 4 Real-Life Cases by Industry

A/B Testing Examples: 4 Real-Life Cases by Industry

A/B testing examples

A quick analysis of some notable A/B testing examples – divided by industry.

1. A/B testing example – Media & Publishing Industry

For businesses that operate in the media/publishing industry, increases in readership, subscriptions, time on page, video views, social sharing, etc. are among the main goals to achieve. As a result, you may want to conduct A/B testing on variations of email sign-up modals, recommended content, social sharing buttons, highlighting subscription offers, and other promotional choices.

Anyone who has used Netflix’s services can attest to the quality of their viewing experience. However, not everyone understands how they are able to make it happen. The answer is simple – Netflix uses an organized A/B testing approach to create a wonderful user experience, which other organizations still fail to deliver despite substantial efforts. Before deploying any changes, Netflix conducts a rigorous A/B testing procedure – a great example is their use of personalization.

Having a look at Netflix’s homepage should give you an idea of how serious Netflix is when it comes to personalization. They customize the homepage for each user based on their profile – in order to deliver the most optimal experience. Based on the users’ streaming history and preferences, they decide how many rows to be shown on the site – and which shows/movies are displayed.

Source: Netflix Tech Blog

The same process is also applied to media title pages. Netflix displays what titles we’re most likely to view, the thumbnails we see on these pages, the title text that entices us to click, and whether or not social proof helps us make a decision, etc.

2. A/B Testing example – eCommerce Industry

Online stores can use A/B testing to increase average order value (AOV), improve their checkout funnel, lower cart abandonment rates, and so on. You could experiment with the way shipping costs are shown – as well as when, whether, and how the free shipping feature is highlighted, text and color changes on the payment or checkout pages, and the visibility of reviews/ ratings, among other things.

Amazon is at the forefront of conversion optimization in the eCommerce business, partly due to their massive scale and unwavering commitment to offering the best customer experience. One of the most well-known methods they introduced was ‘1-Click Ordering.’, which was introduced in the late 1990s after extensive testing and analysis.

This feature allows consumers to make purchases without having to use the shopping cart. Users simply need to click the button after entering their default billing card information and shipping address, then wait for the ordered things to arrive. When placing orders, they do not have to re-enter their billing and shipping information. As a result, customers could no longer reject the convenience of buying and go to another store with 1-Click Ordering.

This improvement had such a significant impact on Amazon’s company that it was patented (since expired) in 1999. In fact, Apple purchased a license for the same in 2000 to use for their online store.

Source: Amazon

Amazon is able to provide an optimal user experience thanks to its continual A/B testing efforts. Every change to the website is first tested with the target audience before being implemented. If you look at Amazon’s buying funnel, you’ll note that – while it resembles other websites’ purchase funnels in many ways, each and every aspect is properly optimized and meets the audience’s expectations.

Every page provides the most important information and directs customers to the next logical step in the conversion funnel. Additionally, each step is streamlined to its maximum potential – using significant user insights and website data to fit their users’ expectations.

For instance, let’s have a look at Amazon’s ubiquitous shopping cart. A small cart icon appears in the top right corner of Amazon’s homepage and remains visible – no matter what page you are on.

The icon isn’t merely a reminder of adding products or a shortcut to the cart. In its current version, 5 options are available:

  • Continue shopping (if there are no products added to the cart)
  • Find out about today’s offers (if there are no products added to the cart)
  • Wish List (if no products have been added to the cart)
  • Proceed to the payment page (when there are products in the cart)
  • Sign in to turn on 1-Click Checkout (if there are products in the cart)

3. A/B Testing example – Travel Industry

A/B testing can help increase the number of successful bookings on your website/ mobile app, as well as your revenue from ancillary transactions. You can test various elements – your home page search modals, search results page, supplementary product presentation, checkout progress bar, and so on.

Booking.com is a prime A/B testing example for the travel sector. Since its inception, the company has continuously conducted A/B testing in its day-to-day operations. By eliminating HiPPOs and prioritizing data over everything else, they were able to boost their testing velocity to its current level. To speed up the testing process, all of their workers were given the opportunity to run tests on ideas they thought could help the company develop.

In 2017, Booking.com aimed to expand its reach by including vacation rental units alongside hotels. They teamed up with Outbrain, a native advertising platform, to help with their global property owner registration.

As the team observed during the first few days of the launch, they learned that many property owners had completed the first sign-up step – but were stuck in the next steps. The process was done using sites designed for their native campaigns’ paid search at the time.

The teams then worked on building three different landing page variants for Booking.com. Social proof, honors and recognitions, user rewards, and other details were added to the versions. The test lasted two weeks and resulted in a 25% increase in owner registration.

4. A/B Testing example – B2B/SaaS Industry

The goal of SaaS A/B testing is to improve conversions and user experience. You may test your lead form elements, free trial sign-up flow, homepage messaging, CTA text, and social proof on the home page, among other things.

POSist, a leading SaaS-based restaurant management platform with over 5,000 customers, sought to boost the number of demo requests they received.

The most significant pages in their buying funnel are the homepage and Contact Us page. With the aim of lowering bounce rates on these pages, the team produced 2 different homepage versions – as well as 2 different Contact Us page variations. Take a look at the changes that have been made to the homepage:

The POSist team concluded that by adding more relevant and conversion-focused material to the website, they will be able to improve user experience and conversion rates. As a result, they generated two versions to compare to the control. This is how the variations appeared:

Control was pitted against Variation 1, and the latter was chosen. To better improve the page, version one was pitted against variation two, with variation 2 being the winner. The new variation resulted in a 5% increase in page views.

Conclusion

Hopefully, by the end of the A/B testing example analysis above, you should be able to realize the importance of implementing it into your business strategy.  For those who are looking for a software development agency to help with your website/app development project, don’t hesitate to reach out to JSLancer – our team will be more than happy to provide a FREE consultation on how we can help you visualize your goals.

JSLancer Blog - Sharing knowledge on technology trends & software development