
Background
Hypothesis
By amending the PLP copy to more clearly specify that users need to “choose” either a nicotine level or device colour and remove the inactive CTA we expect to see an uplift in add to basket.Design Changes
- On page load of PLP, the grey inactive CTAs below each product are removed.
- User must select either product colour (E-Cigarettes) or nicotine level (liquids) before the yellow add to basket CTA appears.
- On E-Cigarettes PLP, the “Couleur: None” text is removed until a colour is selected. It then appears displaying the currently selected device colour.
- Also on E-Cigarettes PLP, on page load before a colour is selected, the “Choisissez votre couleur” text is displayed to indicate that a colour must be chosen before add to basket can take place.
- Once colour is selected, the message is replaced with the appropriate colour as described above.
- On Liquids PLP, the “Taux de Nicotine” copy is updated to “Choisir le taux de Nictone” to clearly indicate that a nictone level must be chosen before add to basket can take place.
Targeting – Pages
- Liquids PLP
- E-cigarette PLP
Test Goal
- Add to Basket
- Conversion Rate
Devices
- Mobile
Audiences
- All Visitors
#1. Which one do you think is the winner?
Results
Congrats! You got it right.
Winner: Variation 1
Start: 23/08/2021
End: 13/09/2021
Duration: 21 days
Control visitors: 11,251
Variation visitors: 11,379
Highlights
- Following the 3 week experiment, we observed an 8.9% uplift in conversion rate (transactions) for visitors exposed to variation 1. This metric reached 93% statistical significance.
- The uplift in overall conversion rate was driven by an improvement in PLP exit and bounce rates for both the e-liquids and e-cigarettes PLPs. Due to the improved layout, users were far less likely to exit the site from these pages, and we saw an improvement in users progressing into the checkout funnel, with a 5.7% uplift in unique checkout visits
- The 3.1% uplift in PDP views indicates that the frustration observed during usability testing around the correct way to navigate to PDP has been eased by removing the inactive CTAs, which were not clickable in control.
- The uplift in overall conversion rate observed in this experiment is worth an estimated €85.1k in annual incremental revenue. This has been calculated using the last 12 months of GA revenue data for mobile users that visited a PLP during their session.
Recommendations
Our recommendation is to deploy Variation 1 to 100% of mobile visitors and to raise a ticket for the changes featured in this experiment to be implemented
Unfortunately, you didn’t get it right. Variation 1 was the winner.
Winner: Variation 1
Start: 23/08/2021
End: 13/09/2021
Duration: 21 days
Control visitors: 11,251
Variation visitors: 11,379
Highlights
- Following the 3 week experiment, we observed an 8.9% uplift in conversion rate (transactions) for visitors exposed to variation 1. This metric reached 93% statistical significance.
- The uplift in overall conversion rate was driven by an improvement in PLP exit and bounce rates for both the e-liquids and e-cigarettes PLPs. Due to the improved layout, users were far less likely to exit the site from these pages, and we saw an improvement in users progressing into the checkout funnel, with a 5.7% uplift in unique checkout visits
- The 3.1% uplift in PDP views indicates that the frustration observed during usability testing around the correct way to navigate to PDP has been eased by removing the inactive CTAs, which were not clickable in control.
- The uplift in overall conversion rate observed in this experiment is worth an estimated €85.1k in annual incremental revenue. This has been calculated using the last 12 months of GA revenue data for mobile users that visited a PLP during their session.
Recommendations
Our recommendation is to deploy Variation 1 to 100% of mobile visitors and to raise a ticket for the changes featured in this experiment to be implemented
Get a copy of the full report
You can download the full report in pdf format from the link down below.

