PCA: Spend & Save Product: Sticky Apply CTA
Background
Currently only around 30% of visitors to the Spend & Save product page are exposed to the “ready to apply” section of the page. We believe this is since this content is right at the bottom of the page and requires a significant amount of scroll effort from users, particularly on mobile. This is an opportunity to boost the prominence of this content to encourage a higher proportion of users into the application flow for this product.
Hypothesis
By adding a sticky “ready to apply” banner to the Spend & Save product page, we expect to see an uplift in the app start rate due to increased interaction with the apply now section of the product page.Design Changes
- A full width banner sticky to the top of the browser window is added to the page. The “Ready to apply?” CTA should anchor the user to the application section of the same page on click.
- The sticky banner is only displayed when the “Open a spend & save account” CTA at the top of the page is not visible, i.e. the banner only appears as users scroll down the page. If users scroll back to expose the “Open a spend & save account” CTA at the top of the page then the sticky banner is removed.
Targeting
- Spend & Save product page
Test Goal
- Increase App Start for “Spend & Save” account
- Increase App Completes for “Spend & Save” account
- Reduce Exit Rate for the Spend & Save product page
Devices
- Mobile
- Desktop
Audiences
- All visitors (Excluding internal TSB IPs)
#1. Which one do you think is the winner?
Results
Congrats! You got it right.
Winner: Variation 1
Start: 12/01/22
End: 01/02/22
Duration: 21 days
Control Visitors: 12,524
V1 Visitors: 12,822
Highlights
- We observed a statistically significant uplift in the primary metric of App Start of 5.27% vs. the control experience. We believe this is due to significant interaction with the sticky apply banner, which anchored users to the apply section on the Spend & Save product page.
- We saw a very similar uplift in App Submits of 5.43% vs. the control experience, indicating that not only were more users starting the application process, but a higher proportion of total visitors were also attempting to complete the application.
- The uplift in app start and submit rates resulted in an increase of 2% in the volume of completed applications. However, further analysis has identified that the overall app complete rate stayed flat vs. the control experience due to a significant uplift in declined applications ( 14.6%). The reason for the uplift in declines is unclear, but a potential cause could be the lack of requirements information within the banner itself, such as Minimum Age etc.
Recommendations
There is clear evidence that the addition of the sticky Apply banner on the Spend & Save product page encourages a higher proportion of visitors to start and submit an application for this account. The key driver for this is interaction with the sticky banner itself, which drove a significantly higher exposure rate for the apply now section of the page. However, due to a significant uplift in app declines, there is no evidence that the banner drives additional business in terms of successful account applications. We recommend analysing specific decline reasons once available to understand any behaviour changes that might be driving this.
We would therefore recommend a follow up experiment to replace the card image in the banner with a set of key minimum requirements to open a spend & save account. We believe this could reduce the app declines seen in the original test.
Unfortunately, you didn’t get it right. Variation 1 was the winner.
Winner: Variation 1
Start: 12/01/22
End: 01/02/22
Duration: 21 days
Control Visitors: 12,524
V1 Visitors: 12,822
Highlights
- We observed a statistically significant uplift in the primary metric of App Start of 5.27% vs. the control experience. We believe this is due to significant interaction with the sticky apply banner, which anchored users to the apply section on the Spend & Save product page.
- We saw a very similar uplift in App Submits of 5.43% vs. the control experience, indicating that not only were more users starting the application process, but a higher proportion of total visitors were also attempting to complete the application.
- The uplift in app start and submit rates resulted in an increase of 2% in the volume of completed applications. However, further analysis has identified that the overall app complete rate stayed flat vs. the control experience due to a significant uplift in declined applications ( 14.6%). The reason for the uplift in declines is unclear, but a potential cause could be the lack of requirements information within the banner itself, such as Minimum Age etc.
Recommendations
There is clear evidence that the addition of the sticky Apply banner on the Spend & Save product page encourages a higher proportion of visitors to start and submit an application for this account. The key driver for this is interaction with the sticky banner itself, which drove a significantly higher exposure rate for the apply now section of the page. However, due to a significant uplift in app declines, there is no evidence that the banner drives additional business in terms of successful account applications. We recommend analysing specific decline reasons once available to understand any behaviour changes that might be driving this.
We would therefore recommend a follow up experiment to replace the card image in the banner with a set of key minimum requirements to open a spend & save account. We believe this could reduce the app declines seen in the original test.
Get a copy of the full report
You can download the full report in pdf format from the link down below.
