Having finalised the information architecture, development of the live site follows. Once development is complete, we move next into a phase of optimisation as we seek to maximise the performance of the site in line with the business goals we have identified upfront. There are a number of valuable use cases for ongoing user testing in this optimisation phase.
Some key areas of focus include benchmarking, True Intent studies and A/B testing. We will now explore each of these applications in detail.
Benchmarking the performance of the live site against key business goals over time provides a great yardstick to determine if goals are being reached. This approach will also validate how far the needle of performance moves over time when further changes are implemented. Organisations like to see how their site fares against the competition; we are able to carry out testing on any other site we wish and compare these results to the performance of our own site.
Given the EverWear site has now been launched, we were keen to run testing on the site to ascertain an initial benchmark. We were able to create a new project which replicated the exact tasks and questions we had used in the earlier prototype testing exercise. Why would we need to repeat testing again, when we had already executed testing on the prototype?
There are two reasons why repeating testing at this stage is valuable:
We can assess the impact of changes made to the information architecture, and;
Ascertain performance metrics for the site, now that all levels of the site design have been built out and the design refined
For the last point, the fact the live site is now fully functional means that users are able to progress down many more paths than were offered in the prototype. Our focus is now establishing benchmarks for natural customer behaviour and feedback on the live site for future comparison.
For this initial round of live site testing, we validated that performance for each of the three tasks were strong:
Moving forward, we can re-run this testing benchmark following implementation of changes to determine their impact on task success, user experience and advocacy.
Another effective tool for assessing the user experience EverWear offers is to determine exactly who is visiting the site are and what tasks or goals they are aim to accomplish on it. A True Intent Study helps identify this by interviewing actual site visitors. True intent studies provide an alternative to regular benchmarking studies.
What’s a true intent study?
As the name suggests, a true intent study aims at understanding a user’s objective as they browse your site. What are they there to do? And are they able to achieve it?
A true intent study is structured to:
As true intent studies pose open-ended questions that do not make assumptions about the tasks/goals your users aim to accomplish, we can learn something surprising about our site. The study helps gather data that we probably wouldn’t have gathered if relying strictly on a highly controlled test with specific tasks or scenarios.
Conducting A/B Testing using Loop11 provides a clear understanding of what is happening on each design being tested as well as why. Traditional A/B testing on the other hand simply provides the what.
This form of testing can be very insightful when you have two different designs (A and B), both having certain benefits, but you need to know which one has an edge over the other, or which one has a higher conversion rate. If you are looking for answers to questions like “Which design results in the most click-throughs by users?” or “Which design layout results in more sales?”
A/B Testing is an effective way of testing how certain design changes (big or small) in the existing product can produce an impact on your returns. The main advantage of performing such a test is that you can compare between two versions of the same product where the difference in design elements can be as simple as the colour of a particular button or as expansive as testing completely different website designs.
The three approaches described here provide effective vehicles for your team to engage in a process continual design learning.
Congratulations on making your way through to the end of this course!
Hopefully the material we have presented wrapped around our fictitious EverWear example provides you with some clear structure and guidance as you approach setting up and conducting user testing as well as analysing the results which come out the other end.
There is no time like the present to get started, I wish you the very best with your testing moving forward!
Let us know how you go!