Laboratory-based usability testing has been around for many years. The effectiveness of lab-based testing in uncovering usability problems with websites and applications is widely accepted. However, in recent years the influx of remote usability testing tools has started to change the way usability testing is conducted. One reason for this has been the cost associated with traditional lab-based testing, which can become very expensive. Other reasons have centered on the shortcomings of lab testing, in particular the limited number of participant commonly used.
Here are seven important things that you can do with remote user testing that you can’t do with lab based testing.
1. Test early, test often… affordably
Remote testing is much more cost effective than lab based testing, which means you can test at every stage of the development lifecycle, from wireframes to a live website. Or conduct benchmarking studies on a weekly or monthly basis to see how changes to your website affect usability. Lab-based testing can cost tens of thousands of dollars, which usually results in lab-based testing not being run very often by most companies. Remote testing tools usually cost just a few hundred dollars, making it easy to test on a much more frequent basis, thus creating the opportunity to run more regular studies.
2. True Intent Studies
True intent studies focus around a user’s real aim when using a website. What is their goal on your website, how do they go about it, and did they achieve their goal? These are the main objectives. True intent studies can only be run online because you must intercept users as they visit your website so you can observe their natural behaviour.
3. Get large numbers of “hard to get” users in their own environment
You can test hundreds of people simultaneously, in their natural environment. Whether you need a doctor or French speaker in China, online panels let you tap in to the thousands of unique demographics out there.
4. Get more in-house people involved
Remote usability testing is a great way of introducing usability testing into a company using limited resources and budget. Typically, most companies only involve a few people in lab-based usability studies. Remote online testing enables other parties to reap the benefits and even get involved. Quantitative metrics will show your IT guys why changes are needed to your website, and the finance guys will have the stats they need to understand why money needs to be spent.
5. Get statistically significant data
Lab-based usability testing is all about identification of usability issues and qualitative reasoning, not about quantification. Due to the small number of participants in lab testing (usually 5-10) compared to the hundreds or even thousands in remote testing, one simply can’t generate the quantitative reports that high level company decision makers may want to see. However, with remote user testing you can quash any in-house differences of opinion by getting the hard facts about your website’s user experience.
6. Special technology
Some interfaces just don’t make any sense to test outside their intended usage environment. Maybe you are running a user test on Facebook. The test involves uploading personal work with personal email contacts, pictures, videos and more. For this, you need the users to have their own email contacts, photos, videos and more. Having them bring their laptop or media to a lab isn’t going to do the study justice. Remote testing would allow them to use their own equipment at their own place…saving time, money and improving the quality of the results.
Or, let’s say you’re testing a recipe website that guides users step-by-step through preparing a meal; it wouldn’t make much sense to take people out of their kitchen, where they’re unable to perform the task of interest. When this is the case, remote research is usually the most practical solution, unless the users also lack the necessary equipment. We also call this the participant’s “technological ecosystem” because it implies that their devices and computing environment have an impact on how they interact.
7. Online accessibility testing
Now, let’s say you’re still testing that recipe website, except this time you need to test it for accessibility. Meaning you need to determine if there are any barriers that might prevent people with unique physical or mental needs from using the step-by-step recipe website in their own homes. It would be virtually impossible to replicate the technology and assistive devices of users with special needs, so the best way to do this is online.