How many participants should be used for online, quantitative usability testing?

How many participants should be used for Online, Quantitative Usability Testing?

Qualitative usability testing has traditionally been based around small sample sizes of 5-20 participants. However, the growth of online testing tools and quantitative usability research is changing the game. Whist many experts agreed that for qualitative, lab-based testing small samples sizes of 5-20 participants is sufficient.  Online user testing has created a new wave of analysis such as benchmarking, A/B testing, competitor comparison, validating and much more. These kinds of quantitative analyses require larger numbers of participants to validate the data.

For example: Let’s say your company is testing two different versions of wireframes so management can decide and approve one to implement and allocate resources to. It would not be very good practice to use 10 participants and get a 60%-40% success rate. It would be very hard to validate a study and implement a strategy based on 10 participants. However, if 500-1000 participants were used, that data would be a lot more accurate, and management would have valid data to approve of the findings.

Some specialists, such as Usability Sciences, recommend up to several thousand participants for more high level quantitative testing such as, click-stream data, or multiple cross-tabulations. They have broken down the % error for margin based certain numbers of participants. Read the full article here.

Happy Testing!

Loop11 – A look back at 2011

2011 was a big year, not just for world despots, but for Loop11 too. Whilst global dictators everywhere fell to revolutions, Loop11 contributed its part to the remote, unmoderated usability testing revolution. As revealed in the recent UPA biennial survey the use of remote unmoderated testing grew by 28%. Our own usage statistics also show some interesting figures which we thought we would share with you here.

Here are some big numbers about Loop11 in 2011:

60,456 participants:

That’s right! 60,456 participants from all over the world took part in Loop11 usability testing projects. From Brazil to Japan and students to retirees, people of all ages and places participated in Loop11 usability tests.

2165 usability testing projects created:

2165 different projects were created using Loop11. Whether our customers wanted to test a current website, benchmark their competitors or test a new design, Loop11’s various capabilities allowed them to conduct a variety of usability studies.

34 languages:

Projects were created in 34 of the 42 different languages available. Of the 2165 projects created, 72% were in English followed by 4% in Spanish, 3% in German and 2% in Portuguese.

Over 30 countries:

Our customers from all around the world used Loop11 to conduct remote usability studies. From China to Australia, Loop11 was used by people everywhere and anywhere to conduct remote usability testing!

Dozens of different devices:

As more and more people used Loop11, one of our best features really came to fore. Loop11 was used to conduct usability testing on an array of different internet enabled devices. PCs, iPhones, iPads, Androids and all sorts of other internet enabled devices. We even had someone test the usability of an ATM interface by putting the interface online and using an iPad as the touchscreen. Now that is genius!

4,300 new customers:

Overall, it was a great year for Loop11. With 4300 new customers who signing up  to Loop11.

We’re sure 2012 will be an even better year. With the growing demand for remote usability testing, and with more new cutting edge features and quirky promotions coming your way, we aim to raise the bar once again.

Happy New Year and Happy Testing!

A Shout Out for Multilingual Usability Testing

As you are reading this, dozens of our clients are running remote usability studies in an array of different languages. In the 2 years that Loop11 has operated, clients have run all sorts of usability tests in over 20 languages. Some of our clients in fact conduct the exact same test in multiple languages, which makes a lot sense if your website is multilingual. As you can see from the chart below, English is still the predominant language, but clearly the rest of the world is coming.

So we wanted to give a shout out to all the languages already being used in Loop11, such as; French, German, Russian, Suomi, Spanish, Italian, Japanese, Hebrew, Swedish, Czech, Polish, Chinese, Portuguese and even Bahasa. But more importantly, we want to remind people that Loop11 is capable of running user tests in over 40 languages, which means that there many more languages yet to be tried in Loop11.

Got a website in a foreign language you want tested? Try Loop11 for free.

Usability vs SEO: Getting The Right Balance

There is a misguided notion that good SEO and usability need to clash. Some companies insist on focusing their resources on usability rather than SEO, and other companies take the opposite approach. There is no doubt that SEO and usability have two completely different goals. The goal of SEO is to rank highly in search engines for specific keywords, and the goal of usability is to make the online experience as user-friendly as possible. Traditionally, SEO friendly pages were usually filled with text links and keywords, making the webpage virtually impossible to use. However with the changes to the way Google indexes pages, good SEO and good usability can live side by side. These days, it’s all about defining goals, and striking the perfect balance.

So we put together a checklist of ten things to consider when creating the perfect balance between good SEO and good usability:

  1. 1. Identify Goals.

What is the goal of your website? Impressions, sign-ups or enquiries? Whether your website’s goal is conversion based or impression based should determine which direction to take regarding SEO and usability. If your website is all about impressions for display ads, then you probably won’t care much for usability.

  1. 2. Identify Your Audience.

Defining your audience will help validate your design, branding and marketing. Research this thoroughly.

  1. 3. Checkout Your Competitors.

Seeing what strategies your competitor’s are using will enlighten and inspire you. Are their website’s SEO or usability heavy? Are there any opportunities to exploit?

  1. 4. Usability Testing.

This is a crucial step. Does your website work? Is it usable? Depending on your experience and resources, there are various usability testing methods available, but we obviously recommend Loop11 as a usability testing solution for both novices and seasoned professionals. Any findings relating to bad navigation or usability should be addressed, and unless your website’s success is based on impressions, usability should over-ride SEO.

  1. 5. 508 Compliance.

Making your website 508 compliant will help both usability and SEO. Not only will pages load correctly in all browsers for all types of users, but good usability also means visitors will be able to use the site. From an SEO point of view, Google factors in the duration and pages per visits when indexing pages now.

  1. 6. Titles / Heading Tags.

This is potentially the most important area to strike a perfect balance between SEO and usability. The title tag is the most important on page factor for search engine indexing. But it is also the first thing the user sees. Obviously, headers are secondary to titles, but are still important. A good title / headline tag focuses on 1-2 keyword themes… nothing more! Headers and titles should be an extension of your keyword themes, reinforcing them. Every page of your website should have a different title. This makes both good SEO and good usability.

  1. 7. Keywords.

Once you have identified your main keyword themes. Don’t be tempted to stuff them all over your website. These days it’s bad SEO and usability. This applies to content, as well as metadata such as “alt” tags and page descriptions. Use tags exactly how they are meant to be used. For example, the “alt” tag should state exactly what that image is.

Your home page can’t be everything to everyone. So, identify 3-4 keyword themes, and use them on the relevant areas of your website. Just as importantly, make sure the content makes sense and reads correctly.

  1. 8. Landing Doorway Pages.

These days, this is both bad usability and SEO! If you have any, get rid of them! The Google-bot is now able to differentiate between genuine webpages and pages used to spamdex.

  1. 9. Links.

When creating internal links, it’s important to strike a balance between relevant keywords and good communication. Make sure your internal links are relevant. For example, if you have a link on your homepage going to a page about brown dogs. It would make good usability and SEO to label it “Brown Dogs” or “Learn More About Brown Dogs” rather than “click here for more dogs”

Dealing with external links has very little to do with usability. The only important SEO factor, is quantity, not quality. Logically, If there is a link to your website on, Google will weight it a lot more than a link on a small time blog.

  1. 10. Structure

A well thought-out structure with the relevant names / titles / subdomains will help both usability and SEO. An internal linking structure that links to your most important pages should also be considered. Consider a bread-crumb navigational menu for both good SEO and usability.

« Previous Page
Want more inspiration?
Join the Fab-UX 5 newsletter!

Five links to amazing UX articles,sent to you once a week.

No SPAM, just pure UX gold!

No Thanks