How many participants should be used for online, quantitative usability testing?

How many participants should be used for Online, Quantitative Usability Testing?

Qualitative usability testing has traditionally been based around small sample sizes of 5-20 participants. However, the growth of online testing tools and quantitative usability research is changing the game. Whist many experts agreed that for qualitative, lab-based testing small samples sizes of 5-20 participants is sufficient.  Online user testing has created a new wave of analysis such as benchmarking, A/B testing, competitor comparison, validating and much more. These kinds of quantitative analyses require larger numbers of participants to validate the data.

For example: Let’s say your company is testing two different versions of wireframes so management can decide and approve one to implement and allocate resources to. It would not be very good practice to use 10 participants and get a 60%-40% success rate. It would be very hard to validate a study and implement a strategy based on 10 participants. However, if 500-1000 participants were used, that data would be a lot more accurate, and management would have valid data to approve of the findings.

Some specialists, such as Usability Sciences, recommend up to several thousand participants for more high level quantitative testing such as, click-stream data, or multiple cross-tabulations. They have broken down the % error for margin based certain numbers of participants. Read the full article here.

Happy Testing!

Loop11 – A look back at 2011

2011 was a big year, not just for world despots, but for Loop11 too. Whilst global dictators everywhere fell to revolutions, Loop11 contributed its part to the remote, unmoderated usability testing revolution. As revealed in the recent UPA biennial survey the use of remote unmoderated testing grew by 28%. Our own usage statistics also show some interesting figures which we thought we would share with you here.

Here are some big numbers about Loop11 in 2011:

60,456 participants:

That’s right! 60,456 participants from all over the world took part in Loop11 usability testing projects. From Brazil to Japan and students to retirees, people of all ages and places participated in Loop11 usability tests.

2165 usability testing projects created:

2165 different projects were created using Loop11. Whether our customers wanted to test a current website, benchmark their competitors or test a new design, Loop11’s various capabilities allowed them to conduct a variety of usability studies.

34 languages:

Projects were created in 34 of the 42 different languages available. Of the 2165 projects created, 72% were in English followed by 4% in Spanish, 3% in German and 2% in Portuguese.

Over 30 countries:

Our customers from all around the world used Loop11 to conduct remote usability studies. From China to Australia, Loop11 was used by people everywhere and anywhere to conduct remote usability testing!

Dozens of different devices:

As more and more people used Loop11, one of our best features really came to fore. Loop11 was used to conduct usability testing on an array of different internet enabled devices. PCs, iPhones, iPads, Androids and all sorts of other internet enabled devices. We even had someone test the usability of an ATM interface by putting the interface online and using an iPad as the touchscreen. Now that is genius!

4,300 new customers:

Overall, it was a great year for Loop11. With 4300 new customers who signing up  to Loop11.

We’re sure 2012 will be an even better year. With the growing demand for remote usability testing, and with more new cutting edge features and quirky promotions coming your way, we aim to raise the bar once again.

Happy New Year and Happy Testing!

UXmas Bundle

We’re celebrating Christmas nice and early this year. If you planned on spending your left over quarterly budget on eggnog and Christmas decorations, we’re hoping that you do the responsible thing instead and buy yourself a UXmas present.

We called it UXmas because it includes four great tools at a gift of a price. They are User eXperience enhancing tools that will make life easier for you and make your site better, easier and more effective.

So here’s our UXmas offer:

You get 3 user testing projects for $700, saving $350.  If you’ve already got an account we’ll credit the extra time to it.

You get to vote on which three of the following three excellent tools you want us to include for free. Each option has 12 months access and is fully featured:

    • OpenHallway (Worth $624)

    Record screencast video and audio of users testing your website, locally or remotely – all from the comfort of your browser

    • ConceptShare (Worth $600)

    Markup visual designs collaboratively, sketch directly onto images, compare alternatives side-by-side and collaborate and get approval remotely.

    • HotGloo (Worth $336)

    Rapid collaborative wireframing – design your page layout and focus on your message, your navigation and your interaction before you add graphics.

    • SnapEngage (Worth $540)

    Live chat with your site visitors and customers.

    • Websort (Worth $2,499)

    Card sorting is a simple but effective way to discover how people think your site content should be organized.

    • Plainframe (Worth $1,499)

    Allows you to easily and quickly put your navigation structure into a plain, clickable site.

    http://uxmasbundle.com/nominations/84ac4

    Hopefully that covers all the bases.  $1,050 worth of user testing projects (3 studies), for $700, and 12 months access to the three most popular UX tools for free.

    The UXmas offer will only be available for two days starting on Thursday 15th December, but you will be able to request an invoice/purchase order and pay within 21 days. Please start talking to whoever needs to authorise the payment and any other teams that may want to use the bonus tools.

    Merry Christmas and Happy Testing!

« Previous PageNext Page »