How many participants should be used for Online, Quantitative Usability Testing?

Qualitative usability testing has traditionally been based around small sample sizes of 5-20 participants. However, the growth of online testing tools and quantitative usability research is changing the game. Whist many experts agreed that for qualitative, lab-based testing small samples sizes of 5-20 participants is sufficient.  Online user testing has created a new wave of analysis such as benchmarking, A/B testing, competitor comparison, validating and much more. These kinds of quantitative analyses require larger numbers of participants to validate the data.

For example: Let’s say your company is testing two different versions of wireframes so management can decide and approve one to implement and allocate resources to. It would not be very good practice to use 10 participants and get a 60%-40% success rate. It would be very hard to validate a study and implement a strategy based on 10 participants. However, if 500-1000 participants were used, that data would be a lot more accurate, and management would have valid data to approve of the findings.

Some specialists, such as Usability Sciences, recommend up to several thousand participants for more high level quantitative testing such as, click-stream data, or multiple cross-tabulations. They have broken down the % error for margin based certain numbers of participants. Read the full article here.

Happy Testing!

Posted in Loop11 Articles | No Comments »

2011 was a big year, not just for world despots, but for Loop¹¹ too. Whilst global dictators everywhere fell to revolutions, Loop¹¹ contributed its part to the remote, unmoderated usability testing revolution. As revealed in the recent UPA biennial survey the use of remote unmoderated testing grew by 28%. Our own usage statistics also show some interesting figures which we thought we would share with you here.

Here are some big numbers about Loop¹¹ in 2011:

… 60,456 participants:

That’s right! 60,456 participants from all over the world took part in Loop¹¹ usability testing projects. From Brazil to Japan and students to retirees, people of all ages and places participated in Loop¹¹ usability tests.

… 2165 usability testing projects created:

2165 different projects were created using Loop¹¹. Whether our customers wanted to test a current website, benchmark their competitors or test a new design, Loop¹¹’s various capabilities allowed them to conduct a variety of usability studies.

… 34 languages:

Projects were created in 34 of the 42 different languages available. Of the 2165 projects created, 72% were in English followed by 4% in Spanish, 3% in German and 2% in Portuguese.

… Over 30 countries:

Our customers from all around the world used Loop¹¹ to conduct remote usability studies. From China to Australia, Loop¹¹ was used by people everywhere and anywhere to conduct remote usability testing!

… Dozens of different devices:

As more and more people used Loop¹¹, one of our best features really came to fore. Loop¹¹ was used to conduct usability testing on an array of different internet enabled devices. PCs, iPhones, iPads, Androids and all sorts of other internet enabled devices. We even had someone test the usability of an ATM interface by putting the interface online and using an iPad as the touchscreen. Now that is genius!

… 4,300 new customers:

Overall, it was a great year for Loop¹¹. With 4300 new customers who signing up  to Loop¹¹.

We’re sure 2012 will be an even better year. With the growing demand for remote usability testing, and with more new cutting edge features and quirky promotions coming your way, we aim to raise the bar once again.

Happy New Year and Happy Testing!

Posted in Loop11 Articles | No Comments »

We’re celebrating Christmas nice and early this year. If you planned on spending your left over quarterly budget on eggnog and Christmas decorations, we’re hoping that you do the responsible thing instead and buy yourself a UXmas present.

We called it UXmas because it includes four great tools at a gift of a price. They are User eXperience enhancing tools that will make life easier for you and make your site better, easier and more effective.

So here’s our UXmas offer:

You get 3 user testing projects for $700, saving $350.  If you’ve already got an account we’ll credit the extra time to it.

You get to vote on which three of the following three excellent tools you want us to include for free. Each option has 12 months access and is fully featured:

• OpenHallway (Worth $624)

Record screencast video and audio of users testing your website, locally or remotely – all from the comfort of your browser

• ConceptShare (Worth $600)

Markup visual designs collaboratively, sketch directly onto images, compare alternatives side-by-side and collaborate and get approval remotely.

• HotGloo (Worth $336)

Rapid collaborative wireframing – design your page layout and focus on your message, your navigation and your interaction before you add graphics.

• SnapEngage (Worth $540)

Live chat with your site visitors and customers.

• Websort (Worth $2,499)

Card sorting is a simple but effective way to discover how people think your site content should be organized.

• Plainframe (Worth $1,499)

Allows you to easily and quickly put your navigation structure into a plain, clickable site.

http://uxmasbundle.com/nominations/84ac4

Hopefully that covers all the bases.  $1,050 worth of user testing projects (3 studies), for $700, and 12 months access to the three most popular UX tools for free.

The UXmas offer will only be available for two days starting on Thursday 15th December, but you will be able to request an invoice/purchase order and pay within 21 days. Please start talking to whoever needs to authorise the payment and any other teams that may want to use the bonus tools.

Merry Christmas and Happy Testing!

Posted in Events | 2 Comments »

Loop¹¹ will be conducting a scheduled server upgrade on Saturday December 3rd, 2011 beginning at 21:00 PM UTC (Co-ordinated Universal Time), and lasting up to 1 hour.

During this time, the Loop11.com website and systems, will be temporarily inaccessible. Any content hosted on Loop11.com will not be accessible by you or your project participants. During this time a message will be posted on the Loop¹¹ website alerting visitors of the scheduled maintenance. This should only take an hour, and once restored, all your content and data will be back online.

Schedule:

• Eastern Daylight Time (North America): Saturday, July 17: 4:00pm

• Greenwich Mean Time (Europe): Saturday, July 17: 9:00pm

• Eastern Standard Time (Australia): Sunday, July 18: 10:00am

We appreciate your patience and can assure you the service will be restored as quickly as possible. We also wanted to point out some recent improvements:

• Participant instructions page can now be skipped. For anyone running True Intent studies,

the original instructions to participants were not quite suitable. Instructions can now be completely skipped.

• Custom IDs. If you want to track participants so you know who to provide incentives to, you

can customise the end of the URL with a unique ID for each participant. Learn more.

For additional time zones, please reference www.timeanddate.com.

We apologise for any inconvenience caused, if you have any queries relating to this notice please email us at Support@Loop11.com.

Posted in News | Comments Off

One of the most common uses of Loop¹¹ is to test the usability of wireframes, prototypes and concepts. This newfound power has surprised some people, as for the first time they can now test the usability of wireframes, as easily as they can a website. You may have read the Boeing Credit Union wireframe case study, and wondered how it was done. Well, for those of you wondering, this is how to test wireframes using Loop¹¹.

Well for those wondering how, this is how to test wireframes using Loop¹¹:

1. 1.  Design some wireframes
2. 2. Export your wireframes in HTML
3. 3. Upload the exported wireframes to a public domain
4. 4. In Loop¹¹, enter the URL of the wireframe as the starting point URL (see below)

Create some tasks and questions to suitably test their usability and Tthat’s pretty much it! Now test it as you would a normal website. Now there are no excuses, we expect a massive influx of wireframe usability tests now.

If you would like to know more about wireframe testing, checkout our case studies here or contact as at support@Loop11.com.

Happy Testing!

Posted in How To Articles | Comments Off

As you are reading this, dozens of our clients are running remote usability studies in an array of different languages. In the 2 years that Loop¹¹ has operated, clients have run all sorts of usability tests in over 20 languages. Some of our clients in fact conduct the exact same test in multiple languages, which makes a lot sense if your website is multilingual. As you can see from the chart below, English is still the predominant language, but clearly the rest of the world is coming.

So we wanted to give a shout out to all the languages already being used in Loop¹¹, such as; French, German, Russian, Suomi, Spanish, Italian, Japanese, Hebrew, Swedish, Czech, Polish, Chinese, Portuguese and even Bahasa. But more importantly, we want to remind people that Loop¹¹ is capable of running user tests in over 40 languages, which means that there many more languages yet to be tried in Loop¹¹.

Got a website in a foreign language you want tested? Try Loop¹¹ for free.

Posted in Loop11 Articles | Comments Off

Want to get Loop¹¹ credits for as low as $11? Well here’s your chance! To celebrate 11.11.2011… Loop¹¹ is offering the usability testing opportunity of the century!

Between October 10, 2011 and November 10^th 2011, the more @Loop11 tweets there are the bigger the discount will be on November 11^th, 2011. You won’t get a chance like this for another 100 years!

We have set five thresholds:

• 100 Tweets = $300
• 250 Tweets = $250
• 500 Tweets = $100
• 750 Tweets = $50
• 1000 Tweets = $11

This promotion starts on October 10th, 2011.Visit our promo page to learn more and Tweet about it.

Happy tweeting and testing!

Posted in Events | Comments Off

There is a misguided notion that good SEO and usability need to clash. Some companies insist on focusing their resources on usability rather than SEO, and other companies take the opposite approach. There is no doubt that SEO and usability have two completely different goals. The goal of SEO is to rank highly in search engines for specific keywords, and the goal of usability is to make the online experience as user-friendly as possible. Traditionally, SEO friendly pages were usually filled with text links and keywords, making the webpage virtually impossible to use. However with the changes to the way Google indexes pages, good SEO and good usability can live side by side. These days, it’s all about defining goals, and striking the perfect balance.

So we put together a checklist of ten things to consider when creating the perfect balance between good SEO and good usability:

1. 1. Identify Goals.

What is the goal of your website? Impressions, sign-ups or enquiries? Whether your website’s goal is conversion based or impression based should determine which direction to take regarding SEO and usability. If your website is all about impressions for display ads, then you probably won’t care much for usability.

1. 2. Identify Your Audience.

Defining your audience will help validate your design, branding and marketing. Research this thoroughly.

1. 3. Checkout Your Competitors.

Seeing what strategies your competitor’s are using will enlighten and inspire you. Are their website’s SEO or usability heavy? Are there any opportunities to exploit?

1. 4. Usability Testing.

This is a crucial step. Does your website work? Is it usable? Depending on your experience and resources, there are various usability testing methods available, but we obviously recommend Loop¹¹ as a usability testing solution for both novices and seasoned professionals. Any findings relating to bad navigation or usability should be addressed, and unless your website’s success is based on impressions, usability should over-ride SEO.

1. 5. 508 Compliance.

Making your website 508 compliant will help both usability and SEO. Not only will pages load correctly in all browsers for all types of users, but good usability also means visitors will be able to use the site. From an SEO point of view, Google factors in the duration and pages per visits when indexing pages now.

1. 6. Titles / Heading Tags.

This is potentially the most important area to strike a perfect balance between SEO and usability. The title tag is the most important on page factor for search engine indexing. But it is also the first thing the user sees. Obviously, headers are secondary to titles, but are still important. A good title / headline tag focuses on 1-2 keyword themes… nothing more! Headers and titles should be an extension of your keyword themes, reinforcing them. Every page of your website should have a different title. This makes both good SEO and good usability.

1. 7. Keywords.

Once you have identified your main keyword themes. Don’t be tempted to stuff them all over your website. These days it’s bad SEO and usability. This applies to content, as well as metadata such as “alt” tags and page descriptions. Use tags exactly how they are meant to be used. For example, the “alt” tag should state exactly what that image is.

Your home page can’t be everything to everyone. So, identify 3-4 keyword themes, and use them on the relevant areas of your website. Just as importantly, make sure the content makes sense and reads correctly.

1. 8. Landing Doorway Pages.

These days, this is both bad usability and SEO! If you have any, get rid of them! The Google-bot is now able to differentiate between genuine webpages and pages used to spamdex.

1. 9. Links.

When creating internal links, it’s important to strike a balance between relevant keywords and good communication. Make sure your internal links are relevant. For example, if you have a link on your homepage going to a page about brown dogs. It would make good usability and SEO to label it “Brown Dogs” or “Learn More About Brown Dogs” rather than “click here for more dogs”

Dealing with external links has very little to do with usability. The only important SEO factor, is quantity, not quality. Logically, If there is a link to your website on cnet.com, Google will weight it a lot more than a link on a small time blog.

1. 10. Structure

A well thought-out structure with the relevant names / titles / subdomains will help both usability and SEO. An internal linking structure that links to your most important pages should also be considered. Consider a bread-crumb navigational menu for both good SEO and usability.

Posted in Loop11 Articles | Comments Off

Jeff Sauro at Measuring Usability, has put together another great blog post comparing preferred methods of usability testing by usability professionals over the past six years. Based on the last three biennial UPA surveys, Jeff determined that there was a 28% increase in people using remote, unmoderated usability testing tools like Loop¹¹. Jeff states that the reason for the increase is that remote testing tools are an inexpensive and effective way of measuring task based, quantitative usability data.

Read the full post here: The Methods UX Professionals Use – MeasuringUsability.com

Also check out: 2011 UPA Survey Results

Posted in News | Comments Off

Loop¹¹ will be conducting scheduled maintenance on Sunday July 17th, 2011 beginning at 22:00 PM UTC (Co-ordinated Universal Time), and lasting up to 4 hours.

During this time, the Loop11.com website will be temporarily inaccessible. Any content hosted on Loop11.com will not be accessible by you or your project participants. During this time a message will be posted on the Loop¹¹ website alerting visitors of the scheduled maintenance.

This is a major service upgrade including database improvements and server updates. We appreciate your patience and can assure you the service will be restored as quickly as possible.

Schedule:

• Eastern Daylight Time (North America): Sunday, July 17: 6:00pm

• Greenwich Mean Time (Europe): Sunday, July 17: 11:00pm

• Eastern Standard Time (Australia): Monday, July 18: 8:00am

For additional time zones, please reference www.timeanddate.com.

We apologise for any inconvenience caused, if you have any queries relating to this notice please email us at Support@Loop11.com.

Posted in Events, News | Comments Off