Loop11 FAQs:


I’m new to user testing. How does this work?
What is usability testing?
Why is usability testing important?
Why should I choose Loop11?
When should I do user testing in the development process?
What does remote user testing mean?
Can I do a trial before I pay for a user test?
Who is user testing for?
What’s the difference between qualitative and quantitative usability testing?
What interfaces can I test with Loop11?
Who uses loop11?
What’s the difference between online usability testing and lab-based (face-to-face) testing
Why do user testing instead of A/B testing?
Can I test competitors’ websites?
How do I get participants for my user test?
How do I select the tasks for testing?
How do I know which licence is right for me?
How do I analyze the data?
How do I set up month-by-month benchmarking?
What are the basic steps to running a Loop11 study?
How many participants should be used for online, quantitative usability testing?

I’m new to user testing. How does this work?

Loop11 is an application that lets you run your own usability studies without the need for a usability lab, specialized equipment and a moderator. Simply determine the tasks you want to test on your website, recruit some participants and launch your study. Loop11 does all the rest and provides you with real time data giving you an understanding of the usability of your website and where you need to improve or make changes.

back to top

What is usability testing?

Usability testing is a technique used to ensure that the users of a website (or interface) can carry out the intended tasks efficiently, effectively and satisfactorily.

back to top

Why is usability testing important?

On the web, usability is a necessary condition for survival. If a website is difficult to use, people leave. If the homepage fails to clearly state what a company offers and what users can do on the site, people leave. If users get lost on a website, they leave. If a website’s information is hard to read or doesn’t answer users’ key questions, they leave. Note a pattern here? There’s no such thing as a user reading a website manual or otherwise spending much time trying to figure out an interface. There are plenty of other websites available; leaving is the first line of defence when users encounter a difficulty

back to top

Why should I choose Loop11?

Loop11 provides the fastest and easiest way for you to set up and run a usability project. You could have results back within 24 hours! There’s no code to insert and no IT or usability knowledge required. Our step-by-step approach will guide you through the process, but we’ve got awesome support available if you need it

back to top

When should I do user testing in the development process?

Usability testing can play a role in each stage of the development process. The resulting need for multiple studies is one reason for making individual studies fast and cheap. Here are the main stages where you could conduct testing:

  1. Before starting the new design, test the old design to identify the good parts that you should keep or emphasize, and the bad parts that give users trouble.
  2. Test your competitors’ designs to get cheap data on a range of alternative interfaces that have similar features to your own.
  3. Conduct a True Intent study to see how users behave in their natural habitat.
  4. Conduct information architecture testing to give you the insights you need to design or refine a great information architecture.
  5. Make wireframe prototypes of one or more new design ideas and test them. The less time you invest in these design ideas the better, because you’ll need to change them all based on the test results.
  6. Refine the design ideas that test best through multiple iterations, gradually moving from low-fidelity prototyping to high-fidelity representations. Test each iteration.
  7. Conduct accessibility testing to ensure your website can be used by users with a range of assistive technologies, such as screen readers or speech recognition software.
  8. Once you decide on and implement the final design, test it. Subtle usability problems always creep in during implementation.

back to top

What does remote user testing mean?

Remote user testing means that you can test the usability of any website from any computer, with users anywhere in the world. Traditional lab-based testing requires a moderator, users and the website to be in the same room at the same time.

back to top

Can I do a trial before I pay for a user test?

Yes! Your first project is free. It’s limited to a maximum of 5 tasks and 2 questions to give you a feel for the Loop11 experience. Once you’re ready to upgrade, these restrictions go away. You can start your free trial here.

back to top

Who is user testing for?

Usability testing is great for anyone involved in the design, development, or marketing of a website.

back to top

What’s the difference between qualitative and quantitative usability testing?

The difference between quantitative and qualitative research is often explained using contrasting terminology, like “hard vs. soft”, “numeric vs. non-numeric”, “statistics vs. insights”, “measure vs. explore”, “what vs. why”.

Broadly speaking, quantitative research can provide path and performance analyses by capturing the “what” of user behavior. It enables you to study your users’ experience by the numbers (e.g. how many people were able to buy a product successfully on my website?). In contrast, a goal of qualitative research is to gain valuable insight into the thought processes – the “why” – behind user’s actions.

back to top

What interfaces can I test with Loop11?

Any website or interface that is hosted on a domain can be tested with Loop11. This includes live websites, wireframes and prototypes of any interface.

back to top

Who uses Loop11?

Thousands of companies use Loop11, including UX consultancies, agencies, government organizations, the education space, Not-for-profits, etc.

Loop11 is used by usability consultancies as a complimentary tool to lab-based testing, web designers and developers to test prototypes during the development process and other online businesses to test and assess the usability of their websites.

back to top

What’s the difference between online usability testing and lab-based (face-to-face) testing?

Laboratory-based usability testing has been around for many years. The effectiveness of lab-based testing in uncovering usability problems with websites and applications is widely accepted. However, in recent years the influx of remote usability testing tools has started to change the way usability testing is conducted. One reason for this has been the cost associated with traditional lab-based testing, which can become very expensive. Other reasons have centered on the shortcomings of lab testing, in particular the limited number of participant commonly used.

Here are seven important things that you can do with remote user testing that you can’t do with lab based testing.

1. Test early, test often… affordably

Remote testing is much more cost effective than lab based testing, which means you can test at every stage of the development lifecycle, from wireframes to a live website. Or conduct benchmarking studies on a weekly or monthly basis to see how changes to your website affect usability. Lab-based testing can cost tens of thousands of dollars, which usually results in lab-based testing not being run very often by most companies. Remote testing tools usually cost just a few hundred dollars, making it easy to test on a much more frequent basis, thus creating the opportunity to run more regular studies.

2. True Intent Studies

True intent studies focus around a user’s real aim when using a website. What is their goal on your website, how do they go about it, and did they achieve their goal? These are the main objectives. True intent studies can only be run online because you must intercept users as they visit your website so you can observe their natural behaviour.

3. Get large numbers of “hard to get” users in their own environment

You can test hundreds of people simultaneously, in their natural environment. Whether you need a doctor or French speaker in China, online panels let you tap in to the thousands of unique demographics out there.

4. Get more in-house people involved

Remote usability testing is a great way of introducing usability testing into a company using limited resources and budget. Typically, most companies only involve a few people in lab-based usability studies. Remote online testing enables other parties to reap the benefits and even get involved. Quantitative metrics will show your IT guys why changes are needed to your website, and the finance guys will have the stats they need to understand why money needs to be spent.

5. Get statistically significant data

Lab-based usability testing is all about identification of usability issues and qualitative reasoning, not about quantification. Due to the small number of participants in lab testing (usually 5-10) compared to the hundreds or even thousands in remote testing, one simply can’t generate the quantitative reports that high level company decision makers may want to see. However, with remote user testing you can quash any in-house differences of opinion by getting the hard facts about your website’s user experience.

6. Special technology

Some interfaces just don’t make any sense to test outside their intended usage environment. Maybe you are running a user test on Facebook. The test involves uploading personal work with personal email contacts, pictures, videos and more. For this, you need the users to have their own email contacts, photos, videos and more. Having them bring their laptop or media to a lab isn’t going to do the study justice. Remote testing would allow them to use their own equipment at their own place…saving time, money and improving the quality of the results.

Or, let’s say you’re testing a recipe website that guides users step-by-step through preparing a meal; it wouldn’t make much sense to take people out of their kitchen, where they’re unable to perform the task of interest. When this is the case, remote research is usually the most practical solution, unless the users also lack the necessary equipment. We also call this the participant’s “technological ecosystem” because it implies that their devices and computing environment have an impact on how they interact.

7. Online accessibility testing

Now, let’s say you’re still testing that recipe website, except this time you need to test it for accessibility. Meaning you need to determine if there are any barriers that might prevent people with unique physical or mental needs from using the step-by-step recipe website in their own homes. It would be virtually impossible to replicate the technology and assistive devices of users with special needs, so the best way to do this is online.

back to top

Why do user testing instead of A/B testing?

Generally, A/B testing is for testing a single page of a website or even a single element on a single page of a website. In contrast, user testing is designed to test entire website and the whole journey that a user takes to complete the goal they had when they first visited the website.

back to top

Can I test competitors’ websites?

You sure can! Because we do not require you to insert code into your website you can run user tests on any website, including your competitors. It’s the perfect way to run benchmarking studies!

back to top

How do I get participants for my user test?

There are many places you can find participants for your user testing. The easiest – and cheapest – place to find them is via social media. If you have a Twitter, Facebook, LinkedIn or other social media account you can simply send your Loop11 project URL out to all your followers and fans asking for their feedback. Similarly, if you have a database of members you can email them directly.

Alternatively, you can create a pop-up invitation in Loop11 to intercept your current website users and ask them to participate in your study.

Finally, if your project requires having participants that meet certain demographic criteria then there are many market research panels that will source participants to meet your exact requirements. Usually these panels charge a small fee for locating your participants. The following companies can provide participants for your projects:

back to top

How do I select the tasks for testing?

Usability testing is about watching users do tasks which should simulate actual usage of your website. An initial usability test should focus on core functionality – areas that will see the most usage. If you are tasked with testing a specific piece of functionality, then naturally your test would need to focus on that area.

For example, if you were testing a rental car website, the core tasks could be: renting a car, renting a car with additional options, finding the address of a rental location and finding the opening hours of a rental location on a Tuesday morning.

Sometimes you have to test more than the core tasks, even if it is the initial usability test. As a balance you can have some core tasks tested first and if there is still time during a test session, test a set of peripheral tasks that include the fringe functionality.

back to top

How do I know which licence is right for me?

Although we have three different annual licence price points, the features of each licence are identical. The difference in price depends on the type of organization you are and the number of employees you have, as detailed below.

Annual Licence Pricing
MICRO SME ENTERPRISE
$1,900 $4,900 $9,900
Available to:

  • Micro businesses (less than 10 employees)
  • Not-for-profit organizations
  • Educational Institutions
  • Government
Available to:

  • Small and medium businesses (less than 100 employees)
Available to:

  • Everyone else
pricing page annual shadow

back to top

How do I analyze the data?

The moment a participant completes an evaluation, the responses can be viewed and reports are generated. Our reports make it easy for you to see how well each task is performed by your participants.

  • Understand what percentage of participants have difficulty navigating your website
  • See where participants are failing your most important tasks
  • Dive into individual responses to view details of a particular participant or to read the comments of open-ended questions.
  • View heatmaps showing you where participants click on an individual page
  • Generate clickstream data to see how participants really navigate your website.
  • back to top

    How do I set up month-by-month benchmarking?

    Conducting monthly benchmarking of your own website is a great way to ensure the usability of your website remains in good standing. Running regular benchmarking studies is an easy as setting up the key tasks you want to observe and copying the entire project each month.

    back to top

    What are the basic steps to running a Loop11 study?

    1. Determine what you want to test: What parts of the application and what functions.
    2. Why are you testing?
      • Do you want a benchmark of usability and performance?
      • Are you trying to find and fix as many usability problems as possible?
      • Are you comparing an application with a competing product?
    3. Write a minimum of 5-10 task scenarios that have users attempting realistic tasks in which they interact with the parts of the application you want to test.
    4. Recruit a set of users to test. At a minimum aim for between 50 and 100 in total. For goal 2b, test a subset of your users as part of an iterative testing approach.
    5. Launch your project.
    6. Analyze the results.

    back to top

    How many participants should be used for online, quantitative usability testing?

    Qualitative usability testing has traditionally been based around small sample sizes of 5-20 participants. However, the growth of online testing tools and quantitative usability research is changing the game. Whist many experts agreed that for qualitative, lab-based testing small samples sizes of 5-20 participants is sufficient. Online user testing has created a new wave of analysis such as benchmarking, A/B testing, competitor comparison, validating and much more. These kinds of quantitative analyses require larger numbers of participants to validate the data.

    For example: Let’s say your company is testing two different versions of wireframes so management can decide and approve one to implement and allocate resources to. It would not be very good practice to use 10 participants and get a 60%-40% success rate. It would be very hard to validate a study and implement a strategy based on 10 participants. However, if 500-1000 participants were used, that data would be a lot more accurate, and management would have valid data to approve of the findings.

    The maximum number of participants you can have in a Loop11 project is 1,000.

    back to top