Three challenges to overcome for awesome remote moderated user testing results

Coronavirus lock-down measures may have stopped the global economy in its tracks, but workers around the world were quick to adapt to new technologies, pick up where they left off, and carry on.


Many UX designers and researchers made the switch to remote moderated user testing to keep their website design projects on track.


If you’re more experienced with face-to-face research, you’ll need to tweak your approach – and be prepared for some new challenges – in order to get the most out of UX research using online usability testing.


Challenge 1: Managing technical difficulties


Sometimes, the old adage that it’s the user, not the tech, at fault, is not always true. Occasionally, it really is the tech. No amount of switching on and off your PC (or checking you’re plugged in) can solve a poor internet connection.


With the increased web traffic that has resulted from the ‘work from home’ directives, internet connections have been patchy. During usability testing, this can result in a hiccups in the recording or cause difficulty hearing the participants. If you lose your connection completely, try hot spotting from your phone to computer. If that’s not possible – stay calm and try again later.


There are technical obstacles that can be avoided if you take the time to do a little planning in advance. Firstly, make sure everyone involved has installed the right technology. Then, do a test the day before the real session: a sound check with your microphone, a trial with the video camera, and a general exploration of how the testing platform records and plays back.


Use a friend or colleague as a guinea pig and take them through the whole testing process. Forward-prep has the added bonus of making you feel a lot more confident rolling out the testing with strangers.


Challenge 2: Missing the non-verbal, visual cues


In a face-to-face UX test, you pick up on silent cues that speak volumes: the frown that suggests a webpage is confusing, a smile that indicates the right resource has been located, a yawn of boredom over a popup offer that was supposed to be exciting. When you’re remote testing, how do you replicate this added layer of feedback to give your research depth?


Prepare questions in advance


If your moderator is a people person, they’ll naturally engage in a dialogue that draws the participant into a conversation to probe their reactions.


For the rest of us, preparation is key. Brainstorm a list of questions in advance, and then hone in on those that are simple to express, open-ended, and spark meta-thinking – thinking about thinking.


Ask open-ended questions


Avoid closed questions (they’re the ones with a ‘yes/no’ answer). Instead of asking ‘Did you find it easy to navigate to this page?’ use ‘What were you expecting to see on the site to help you find your way here?’. This technique helps your participant verbalize the mental process they went through as their eyes scanned your website.


Avoid leading questions


Don’t ask ‘Do the colors of the site suggest this is a luxury brand’? You’ve put the idea of ‘luxury’ into their mind. Instead, try ‘What does this pink color and cursive font remind you of?” Some people have trouble verbalizing their feelings and thoughts. If this question causes stunned silence, have a multiple-choice question up your sleeve. You can follow with “Do you think this site is selling a luxury product, a product that will appeal to millennials, or something for those who are budget-conscious?”


Choose a UX tool that includes video


Face-to-face contact can never be beaten for the highest quality human interaction. It’s the same for usability testing. Find a UX tool that offers both audio and video recordings if you want to observe your participants in action. You can then round out your questioning with references to facial expressions, pauses, or confident movements. Most Loop11 customers use both our audio and video features to capture the full range of user experience in their testing.


Challenge 3: Recruiting the right participants

Almost half of all UX professionals say that recruiting participants is one of their biggest challenges.


Outsourcing recruitment to a specialist firm can save time and leave you to do what you do best – design and carry out the study! For outsourcing to be effective though, you need to give the firm a clear brief. Outline the context of your study and what you hope to achieve, and list the key demographics of your target audience.


Recruitment is far more than a ‘bums on seats’ exercise. You want to find the right participants, or the feedback you get may be compromised. When Loop11 sources participants to join our ever-growing user pool, we run stringent qualification tests. We make sure people are categorized correctly so they can be directed to the right kinds of projects. We also ensure they are equipped and prepared to undertake the tests. In particular, it’s important to make sure that any participants a third-party recruits for you have signed off on allowing you access to their microphone and/or video. People are conscious of their privacy and you don’t want them pulling out of a test at the last minute if this hasn’t been covered off at the outset.


On your way to successful remote moderated user testing

Remote user testing is a juggling act. You need to keep the conversation going, watch the user’s onscreen behavior, take notes, listen to comments from your team and keep an eye on the time. You also have to exude an aura of calmness and control to set the participant at ease and get the best results.

Practice, preparation, and patience (if tech issues arise) are key to a perfect remote user testing session.

If you’d like to experiment with remote moderated usability testing, jump on our free 14-day trial now.

If you’re new to user testing, get in touch and we can tell you more about how our customers are using remote testing to get awesome feedback on their designs.

Meet the New Loop11



Over the past months, our team has been working hard to rebuild Loop11 from the ground up – introducing new features and significantly improving the experience. With more complete functionality than ever before, our latest updates help you capture customer feedback on your web designs more efficiently than ever before.

If you can’t wait to see it – click here to meet the new Loop11 right now! OR take a self-guided tour. You can always switch back to the old Loop11 by clicking ‘Switch to Live’ in the left-hand navigation.

What’s new?

Apart from a completely new look, included in this release are the following new features:

  • More intuitive project creation process
  • Segment your data by more variables
  • Unlaunch and edit projects at any stage
  • Automatic NPS, SUS and lostness calculations
  • Include wildcards for task success URLs

And watch this space for our new website – coming soon!

Introducing the new Loop11 features…

More intuitive project creation process

The project creation process has been streamlined so you can create and launch projects in minutes. What was already easy is now even easier!


Segment your data by more variables

You’ve always had the ability to filter data so you can eliminate ‘speeders and cheaters’ from your results. Now, you can segment your data so you can explore your results by different variables, like mobile versus desktop or iPhone versus Android. There are eight different variables you can filter by and we’ll continue to add more.

Unlaunch and edit projects at any stage

Making changes to a launched project was never a possibility – unless you kindly requested a change from one of our helpful support staff! Now, you can make changes and edits to a launched project yourself. You can also completely unlaunch a project, too.



Automatic NPS, SUS and ‘lostness’ calculations

If you included NPS and SUS in your project we’ll calculate them automatically for you so you don’t need to struggle through the calculations yourself.  Lostness – the measure of efficiency using a digital product or service – will be calculated for every task – automagically.

Include wildcards for task success URLs

Remember entering multiple URLs during task creation? It was a big job, so we’ve made sure that Success URLs can now include wildcards. Now, you won’t need to enter multiple success URLs during task creation. You can still re-categorize URLs retrospectively, but you won’t need to do it as often anymore.

What features are still the same?

We’re still the same Loop11 on the inside: leading-edge user-testing, brilliant customer support, and a huge passion for all things UX. And we still offer:

  • Moderated AND unmoderated user testing
  • Recording of video and audio
  • First click tests, time-based tests, A/B testing, benchmarking, mobile and tablet testing, and more…
  • Upload images for feedback in addition to testing URLs
  • The Loop11 paid tester panel that you can recruit from, based all around the world

Finally, we’d like to say a huge thank-you to all our beta testers who helped test the new and improved Loop11.

This is just the beginning of a range of new additions. We’d love to hear what you think of the new features so far.

Happy testing!


Introducing Loop11’s Tester Panel

For the past 12 months we have been building a panel of paid testers from around the world who have had to perform stringent qualification tests to ensure they are ready and able to complete usability studies on our platform.

We were wanting to build out our panel tools further before launching, however, in the current global climate (read more about our COVID-19 response here) we want to start offering work to our panel of website testers to help ease any financial burden they might be facing.

So, here is how you can use our panel of testers. When you are creating your study, reach out to us on support @ loop11 dot com and let us know what your requirement requirements are. Things like:

  • Demographics (age, gender, location, martial status, smoking behaviour, etc)
  • Total numbers
  • Budget

We will also need to see the study design in your account so we can confirm a price. The longer your study, the higher the cost.

With cost in mind though, we’re going to be waving most of our charges and passing the money straight on to the website testers. We normally charge $1-2 per minute. We will be cutting that in half to between $0.50 – $1 per minute.

Please reach out with any questions, or feel free to start booking in panel participants with us on support @ loop11 dot com

Our Response to the Coronavirus

Over the past weeks we have been watching the concerning spread of the Coronavirus along with the rest of you. As more and more Governments encourage people to limit personal contact, we’ve decided to take action to help facilitate remote moderated testing so that UX professionals can continue to get the job done while in relative isolation.

Until the end of May 2020 we are:

  1. Increasing the limits on our Rapid Insights plan
    From 5 participants per project to 15 and from 3 projects per month to 6.
  2. Cutting the price of our monthly Pro subscription for new customers
    $249 per month is now $199 per month
  3. Extending all trials of Loop11 from 14 to 25 days

To take advantage of the Pro subscription offer please use the coupon code below when purchasing your subscription.


For those teaching or studying UX-related courses in schools, universities, or educational institutions please be aware that Loop11 has and always will be free for you to use.

Stay healthy, and thank you for using Loop11.

Toby Biddle
CEO, Loop11

P.S. Please forward this to the teachers you know or anyone else you know who think could benefit – we’d love to help as many people as we can.


Fab-UX 5 – Issue 126

The weekly UX Design newsletter from Loop11.

Fab-UX 5 – Issue 125

The weekly UX Design newsletter from Loop11.

Fab-UX 5 – Issue 123

The weekly UX Design newsletter from Loop11.

Fab-UX 5 – Issue 122

The weekly UX Design newsletter from Loop11.

Next Page »
Want more inspiration?
Join the Fab-UX 5 newsletter!

Five links to amazing UX articles,sent to you once a week.

No SPAM, just pure UX gold!

No Thanks