My gut feel is that many of you who will read this title, and the article, might be confused as to why this needs to be written. The delightful thing about many UXers is their high level of empathy and general emotional intelligence. It’s what makes us good at what we do. However, the truth of the matter is not everyone running user research thinks, or cares, about best practice.

Many considerations that I’ll discuss below are red lines we intuitively know not to cross, and more than likely, never even considered breaching. However, as more and more professionals pile into the world of product development, with differing skillsets, we’re starting to see more user tests ran by people who don’t know what they’re doing and in the process raise ethical concerns.

Further down in this article I have included an example of gobsmackingly poor ethics within a user test, but before I get there I’ll cover off some of the things we think about as makers of user testing software. (more…)

At Loop11 we like to practice dog fooding.

Recently we ran two usability studies to gain comparative benchmarking data on an existing design and then comparing that to a new one.

Since the new design was yet to be released, we created two InVision prototypes, one for each design. We then created 4 tasks and 5 questions and generated two identical studies, one for each prototype.

Next, we set about running 100 participants though the prototypes, 50 on each, to see if our new design had created a better overall experience for participants.

Disappointment Meets Confusion

An hour after launch we had the results back from the two studies so I set about consuming the reports. It didn’t take me long to see that the average page views per task were higher for our new design.

As I’m sure many of you can attest, it hurts a little when something you’ve put a lot of effort into and believe is better proves to be worse. But in the interest of creating a better piece of software, I swallowed my pride and took a look at some of the highest page count participants to see how we’d failed.

I focussed in on two participants who were large outliers, both having recorded roughly 3 times the amount of page views than their next nearest participant. These two participants alone were enough to elevate the page count averages to the point where the new design was out performed by the old design.

As I watched the videos of these two participants I only became more confused. (more…)

Search Engine Optimization (SEO) has long been viewed as one of the most important factors in determining the success of online companies and their websites. Leading tools, such as MOZ, and their suite of products, would facilitate SEO professionals diving into the depths of search engine results and pulling out valuable nuggets of SEO gold that could then be applied to their website in order to climb up the search rankings.

Traditionally SEO work has involved looking at large sets of in-personal data, usually mined from Google Analytics and/or it’s general rankings. The problem – often this information lacks color. You can see the ‘what’ but it’s not always clear why the users are thinking and acting like they are.

SEFUS… ah … gazoontite?

This is where Search Engine Findability User Studies (SEFUS) come in and save the day. (more…)

Code is a developer’s problem… right? Mmmm, maybe, but when it goes wrong it can sure derail your user testing in a hurry.

There is one inconspicuous line of code that can ruin your UX testing and, as UX professional, it’s your responsibility, not your developer’s, to get right.

What am I referring to? (more…)

“What a waste of time” the researcher says as they throw their hands in the air.

This scene is more common than we, as UX professionals, would like to admit. One of the biggest frustrations for user researchers revolves around participants either not turning up, or dropping out mid-way through a remote user testing session.

We often get support tickets, at Loop11, from customers desperate to improve their completion rates and looking for tips to ensure they are as efficient as possible.

In response to this we decided to dig into the details and see if we could pull out any commonalities consistently associated with high performing tests with low dropout rates.

For this task we pulled the most recent 1,000 user tests which had at least 10 participants complete the study. We then cut the data every which way we could think of to draw out tidbits that will help you run better user tests.

As a point of reference, in these 1,000 usability tests the average completion rate was 59% and the median was 63%. The average number of participants in a study was 103 and the median was 31. Last but not least, the average duration that a participant would take to complete a user test was 23 minutes.

So without further ado, here are the top 3 tips for ensuring the majority of participants who begin your testing process successfully finish, giving you those valuable insights. (more…)

Some of the most common questions we receive at Loop11 from customers revolve around participant recruitment. Since Loop11 allows you to use your own lists, integrate with 3rd party panels, or recruit via pop-up intercepts on your website, there are a lot of choices.

Freedom is great but can sometimes lead to indecision. We’ve decided to put together the top 10 questions we receive relating to recruitment of participants and share our take on the answers.

For the purpose of this article, when we use the term ‘panel’ it refers to a third party which anyone can go to in order to recruit participants for their study. Generally speaking, you will provide your demographic requirements and the panel company will provide you a per participant quote.

(more…)

I realize the title of this article sounds unbelievably meta but stick with me.

This was exactly what we had to do when reimagining the Loop¹¹ participant interface used during usability studies run by our customers.

The context you need for this article is that Loop¹¹ is a remote usability testing platform used by UX professionals around the world to test websites and prototypes within their browsers on desktop, mobile and tablet. The results are reports which feature both quantitative (heatmaps, click steams, success rates, etc) and qualitative data (video, audio, survey questions).

When Loop¹¹’s participant interface was created a number of years ago it was cutting edge, worked beautifully and generally fitted the aesthetic of most modern websites. Flash forward to 2017 and you could pretty much reverse all of the statements in the prior sentence. This didn’t happen overnight, but it grew in importance as technology evolved surrounding security and testing on mobile devices became more and more popular.

In this article I’m going to talk about some of the problems we identified with our old UI and how we looked to solve them. I won’t dive into any of the technical challenges and changes, rather, I will focus more on the UX considerations.

(more…)

If you are reading this then you’ve probably just been given the job of improving a website, app, product, or maybe even a real-world service. Like with most things that are new, you aren’t sure where to start and might be a little overwhelmed with the amount of information Google is throwing back at your search queries.

What I hope to achieve in this article is give you the bare basics of user testing so that you can move forward and get a little experience of your own, get some positive results and then move deeper into the world of UX.

The first thing you need to do is: (more…)