The importance of planning in guerilla testing

When talking about user testing in the UX circles, there’s formal user testing, and then there’s guerilla testing. Formal user testing is the subject of many research papers and studies, but there is relatively little guideline on how guerilla testing should be conducted. Although many UX purists frown on the practice, guerilla testing has become a useful tool for the everyday UX practitioner. Whether they be struggling with tight UX budgets or looming project deadlines, guerilla testing can help fast track the research and testing phases of their UX design cycle. In fact some UX practitioners might even refer to the practice as the art of guerilla testing.

In essence, guerilla testing is user research done using a lean and agile approach. While this means making the user testing simple, short and relatively flexible, it doesn’t mean going about this in a totally unstructured, undocumented and unplanned fashion. What formal testing tries to avoid by not conducting research and testing in a haphazard manner is the risk of introducing potentially costly design changes that may not lead to any benefits to the end user.

Yet, the danger of guerilla testing comes from poorly planned and executed tests that are not reliable, consistent or meaningful. In this post, we’ll explore some pitfalls of guerilla testing “in the wild,” a few tactics to avoid or minimize the methodology’s weaknesses and tips to improve planning for all research and testing.

Plan B – catering for the uncontrollable

The first and most important difference between guerilla testing and other standard user testing processes is the lack of a controlled environment. You may have carefully picked out a time, location and worked out the groups of people that you want to survey, but when you do turn up, things may not go according to plan. What will you do if some unexpected event happens that week, or if the location becomes unavailable or too crowded/noisy/distracting, and what if you don’t end up running into the type of people you want to speak to? So the first thing to remember when carrying out guerilla testing is always to have at least one backup plan, or even better, to have a plan B and C.

Consistency – stay true to your Q’s

Another common problem with guerilla testing is the temptation to go with the flow when querying the user. Often a particular question or comment triggers interesting insights or unexpected findings, and you become fixated with getting to the bottom of it. This can cause a few different issues, such as not being able to compare results because you haven’t asked the same type of questions, or the questions were asked in ways that produced varying results, or you introduce new variables and behaviour triggers that were not present for other users. In the end you find yourself unable to reconcile all the findings and draw a neat conclusion. So the takeaway message here is to have a focus in mind, stick to the main questions and resist the temptation to chase loose ends. If you must, circle back at the end to dig deeper.

Beyond paper – rich data capture

One of the details that often gets overlooked in guerilla testing is the capturing of information accurately and reliably. Guerilla testing doesn’t mean you are only restricted to pencil and paper.  Although you can get a lot out of paper wireframes, it is not easy for the testers to capture all the feedback on the paper itself or on sticky notes. There’s no shame in taking a PowerPoint presentation or even a semi-interactive prototype into the field. You might even consider a tablet running a usability testing web application if you have the luxury to do so. This way you will be able to have all your data and results captured and stored neatly in one place for you to review later. It also means no more deciphering handwritten notes scribbled down while your mind is focusing on what the user is telling you.

Guerilla testing – what’s it to you?

Last but not least, you should consider what the guerilla test means for you in the grand scheme of things. Not every organization is going to need a research and testing framework document to formalize and standardize the process, but chance are, if you find this to be a useful research tool then you’ll want to know that you can extract reliable and consistent results from your test subjects. So do spend a bit more time thinking about how to eliminate the environmental variables that might affect your users (e.g. testing a weather app on a hot versus cold day might affect people’s mood), consider when and how you approach people to conduct the test, and try to keep the test items simple with a clear focus on the answers you want to get.

With all of these planning details in mind, you should have better luck finding the right balance between flexibility and consistency for your guerilla tests.

Create a more intuitive website structure with this 5-star card sorting course (get 50% off)

Good information architecture is key to creating engaging, easy-to-use and intuitive websites. We invite you to take this useful online course, brought to you by U1 Group (a UX consultancy firm), comprised of easy-to-follow lectures that will walk you through the card sorting process.

What is card sorting?

Card sorting is a professional exercise undertaken with participants that determines the best navigational structure (formally called ‘information architecture’) for your site. Card sorting can be carried out face-to-face, or online using a software program called OptimalSort. In this course, you will learn how to master both methods.

How to get 50% off the online course

Loop11 fans get a 50% discount on this course by using the code “Loop11” (the code is automatically applied when you follow any of the links in this email).

Why you should check it out:

  • Lifetime access to 14 lectures
  • Join a community of 150+ students
  • 30 day money back guarantee
  • Mobile accessibility
  • Certificate of completion

Carrying out card sorting prior to web design and development enables you to:

  • See how users map out relationships between your content
  • Figure out where users think content should (or shouldn’t) sit
  • Observe what navigation paths don’t make sense to users and why
  • Learn the language visitors use to categorize information

Check out the free preview to see how this course will provide you valuable insights about creating & testing intuitive website structures.

 

How to use Tree Testing to Test the Information Architecture of Your Website or App

Importance of testing IA

Back when I was a kid, I was taught that raw bits of unorganized facts that needed to be processed are called data, and when data is processed and organized into something sensible, or presented to us in a given context so as to make it useful, it is called information. What I wasn’t taught as a kid was that even information needed some sort of organization, so that we could achieve a consistency in task flow.

One of the biggest challenges faced while building a website or an app is organization of content. If your content is not findable or accessible, no matter how pretty or full of bells and whistles your website or app is, your users are going to run away, and conversion rates are going to come down. Testing the organization of content (or information architecture as we call it), thus, becomes very necessary at the early stages of product development lifecycle.

Hello Tree Testing

There are several different ways in which the information architecture (IA) of a particular website/app can be tested. What if I told you there is a simple yet bulletproof technique to carry out such a test? Yes, we are talking about Tree Testing – one of the simplest ways to test the IA of an application.

So what is a ‘tree’? Typically, every website that has more than a few pages translates into a structure that categorizes pages into groups and sub-groups forming some sort of hierarchy of content. This hierarchy of content or ‘tree’ can be formed by the usual IA/User Research techniques (Read: Card Sorting). Once this tree has been formed, it needs to be cross-checked to make sure that everything is perfect. This is where Tree Testing comes into play.

Tree testing is an effective way to assess the findability, labeling & organization of your website’s/app’s structure. Unlike traditional usability testing, tree testing is not done on the website itself; instead, a simplified text version of the site’s structure is used. The prime focus is to test the navigation system of the website.

The questions to be answered are – “Can users find what they are looking for?”, “Does the navigation system make sense to users?”, “Can they choose between menu items, without having to think too much?”, etc. Factors like visual design, motion design, etc. are taken out of the picture.

Typically, a tree test is conducted prior to building a prototype to make sure that users are able to navigate easily through the ‘tree’ (hierarchy of content).

 

Why you should do it

There are various advantages of adopting this process. Few of them are clubbed together below:

  • It allows you to visually test the navigation and findability of your website/app.
  • It allows you to identify navigational issues prior to building a prototype or a dynamic website.
  • It allows you to analyze all attempts where users had trouble navigating before you go live.
  • It allows you to gauge how well users can find items in the ‘tree’.
  • It allows you to determine the ease with which users/participants complete the given tasks successfully.
  • How to go about performing a Tree Test

Here’s a short guide on how to go about performing a Tree Test: Let’s consider a hypothetical situation where you want to test the information architecture of an e-commerce website that sells hair and skin care products. Let’s assume that you’ve already performed a card sort and have come up with a navigation system that seems to be appropriate. The next step is to cross check and make sure everything is perfect. Here’s what you do.

  • Give your users/participants a “find it” task (Example: “Look for American Crew Daily Shampoo”).
  • Show them a text version of the top tier of the menu items of your website.
  • Once they choose a menu item, show them the list of items under that particular category (This is the next tier in your tree).
  • Let them continue to move down through the tree, backtracking if necessary – until they successfully complete the given task or until they give up.
  • Give them several tasks in this manner, every time starting back at the top of the tree.

 

image

Analyze and implement the findings/results.

  • Conclusion Proper analysis of the findings will answer the following questions:
  • Did the users/participants succeed in completing the given task(s)?
  • Did they backtrack? If yes, then where and the number of times they needed to do so?
  • How fast did they click?
  • Which sections need a rework?

The most important task at hand next is to implement these findings/results. Redesign the structure of content using these findings and perform the test once again. If the user interaction is found to be smooth and error free, you are good to go.

Although Tree Testing might seem like overkill to some of us, but it does reveal major flaws in your website’s/app’s structure, and lets you define a more reliable site structure and navigation by validating the results derived from IA techniques like Card Sorting, etc..

 

Interested in learning more?  Get a 50% discount on this 5-star course on Developing an Information Architecture with Card Sorting.  Join a community of 100+ students & receive lifetime access to 14 easy-to-follow lectures from the user experience experts U1 Group (a UX consultancy firm).

 

Arijit Banerjee is a UI & UX Enthusiast. Although a power systems engineer by education, he has always found himself inclined toward the world of UX.  He has been associated with several firms and has helped define experiences across a wide range of products.  Apart from that, he’s a terrible singer, a dog lover, and an out and out foodie with decent culinary skills.  You can visit his website or follow him on Twitter.

« Previous PageNext Page »