Samaritans is a charity in the UK and Ireland specialising in confidential emotional support. It offers its invaluable service 24 hours a day for people who are experiencing feelings of emotional distress, despair or suicidal thoughts, using trained volunteers to respond and lend help to phone calls, emails and letters.
Relying entirely on donations and volunteer help, Samaritans’ website is the crucial portal through which the charity operates. To improve the charity’s on-line portal, Samaritans sought the services of user-centred digital communications’ specialists, SiftGroups, with a view to redesigning its website.
It was unknown whether the established Samaritans’ website was meeting the needs of current donors, volunteers and those needing emotional assistance.
SiftGroups ran a project to explore the usability of Samaritans’ established website with a view to informing how to shape its approach to the forthcoming website redesign.
SiftGroups conducted an online usability appraisal of the current Samaritans website with 149 participants, providing an overall snapshot of how well core strategic tasks were performed by the site’s main audience types. The key areas SiftGroups assessed included vitally important and fundamental site tasks for the charity:
- • Getting Help
- • Making Donations and
- • Becoming a Volunteer.
The study was distributed to both “warm” and “cold” users that came from Samaritans’ wider internal email database. Help tasks and questions were devised linking participants to the actual site to better determine user satisfaction, validate task completion and enable SiftGroup’s to form a strategic recommendation for the navigation of the new site design.
THE ANALYSIS AND RESULTS
Upon completion of the evaluation, the real-time reporting capability allowed SiftGroups to track and evaluate the task completion rates and monitor the navigation path of participants. The results showed that only half of the participants successfully completed the tasks, identifying in more detail those areas requiring addressing for the site remodeling.
In particular, the results showed a significant issue with the current site navigation when it came to those in need of finding ways of talking to friends they’re worried about with 66% of participants either failing or abandoning the task of “talking to someone” altogether.
The results also revealed the need to improve the retrieval of the all important real life case studies, which encourage those thinking of seeking help as well as volunteers and donors:
As a result of the study, SiftGroups was able to provide a strategic recommendation for an improved usability experience using hard statistics to back up new navigation models.
LOOKING TO DO SOME SIMILAR, EFFECTIVE USABILITY TESTING?
With its proven track record, Loop11 will help you get the most out of your website. Providing the ultimate online usability testing experience, Loop11 will reveal some startling home truths about your website usability generating quantifiable results and delivering real time performance without the cost of lab-based testing sessions. It’s fast, affordable, simple to use and easy to get started straight away from the comfort of your own office. Sign-up now or discover another Loop11 success story here.
The guys at Balsamiq asked me recently to write a brief article for their customers on how to conduct usability testing with Loop11 on prototypes and wireframes built with Balsamiq. My article below shows the steps involved in putting a simple Loop11 project together to test a prototype of the Kayak website.
We’re excited to announce the first of a number of new features for Loop11 in 2012 – Clickstream Analysis. The clickstream analysis will replace the ‘Most common navigation path’ by allowing you to analyze task navigation graphically and instantly understand how visitors navigate a task through your website.
The clickstream report provides a graphical representation of participants’ navigation through the website so you can see their journey, as well as the path they took before abandoning or failing a task.
You’ll notice that we made the visualization highly interactive so you can interact with the graph to highlight different pathways, and to see detailed information about specific pages. For example, if you want to dive deeper into your pages, you can hover over the node to see more information at a glance.
Below you can see an example of a task that performed well in usability testing (in this instance with a task completion rate of 92%). A quick look at the analysis shows that 90% of participants went directly to the success page from the homepage. The orange lines are a visual indication of the magnitude of participants who failed the task at different points in their journey through the website. Navigation through the website is clean and uncomplicated, which should be the case when the participant has a clear direction.
By contrast, the clickstream below is for a task that was performed comparatively poorly, where the task completion rate was only 32%. In this example, there is clearly confusion as to where participants should navigate from the home page to complete the task.
This is our first step in tackling clickstream analysis and we look forward to hearing your feedback as you begin to use the report in the coming weeks. We’re excited to bring you the first of a number of new features for the year, so stayed tuned for more!
As always, we welcome your input on how we can make the clickstream analysis more useful for you, so let us know in the comments below.
How many participants should be used for Online, Quantitative Usability Testing?
Qualitative usability testing has traditionally been based around small sample sizes of 5-20 participants. However, the growth of online testing tools and quantitative usability research is changing the game. Whist many experts agreed that for qualitative, lab-based testing small samples sizes of 5-20 participants is sufficient. Online user testing has created a new wave of analysis such as benchmarking, A/B testing, competitor comparison, validating and much more. These kinds of quantitative analyses require larger numbers of participants to validate the data.
For example: Let’s say your company is testing two different versions of wireframes so management can decide and approve one to implement and allocate resources to. It would not be very good practice to use 10 participants and get a 60%-40% success rate. It would be very hard to validate a study and implement a strategy based on 10 participants. However, if 500-1000 participants were used, that data would be a lot more accurate, and management would have valid data to approve of the findings.
Some specialists, such as Usability Sciences, recommend up to several thousand participants for more high level quantitative testing such as, click-stream data, or multiple cross-tabulations. They have broken down the % error for margin based certain numbers of participants. Read the full article here.
2011 was a big year, not just for world despots, but for Loop11 too. Whilst global dictators everywhere fell to revolutions, Loop11 contributed its part to the remote, unmoderated usability testing revolution. As revealed in the recent UPA biennial survey the use of remote unmoderated testing grew by 28%. Our own usage statistics also show some interesting figures which we thought we would share with you here.
Here are some big numbers about Loop11 in 2011:
… 60,456 participants:
That’s right! 60,456 participants from all over the world took part in Loop11 usability testing projects. From Brazil to Japan and students to retirees, people of all ages and places participated in Loop11 usability tests.
… 2165 usability testing projects created:
2165 different projects were created using Loop11. Whether our customers wanted to test a current website, benchmark their competitors or test a new design, Loop11’s various capabilities allowed them to conduct a variety of usability studies.
… 34 languages:
Projects were created in 34 of the 42 different languages available. Of the 2165 projects created, 72% were in English followed by 4% in Spanish, 3% in German and 2% in Portuguese.
… Over 30 countries:
Our customers from all around the world used Loop11 to conduct remote usability studies. From China to Australia, Loop11 was used by people everywhere and anywhere to conduct remote usability testing!
… Dozens of different devices:
As more and more people used Loop11, one of our best features really came to fore. Loop11 was used to conduct usability testing on an array of different internet enabled devices. PCs, iPhones, iPads, Androids and all sorts of other internet enabled devices. We even had someone test the usability of an ATM interface by putting the interface online and using an iPad as the touchscreen. Now that is genius!
… 4,300 new customers:
Overall, it was a great year for Loop11. With 4300 new customers who signing up to Loop11.
We’re sure 2012 will be an even better year. With the growing demand for remote usability testing, and with more new cutting edge features and quirky promotions coming your way, we aim to raise the bar once again.
Happy New Year and Happy Testing!
We’re celebrating Christmas nice and early this year. If you planned on spending your left over quarterly budget on eggnog and Christmas decorations, we’re hoping that you do the responsible thing instead and buy yourself a UXmas present.
We called it UXmas because it includes four great tools at a gift of a price. They are User eXperience enhancing tools that will make life easier for you and make your site better, easier and more effective.
So here’s our UXmas offer:
You get 3 user testing projects for $700, saving $350. If you’ve already got an account we’ll credit the extra time to it.
You get to vote on which three of the following three excellent tools you want us to include for free. Each option has 12 months access and is fully featured:
• OpenHallway (Worth $624)
Record screencast video and audio of users testing your website, locally or remotely – all from the comfort of your browser
• ConceptShare (Worth $600)
Markup visual designs collaboratively, sketch directly onto images, compare alternatives side-by-side and collaborate and get approval remotely.
• HotGloo (Worth $336)
Rapid collaborative wireframing – design your page layout and focus on your message, your navigation and your interaction before you add graphics.
• SnapEngage (Worth $540)
Live chat with your site visitors and customers.
• Websort (Worth $2,499)
Card sorting is a simple but effective way to discover how people think your site content should be organized.
• Plainframe (Worth $1,499)
Allows you to easily and quickly put your navigation structure into a plain, clickable site.
Hopefully that covers all the bases. $1,050 worth of user testing projects (3 studies), for $700, and 12 months access to the three most popular UX tools for free.
The UXmas offer will only be available for two days starting on Thursday 15th December, but you will be able to request an invoice/purchase order and pay within 21 days. Please start talking to whoever needs to authorise the payment and any other teams that may want to use the bonus tools.
Merry Christmas and Happy Testing!
Loop11 will be conducting a scheduled server upgrade on Saturday December 3rd, 2011 beginning at 21:00 PM UTC (Co-ordinated Universal Time), and lasting up to 1 hour.
During this time, the Loop11.com website and systems, will be temporarily inaccessible. Any content hosted on Loop11.com will not be accessible by you or your project participants. During this time a message will be posted on the Loop11 website alerting visitors of the scheduled maintenance. This should only take an hour, and once restored, all your content and data will be back online.
• Eastern Daylight Time (North America): Saturday, July 17: 4:00pm
• Greenwich Mean Time (Europe): Saturday, July 17: 9:00pm
• Eastern Standard Time (Australia): Sunday, July 18: 10:00am
We appreciate your patience and can assure you the service will be restored as quickly as possible. We also wanted to point out some recent improvements:
• Participant instructions page can now be skipped. For anyone running True Intent studies,
the original instructions to participants were not quite suitable. Instructions can now be completely skipped.
• Custom IDs. If you want to track participants so you know who to provide incentives to, you
can customise the end of the URL with a unique ID for each participant. Learn more.
For additional time zones, please reference www.timeanddate.com.
We apologise for any inconvenience caused, if you have any queries relating to this notice please email us at Support@Loop11.com.
One of the most common uses of Loop11 is to test the usability of wireframes, prototypes and concepts. This newfound power has surprised some people, as for the first time they can now test the usability of wireframes, as easily as they can a website. You may have read the Boeing Credit Union wireframe case study, and wondered how it was done. Well, for those of you wondering, this is how to test wireframes using Loop11.
Well for those wondering how, this is how to test wireframes using Loop11:
- 1. Design some wireframes
- 2. Export your wireframes in HTML
- 3. Upload the exported wireframes to a public domain
- 4. In Loop11, enter the URL of the wireframe as the starting point URL (see below)
Create some tasks and questions to suitably test their usability and Tthat’s pretty much it! Now test it as you would a normal website. Now there are no excuses, we expect a massive influx of wireframe usability tests now.
If you would like to know more about wireframe testing, checkout our case studies here or contact as at support@Loop11.com.
As you are reading this, dozens of our clients are running remote usability studies in an array of different languages. In the 2 years that Loop11 has operated, clients have run all sorts of usability tests in over 20 languages. Some of our clients in fact conduct the exact same test in multiple languages, which makes a lot sense if your website is multilingual. As you can see from the chart below, English is still the predominant language, but clearly the rest of the world is coming.
So we wanted to give a shout out to all the languages already being used in Loop11, such as; French, German, Russian, Suomi, Spanish, Italian, Japanese, Hebrew, Swedish, Czech, Polish, Chinese, Portuguese and even Bahasa. But more importantly, we want to remind people that Loop11 is capable of running user tests in over 40 languages, which means that there many more languages yet to be tried in Loop11.
Got a website in a foreign language you want tested? Try Loop11 for free.
Between October 10, 2011 and November 10th 2011, the more @Loop11 tweets there are the bigger the discount will be on November 11th, 2011. You won’t get a chance like this for another 100 years!
We have set five thresholds:
- 100 Tweets = $300
- 250 Tweets = $250
- 500 Tweets = $100
- 750 Tweets = $50
- 1000 Tweets = $11
Happy tweeting and testing!