Posts Tagged ‘usability testing’

Project Profile: BECU (Boeing Employees’) Credit Union

Friday, July 23rd, 2010

We often sneak a quick look at the projects our members are running and we came across this beauty by the clever guys at the interactive agency ZAAZ, who did some re-design work on the BECU (Boeing Employee’s Credit Union) website (www.becu.org).

Their project was designed to give them the answer to just one question, “What do we call the place where our website visitors go to do their…you know…thingamybob banking that you do when you don’t do it at one of the BECU Neighborhood Financial Centers?”

A very simple project was constructed to get them precisely that one answer. Here’s what they did…

Five different hi-fidelity wireframe designs of the BECU homepage were created and hosted on a staging server. Three designs were identical except for the label to this unnamed section of the website. One placed the link in the primary navigation, rather than the utility navigation at the top, and the final design placed the link in the footer.

The alternative labels for the section were:

  • Remote Account
  • Access Mobile & Online Banking
  • Remote Banking, and
  • Online Banking (for the one in the footer)

Five separate projects were set up so there was no task order bias and the task was worded the same way for each project:

“Use the site to find how to view your banking information using your internet-enabled cell phone.”

During the evalution, when participants selected a link that did not direct them to the ‘correct’ location they were presented with the screen below, which is particularly handy as navigating through a wireframed website where only the home page had any functional purpose could have been disasterously confusing for participants.

Four follow-up questions were asked, again always the same for each project.  They were:

  1. How difficult was it to complete this task?
  2. How certain did you feel that the “[Name of the section]” link would take you to the information you were looking for?
  3. What specific information would you expect to see when you click on ” Name of the section”?
  4. What, if anything, would be a better name or label for this information?

All in all a quick, simple and inexpensive way of getting a precise link label to a section of their website growing in importance.

As for the results of the testing…well, go to www.becu.org to find out which option provided the best experience.

Tags: , , , ,
Posted in Reports & Articles | Comments Off

New Feature: Participant Filtering

Friday, July 2nd, 2010

Today we’re unveiling an exciting new feature to help add more value to your project reporting – participant filtering.

When we first launched Loop11 we made the decision to only show the results of participants who fully completed user tests in the reporting. We had a strong belief (and still do) that participants who do not fully complete user tests are not giving your tasks and questions their best efforts. So we excluded them from reporting altogether.

Since launching we’ve had many, many, many requests from our members to make the results of partially completed user tests available. We listened, and so that’s what we’ve done.

Here’s a bit of a tutorial of how it works:

  •   When you log into your account you will now find in your list of Launched Projects a column titled ‘Participants  Start/Finish’ (as shown below). This indicates the total number of participants who commenced your user test  followed by the total number who completed it.

  •   By default, your reporting will only show you the results of participants who fully complete your user tests. But if you want to change that so you can see the results off all participants up until they drop out of your user test you can now go to Settings, tick the box under ‘Include participants in the reporting that have partially completed  the user test’, then click Save Settings.

  •   In your reporting you’ll now see an indication of the total number of participants who completed each task. The early tasks will always have more as participants drop out of your project.
  •   On the right hand side, we’ve included a Settings Indicator that will always tell you whether you are looking at all participants or completed participants so you don’t have to remember.
  •   NOTE: If you set a quota of 100 participants, for example, your project will still only close after 100 FULLY COMPLETED user tests have been collected.

  •   When you’re analysing participants individually, you’ll also be told which participants were guilty of not fully completing your user test.

In the next couple of weeks we’ll be adding a few more useful features to Settings so you can clean up your data by excluding the results of participants who fall outside certain time-based and click-based thresholds that you can customise.

Stay tuned…and happy testing!

Tags: , , , , ,
Posted in Reports & Articles | 5 Comments »

Loop11 Updates: Track participants, Re-categorise multiple URLs, Pop-up invitation control, Improved reporting and more

Tuesday, June 22nd, 2010

It’s been a while since we posted an update about Loop11, but we’ve just launched some new features, and we wanted to let you know about them.

Apart from the many minor (oh, alright…and a couple of major) bug fixes we made recently, we’ve also improved the performance of our servers so page load speeds should be faster.

The other key changes include:

Track participants using unique IDs

If you want to track participants so you know who to provide incentives to you can customise the end of the URL with a unique ID for each participant. You can find this new feature in Step 3 – User Test Options and full details of the process here.

Re-Categorise multiple URLs at a time

Instead of re-categorising one URL at a time you can now re-categorise multiple URLs at the same time. You can find this new feature in the reporting of any task.

Pop-up invitation controls

If you are recruiting your participants by intercepting them via your website, you now have a feature allowing you to control the percentage of visitors that you ‘invite’. You can find this new feature in Step 4 – Invite Participants: Create a pop-up invitation for your own website.

Individual participant path analysis in exports

The full path analysis of individual participants was never made available in the exported reports. Don’t ask why. But it is there now.

Individual responses for Rating Scale Matrix questions

The results for rating scale matrix questions, while always available at an aggregated level were never available so you could see the individual participant responses. These can now be found in your reporting under Participants.

Most of these changes have come about from customer feedback. Just give us your opinion and we will happily take it on board!

Happy Testing!

PS: The guys at UserFirst have elaborated on these features on their blog. The post is located here: http://www.userfirst.com/our-blog/2010/06/28/unmoderated-usability-study/

Tags: , , , , , , ,
Posted in Reports & Articles | 2 Comments »

Planes, trains and usability conferences.

Monday, June 7th, 2010

May was a crazy month for us at Loop11. After kicking the month off at the Conversion Conference in San Jose, we came back to Australia for a few days, before heading off to Europe for a whirlwind tour. With the Icelandic volcano wreaking havoc, and British Airways staff going on strike, the odds of us getting stranded were high. However, with some perseverance and ingenuity, we prevailed and made it to every destination on time.

We attended the NNG conference in London for 2 few days to exhibit, then whizzed off to Munich for the “piece de resistance” for usability professionals; the 2010 UPA conference in Munich. This was the first time the annual UPA conference was held outside the USA. So we were a bit unsure of what to expect. But it turned out to be a great event, not just for UPA but for Loop11 too. We talked to a lot of usability and IT professionals wanting to know more about Loop11 and giving us very insightful feedback. Based on our discussions with people in Munich, we will be fast tracking some new features and offers to our members.

The most interesting thing about UPA 2010 in Munich, is how many people already knew about Loop11 and how many people were seeking us out to give specific feedback, or discuss how they want use Loop11 more in-depth. Maybe the fact we gave away an iPad helped generate a crowd. Congratulations to Maik Klotz for winning our iPad giveaway! Maik was one of the first people in Germany to own an iPad.

In the end in May, the Loop11 team spent over 150 hours travelling, talked to over 400 people and watched over 30 in flights movies. It was all worth it, as the events were a rousing success for Loop11, and we’ll definitely be attending them next year.

Tags: , , , , , , , , ,
Posted in Events | Comments Off

eMetrics & Conversion Conference 2010 – San Jose

Wednesday, May 12th, 2010

We recently exhibited at the 2010 eMetrics / Conversion Conference in San Jose. It was great being in the Silicon Valley and meeting  all sorts of internet professionals. Our booth was a rousing success and the feedback we got was very positive. We even gave away an iPad on the last day. Congratulations to Rob Love from OpinionLab. The event was well organized, and we’ll definitly try to make it next year. Here are some pictures we took of the event.

The Loop11 Booth

What was with the Hippo at Google's booth?

Henri talking to Scott Calise from MTV

Our Loop11 stickers were very popular and seemed to be everywhere

Tags: , , , , , ,
Posted in Events | 2 Comments »

Loop11 Road Trip

Wednesday, March 17th, 2010

In May, we will be hitting the road to showcase Loop11 at three different events. So come and meet the Loop11 team, and receive exclusive offers and discounts!  The three events we are exhibiting at are:

-          Conversion Conference, San Jose, USA (May 4th-5th)

-          NNG London, London, UK ( May 19th-20th)

-          UPA 2010 Conference, Munich, Germany (May 22nd-28th)

Here’s your chance to come and meet us. Get all the latest Loop11 info and a special “buy one get one free” offer available exclusively to people who visit our booth us at these events. We’re also happy to answer any questions or discuss any issues or feedback you may have.

So, if you’re around, pop by our stand and talk to us. We’ll give you all the latest gossip and the special “buy one get one free” deal.

Happy testing!

Tags: , , , , ,
Posted in Events | Comments Off

Get 500 free participants for your next project! One Week Only!

Monday, March 8th, 2010

Yes, that’s right, for every Loop11 credit purchased during the week commencing March 8th 2010, we will arrange for 500 participants to complete your user test! It’s as simple as that.

How this will work:

Just purchase a credit during the week commencing March 8th 2010 and ending March 14th 2010. You can use your credit at any time during that week or at any time in the future.  Contact us when you’re ready to launch it and we will arrange for 500 participants to complete your user test…at no cost to you.  Participants will be randomly sourced from MechanicalTurk.com.  To purchase a usability testing credit and get your 500 free participants, go here: https://www.loop11.com/account/purchase/

Happy Testing

Tags: , , ,
Posted in Events | Comments Off

72 Questions and Answers for Measuring Usability

Thursday, March 4th, 2010

Jeff Sauro from  Measuring Usability recently wrote an in-depth report about quantitative usability testing. We think it’s a great guide for people wanting to conduct their own usability studies (however  the report does cost USD $72 to purchase). In the report, Jeff clearly identifies the benefits of quantitative usability testing as well as explaining the best way to conduct a quantitative usability study.

What are the advantages of a Quantitative Usability Test?

Jeff explains that if you can’t measure it, you can’t manage it. Quantifying usability means making better decisions about usability data. By quantifying the reduction in problems and the increases in efficiency, decision makers will make better decisions regarding their website.

What are common usability metrics?

Although there is an international standard for measuring usability (ISO 9241), the standard leaves open the question of how to measure effectiveness, efficiency and satisfaction. The following metrics are the most common ways of measuring usability.

-          Usability Problems

-          Time on Task

-          Completion And Failure Rates

-          Errors

-          Satisfaction Ratings

-          Clicks / Pageviews / Navigation Path

What are the basic steps for conducting a quantitative usability test?

  1. Recruit participants
  2. Set-up and create a test
  3. Launch the test
  4. Analyse results

To purchase the report “72 questions and answers for measuring usability” go here: http://www.measuringusability.com/products/quantitativeReport

So what are you waiting for? Now you know why and how to conduct a quantitative usability study and with Loop11 your first project is free! Or for more information visit our F.A.Q.

Happy Testing!

Tags: , , ,
Posted in Reports & Articles | 1 Comment »

Online Usability Testing – Who’s more reliable?

Friday, February 12th, 2010

We are often asked if using different types of participants will affect the quality of data from online usability studies.  For example, does a lab-based participant perform tasks the same way a participant from an online study does? Can we really trust data coming from online usability studies where participants might be getting involved just for the money?

These are all interesting points which were recently explored in a usability case study. The case study compared usability data from premium lab-based participants with cheaper online participants. The results were quite interesting. One might presume that the higher paid lab-based user would provide better quality data than their cheaper, online counterparts.  However this was not the case.  The results showed that satisfaction scores and completion rates obtained from the professional users were nearly identical and statistically indistinguishable to the lab-based users.

To read the full case study with analysis go here: www.measuringusability.com

Tags: , , , ,
Posted in Reports & Articles | Comments Off