ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo
The User Is Always Right (Usually)
4 Research Methods That Get Results
by Michael Hartman, Sandstorm Design?
The User Is Always Right (Usually): 4 User Research Methods That Get Results
A little about Sandstorm?
? Chicago-based UX and digital marketing agency
? Drupal development shop
? 1500+ in depth interviews and usability studies
? Industry Expertise
¨C Healthcare
¨C Financial Services
¨C Associations
¨C Technology
¨C Manufacturing
The User Is Always Right (Usually): 4 User Research Methods That Get Results
What we¡¯ll cover
In Depth
User Interviews
Heuristic
Analysis
Card Sorting
& Tree Testing
Usability
Studies
In Depth User Interviews
Why Conduct In Depth User Interviews
? Uncover user goals and use cases
? Content requirements
? Functional requirements
What to Expect from In Depth Interviews?
? User stories and requirements
? Personas
? Allocate budget to most beneficial functionality
? Identify opportunities to meet previously unknown needs
? Align organization goals with user goals
? First step in design process
? Anytime is better than never
? If you haven¡¯t spoken with your users yet, do it now
When Conduct In Depth User Interviews?
1. Identify research goals
2. Identify your user groups
3. Write protocol
4. Recruit and schedule
5. Conduct interviews
6. Analyze results, user stories, requirements, personas
How to Conduct In Depth User Interviews
1. Ask a mix of open-ended questions, behavior
based questions and quantified questions
2. Allow space for follow up and probing
3. Be flexible yet consistent
Tips for In Depth User Interviews
Case Study: Health Care
Card Sorting & Tree Testing
Why Do Card Sorting and Tree Testing
Build a user centered site structure and navigation
? Create a new navigation system and menu structure
? Test an existing navigation system and menu structure
? Identify user-centric labels for your navigation
and content
When to Conduct Card Sorting or Tree Testing
? At the beginning of the design process
? Anytime you realize your current navigation sucks
How to Conduct Card Sorting
? Identify / catalog your content
? Choose sample of content for testing
? Conduct the study ¨C online or in-person
? Analyze results and build your new navigation
How to Conduct Tree Testing
? Build a sample menu structure of your site (the tree)
? Ask your users ¡°where would you go to find X?¡±
? Adjust you menu structure accordingly
Tips for Card Sorting and Tree Testing
? At least 16-20 users
? The online tools are worth the cost
? Test users not stakeholders
Case Study: Neurosurgery
Case Study: Neurosurgery
Usability Studies
Why Conduct Usability Testing
? Add user context to your QA process
? Test forms and processes
? Test mobile and touch devices
5 ¨C 6 Users Per User Group
Reference: http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
? Identify confusing or unclear language and
navigation labels
? Identify confusing or broken processes
? Identify inconsistencies between multi-device versions
of your site (mobile, tablet, desktop)
? Identify issues with the ¡°findability¡± of content
What to Expect from Usability Testing
When to Test in the Design Process?
? Early and often is the best practice...but definitely before
launch
? Informal usability testing with your team
? Test wireframes and prototypes
? A/B testing on user interface concepts
How to Conduct Usability Testing
? Identify the goal of your study and key tasks
? Identify your users and participant criteria
? Write test protocol
? Recruit users
? Conduct study
? Analyze results and make improvements
Tips for Usability Testing
? Focus on conversion points
? Think aloud protocol
? Record if possible
? Allow room for follow up questions and clarifications
? Don¡¯t interfere; observe
? Test mobile, touch and desktop experiences
? Involve users not stakeholders
Case Study: Radio Flyer
Heuristic Analysis
Heuristic Analysis
? A checklist of heuristics with pass / fail / needs
improvement scoring
? Reasonable substitute for a usability study when it¡¯s not
possible to recruit users
? Check for 508 compliance
? Identify usability issues when testing with real users
is not possible
? Checks for site structure, 508 compliance, SEO best
practices, etc
? Benchmark your site against recognized
usability standards
What to Expect from a Heuristic Analysis
When to Conduct a Heuristic Analysis
? When you don¡¯t have time or budget for a usability study
? When your site needs to meet certain compliance
standards (508 or WCAG 2.0)
How to Conduct a Heuristic Analysis
? Define heuristics, several good lists available online
? Conduct analysis, when practical have multiple people
review the site
? Follow your list of heuristics
? Analyze results and make improvements
Tips for Conducting a Heuristic Analysis
? Several good checklists available online
Case Study: Bermuda Monetary Authority
Interpretation & Analysis
Questions?
Presented by:
Michael Hartman
Director of Technology & Usability
Sandstorm?
773.348.4200
mhartman@sandstormdesign.com
?
The User Is Always Right (Usually): 4 User Research Methods That Get Results

More Related Content

The User Is Always Right (Usually): 4 User Research Methods That Get Results

  • 1. The User Is Always Right (Usually) 4 Research Methods That Get Results by Michael Hartman, Sandstorm Design?
  • 3. A little about Sandstorm? ? Chicago-based UX and digital marketing agency ? Drupal development shop ? 1500+ in depth interviews and usability studies ? Industry Expertise ¨C Healthcare ¨C Financial Services ¨C Associations ¨C Technology ¨C Manufacturing
  • 5. What we¡¯ll cover In Depth User Interviews Heuristic Analysis Card Sorting & Tree Testing Usability Studies
  • 6. In Depth User Interviews
  • 7. Why Conduct In Depth User Interviews ? Uncover user goals and use cases ? Content requirements ? Functional requirements
  • 8. What to Expect from In Depth Interviews? ? User stories and requirements ? Personas ? Allocate budget to most beneficial functionality ? Identify opportunities to meet previously unknown needs ? Align organization goals with user goals
  • 9. ? First step in design process ? Anytime is better than never ? If you haven¡¯t spoken with your users yet, do it now When Conduct In Depth User Interviews?
  • 10. 1. Identify research goals 2. Identify your user groups 3. Write protocol 4. Recruit and schedule 5. Conduct interviews 6. Analyze results, user stories, requirements, personas How to Conduct In Depth User Interviews
  • 11. 1. Ask a mix of open-ended questions, behavior based questions and quantified questions 2. Allow space for follow up and probing 3. Be flexible yet consistent Tips for In Depth User Interviews
  • 13. Card Sorting & Tree Testing
  • 14. Why Do Card Sorting and Tree Testing Build a user centered site structure and navigation ? Create a new navigation system and menu structure ? Test an existing navigation system and menu structure ? Identify user-centric labels for your navigation and content
  • 15. When to Conduct Card Sorting or Tree Testing ? At the beginning of the design process ? Anytime you realize your current navigation sucks
  • 16. How to Conduct Card Sorting ? Identify / catalog your content ? Choose sample of content for testing ? Conduct the study ¨C online or in-person ? Analyze results and build your new navigation
  • 17. How to Conduct Tree Testing ? Build a sample menu structure of your site (the tree) ? Ask your users ¡°where would you go to find X?¡± ? Adjust you menu structure accordingly
  • 18. Tips for Card Sorting and Tree Testing ? At least 16-20 users ? The online tools are worth the cost ? Test users not stakeholders
  • 22. Why Conduct Usability Testing ? Add user context to your QA process ? Test forms and processes ? Test mobile and touch devices
  • 23. 5 ¨C 6 Users Per User Group Reference: http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  • 24. ? Identify confusing or unclear language and navigation labels ? Identify confusing or broken processes ? Identify inconsistencies between multi-device versions of your site (mobile, tablet, desktop) ? Identify issues with the ¡°findability¡± of content What to Expect from Usability Testing
  • 25. When to Test in the Design Process? ? Early and often is the best practice...but definitely before launch ? Informal usability testing with your team ? Test wireframes and prototypes ? A/B testing on user interface concepts
  • 26. How to Conduct Usability Testing ? Identify the goal of your study and key tasks ? Identify your users and participant criteria ? Write test protocol ? Recruit users ? Conduct study ? Analyze results and make improvements
  • 27. Tips for Usability Testing ? Focus on conversion points ? Think aloud protocol ? Record if possible ? Allow room for follow up questions and clarifications ? Don¡¯t interfere; observe ? Test mobile, touch and desktop experiences ? Involve users not stakeholders
  • 30. Heuristic Analysis ? A checklist of heuristics with pass / fail / needs improvement scoring ? Reasonable substitute for a usability study when it¡¯s not possible to recruit users ? Check for 508 compliance
  • 31. ? Identify usability issues when testing with real users is not possible ? Checks for site structure, 508 compliance, SEO best practices, etc ? Benchmark your site against recognized usability standards What to Expect from a Heuristic Analysis
  • 32. When to Conduct a Heuristic Analysis ? When you don¡¯t have time or budget for a usability study ? When your site needs to meet certain compliance standards (508 or WCAG 2.0)
  • 33. How to Conduct a Heuristic Analysis ? Define heuristics, several good lists available online ? Conduct analysis, when practical have multiple people review the site ? Follow your list of heuristics ? Analyze results and make improvements
  • 34. Tips for Conducting a Heuristic Analysis ? Several good checklists available online
  • 35. Case Study: Bermuda Monetary Authority
  • 37. Questions? Presented by: Michael Hartman Director of Technology & Usability Sandstorm? 773.348.4200 mhartman@sandstormdesign.com ?

Editor's Notes

  1. Technology and Usability Director at Sandstorm Design Lead usability study and user research practice Also do UX ¨C wireframing, user flows, etc Also lead projects / Drupal
  2. Boutique design agency in Chicago Il Association space Key Industries Healthcare Finance Manufacturing Government (Tax and financial reporting) Usability Studies and User Research for B2B B2C E-Commerce Clients include AT&T, Radio Flyer, Time Warner Cable, The Metropolitan Opera, American Medical Association
  3. Your users help write your user stories If you want to answer any of the following questions, we recommend In Depth User Interviews: How do I understand my users? What features would bring the most benefit to my site/users? What do users think about our brand compared to our competitors? How should we be engaging our customers?
  4. Indentify your research goals. What questions are you trying to answer? Determine what types of users (user groups) will participate in the study. A user group is a set of users who have similar goals or use cases on your site or application. This is different from demographics. Write a protocol, that¡¯s a fancy word for the list of questions your going to ask your users. Recruit and schedule the interviews. Interviews can be conducted over the phone to make it convenient for the participants. We recommend offering a gratuity or incentive to participate. Conduct the interviews, 30 to 45 minutes each should be good. Analyze the results and develop your user stories, requirements and/or personas. The results can also be helpful in making business decisions about the scope of your project.
  5. For example, what¡¯s the primary reason you visit example.com? Tell me about the last time you visited example.com, what did you visit for? Tell me 3 things you like about example.com? Tell me 3 things you would like to see improved with example.com? questions like, can you tell me more about that? Can you give me an example? , follow up questions may vary but be sure to follow your protocol with all participants. You¡¯re looking to identify trends, so you¡¯ll need to be consistent in your research methods
  6. Card sorting and tree testing are the yin and yang of determining and testing your navigation and menu structure. Card sorting exercises consist of writing examples of your content on cards and having your users sort those cards into groups and label them. Tree testing works in the other direction where you present the user with a navigation structure and ask them to find particular piece of content. Both are quick and easy ways to arrive at an effective navigation structure and there are several great online tools for conducting both of these exercises. If you¡¯re looking to solve any of the following, card sorting and/or tree testing will help: You¡¯ve heard feedback that your content is hard to find. You¡¯re not sure what to label a section or type of content? Your navigation structure is overly complicated. (hint: it shouldn¡¯t be complicated at all). ?
  7. How can we get more users to complete the check out process? How can we get users to accurately complete each step of this form? How does our site perform on mobile devices? How can we help our users learn more about what we have to offer?
  8. particularly useful for check-out and registration processes or any conversion points
  9. at Sandstorm we start the testing process as soon as we have enough wireframes or a prototype to start getting feedback. Usability testing early in the process can help indentify issues before budget is spent developing something that¡¯s not optimal. It¡¯s also the tool to use if parts of your site aren¡¯t performing as expected. Even when your site is performing well, you¡¯ll want to make sure your site is optimized for your users.
  10. Identify the goal of your study and the key tasks you want to test, don¡¯t try to test too much in one study. If you want to test a lot of areas, it¡¯s better to do multiple studies. Identify your users and participant criteria; make sure you¡¯re testing with people who would use your site in the ¡®real¡¯ world. Write the testing protocol, the list of scenarios you want to test. Recruit users ¨C we recommend offering a gratuity for participation.
  11. Only offer help when it is clear the users has failed the task and you need to move on.
  12. A heuristic analysis is a good substitute for a usability study when a study with your site users is not possible. It can be completed faster and requires fewer resources and therefore less of your budget. Keep in mind, a usability professional will catch most of a site¡¯s usability issues, the downside is you¡¯ll be missing the context and nuances of testing with real site users, particularly in uncovering issues with content and labeling.
  13. Will identify some things a usability study will not find H1 tags, page titles, alt text on images, branding, etc
  14. at Sandstorm we start the testing process as soon as we have enough wireframes or a prototype to start getting feedback. Usability testing early in the process can help indentify issues before budget is spent developing something that¡¯s not optimal. It¡¯s also the tool to use if parts of your site aren¡¯t performing as expected. Even when your site is performing well, you¡¯ll want to make sure your site is optimized for your users.
  15. Define your heuristics, there are several good lists available online, we¡¯ve adapted and appended these lists to create our own set of heuristics. We look at things like: Are navigation and page titles easy to find and use? Are links easy to identify? Are font sizes and spacing easily readable? Is the color contrast between design elements stark enough for easy legibility? Is it clear what each action does? Is it clear what path to take? Are error messages clear and easy to understand? Does the site work well on multiple devices and smaller screens? Conduct the analysis, we use a Google form to list the heuristics and note our comments. When practical we have more than one usability expert conduct the analysis and compare notes. Analyze the results and make improvements to your site.
  16. Define your heuristics, there are several good lists available online, we¡¯ve adapted and appended these lists to create our own set of heuristics. We look at things like: Are navigation and page titles easy to find and use? Are links easy to identify? Are font sizes and spacing easily readable? Is the color contrast between design elements stark enough for easy legibility? Is it clear what each action does? Is it clear what path to take? Are error messages clear and easy to understand? Does the site work well on multiple devices and smaller screens? Conduct the analysis, we use a Google form to list the heuristics and note our comments. When practical we have more than one usability expert conduct the analysis and compare notes. Analyze the results and make improvements to your site.
  17. Only offer help when it is clear the users has failed the task and you need to move on.
  18. If you don¡¯t have the budget for recruiting real users, at least find someone who is not familiar with the project.