際際滷

際際滷Share a Scribd company logo

Make better surveys

Jeremy Rosenberg
Jeremy Rosenberg

My UX Camp Brighton 2016 on why writing a good survey is hard and tips on how to create better ones. UX research & design www.jeremyrosenberg.co.uk

1 of 25
Download to read offline
Make better surveys Jeremy Rosenberg @jeremy74
@jeremy74 About me UX researcher & designer for 7 years Agency and client side BSc. in Psychology & Cognitive Neuroscience MSc. in Human Computer Interaction (HCI) Published psychology study in peer reviewed journal Run many surveys!
@jeremy74 Surveys are hard Face to face conversations are rich interactions We seek feedback and modulate our responses We can know when there is shared understanding Surveys are rigid and prone to misunderstanding
@jeremy74 Is a survey the best methodology? Avoid doing it instead of direct user contact e.g. interviews Are there better ways to answer your question?
@jeremy74 Be clear about the purpose Be clear about what you want to 鍖nd out What will you do with the information Hypotheses
@jeremy74 A bad survey doesnt smell Often dont know if the questions were bad/missed asking the right things
@jeremy74 Be ruthless with question choice People will only answer so many questions Justify every question Respect the users time Branch it
@jeremy74
@jeremy74 Dont assume people can/want to answer Adds noise to the data or increases drop-out Allow skipping Include active skipping e.g. n/a Include other option
@jeremy74 Put boring questions at the end Improves completion rates Completion tendency/sunk cost Still get partial info for drop outs Roberson, M.T. & Sundstrom, E. (1990). Questionnaire design, return rates, and response favorableness in an employee attitude questionnaire. Journal of Applied Psychology, 75, 354-357. Martin, J.D. & McConnell, J.P. (1970). Mail questionnaire response induction: the e鍖ect of four variables on the response of a random sample to a di鍖cult questionnaire. Social Science Quarterly, 51, 409-414.
@jeremy74 Be clear/speci鍖c How often do you cook?
@jeremy74 Be clear/speci鍖c How often do you cook? How often do you prepare a meal?
@jeremy74 Be clear/speci鍖c How often do you cook? How often do you prepare a meal? How often do you prepare a meal, excluding ready-meals
@jeremy74 Be clear/speci鍖c How often do you cook? How often do you prepare a meal? How often do you prepare a meal, excluding ready-meals How often do you prepare an evening meal, excluding ready meals?
@jeremy74 Be clear/speci鍖c How many times did you prepare an evening meal, excluding ready-meals in the last 7 days?
@jeremy74 Be clear/speci鍖c How many times did you prepare an evening meal, excluding ready-meals in the last 7 days? Did you prepare any evening meals, excluding ready-meals, in the last 7 days? then, if so, How many did you prepare?
@jeremy74 Subjective questions are unreliable Like usability testing, behaviour is more reliable than attitudes/preferences Bertrand, M. & Mullainathan, S. (2001). Do people mean what they say? Implications for subjective survey data. American Economic Review, 91, 67-72.
@jeremy74 Anticipate biases Is sustainable seafood important to you versus Are you interested in any of the following?
@jeremy74 Free text responses Be judicious Do you really have no idea of range of responses? Free text as an answer option
@jeremy74 Other things to get right Avoid leading questions Avoid double barrelled questions Avoid absolutes Target the appropriate audience Reduce reliance on memory Consider cost/bene鍖t of open ended questions Map it out if its complicated
@jeremy74 Other things to get right Consider a pilot study Randomise question order, and answer options, where possible Make sure it works on mobile Map it out if its complicated
@jeremy74
@jeremy74 Other things to get right Consider a pilot study Randomise question order, and answer options, where possible Make sure it works on mobile Map it out if its complicated
@jeremy74
@jeremy74 Thanks Questions or tips to share? Will Tweet link to slides @jeremy74

More Related Content

Make better surveys

  • 1. Make better surveys Jeremy Rosenberg @jeremy74
  • 2. @jeremy74 About me UX researcher & designer for 7 years Agency and client side BSc. in Psychology & Cognitive Neuroscience MSc. in Human Computer Interaction (HCI) Published psychology study in peer reviewed journal Run many surveys!
  • 3. @jeremy74 Surveys are hard Face to face conversations are rich interactions We seek feedback and modulate our responses We can know when there is shared understanding Surveys are rigid and prone to misunderstanding
  • 4. @jeremy74 Is a survey the best methodology? Avoid doing it instead of direct user contact e.g. interviews Are there better ways to answer your question?
  • 5. @jeremy74 Be clear about the purpose Be clear about what you want to 鍖nd out What will you do with the information Hypotheses
  • 6. @jeremy74 A bad survey doesnt smell Often dont know if the questions were bad/missed asking the right things
  • 7. @jeremy74 Be ruthless with question choice People will only answer so many questions Justify every question Respect the users time Branch it
  • 9. @jeremy74 Dont assume people can/want to answer Adds noise to the data or increases drop-out Allow skipping Include active skipping e.g. n/a Include other option
  • 10. @jeremy74 Put boring questions at the end Improves completion rates Completion tendency/sunk cost Still get partial info for drop outs Roberson, M.T. & Sundstrom, E. (1990). Questionnaire design, return rates, and response favorableness in an employee attitude questionnaire. Journal of Applied Psychology, 75, 354-357. Martin, J.D. & McConnell, J.P. (1970). Mail questionnaire response induction: the e鍖ect of four variables on the response of a random sample to a di鍖cult questionnaire. Social Science Quarterly, 51, 409-414.
  • 12. @jeremy74 Be clear/speci鍖c How often do you cook? How often do you prepare a meal?
  • 13. @jeremy74 Be clear/speci鍖c How often do you cook? How often do you prepare a meal? How often do you prepare a meal, excluding ready-meals
  • 14. @jeremy74 Be clear/speci鍖c How often do you cook? How often do you prepare a meal? How often do you prepare a meal, excluding ready-meals How often do you prepare an evening meal, excluding ready meals?
  • 15. @jeremy74 Be clear/speci鍖c How many times did you prepare an evening meal, excluding ready-meals in the last 7 days?
  • 16. @jeremy74 Be clear/speci鍖c How many times did you prepare an evening meal, excluding ready-meals in the last 7 days? Did you prepare any evening meals, excluding ready-meals, in the last 7 days? then, if so, How many did you prepare?
  • 17. @jeremy74 Subjective questions are unreliable Like usability testing, behaviour is more reliable than attitudes/preferences Bertrand, M. & Mullainathan, S. (2001). Do people mean what they say? Implications for subjective survey data. American Economic Review, 91, 67-72.
  • 18. @jeremy74 Anticipate biases Is sustainable seafood important to you versus Are you interested in any of the following?
  • 19. @jeremy74 Free text responses Be judicious Do you really have no idea of range of responses? Free text as an answer option
  • 20. @jeremy74 Other things to get right Avoid leading questions Avoid double barrelled questions Avoid absolutes Target the appropriate audience Reduce reliance on memory Consider cost/bene鍖t of open ended questions Map it out if its complicated
  • 21. @jeremy74 Other things to get right Consider a pilot study Randomise question order, and answer options, where possible Make sure it works on mobile Map it out if its complicated
  • 23. @jeremy74 Other things to get right Consider a pilot study Randomise question order, and answer options, where possible Make sure it works on mobile Map it out if its complicated
  • 25. @jeremy74 Thanks Questions or tips to share? Will Tweet link to slides @jeremy74