2009 CDC Conference Presentation Ellery
- 1. 3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009INShape Indiana: A Case Study in Evaluation Strategies for Evaluating Health-Related Website UsabilityPeter J. Ellery Fisher Institute for Wellness and GerontologyJennifer BottMiller School of BusinessKristin RitcheySchool of PsychologyLori ByersCollege of Communication, Information and MediaJane Ellery Fisher Institute for Wellness and Gerontology
- 2. 23 Key Strategies for Evaluating Website Usability3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009Use a Theoretically Based Approach to Your Assessment and Evaluation Process.
- 3. Use Micro Qualitative Research Methods to Determine the Interactive Experience of the User.
- 4. Use Quantitative Macro Research Methods to Determine the Overall Experience of your Clients as a Group.3 Essential Strategies Needed for the Effective Assessment and Evaluation of Health Information Websites
- 5. The Importance of a Theoretically-Based Approach to Determining Website Usability?3
- 6. 4The Importance of a Theoretical Framework3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009Without a theoretical frame of reference, usability data is just a series of numbers, words, or as in this case black marks.
- 7. 5The Importance of a Theoretical Framework3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009Having a frame of reference also helps prevent data from being distorted by other data, and from incorrect interpretations from being made.
- 8. 6The Importance of a Theoretical Framework3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009Do you have the time and resources needed to cope with the problems you face on a daily basis?Salutogenesis: A Theoretical Framework for Determining UsabilityIs the world around you ordered, consistent, structured, and clear? Does it conform with the way you understand the world to be?Is what you do adding value and meaning to your life? MeaningfulComprehensible
- 9. 7The Importance of a Theoretical Framework3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009GRRs are properties of a person, a collective, or a situation that allow individuals to successfully cope and thrive.
- 10. The one quality GRRs have in common is that they all foster repeated life experiences that help one see the world as making sense, cognitively, interactively and emotionally.
- 11. Some examples of GRRs include time, money, ego, cultural background, family support, intelligence, etc.General Resistance Resources (GRRs) give rise to, or reinforce, a strong Sense of Coherence. Deficits in GRRs Diminish our Sense of Coherence.
- 12. INShape Indiana: Assessing Website Usability using Qualitative or Micro Research Methods8Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
- 13. 9Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Program OverviewIndiana State Government supported health and wellness program that is promoted through Indiana businesses and organizations.
- 14. Primarily a web-based health and wellness promotion program that also utilizes motivational/informational emails to participants.
- 15. The program currently has a participant database for over 63,000 Hoosiers of which approximately 70% are female participants.10Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Study DesignThe Qualitative Portion of the Study Design Involved:Developing a Task-Based Interview Guide that included questions related to how meaningful, manageable and comprehensible the website was to the subject;
- 16. Selection of a demographically similar subject sample;
- 17. Interviewing and recordingsubjects while attempting to complete the tasks from the interview guide on the INShape Indiana website;
- 18. CollectingASL eye tracking technology data showing user eye movement around the web pages; and,
- 19. The use of Morae Usability software to capture subject voice and facial expression while the website is being used; 11Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Interview Task ListThe participants in the study were asked to complete tasks and provide their thoughts and opinions related to different aspects of the website.
- 20. Tasks and activities were developed based on the following website sections:
- 21. The landing page or initial home page for the website
- 22. Signing up and the support information section of the website
- 23. The tools and resources section of the website
- 24. The current program support information
- 25. Participants completed tasks and activities that were considered typical and common to users of this kind of website.12Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Subject Selection and Characteristics16 volunteer subjects participated in the study.
- 26. Group demographics and characteristics included:
- 27. All were Ball State University employees currently enrolled in INShape Indiana
- 28. 3 to 1 Female to Male ratio (which was representative of the INShape Indiana Membership gender makeup)
- 29. A variety of education levels
- 30. 3 (Some College Education); 4 (Bachelors Degree); & 9 (Graduate or Terminal Degree)
- 31. A variety of age levels
- 32. 3 (18-24); 4 (25-34); 3 (35-44); 0 (45-54); & 6 (55 and older)
- 33. A variety of computer use and weekly internet access time
- 34. 2 (2 to 3 times per week); 3 (4 to 5 times per week); & 11 (everyday)
- 35. All but one had been to the website prior to the study13Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Examples of What the Subjects Were Looking At Links to the left and right were readily noticed;
- 36. And, the central image does draw viewer attention.
- 37. However, the web page content did not appear to catch the subjects attention very much.14Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009What the Subjects Were Looking At Links to the left and right were readily noticed;
- 38. And, the central image does draw viewer attention.
- 39. However, the web page content did not appear to catch the subjects attention very much.15Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009What the Subjects Were Looking At Links to the left and right were readily noticed;
- 40. And, the central image does draw viewer attention.
- 41. However, the web page content did not appear to catch the subjects attention very much.16Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Comments and Eye Tracking Example for the Subscription and Supporting Information Section ActivitiesUnsubscribing was difficult. Separate it from the subscription process.
- 42. Add headlines to your stories to draw readers attention
- 43. INShape Partners is confusing. Consider offering activities and events related to programs being conducted.
- 44. Good information that is relevant to the user related to tobacco cessation, nutrition and physical activity.
- 45. Having the opportunity to ask questions for themselves and for others is a good feature.17Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Comments and Morae Video Example for the Subscription and Supporting Information Section ActivitiesUnsubscribing was difficult. Separate it from the subscription process.
- 46. Add headlines to your stories to draw readers attention
- 47. INShape Partners is confusing. Consider offering activities and events related to programs being conducted.
- 48. Good information that is relevant to the user related to tobacco cessation, nutrition and physical activity.
- 49. Having the opportunity to ask questions for themselves and for others is a good feature.18Assessing the Individual Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Examples of Comments and ConclusionsUnsubscribe needs to be a stand alone link on the web site. (Comprehensible)
- 50. Easy to wander into other sections on the government side of the site. (Manageable)
- 51. Partner page appears to be more about advertising than useful links for the user. (Meaningful)
- 52. Well organized with good information but too much on the landing page (Manageable).
- 53. The calorie counter, newsletter and videos and the website overall is supportive (Meaningful).
- 54. Links, photos and information all need to be relevant to the user. For example, Whats happening in my County? (Comprehensible)INShape Indiana: Website Usability from a Group Perspective19Assessing the Overall Group Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
- 55. 20Assessing the Overall Group Experience3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Toward a Quantitative ToolAdapted from Antonovskys Orientation Toward Life Survey.
- 56. Comprised of rating items (1 to 5) for 31 statements related to the meaningfulness, manageability and comprehensibility of the website.
- 57. Administered on-line to users of the INShape Indiana website and completed by almost 400 participants.
- 58. Responses were used to reduce the number of survey items and begin the validity and reliability testing process.
- 59. Preliminary analysis of the data suggests a one factor structure consistent with Antonovskys suggestion that Meaningfulness, Manageability, and Comprehensibility are interconnected and may be more appropriately viewed as the single factor Sense of Coherence.213rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009ImplicationsFormulating a theoretical framework is essential as we work toward a common understanding of usability.
- 60. Website evaluation should include qualitative research methods that look at individuals actually using the website.
- 61. ASL Eye Tracking and Morae Video are great support tools in qualitative research looking at a users experience on a website but should always be viewed in context with the qualitative data collected from the subject.
- 62. An overall group evaluation of your clientele should provide a benchmark measure of their sense of coherence when using the website, and offer some indication as to those aspects of the website that make it difficult to use.