�ݺ�ߣ

Does the End Justify the Means?
Exploring & Evaluating
Technological Alternatives to
Qualitative Data Collection
Margaret R. Roller, MA
May 17, 2018

Does the End Justify the Means? – May 17, 2018 1Margaret R. Roller
 Qualitative research & typical methods/mode
 Tech solutions:Advantages & drawbacks
 Tech solutions: Method-mode alternatives
Overview

2Margaret R. RollerDoes the End Justify the Means? – May 17, 2018
Reference: Total Quality Framework

Scope
Representation
Cooperation
Data Gathering
Validity
Researcher effects
Participant effects
Reference: TQF Credibility Component

Reference: Text
Roller & Lavrakas, 2015. New York: Guilford Press.
http://bit.ly/TQFbook

Reference: Blog
http://www.researchdesignreview.com

What is Qualitative Research?Qualitative Research: Unique Attributes

Tech Solutions – Advantages
 Representation & cooperation
 Coverage & access (geography & population segment)
 Convenience & flexibility for participants
 Appropriate for sensitive topics
 Relevant form of communication, esp., for certain demographics
 Data accuracy & depth
 Mitigates memory/recall error
 Added dimension via blending approaches, use of visual
elicitation, & multimedia – text, video, images
 Efficient use of resources

Tech Solutions – Potential Drawbacks
 Internet access & comfort with technology
 Managing engagement (skills, time)
 Important visual &/or verbal cues may be missing
 Analysis – Lots of data (in volume & formats)
 Fraud, misrepresentation
“Identity is fluid and potentially multiple on the Internet.”
 Security, confidentiality, anonymity

Tech Solutions:
In-depth Interviews

10
Tech Solutions: In-depth Interviews
EMAIL
WEBCAM
MOBILE
VIDEO
SensitiveTopic
Professionals
Diary/Life Story
Substitute For In-person
Usability
Concurrent Mixed Methods
In-the-moment Research
Location-specific, Geofencing
Study a Process or Event

11
Email
SCOPE DATA GATHERING
CooperationRepresentation
Validity
Researcher
Effects
Participant
Effects
Comfort level
with mode
Length &
perceived
burden may lead
to drop outs
Exclusion of
some segments,
e.g., 65+ & <HS
Depth,
complex data
from follow-up
& “homework”
Participant’s
words & how
text is typed
may bias
Willingness to
discuss sensitive
topic due to
privacy, safe
environment
Mitigates social
desirability
responding
Mitigates
bias due to
ability to
reflect
Access to pop
segment
Flexibility of
location
Geographic
coverage

12
Webcam
Cooperation Validity Participant
Effects
Flexibility of
location
Requires
reminders &
instructions
Social desirability
Depth, ability
to showing
stimuli,
participant
markup
Researcher
Effects
Requires access
& technical
know-how
Willingness to
respond due to
rapport & cues
Consistency,
ability to
follow guide
Bias due to
visual cues
Observers &/or
recording(s) may
stifle responses
Representation
Access to pop
segment
Geographic
coverage

13
MobileVideo
Cooperation Validity Participant
Effects
Engagement
with video
technology
Requires
reminders &
instructions
Selection bias,
choosing
what to share
Mitigates
recall error
Researcher
Effects
Access & comfort
with app –
downloading & use
Geographic
coverage Mitigates on-
site social
interaction
bias
Depth from
verbal, nonverbal,
& contextual cues
Inconsistency,
difficulty
following guide
Altered
behavior
Absence of
broader context
& meaning
Representation
Access to pop
segment

Tech Solutions:
Asynchronous Focus Groups

Tech Solutions: Asynchronous FGs
Asynchronous
Focus Groups
SensitiveTopic
Professionals
Hard-to-reach, disabled

16
Cooperation Validity
Participant
Effects
Flexibility of
location, time,
& device
Requires
reminders &
instructions
Nonresponse –
Unwillingness or
inability to
participate in a
mostly text-
based discussion
Breadth & depth
due to days of
discussion
Researcher
Effects
Access &
comfort with
platform
technology
Platform
features help
mitigate bias
& maintain
consistency
Triangulation,
multi-media
responses – text,
video, image Losing focus
while
managing
engagement
Mitigates bias,
multiple ways
to respond
Representation
Access to pop
segment
Geographic
coverage

Platform Functionality – Discussion Guide

Platform Functionality – Activities

Platform Functionality – Managing Engagement

Tech Solutions:
Observation &
Multimethod/mode

“Day in the Life”
Journey
Specific assignment/task
Tech Solutions:
Observation & Multimethod/mode

24
Mobility Patterns
Among Children
IDIs
Observation
via GPS &
SMS questions
“Day in the Life”
IDIs
using GPS maps &
guided tour as stimuli
AcneTreatment
2-month
multimedia diary
IDIs
via mobile video
+
Brazilian Migrants
6-month online
observation of
migrant communities
IDIs
via email & in-person
+
Tech Solutions:

25
Influence of the Internet
in the Classroom on Literacy
Email IDIs
Diaries
Photos
Journey
+
+
Teachers’ published articles
+
Emotional Effects of
Selecting a College
Webcam IDIs
Participants submit text, images,
video during the course of the
application-selection process
Asynchronous focus group
Tech Solutions:

26
Shopping For & Using Body
Wash Product
Audio responses to
questions on “wallet card”
In-store photos & videos
Specific Assignment/Task
+
In-shower audio responses
to questions on laminated card
In-the-moment Sensorial
Experience with Meatless Burgers
Photo + text
regarding package
1’ video + text
regarding interaction with package &
cooking burger (look, smell, texture)
Text
regarding the eating experience
Tech Solutions:

27
Tech Solutions:
Cooperation Validity
Participant
Effects
Comfort with
technology
(among some)
Tasks/length
may be
burdensome Triangulation,
methodological
& researcher
Researcher
Effects
Bias towards
tech savvy
individuals
Broad geographic
coverage Mitigate bias
by reflecting
on multi data
Depth due to
synergy of
methods, multi-
layered data
Tasks bias
normality &
taint study
environment
Tasks/length
help reflexive
responding
Tasks alters
behavior &
attitudes
Selection bias
Representation

Does the End Justify the Means? – May 17, 2018
Interested in Qualitative Research
& Hard-to-reach Participants?
Recruiting Hard-to-Reach Respondents with Complex
Characteristics for Qualitative Interviews
Daniela Glusberg, Dánae Corado, Casey Langer Tesfaye
AAPOR Thursday May 17th
Time: 4:15 p.m. – 5:45 p.m. Session 10
Location: Governor’s Square 17
Check Out

Questions &
Comments
Margaret R. Roller, MA
Email: rmr@rollerresearch.com
Website: rollerresearch.com
Blog: researchdesignreview.com

Appendix
References

EMAIL IDI
Bowker, N., & Tuffin, K. (2004). Using the online medium for discursive research about people with disabilities. Social Science
Computer Review, 22(2), 228–241.
Egan, J., Chenoweth, L., & McAuliffe, D. (2006). Email-facilitated qualitative interviews with traumatic brain injury survivors: A
new and accessible method. Brain Injury, 20(12), 1283–1294.
Hawkins, J. E. (2018). The Practical Utility Suitability of Email Interviews in Qualitative Research The Practical Utility Suitability of
Email Interviews in Qualitative Research, 23(2), 493–501.
McCoyd, J. L. M., & Kerson, T. S. (2006). Conducting intensive interviews using email: A serendipitous comparative opportunity.
Qualitative Social Work, 5(3), 389–406.
Stacey, K., & Vincent, J. (2011). Evaluation of an electronic interview with multimedia stimulus materials for gaining in depth
responses from professionals. Qualitative Research, 11(5), 605–624.
ASYNCHRONOUS FOCUS GROUP
Tates, K., Zwaanswijk, M., Otten, R., van Dulmen, S., Hoogerbrugge, P. M., Kamps, W. A., & Bensing, J. M. (2009). Online focus
groups as a tool to collect data in hard-to-include populations: Examples from paediatric oncology. BMC Medical Research
Methodology, 9(1), 15.
Harmsen, I. A., Mollema, L., Ruiter, R. A., Paulussen, T. G., de Melker, H. E., & Kok, G. (2013). Why parents refuse childhood
vaccination: a qualitative study using online focus groups. BMC Public Health, 13, 1183.
References

OBSERVATION & MULTIMETHOD/MODE
Christensen, P., Mikkelsen, M. R., Nielsen, T. A. S., & Harder, H. (2011). Children, mobility, and space: Using GPS and mobile
phone technologies in ethnographic research. Journal of Mixed Methods Research, 5(3), 227–246.
Hancock, K. (2012). Online qual studies health care foundation to shift its focus. Quirk’s Marketing Research Review, October,
Article ID 20121006.
Karchmer, R. A. (2001). The journey ahead: Thirteen teachers report how the Internet influences literacy and literacy instruction
in their K-12 classrooms. Reading Research Quarterly, 36(4), 442–466.
Schrooten, M. (2012). Moving ethnography online: Researching Brazilian migrants’ online togetherness. Ethnic and Racial
Studies, 35(10), 1794–1809.
OTHER
James, N., & Busher, H. (2009). Online interviewing. London: Sage Publications.
“Minute Maid Debuts New Campaign That Celebrates the Good in Everyday Family Moments,”
https://www.businesswire.com/news/home/20180409006050/en/Minute-Maid-Debuts-New-Campaign-Celebrates-Good
(Accessed April 10, 2018)
http://www.pewinternet.org/fact-sheet/mobile/ (Accessed April 17, 2018)
References

�ݺ�ߣ

Technological Alternatives to Qualitative Data Collection

More Related Content

Technological Alternatives to Qualitative Data Collection