際際滷

際際滷Share a Scribd company logo
C O D I N G P U B L I C S A N D C O D E A S E T H N O G R A P H I C A R T E F A C T
S Y 0 6 : M A K E R C U L T U R E S , C O D E A N D P U B L I C N E S S -
E X P L O R I N G ( P O S T - ) D I G I T A L S P H E R E S A N D P R A C T I C E S
Dan Verst辰ndig | Otto-von-Guericke-University of Magdeburg/Germany
@danvers | ebdw.ovgu.de
Going Public? Ethnography in Education and Social Work and Its Publics
October 31st  November 2nd, 2019
Martin-Luther-University Halle-Wittenberg, Germany
W H Y ?
I N O R D E R T O U N D E R S T A N D D I G I T A L D A T A A N D D I G I T A L I T Y ,
W E N E E D T O U N D E R S T A N D D I G I T A L C U L T U R E S .
REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html
a 2-dimensional relation
between researcher and the field
REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html
#!/usr/bin/env python
import twitter, random
api = twitter.Api(consumer_key='xx', consumer_secret='xx', access_token_key='xx', access_token_secret='xx)
friends = api.GetFriends()
for friend in friends:
friendName = friend.GetScreenName()
friend2 = random.choice(friends).GetScreenName()
message = "%s wants to meet in the main public square tomorrow" % (friend2,)
api.PostDirectMessage(friendName, message)
api.DestroyFriendship(friendName)
Cox & McLean 2013, 97
articulation
REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html
#!/usr/bin/env python
import twitter, random
api = twitter.Api(consumer_key='xx', consumer_secret='xx', access_token_key='xx', access_token_secret='xx)
friends = api.GetFriends()
for friend in friends:
friendName = friend.GetScreenName()
friend2 = random.choice(friends).GetScreenName()
message = "%s wants to meet in the main public square tomorrow" % (friend2,)
api.PostDirectMessage(friendName, message)
api.DestroyFriendship(friendName)
Cox & McLean 2013, 97
algorithmic articulation
REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html
#!/usr/bin/env python
import twitter, random
api = twitter.Api(consumer_key='xx', consumer_secret='xx', access_token_key='xx', access_token_secret='xx)
friends = api.GetFriends()
for friend in friends:
friendName = friend.GetScreenName()
friend2 = random.choice(friends).GetScreenName()
message = "%s wants to meet in the main public square tomorrow" % (friend2,)
api.PostDirectMessage(friendName, message)
api.DestroyFriendship(friendName)
Cox & Mclean 2013, 97
a 3-dimensional relation
between researcher, field and digital architecture
[] publicness is constituted not simply by speaking, writing, arguing and
protesting but also through modification of the domain or platform
through which these practices are enacted, making both technology and
the law unstable.
Cox & McLean 2013, p.93
H O W ?
D O I N G D I G I T A L E T H N O G R A P H Y B Y F O C U S I N G O N
S T R U C T U R A L A S P E C T S A N D S O C I A L C O N T E X T S
C H A P T E R I
Coding publics and code as ethnographic artefact
 digital ethnography (?) follow the digital footprints.
Christiaan Colen https://flic.kr/p/wuu37e
Kozinets, R. V. (1998). On Netnography: Initial Reflections on Consumer Research Investigations of Cyberculture.
Miller, D., & Slater, D. (2000). The Internet: An ethnographic approach.
Marotzki, W. (2003). Online-Ethnographie  Wege und Ergebnisse zur Forschung im Kulturraum Internet.
Ard辿vol, E., & G坦mez-Cruz, E. (2013). Digital Ethnography and Media Practices.
Brunton, F., & Coleman, G. (2014). Closer to the Metal.
Coleman, G. (2017). From Internet Farming to Weapons of the Geek.
Seaver, N. (2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems.
Cousineau, L. S., Oakes, H., & Johnson, C. W. (2018). Appnography: Digital ethnography for app-based culture.
Matzner. T. (2017). Opening Black Boxes is Not Enough  Data-based Surveillance in Discipline and Punish and Today
Hine, C. (2000). Virtual Ethnography.
Christiaan Colen https://flic.kr/p/wuu37e
Kozinets, R. V. (1998). On Netnography: Initial Reflections on Consumer Research Investigations of Cyberculture.
Miller, D., & Slater, D. (2000). The Internet: An ethnographic approach.
Marotzki, W. (2003). Online-Ethnographie  Wege und Ergebnisse zur Forschung im Kulturraum Internet.
Ard辿vol, E., & G坦mez-Cruz, E. (2013). Digital Ethnography and Media Practices.
Brunton, F., & Coleman, G. (2014). Closer to the Metal.
Coleman, G. (2017). From Internet Farming to Weapons of the Geek.
Seaver, N. (2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems.
Cousineau, L. S., Oakes, H., & Johnson, C. W. (2018). Appnography: Digital ethnography for app-based culture.
Matzner. T. (2017). Opening Black Boxes is Not Enough  Data-based Surveillance in Discipline and Punish and Today
Hine, C. (2000). Virtual Ethnography.
Algorithms in culture
They may shape culture (by altering the flows of cultural material), and they may be
shaped by culture (by embodying the biases of their creators), but this relationship is like
that between a rock and the stream it is sitting in: the rock is not part of the stream, though
the stream may jostle and erode it and the rock may produce ripples and eddies in the
stream. In this view, algorithms can affect culture and culture can affect algorithms
because they are distinct.
Seaver 2017
Michael Mueller https://flic.kr/p/SYpMQs
Algorithms as culture
algorithms are not technical rocks in a cultural stream, but are rather just more water. Like
other aspects of culture, algorithms are enacted by practices which do not heed a strong
distinction between technical and non-technical concerns, but rather blend them
together.
Seaver 2017
NASA Goddard Space Flight Center https://flic.kr/p/976zug
Coding publics and code as ethnographic artefact
Understanding what were seeing.
- Christine Hine (today)
@Trump2Cash
https://github.com/maxbbraun/trump2cash
Coding publics and code as ethnographic artefact
Toyotas NYSE:TM stock price on January 5th 2017
Toyotas NYSE:TM stock price on January 5th 2017
matters
Contextuality
Coding publics and code as ethnographic artefact
Coding publics and code as ethnographic artefact
H O W ?
U S I N G D I G I T A L T O O L S A N D F R A M E W O R K S T O W O R K
W I T H I N A D I G I T A L E N V I R O N M E N T
C H A P T E R I I
1 tweet is more than 280 characters.
D A T A C O L L E C T I O N ?
U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
1 tweet is more than 280 characters.
D A T A C O L L E C T I O N ?
U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
1 tweet is more than 280 characters.
D A T A C O L L E C T I O N ?
U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
{
"created_at":"Thu Apr 06 15:24:15 +0000 2017",
"id": 850006245121695744,
"id_str": "850006245121695744",
"text": "1/ Today were sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP",
}
1 tweet is more than 280 characters.
D A T A C O L L E C T I O N ?
U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
"place":
{
"attributes":{},
"bounding_box":
{
"coordinates":
[[
[-77.119759,38.791645],
[-76.909393,38.791645],
[-76.909393,38.995548],
[-77.119759,38.995548]
]],
"type":"Polygon"
},
"country":"United States",
"country_code":"US",
"full_name":"Washington, DC",
"id":"01fbe706f872cb32",
"name":"Washington",
"place_type":"city",
"url":"http://api.twitter.com/1/geo/id/0172cb32.json"
}
D A T A C O L L E C T I O N ?
U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
We can collect the data.
D A T A C O L L E C T I O N ?
U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
We can collect the data.
But when should we and what
are the ethical implications?
C O D I N G P U B L I C S A N D C O D E A S
E T H N O G R A P H I C A R T E F A C T
A ( T E M P O R A R Y ) C O N C L U S I O N
Ethnography: Understanding algorithms as culture.
There is not only one definition of algorithms and thats fine.
Gathering data might technically work, but reconstructing contextuality might not.
Open Science: using/dev. digital methods to get and share not only findings but also the tools.
C O D I N G P U B L I C S A N D C O D E A S E T H N O G R A P H I C A R T E F A C T
THANK YOU!
Dan Verst辰ndig | Otto-von-Guericke-University of Magdeburg/Germany
@danvers | ebdw.ovgu.de
Going Public? Ethnography in Education and Social Work and Its Publics
October 31st  November 2nd, 2019
Martin-Luther-University Halle-Wittenberg, Germany
Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society,
20(3), 973989. https://doi.org/10.1177/1461444816676645
Ard辿vol, E., & G坦mez-Cruz, E. (2013). Digital Ethnography and Media Practices. In A. N. Valdivia (Eds.), The International Encyclopedia of Media Studies (p. 498518).
https://doi.org/10.1002/9781444361506.wbiems193
Bollig, S., Honig, M.-S., Neumann, S., & Seele, C. (Eds.). (2015). MultiPluriTrans in educational ethnography: Approaching the multimodality, plurality and translocality of
educational realities. Bielefeld: Transcript Verlag.
Brunton, F., & Coleman, G. (2014). Closer to the Metal. In T. Gillespie, P. J. Boczkowski, & K. A. Foot (Hrsg.), Media Technologies (S. 7798).
https://doi.org/10.7551/mitpress/9780262525374.003.0004
Coleman, G. (2017). From Internet Farming to Weapons of the Geek. Current Anthropology, 58(15), 91102. https://doi.org/10.1086/688697
Cousineau, L. S., Oakes, H., & Johnson, C. W. (2018). Appnography: Digital ethnography for app-based culture. In D. Parry, C. W. Johnson, & S. Fullagar (Eds.), Digital
Dilemmas: Transforming gender identities and power relations in everyday life (p. 95117). Palgrave Press.
Hoffmeister, A., Marguin, S., & Schendzielorz, C. (2018). Feldnotizen 2.0. ber Digitalit辰t in der ethnografischen Beobachtungspraxis. Artikel des Sonderbandes 3 der
Zeitschrift f端r digitale Geisteswissenschaften. Available under http://www.zfdg.de/sonderband/3. https://doi.org/10.17175/sb003_007
Kozinets, R. V. (1998). On Netnography: Initial Reflections on Consumer Research Investigations of Cyberculture. ACR North American Advances, NA-25. Abgerufen von
http://acrwebsite.org/volumes/8180/volumes/v25/NA-25
Marotzki, W. (2003). Online-Ethnographie  Wege und Ergebnisse zur Forschung im Kulturraum Internet. MedienP辰d. Retro: Jahrbuch Medienp辰dagogik 3, 149165.
http://dx.doi.org/10.21240/mpaed/retro/2017.07.09.X
Matzner, T. (2017). Opening Black Boxes Is Not Enough  Data-based Surveillance In Discipline and Punish And Today. Foucault Studies, 0(23), 2745.
https://doi.org/10.22439/fs.v0i0.5340
Miller, D. (2012). Das wilde Netzwerk: Ein ethnologischer Blick auf Facebook. Berlin: Suhrkamp.
Miller, D., & Slater, D. (2000). The Internet: An ethnographic approach. Oxford ; New York: Berg.
Munger, K., & Phillips, J. (2019). A Supply and Demand Framework for YouTube Politics. Available under: https://osf.io/73jys/download
Parry, D. C., Johnson, C. W., & Fullagar, S. (2019). Digital dilemmas: Transforming gender identities and power relations in everyday life. Abgerufen von
http://link.springer.com/10.1007/978-3-319-95300-7
Seaver, N. (2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Big Data & Society, 4(2), 205395171773810.
https://doi.org/10.1177/2053951717738104
R E F E R E N C E S

More Related Content

Coding publics and code as ethnographic artefact

  • 1. C O D I N G P U B L I C S A N D C O D E A S E T H N O G R A P H I C A R T E F A C T S Y 0 6 : M A K E R C U L T U R E S , C O D E A N D P U B L I C N E S S - E X P L O R I N G ( P O S T - ) D I G I T A L S P H E R E S A N D P R A C T I C E S Dan Verst辰ndig | Otto-von-Guericke-University of Magdeburg/Germany @danvers | ebdw.ovgu.de Going Public? Ethnography in Education and Social Work and Its Publics October 31st November 2nd, 2019 Martin-Luther-University Halle-Wittenberg, Germany
  • 2. W H Y ? I N O R D E R T O U N D E R S T A N D D I G I T A L D A T A A N D D I G I T A L I T Y , W E N E E D T O U N D E R S T A N D D I G I T A L C U L T U R E S .
  • 3. REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html a 2-dimensional relation between researcher and the field
  • 4. REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html #!/usr/bin/env python import twitter, random api = twitter.Api(consumer_key='xx', consumer_secret='xx', access_token_key='xx', access_token_secret='xx) friends = api.GetFriends() for friend in friends: friendName = friend.GetScreenName() friend2 = random.choice(friends).GetScreenName() message = "%s wants to meet in the main public square tomorrow" % (friend2,) api.PostDirectMessage(friendName, message) api.DestroyFriendship(friendName) Cox & McLean 2013, 97 articulation
  • 5. REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html #!/usr/bin/env python import twitter, random api = twitter.Api(consumer_key='xx', consumer_secret='xx', access_token_key='xx', access_token_secret='xx) friends = api.GetFriends() for friend in friends: friendName = friend.GetScreenName() friend2 = random.choice(friends).GetScreenName() message = "%s wants to meet in the main public square tomorrow" % (friend2,) api.PostDirectMessage(friendName, message) api.DestroyFriendship(friendName) Cox & McLean 2013, 97 algorithmic articulation
  • 6. REUTERS/Konstantin Chernichkin via http://archive.boston.com/bigpicture/2010/11/in_protest.html #!/usr/bin/env python import twitter, random api = twitter.Api(consumer_key='xx', consumer_secret='xx', access_token_key='xx', access_token_secret='xx) friends = api.GetFriends() for friend in friends: friendName = friend.GetScreenName() friend2 = random.choice(friends).GetScreenName() message = "%s wants to meet in the main public square tomorrow" % (friend2,) api.PostDirectMessage(friendName, message) api.DestroyFriendship(friendName) Cox & Mclean 2013, 97 a 3-dimensional relation between researcher, field and digital architecture
  • 7. [] publicness is constituted not simply by speaking, writing, arguing and protesting but also through modification of the domain or platform through which these practices are enacted, making both technology and the law unstable. Cox & McLean 2013, p.93
  • 8. H O W ? D O I N G D I G I T A L E T H N O G R A P H Y B Y F O C U S I N G O N S T R U C T U R A L A S P E C T S A N D S O C I A L C O N T E X T S C H A P T E R I
  • 10. digital ethnography (?) follow the digital footprints.
  • 11. Christiaan Colen https://flic.kr/p/wuu37e Kozinets, R. V. (1998). On Netnography: Initial Reflections on Consumer Research Investigations of Cyberculture. Miller, D., & Slater, D. (2000). The Internet: An ethnographic approach. Marotzki, W. (2003). Online-Ethnographie Wege und Ergebnisse zur Forschung im Kulturraum Internet. Ard辿vol, E., & G坦mez-Cruz, E. (2013). Digital Ethnography and Media Practices. Brunton, F., & Coleman, G. (2014). Closer to the Metal. Coleman, G. (2017). From Internet Farming to Weapons of the Geek. Seaver, N. (2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Cousineau, L. S., Oakes, H., & Johnson, C. W. (2018). Appnography: Digital ethnography for app-based culture. Matzner. T. (2017). Opening Black Boxes is Not Enough Data-based Surveillance in Discipline and Punish and Today Hine, C. (2000). Virtual Ethnography.
  • 12. Christiaan Colen https://flic.kr/p/wuu37e Kozinets, R. V. (1998). On Netnography: Initial Reflections on Consumer Research Investigations of Cyberculture. Miller, D., & Slater, D. (2000). The Internet: An ethnographic approach. Marotzki, W. (2003). Online-Ethnographie Wege und Ergebnisse zur Forschung im Kulturraum Internet. Ard辿vol, E., & G坦mez-Cruz, E. (2013). Digital Ethnography and Media Practices. Brunton, F., & Coleman, G. (2014). Closer to the Metal. Coleman, G. (2017). From Internet Farming to Weapons of the Geek. Seaver, N. (2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Cousineau, L. S., Oakes, H., & Johnson, C. W. (2018). Appnography: Digital ethnography for app-based culture. Matzner. T. (2017). Opening Black Boxes is Not Enough Data-based Surveillance in Discipline and Punish and Today Hine, C. (2000). Virtual Ethnography.
  • 13. Algorithms in culture They may shape culture (by altering the flows of cultural material), and they may be shaped by culture (by embodying the biases of their creators), but this relationship is like that between a rock and the stream it is sitting in: the rock is not part of the stream, though the stream may jostle and erode it and the rock may produce ripples and eddies in the stream. In this view, algorithms can affect culture and culture can affect algorithms because they are distinct. Seaver 2017 Michael Mueller https://flic.kr/p/SYpMQs
  • 14. Algorithms as culture algorithms are not technical rocks in a cultural stream, but are rather just more water. Like other aspects of culture, algorithms are enacted by practices which do not heed a strong distinction between technical and non-technical concerns, but rather blend them together. Seaver 2017 NASA Goddard Space Flight Center https://flic.kr/p/976zug
  • 16. Understanding what were seeing. - Christine Hine (today)
  • 19. Toyotas NYSE:TM stock price on January 5th 2017
  • 20. Toyotas NYSE:TM stock price on January 5th 2017 matters Contextuality
  • 23. H O W ? U S I N G D I G I T A L T O O L S A N D F R A M E W O R K S T O W O R K W I T H I N A D I G I T A L E N V I R O N M E N T C H A P T E R I I
  • 24. 1 tweet is more than 280 characters. D A T A C O L L E C T I O N ? U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
  • 25. 1 tweet is more than 280 characters. D A T A C O L L E C T I O N ? U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S
  • 26. 1 tweet is more than 280 characters. D A T A C O L L E C T I O N ? U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S { "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today were sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", }
  • 27. 1 tweet is more than 280 characters. D A T A C O L L E C T I O N ? U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S "place": { "attributes":{}, "bounding_box": { "coordinates": [[ [-77.119759,38.791645], [-76.909393,38.791645], [-76.909393,38.995548], [-77.119759,38.995548] ]], "type":"Polygon" }, "country":"United States", "country_code":"US", "full_name":"Washington, DC", "id":"01fbe706f872cb32", "name":"Washington", "place_type":"city", "url":"http://api.twitter.com/1/geo/id/0172cb32.json" }
  • 28. D A T A C O L L E C T I O N ? U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S We can collect the data.
  • 29. D A T A C O L L E C T I O N ? U S I N G E X I S T I N G T O O L S A N D F R A M E W O R K S We can collect the data. But when should we and what are the ethical implications?
  • 30. C O D I N G P U B L I C S A N D C O D E A S E T H N O G R A P H I C A R T E F A C T A ( T E M P O R A R Y ) C O N C L U S I O N
  • 31. Ethnography: Understanding algorithms as culture. There is not only one definition of algorithms and thats fine. Gathering data might technically work, but reconstructing contextuality might not. Open Science: using/dev. digital methods to get and share not only findings but also the tools.
  • 32. C O D I N G P U B L I C S A N D C O D E A S E T H N O G R A P H I C A R T E F A C T THANK YOU! Dan Verst辰ndig | Otto-von-Guericke-University of Magdeburg/Germany @danvers | ebdw.ovgu.de Going Public? Ethnography in Education and Social Work and Its Publics October 31st November 2nd, 2019 Martin-Luther-University Halle-Wittenberg, Germany
  • 33. Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3), 973989. https://doi.org/10.1177/1461444816676645 Ard辿vol, E., & G坦mez-Cruz, E. (2013). Digital Ethnography and Media Practices. In A. N. Valdivia (Eds.), The International Encyclopedia of Media Studies (p. 498518). https://doi.org/10.1002/9781444361506.wbiems193 Bollig, S., Honig, M.-S., Neumann, S., & Seele, C. (Eds.). (2015). MultiPluriTrans in educational ethnography: Approaching the multimodality, plurality and translocality of educational realities. Bielefeld: Transcript Verlag. Brunton, F., & Coleman, G. (2014). Closer to the Metal. In T. Gillespie, P. J. Boczkowski, & K. A. Foot (Hrsg.), Media Technologies (S. 7798). https://doi.org/10.7551/mitpress/9780262525374.003.0004 Coleman, G. (2017). From Internet Farming to Weapons of the Geek. Current Anthropology, 58(15), 91102. https://doi.org/10.1086/688697 Cousineau, L. S., Oakes, H., & Johnson, C. W. (2018). Appnography: Digital ethnography for app-based culture. In D. Parry, C. W. Johnson, & S. Fullagar (Eds.), Digital Dilemmas: Transforming gender identities and power relations in everyday life (p. 95117). Palgrave Press. Hoffmeister, A., Marguin, S., & Schendzielorz, C. (2018). Feldnotizen 2.0. ber Digitalit辰t in der ethnografischen Beobachtungspraxis. Artikel des Sonderbandes 3 der Zeitschrift f端r digitale Geisteswissenschaften. Available under http://www.zfdg.de/sonderband/3. https://doi.org/10.17175/sb003_007 Kozinets, R. V. (1998). On Netnography: Initial Reflections on Consumer Research Investigations of Cyberculture. ACR North American Advances, NA-25. Abgerufen von http://acrwebsite.org/volumes/8180/volumes/v25/NA-25 Marotzki, W. (2003). Online-Ethnographie Wege und Ergebnisse zur Forschung im Kulturraum Internet. MedienP辰d. Retro: Jahrbuch Medienp辰dagogik 3, 149165. http://dx.doi.org/10.21240/mpaed/retro/2017.07.09.X Matzner, T. (2017). Opening Black Boxes Is Not Enough Data-based Surveillance In Discipline and Punish And Today. Foucault Studies, 0(23), 2745. https://doi.org/10.22439/fs.v0i0.5340 Miller, D. (2012). Das wilde Netzwerk: Ein ethnologischer Blick auf Facebook. Berlin: Suhrkamp. Miller, D., & Slater, D. (2000). The Internet: An ethnographic approach. Oxford ; New York: Berg. Munger, K., & Phillips, J. (2019). A Supply and Demand Framework for YouTube Politics. Available under: https://osf.io/73jys/download Parry, D. C., Johnson, C. W., & Fullagar, S. (2019). Digital dilemmas: Transforming gender identities and power relations in everyday life. Abgerufen von http://link.springer.com/10.1007/978-3-319-95300-7 Seaver, N. (2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Big Data & Society, 4(2), 205395171773810. https://doi.org/10.1177/2053951717738104 R E F E R E N C E S