ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo
Exploring the What, Why, & How of
Social Learning Analytics
OR
Annotation-Centric Assessment of
Blogging in Higher Education
Laura Gogia - @Googleguacamole
Academic Learning Transformation Lab
Virginia Commonwealth University, Richmond VA
ºÝºÝߣs are available on ºÝºÝߣShare.net
http://www.slideshare.net/LauraGogi
a/
Hello. My Name Is...
Do you teach online? Blog and tweet with student-
participants? In public? To what end?
How do you know if you are achieving your goals?
VCU is exploring the intersection of
connected learning & open education
Photo Credit: http://graduate.admissions.vcu.edu/why/
Rampages.us
(VCU Community)
Personal Blog Sites
Emerging-Evolving Course Experience:
? Public ¨C Open Course Website
? Public and Aggregated Student Blogging
? Public Discourse
By introducing students to this sort of loosely
structured, flexible, open digital learning experience,
we hope to promote certain learning dispositions,
including connectivity.
Connectivity
Creating, recognizing, understanding, and acting
on connections made across content, disciplines,
living spheres, people, space and time.
Interdisciplinary Learning ¨C Self-Reflection ¨C Transferability ¨C
Social Learning ¨C Intersection of formal and informal ¨C Holistic
That¡¯s great.
But how do we assess connectivity?
We are thinking hard about how to build
meaningful assessments in our course designs.
(1) Documenting connectivity.
(2) Advancing the learning.
(3) Meeting 21st century goals for assessment.
Social Learning Analytics: A subset of learning
analytics meant to capture, organize, and
demonstrate the inherently social, open, and
connective aspects of networked participatory
learning.
--Ferguson & Buckingham Shum (2011)
Are there ways to take advantage of the
uniquely digital aspects of blogging and
tweeting to assess student blogging and
tweeting in these environments?
Annotations
Symbols & phrases that are distinct from but included within
the communication, meant to demonstrate communicative
intent
Hyperlinks ¨C Embedded Images ¨C Mentions ¨C Hashtags
Initial argument for annotation-centric
assessment:
? While specific to digital, they point to the art of
crafting communication.
? Annotation is an act that lends itself to strategic
reflection (¡°Why did you annotate that?¡±)
? Documentable, extractable, quantifiable.
Strands of Inquiry
1. How are student-participants using annotations in
course-related blogging and tweeting?
1. How can this information be used (organized and
visualized) to inform meaningful assessment?
COURSES
PARTICIPANTS
DATA SOURCES
DATA TYPES
4 ONLINE COURSES
Undergraduate & Graduate
Multidisciplinary & Gen Ed
6-26 Students
280 PARTICIPANTS
Instructors & Assistants (n = 10)
Students (n = 60)
Open Participants (n = 12)
Other Participants (n = 200)
1618 Posts (500 Sampled)
5000 Tweets
POSTS
Hyperlinks (n = 800)
Embedded Images (n = 400)
TWEETS
Hyperlinks (n = 430)
Mentions (n = 3000)
Hashtags (n = 130)
ANALYSES
Descriptive Statistics
Content Analysis
Social Network Analysis
Two Things Happened.
1. Classification systems for describing how
students annotate.
1. Dashboards for documenting and assessing
student performance.
An Example
CC was a graduate level elective which aimed to introduce its
students to community engaged research.
On average, students blogged several times a week in three
formats: (1) digital makes; (2) reflective posts; (3) research
proposal project.
They also engaged in weekly, structured Twitter chats.
Enrollment: 10 students. 12 open participants.
Classification Systems
? Types and sources of
hyperlinked and
embedded materials
? Purpose or impact of the
annotation in the context
of the post
But why did CC students hyperlink?
Types and sources of materials that CC students
used were appropriate for student level and the
course content.
1. To link to supporting documents.
? Traditional citing and referencing.
? Defining, describing, and providing examples in
ways not supported by formal writing styles.
? ¡°This could be filled in, Mad Libs style.¡±
? ¡°...verbal or nonverbal communication...¡±
2. To link to previous work or
experience.
? ¡°As I discussed in a previous post...¡±
? ¡°As I reflect on my proposed research
questions...¡±
3. To provide course context.
? ¡°I am writing this because I am taking a course...¡±
But what about embedded images
and videos?
The pedagogical value of
embedded images and videos
varied, but in a way that can be
defined and organized into a
spectrum.
BASIC LEVEL (IMAGES):
? Serves no obvious purpose other than
contributing to an aesthetic.
INTERMEDIATE LEVEL (IMAGES):
? Provides additional information
? Makes an otherwise unstated them explicit.
? Inspires deeper questions.
ADVANCED LEVEL (IMAGES & VIDEOS):
? Further the narrative (e.g. a table, chart, or
infographic that the student refers to or explains in
the narrative)
? Demonstrate a personal connection to the subject
(e.g. a photograph, graphic, or video the student
made themselves and explains in the narrative)
How does can typologies like
these inform student
assessment?
Rubrics ¨C Peer Assessment ¨C Self Assessment
Annotation-Centric Assessment of Blogging in Higher Education
Annotation-Centric Assessment of Blogging in Higher Education
Annotation-Centric Assessment of Blogging in Higher Education

More Related Content

Annotation-Centric Assessment of Blogging in Higher Education

  • 1. Exploring the What, Why, & How of Social Learning Analytics OR Annotation-Centric Assessment of Blogging in Higher Education Laura Gogia - @Googleguacamole Academic Learning Transformation Lab Virginia Commonwealth University, Richmond VA
  • 2. ºÝºÝߣs are available on ºÝºÝߣShare.net http://www.slideshare.net/LauraGogi a/
  • 3. Hello. My Name Is... Do you teach online? Blog and tweet with student- participants? In public? To what end? How do you know if you are achieving your goals?
  • 4. VCU is exploring the intersection of connected learning & open education Photo Credit: http://graduate.admissions.vcu.edu/why/
  • 6. Emerging-Evolving Course Experience: ? Public ¨C Open Course Website ? Public and Aggregated Student Blogging ? Public Discourse
  • 7. By introducing students to this sort of loosely structured, flexible, open digital learning experience, we hope to promote certain learning dispositions, including connectivity.
  • 8. Connectivity Creating, recognizing, understanding, and acting on connections made across content, disciplines, living spheres, people, space and time. Interdisciplinary Learning ¨C Self-Reflection ¨C Transferability ¨C Social Learning ¨C Intersection of formal and informal ¨C Holistic
  • 9. That¡¯s great. But how do we assess connectivity?
  • 10. We are thinking hard about how to build meaningful assessments in our course designs. (1) Documenting connectivity. (2) Advancing the learning. (3) Meeting 21st century goals for assessment.
  • 11. Social Learning Analytics: A subset of learning analytics meant to capture, organize, and demonstrate the inherently social, open, and connective aspects of networked participatory learning. --Ferguson & Buckingham Shum (2011)
  • 12. Are there ways to take advantage of the uniquely digital aspects of blogging and tweeting to assess student blogging and tweeting in these environments?
  • 13. Annotations Symbols & phrases that are distinct from but included within the communication, meant to demonstrate communicative intent Hyperlinks ¨C Embedded Images ¨C Mentions ¨C Hashtags
  • 14. Initial argument for annotation-centric assessment: ? While specific to digital, they point to the art of crafting communication. ? Annotation is an act that lends itself to strategic reflection (¡°Why did you annotate that?¡±) ? Documentable, extractable, quantifiable.
  • 15. Strands of Inquiry 1. How are student-participants using annotations in course-related blogging and tweeting? 1. How can this information be used (organized and visualized) to inform meaningful assessment?
  • 16. COURSES PARTICIPANTS DATA SOURCES DATA TYPES 4 ONLINE COURSES Undergraduate & Graduate Multidisciplinary & Gen Ed 6-26 Students 280 PARTICIPANTS Instructors & Assistants (n = 10) Students (n = 60) Open Participants (n = 12) Other Participants (n = 200) 1618 Posts (500 Sampled) 5000 Tweets POSTS Hyperlinks (n = 800) Embedded Images (n = 400) TWEETS Hyperlinks (n = 430) Mentions (n = 3000) Hashtags (n = 130) ANALYSES Descriptive Statistics Content Analysis Social Network Analysis
  • 17. Two Things Happened. 1. Classification systems for describing how students annotate. 1. Dashboards for documenting and assessing student performance.
  • 18. An Example CC was a graduate level elective which aimed to introduce its students to community engaged research. On average, students blogged several times a week in three formats: (1) digital makes; (2) reflective posts; (3) research proposal project. They also engaged in weekly, structured Twitter chats. Enrollment: 10 students. 12 open participants.
  • 19. Classification Systems ? Types and sources of hyperlinked and embedded materials ? Purpose or impact of the annotation in the context of the post
  • 20. But why did CC students hyperlink? Types and sources of materials that CC students used were appropriate for student level and the course content.
  • 21. 1. To link to supporting documents. ? Traditional citing and referencing. ? Defining, describing, and providing examples in ways not supported by formal writing styles. ? ¡°This could be filled in, Mad Libs style.¡± ? ¡°...verbal or nonverbal communication...¡±
  • 22. 2. To link to previous work or experience. ? ¡°As I discussed in a previous post...¡± ? ¡°As I reflect on my proposed research questions...¡±
  • 23. 3. To provide course context. ? ¡°I am writing this because I am taking a course...¡±
  • 24. But what about embedded images and videos?
  • 25. The pedagogical value of embedded images and videos varied, but in a way that can be defined and organized into a spectrum.
  • 26. BASIC LEVEL (IMAGES): ? Serves no obvious purpose other than contributing to an aesthetic.
  • 27. INTERMEDIATE LEVEL (IMAGES): ? Provides additional information ? Makes an otherwise unstated them explicit. ? Inspires deeper questions.
  • 28. ADVANCED LEVEL (IMAGES & VIDEOS): ? Further the narrative (e.g. a table, chart, or infographic that the student refers to or explains in the narrative) ? Demonstrate a personal connection to the subject (e.g. a photograph, graphic, or video the student made themselves and explains in the narrative)
  • 29. How does can typologies like these inform student assessment? Rubrics ¨C Peer Assessment ¨C Self Assessment

Editor's Notes

  • #5: Under the direction of Gardner Campbell, the Vice Provost of Learning Innovation and Student Success, my home university, VCU is trying to take learning through public, digital scholarship to a very large scale.
  • #6: Our goal is to build a virtual city-campus, not just for distance learners, but for everyone. Students, faculty, and staff have their personal blog spaces that can be networked at a local level for formal and informal learning. Because these are public spaces, they integrate smoothly with the global network.
  • #7: And a specific kind of course experience seems to be emerging from this experiment ¨C Course materials, activities, and communications are found on a public course website Students complete at least some of their assignments through the act of blogging on their personal websites ¨C these are aggregated by an RSS feed for the course website Certain level of public discourse which counts towards a participation grade, be that commenting on their colleagues¡¯ blogs, Twitter, Discussion Forums,
  • #8: Our reasons for promoting these sorts of experiences are multifold, but we are hoping to promote certain learning dispositions, one of which is connectivity.
  • #9: One of the main learning dispositions is the concept of connectivity ¨C the ability to recognize, understand, and act strategically on connections made across content, people, space, and time.
  • #11: Meaningful assessments are those that Document what you are hoping to achieve Advance the learning of the students by providing actionable feedback Meet goals of 21st century assessment ¨C scalability, feasibility, peer/self assessment
  • #12: Social learning analytics may offer a solution. Ferguson and Buckingham Shum (2011) propose a shift in emphasis from the highly technical, individualized, and algorithmically motivated work of learning analytics to what they define as social learning analytics, a subset of learning analytics meant to capture, organize, and demonstrate the inherently social, open, and connective aspects of networked participatory learning. The authors identify five research methodologies that show promise in advancing digital age learning: content, disposition, context, network, and discourse analytics.
  • #14: My study focuses on the annotations of blogging and tweeting. These are the symbols and phrases that are distinct from but included within the communication, meant to demonstrate communicative intent. Specifically, I focused on hyperlinks, embedded images, mentions, and hashtags
  • #15: My argument for
  • #20: Three types of information are consistently available from clicking on hyperlinks in student works: the types and sources of the materials, and what purpose it serves. To save time I won¡¯t review the types and sources of materials that CC students used ¨C however they were appropriate for student level and the course content.
  • #22: Students hyperlinked to supporting documents. Oftentimes this took the form of traditional citing or referencing, or in the undergraduate courses, it was an ¡°Additional Resources¡± list at the end of the post. However, sometimes, CC students used hyperlinks to define, describe, or provide examples of something in the narrative in ways that wouldn¡¯t be possible on paper. In the two examples listed, students hyperlinked to provide explanations of what they mean by Mad Lib or nonverbal, rather than explaining it in the primary narrative. If they had stopped to explain it, it would have cluttered up the primary narrative.
  • #23: Students also used hyperlinks to link to previous work or experience. They used them to piece together their personal narrative.
  • #24: Students also hyperlinked to the course website, to provide context for what they were writing. This is interesting to me because it speaks to an awareness of audience beyond people who are in the class.
  • #31: But you can also make a dashboard, like this. 1. Students must complete the work, as demonstrated by the number of posts completed. 2. The number hyperlink-embeds per post provides information about quantity. Content type (image- vs. text-based materials) addresses the most basic characteristic of the connections being made. The presence of broken or misdirected links and aesthetic images offer measures of quality control for hyperlinks and embedded images, respectively. While the CC cohort included a high number of exemplary students, there was still some stratification in performance. At the time of sampling, students should have completed eight or nine posts; the variation occurred because the sampling frame did not precisely match the assignment completion schedule. However, students with only seven posts were falling behind. Students are not necessarily ordered with each category. Exemplary student work included hyperlinked or embedded materials frequently, averaging seven to nine hyperlinks per post. In general, these students used fewer aesthetic images and had few, if any, broken links. Average student work included just as many hyperlinks and embedded images as exemplary students, but did not have the same level of quality. Two of the three students in the final grouping had fallen behind on completing assignments. Furthermore, they had neither the frequency nor quality of hyperlinking or embedding, suggesting that intervention or, at least, further review, is required. ?
  • #32: This figure shows a three-post sample work from three exemplars: CC-S4 (exemplary), CC-S1 (average), and CC-S10 (needs improvement). CC-S4 tends to incorporate hyperlinks that have a variety of purposes in each post. For example, exemplary work might be a post that uses hyperlinks to support a brief description of course or personal context, followed by hyperlinks or embedded images to support their argument (description, citations, and illustration), followed by a hyperlinked list of references (references). An average post may have less links, some of which may be broken, concentrated in a list of references. Posts that need significant improvements may only include several hyperlinked references and embedded clip art that punctuate section headings but do little to further the narrative