ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Using Open Badges as a Certification Solution for Evaluators - AEA 2014

?Download as PPTX, PDF?
?
Dan Randall
Dan Randall

This document discusses using open badges as a potential solution for evaluator certification by the American Evaluation Association (AEA). It outlines challenges with certification, describes the Canadian Evaluation Society's credentialed evaluator designation as a working example, and proposes how an open badge system could address AEA's needs. Key benefits of open badges mentioned include transparency, flexibility, rigor, and sustainable management. The document provides examples of open badge structures and metadata to illustrate how badges could recognize different levels of evaluation knowledge, skills, and experience.

1 of 27
Download to read offline
Using Open Badges as a Certification Solution for Evaluators Randall S. Davies Daniel L. Randall Richard E. West BYU Image by photosteve101. Used under CC BY License.
The Issue of Evaluator Certification In 1999 the topic of certification was fervently debated Not everyone feels the AEA needs an evaluator certification Others feel a certification process would be valuable, and that it may become required for practitioners working with government organizations (Smith et al., 2011) AEA is no closer to having an evaluator certification than we were a decade ago. Many seem to have concluded that an evaluator certification system may be overly complicated and controversial (Morris, 2011) AEA
Challenges to Certification Credentials, Certifications, and Licenses An Evaluator Certification would evaluate an individual¡¯s Knowledge, Skills, and Experience, verifying that he or she has attained a certain level of expertise. Any solution AEA might adopt must be ¨C Transparent (Clear Expectations) ¨C Rigorous BUT Flexible and Efficient ¨C Sustainable AEA
Working Examples of an Evaluator Credentialing Designation Canadian Evaluation Society¡¯s (CES) Credentialed Evaluator (CE) designation This is a designation is not a certification of evaluation proficiency or a license to practice evaluation. It simply verifies ¡°the holder has provided evidence of the education and experience required by the CES to be a competent evaluator¡± (http://www.evaluationcanada.ca, np). AEA
CES Credentialed Evaluator Designation The CE designation is voluntary. Candidates must have at least two years of evaluation-related work experience within the last ten years, with letters of reference from clients. They must pass a test requiring a series of narratives (each 150 words or less) adequately addressing at least 70% of the specified evaluator competencies in each of five domains. AEA
CES Credentialed Evaluator Designation, Cont. Members of a credentialing board consider the application and award the designation. In addition, credentialed evaluators are required to complete ongoing training (40 hours every three years) in order to maintain their status. This application and renewal process is implemented though an online system designed to manage applications and candidate accounts, with a fee to cover administration costs. AEA
Issues & Concerns AEA Must Consider Training Concerns: ¨C Degrees vs training, Experience without a degree (capacity building). What combination of training, Skill, and Experience is needed. Assessment Concerns: ¨C Certification Exams vs Credentials, Validity of assessment. Administration Issues: ¨C Costs involved in establishing and maintaining a system AEA
Potential for an Open Badges Solution Transparency: ¨C Badges communicate specifics, Meta-data, Flexibility: ¨C Modification is relatively easy, Skill subsets can be defined, badges can be added or revised individually. Rigor: ¨C Individuals Authorized to Award Badges, Allows for levels of proficiency to be identified (basic ¨C advanced) Management Sustainability: ¨C Top down design, Automation of much of the process possible, Division of labor model AEA
What are ¡°Open Badges¡±? Images from OpenBadges.org
Merit Badges and Digital Badges Boy Scout Merit Badges Digital Badge - Acknowledge accomplishment - Display skills gained - Motivation Same Benefits as physical badges - Typically not sharable - Gamification -
Khan Academy Badges (not Open!) Retrieved from https://www.khanacademy.org/badgeson 11/30/13.
Duolingo (not Open Badges) Screenshot from the Duolingo app on an Android Tablet. 1/24/14. Encourages daily practice Completed badge Progress indicators Competition with others is optional Previously earned badge not at full strength
Open Badges Open Badges Same Affordances as Digital Badges, Plus: - Uses Open Badge Infrastructure (OBI) - Display badges via web - Metadata (Criteria and Evidence links)
Open Badge Metadata
Mozilla Backpack
Backpack Collection Multiple collections can be created. Collections can remain private or can be made public and shared.
Mozilla¡¯s Vision of Credentialing Learn and gain skills in formal and informal settings Collect and Display Badges New Opportunities
Applications Gamification: breaking education into achievable levels where progress is recognized and rewarded Duolingo, Khan Academy, many others Recognizing expertise within a community Coding communities, translation communities Marrying informal and formal learning Concordia, WGU Providing more authentic credentials, tying evidence to criteria to credential IPT Ed Tec
Strong criteria Key Principles of Badging ? A badge¡¯s value is based on the rigor of the criteria for earning it Easy submission ? Make it simple for people who deserve a badge to get it Web-based evidence ? Portfolios, links, Google documents, etc. Allows for evidence to be viewed with a badge Skilled assessments ? It is essential to trust the person making the assessment Open and shareable badge collections ? What¡¯s the point if you can¡¯t share your badges in multiple ways?
iMovie Personal Tech Technology Integration One Possibility for an AEA Badge System AEA
Early Badge System by Mozilla Mozilla¡¯s Webmaker badges plan used the ¡°constellation¡± concept in which smaller badges are earned and added together to receive a higher level badge. Badges image adapted from the Mozilla Open Badges Project. Retrieved from http://erinknight.com/post/29830945702/webmaker-badges on 11/9/12.
Google Sites IPT EdTec Badge System Personal Tech Choice 1 Choice 2 Choice 3 Student selected Internet Communication Technology Student selected Multimedia Technology Student selected Personal Technologies Additional Concepts Mobile Learn Internet Safety Copy right iMovie Lower level badges are not issued for these projects Project level badge not issued for these additional concepts Educational Technology Course Level Badge Project Level Badge Lower Level Badges
iMovie First Iteration Personal Tech Theory Skill Experience Technology Integration Certified Evaluator Level Badge 2nd Level Badge 3rd Level Badges Possible AEA Badge System
Anatomy of a CE Badge (theory, experience, skill, or green for the CE badge). Center of badge Pips act as stars: 1 = Basic 2 = Intermediate 3 = Advanced Each badge indicates what category the badge is from Outer ribbon signifies this badge is the ¡°Certified Evaluator¡± (CE) badge is reserved for images to individualize badges
Pips and the CE Badge At least 1 pip from each category 7 - 9 pips earned from 2nd level Basic Intermediate Advanced 4 - 6 pips earned from 2nd level
To Learn More¡­ Randall, D. L., Harrison, J. B., West, R.E. (2013). Giving credit where credit is due: Designing Open Badges for a technology integration course. TechTrends, 57(6), 88-95. doi:10.1007/s11528-013-0706-5 View our presentation at the Open Education Conference by clicking here.
Thank You Randall S. Davies ¨C randy.davies@byu.edu Daniel L. Randall ¨C dan.randall26@gmail.com Richard E. West ¨C rickwest@byu.edu Contact us with Questions

More Related Content

Using Open Badges as a Certification Solution for Evaluators - AEA 2014

  • 1. Using Open Badges as a Certification Solution for Evaluators Randall S. Davies Daniel L. Randall Richard E. West BYU Image by photosteve101. Used under CC BY License.
  • 2. The Issue of Evaluator Certification In 1999 the topic of certification was fervently debated Not everyone feels the AEA needs an evaluator certification Others feel a certification process would be valuable, and that it may become required for practitioners working with government organizations (Smith et al., 2011) AEA is no closer to having an evaluator certification than we were a decade ago. Many seem to have concluded that an evaluator certification system may be overly complicated and controversial (Morris, 2011) AEA
  • 3. Challenges to Certification Credentials, Certifications, and Licenses An Evaluator Certification would evaluate an individual¡¯s Knowledge, Skills, and Experience, verifying that he or she has attained a certain level of expertise. Any solution AEA might adopt must be ¨C Transparent (Clear Expectations) ¨C Rigorous BUT Flexible and Efficient ¨C Sustainable AEA
  • 4. Working Examples of an Evaluator Credentialing Designation Canadian Evaluation Society¡¯s (CES) Credentialed Evaluator (CE) designation This is a designation is not a certification of evaluation proficiency or a license to practice evaluation. It simply verifies ¡°the holder has provided evidence of the education and experience required by the CES to be a competent evaluator¡± (http://www.evaluationcanada.ca, np). AEA
  • 5. CES Credentialed Evaluator Designation The CE designation is voluntary. Candidates must have at least two years of evaluation-related work experience within the last ten years, with letters of reference from clients. They must pass a test requiring a series of narratives (each 150 words or less) adequately addressing at least 70% of the specified evaluator competencies in each of five domains. AEA
  • 6. CES Credentialed Evaluator Designation, Cont. Members of a credentialing board consider the application and award the designation. In addition, credentialed evaluators are required to complete ongoing training (40 hours every three years) in order to maintain their status. This application and renewal process is implemented though an online system designed to manage applications and candidate accounts, with a fee to cover administration costs. AEA
  • 7. Issues & Concerns AEA Must Consider Training Concerns: ¨C Degrees vs training, Experience without a degree (capacity building). What combination of training, Skill, and Experience is needed. Assessment Concerns: ¨C Certification Exams vs Credentials, Validity of assessment. Administration Issues: ¨C Costs involved in establishing and maintaining a system AEA
  • 8. Potential for an Open Badges Solution Transparency: ¨C Badges communicate specifics, Meta-data, Flexibility: ¨C Modification is relatively easy, Skill subsets can be defined, badges can be added or revised individually. Rigor: ¨C Individuals Authorized to Award Badges, Allows for levels of proficiency to be identified (basic ¨C advanced) Management Sustainability: ¨C Top down design, Automation of much of the process possible, Division of labor model AEA
  • 9. What are ¡°Open Badges¡±? Images from OpenBadges.org
  • 10. Merit Badges and Digital Badges Boy Scout Merit Badges Digital Badge - Acknowledge accomplishment - Display skills gained - Motivation Same Benefits as physical badges - Typically not sharable - Gamification -
  • 11. Khan Academy Badges (not Open!) Retrieved from https://www.khanacademy.org/badgeson 11/30/13.
  • 12. Duolingo (not Open Badges) Screenshot from the Duolingo app on an Android Tablet. 1/24/14. Encourages daily practice Completed badge Progress indicators Competition with others is optional Previously earned badge not at full strength
  • 13. Open Badges Open Badges Same Affordances as Digital Badges, Plus: - Uses Open Badge Infrastructure (OBI) - Display badges via web - Metadata (Criteria and Evidence links)
  • 16. Backpack Collection Multiple collections can be created. Collections can remain private or can be made public and shared.
  • 17. Mozilla¡¯s Vision of Credentialing Learn and gain skills in formal and informal settings Collect and Display Badges New Opportunities
  • 18. Applications Gamification: breaking education into achievable levels where progress is recognized and rewarded Duolingo, Khan Academy, many others Recognizing expertise within a community Coding communities, translation communities Marrying informal and formal learning Concordia, WGU Providing more authentic credentials, tying evidence to criteria to credential IPT Ed Tec
  • 19. Strong criteria Key Principles of Badging ? A badge¡¯s value is based on the rigor of the criteria for earning it Easy submission ? Make it simple for people who deserve a badge to get it Web-based evidence ? Portfolios, links, Google documents, etc. Allows for evidence to be viewed with a badge Skilled assessments ? It is essential to trust the person making the assessment Open and shareable badge collections ? What¡¯s the point if you can¡¯t share your badges in multiple ways?
  • 20. iMovie Personal Tech Technology Integration One Possibility for an AEA Badge System AEA
  • 21. Early Badge System by Mozilla Mozilla¡¯s Webmaker badges plan used the ¡°constellation¡± concept in which smaller badges are earned and added together to receive a higher level badge. Badges image adapted from the Mozilla Open Badges Project. Retrieved from http://erinknight.com/post/29830945702/webmaker-badges on 11/9/12.
  • 22. Google Sites IPT EdTec Badge System Personal Tech Choice 1 Choice 2 Choice 3 Student selected Internet Communication Technology Student selected Multimedia Technology Student selected Personal Technologies Additional Concepts Mobile Learn Internet Safety Copy right iMovie Lower level badges are not issued for these projects Project level badge not issued for these additional concepts Educational Technology Course Level Badge Project Level Badge Lower Level Badges
  • 23. iMovie First Iteration Personal Tech Theory Skill Experience Technology Integration Certified Evaluator Level Badge 2nd Level Badge 3rd Level Badges Possible AEA Badge System
  • 24. Anatomy of a CE Badge (theory, experience, skill, or green for the CE badge). Center of badge Pips act as stars: 1 = Basic 2 = Intermediate 3 = Advanced Each badge indicates what category the badge is from Outer ribbon signifies this badge is the ¡°Certified Evaluator¡± (CE) badge is reserved for images to individualize badges
  • 25. Pips and the CE Badge At least 1 pip from each category 7 - 9 pips earned from 2nd level Basic Intermediate Advanced 4 - 6 pips earned from 2nd level
  • 26. To Learn More¡­ Randall, D. L., Harrison, J. B., West, R.E. (2013). Giving credit where credit is due: Designing Open Badges for a technology integration course. TechTrends, 57(6), 88-95. doi:10.1007/s11528-013-0706-5 View our presentation at the Open Education Conference by clicking here.
  • 27. Thank You Randall S. Davies ¨C randy.davies@byu.edu Daniel L. Randall ¨C dan.randall26@gmail.com Richard E. West ¨C rickwest@byu.edu Contact us with Questions

Editor's Notes

  • #3: purpose of this Session is to open a dialogue in the community of evaluators to explore the possibility, challenges, and benefits of establishing a certification system for AEA evaluators based on digital open badges
  • #8: The Canadian CE designation is definitely a commendable effort that seems to be working; however, it faces several implementation challenges including training confirmation, assessment validity, experience verification, and program administration. Degrees do not always communicate the specific competencies their graduates have gained
  • #10: Rick¡¯s section starts here
  • #21: Dan¡¯s section