�ݺ�ߣ

Editor's Notes

1. Welcome! Thanks for coming today. It truly exciting to speak on the topic of develpmental evaluation and many thanks to the evaluation use tig for making this possible. Today, I want to share the results of a project I”ve been working on for some time, and in doing so, challenge our collective thinking around the space of possibilty created by developmental evaluation. For this paper, i want to focus on the innovation space and on the process of innovating. The slides are already available at www.chiyanlam.com and will be available shortly under the Eval Use TIG elibrary. \n
2. Let me frame with presentation with a quote by Albert Einstein.\n
3. in 1994, Patton made the observation that some clients resisted typical formative/summative evaluation approaches because of the work that they do. They themselves don’t see a point in freezing a program in time in order to have it assessed and evaluated. He described an approach where he worked collaboratively, as part of the team, to help render evaluative data that would help these program staff adapt and evolve their programs. So, the process of engaging in “developmental evaluation” becomes paramount.\n
4. Fast forward to 2011, ... These are concepts that I will only touch on briefly. \n
5. Developmental evaluation is positioned to be a response to evaluators who work in complex space. \n\nFirst described by Patton in 1994, and futher elaborated in 2011, Developmental Evaluation proposes a collaborative and participatory approach to involving the evaluator in the development process. \n\n\n
6. \n\n
7. Developmental evaluation is positioned to be a response to evaluators who work in complex space. \n\nFirst described by Patton in 1994, and futher elaborated in 2011, Developmental Evaluation proposes a collaborative and participatory approach to involving the evaluator in the development process. \n\n\n
8. So what underlies DE is a commitment to reality testing. Patton positions it as one of the approaches one could taken within a utilization-focused framework: formative/summative/developmental. \n
9. You only need to conferences like this one to hear the buzz and curiosity over developmental evaluation. In february, a webinar was offered by UNICEF, and over 500+... \n
10. If we move beyond the excitement and buzz around DE, we see that DE is still very new. There is not a lot of empirical or practical basis to the arguments. If we are serious about the utilization about DE, \n
11. So, in the remaining time, I want us to dig deep into a case of developmental evaluation. \n
12. \n
13. \n
14. The observation that Patton made is very acute back in 1994 --- SI don’t rest, programs don’t stand still, addressing social problems means aiming a moving target that shifts as society changes...\n
15. \n
16. Let’s put it in a learning context. Simple would be teaching you CPR. I”ll keep the steps simple, rehearse it many times, so that you can do it when needed. \nComplicated would be teaching teacher candidates how to plan a lesson while taken into consideration curriculum expectations, learning objectives, isntructional methods/strategies, and assessment methods\nComplex - preparing TC to become a professional. We have many diff. parts (prac, foci, prof classes), think, behave, and participate like a contributing member of the profession. \n
17. \n
18. \n
19. \n
20. \n
21. \n
22. \n
23. \n
24. Document analysis of 10 design meetings over a 10-month period\n reveals the developmental “footprint”\n tracks and identifies specific developmental concerns being unpacked at a certain point in time of the project\n Interviews with the core design team members (2 instructors, 1 lead TA)\n illuminates which aspects of the DE was found helpful by the program designers\n
25. \n
26. \n
27. \n
28. The instructors of the case were responsible for teaching teacher candidates enrolled in a teacher education program classroom assessment. \n\nIn the field of teacher education, particularly in classroom assessment, the field is experiencing a very tricky situation where teachers are not assessing in the ways that we know helps with student learning. Much of what teachers do currently focuses on traditional notions of testing. At the level of teacher education, the problem is felt more acutely, because teacher candidates are not necessarily exhibiting the kinds of practice we would like to see from them. At my institution, we have two instructors responsible for delivering a module in classroom assessment in 7 hours total. In brief, there are many constraints that we’ve to work around, many of which we have little control over.\n\nWhat we do have control over is how we deliver that instruction. After a survey of different options, we were interested in integrating social media into teacher education as a way of building a community of learners. Our thinking was that assessment learning requires learners to actively engage with peers and to challenge their own experiences and conceptions of assessment.\n\nThat was the vision that guided our work. \n
29. So those were the beginning conditions. let me fast forward and describe for you how far we got in the evaluation. Then we’ll look at how the innovation came to be.\n\n
30. \n
31. \n
32. \n
33. The Development was marked by several key representative episodes:\n1 - creating a learning environment\n2 - use of AI; to help explicate values, so to gain clarity into the QUALITY of the program. \n3 - an example of how DE promotes collaboration\n4 - the use of DE findings to engage clients in sense-making in order to formulate next steps. \n
34. When I look back at the data, uncertainty was evident throughout the evaluation. the team was uncertain about...\n
35. ... typically data in an evaluation are made at the program-level, descirbing quality... \n
36. So let’s return now and unpack the case. \n
37. \n
38. \n
39. \n
40. \n
41. \n
42. \n
43. \n
44. \n
45. \n
46. \n