ESRC Evaluation strategy
1 like677 views
The ESRC evaluation strategy focuses on assessing the quality and impact of its research investments across various domains, including training, infrastructure, and international collaboration. It emphasizes accountability, effectiveness, and generating lessons for future activities, with evaluations involving both internal and external reviews. The strategy also includes ongoing assessments of the economic impact of research, aiming to quantify ESRC's contributions to policy and practice.
ESRC Evaluation strategy
- 1. ESRC Evaluation Strategy Mr Luke Moody, Deputy Head of Evaluation, Corporate Strategy and Analysis
- 2. Content ESRC Evaluation Committee ESRC Evaluation Strategy Project Evaluation Large Investment and Policy Evaluation International Benchmarking Impact Evaluation
- 3. ESRC Evaluation Committee Oversees evaluation of quality and impact of all ESRC investments Responsible for advising Council on the successful achievement of its corporate strategy Operates independently of the Policy Committees, reports directly to Council
- 4. Evaluation Strategy cuts across all Committee and Network remits Impact through World Class Research: Evaluations of the academic quality and impact of ESRC research and UK Social Sciences more generally to advise Council on the impact and quality of its research portfolio, and to advise the Research Committee on further investments Impact through Skilled People: Evaluations of ESRC funding schemes and training investments to advise Council and the Training and Skills Committee on the impact and further development of research training initiatives Impact through Infrastructure: Evaluations of ESRC funding schemes and resource investments to advise Council and the Methods and Infrastructure Committee on the impact and further development of research methods and infrastructure initiatives Impact through International Leadership: Evaluation of internationally focused ESRC funding schemes and investments to advise Council and the Policy Committees on the impact and further development of initiatives that promote international collaboration Impact through Partnerships: Evaluations of the impact of ESRC partnerships and investments on policy and practice to advise Council and the Policy Committees on the practical impact of its partnership building, and on ways to increase this contribution
- 5. Why evaluate? The main purposes of evaluation are to: provide an assessment of accountability, i.e. whether public funds were spent as agreed assess whether a project has been conducted effectively, whether it has met its objectives and to make an assessment of the quality and impact of the research provide award holders with some feedback about the management, quality and rigour of the research, and to provide comments on uses or potential uses of the research Learn lessons to inform ongoing and future activities
- 6. Project Evaluation All ESRC grant holders must provide: End of Award Report (3 months after grant end). Impact Report (12 months after grant end). Subject to external peer review and grade. Cumulative results reported annually to Council.
- 7. Investment and Policy Evaluation Annual programme of large investment, scheme and policy evaluations. Consultation with Policy Committees. Undertaken by external evaluators. Reports presented to Council and relevant Committees, alongside synthesis of key findings and recommendations Mechanism in development for tracking recommendations
- 8. International Benchmarking Assess comparative performance of UK Social Science disciplines. In partnership with Learned Societies. Review series aims to: - highlight the standing and contribution of UK disciplines - identify ways of enhancing performance and capacity - promote future research agendas
- 9. Impact Evaluation - Aims To identify and analyse evidence of research impact on policy and practice. To understand how impact is generated, and help the ESRC to improve its performance in this area. To develop impact evaluation methods.
- 10. Impact Evaluation Critical Issues What are the problems? Timing how long to wait? Attribution what role has research played in change? Additionality what is ESRCs contribution? Different types of Impact: Instrumental Conceptual Capacity Building Looking for demonstrable impact on policy and practice
- 11. Impact Evaluation Case Studies Experimental Policy and Practice Case Studies 2005 onwards Aims: to trial methods - what works produce evidence demonstrating impact improve understanding - how does impact occur Testing: wide range of methods wide range of contractors Key Requirements: conceptual framework understanding the how as well as the what
- 12. Drivers of Impact Established relationships and networks with user communities Involving users at all stages of the research Well-planned user-engagement and knowledge exchange strategies Portfolios of research activity that build reputations with research users Good infrastructure and management support Where appropriate, the involvement of intermediaries and knowledge brokers as translators, amplifiers, network providers
- 13. Economic Impact - Pilot Pilot Study: Some success in valuing outputs, but quantifying the wider impact of research was more challenging Two main barriers: Disentangling multiple contributors to policy development General lack of evidence on economic impact of government policies Access to robust evaluation data on the impact of individual policies is needed in order to quantify ESRCs contribution to those policies
- 14. Economic Impact Tracking Back Tracking Back Studies working backwards from policy or practice initiatives that have been subject to evaluation Assessing ESRC research contributions to the initiative Estimating economic value of ESRCs impact, using national evaluation data as a benchmark EMA and Pathways to Work
- 15. Developing Impact Evaluation Conceptual Impact Child Poverty Policy Impact of Infrastructure Economic impact infrastructure investment: the Economic and Social Data Service (ESDS) Impact of Skilled People People-flow impacts in the Welsh Government
- 16. Further Information Taking Stock and Branching Out: http://www.esrc.ac.uk/impacts-and- findings/impact-assessment/developing-impact- evaluation.aspx Policy and Practice Case Studies: http://www.esrc.ac.uk/impacts-and- findings/impact-assessment/policy-practice- impacts.aspx Economic Impact Evaluation: http://www.esrc.ac.uk/impacts-and-
- 17. ESRCs Contacts Speaker: Mr Luke Moody, Deputy Head of Evaluation, Corporate Strategy and Analysis luke.moody@esrc.ac.uk ESRC website www.esrc.ac.uk
Editor's Notes
- #4: The Evaluation Committee is chaired by a Council member. Other members are appointed by Council for two years and are drawn from a range of academic and non-academic backgrounds.油Prof Ann Buchanan, University of Oxford (Chair) Prof Paul Anand, Economics, Open University Prof Tara Fenwick, Stirling Institute of Education, University of Stirling Prof Brian Francis, Mathematics and Statistics, Lancaster University Steven Marwick, Evaluation Support Scotland Jeremy Mayhew, Public User Member Prof Paul Milbourne,油Cardiff University Dr Paul油Nightingale, University of Sussex Jeremy Peat, BBC National Trustee for Scotland Prof Ken Starkey, University of Nottingham Prof Sandra Walklate, University of Liverpool Prof Paul Whiteley, University of Essex Penny Young, National Centre for Social Research Vicki Crossley, ESRC (Secretary)
- #5: Five distinct evaluation areas that cover the full remit of ESRC activity
- #9: Six reviews have been undertaken now: Social Anthropology, Politics and International Studies, Economics, Sociology, Psychology and Human Geography. The Human Geography review is due to be published early in 2013. A forthcoming review of AIM will look more broadly at the Management and Business Studies discipline.
- #13: These outcomes have been fed back into the management of large ESRC research investments.
- #14: Pilot study in 2008; used evidence from two Research Centres: Centre for Economic Performance (CEP) and Centre on Skills, Knowledge and Organisational Performance (SKOPE).
- #16: Focusing on conceptual impacts- how social science changes ideas and the general debate, thinking and culture of a specific issue. A further development was focused on assessing the economic impact of infrastructure resources (ESDS). This focused on the value and use of the datasets held by what is now called the UK Data Service.Looking how social science contributes to the training of skilled individuals, assessing the impact that they make on society. Currently working with the Government Economic Service and Government Social Research. Government Statistical Service assessment will be published service.