際際滷

際際滷Share a Scribd company logo
BSSIP Monitoring and Evaluation
                    Framework

                13 March 2013
Evaluation of the Bojanala Systemic Improvement Project


 Profiles            Baselines              Follow up



Investment
Decisions
                                 Progress     Impact    Sustainability   X

Teacher RBAT &
Diagnostic tests


Teacher
Standardised tests


Learner
performance



Annual OPR


                                                                             2
Results of the RBAT on Math Literacy
        100%

            90%

            80%

            70%

            60%
Score (%)




                                                                           RBAT: July 2010
            50%
                                                                           Diag 1: January 2011
            40%
                                                                           Diag 2: September 2011
            30%                                                            Diag 3: October 2012
            20%

            10%

            0%
                  ML1   ML2   ML3   ML4          ML5     ML6   ML7   ALL
                                    Teacher Identifier




                                                                                                    3
Uses of the tests

 Designing the teacher development interventions

 Adjusting teacher development interventions

 Personal target setting by teachers

 Increasing the demand for teachers among teachers, e.g. on
  mentorships

 Management awareness and decisions about teacher
  development dynamics



                                                               4
BSSIP FET TEACHER DEVELOPMENT DOSAGE: 2010/2011

                                   Q1                 Q2                 Q3                Q4
                                                                                                   YEAR
INTERVENTION DOSAGE      M1 M2 M3 TOTAL   M4   M5 M6 TOTAL   M7   M8 M9 TOTAL M10 M11 M12 TOTAL
                                                                                                  TOTAL

CONTENT DEVELOPMENT         12                 12                 12                               36

SELF-DIRECTED LEARNING                                                                             0

ONSITE MENTORING AND
SUPPORT PER TEACHER
                            2                  2                  2                                6

PROFESSIONAL LEARNING
CLUSTERS
                                                                                                   0

TOTAL HOURS PER TERM
PER TEACHER
                                    14                14                 14                        42

TOTAL HOURS PER YEAR
PER TEACHER
                                                                                                   84




                                                                                                          5
BSSIP FET TEACHER DEVELOPMENT DOSAGE: 2012


                                         Q1                     Q2                     Q3                        Q4

                                                                                                                         YEAR
INTERVENTION DOSAGE      M1   M2   M3   TOTAL   M4   M5   M6   TOTAL   M7   M8   M9   TOTAL   M10   M11   M12   TOTAL
                                                                                                                        TOTAL

CONTENT DEVELOPMENT           19         19          19         19     19              19     19                 19      76

SELF-DIRECTED LEARNING   12   12   12    36     12   12   12    36     12   12   12    36     12    12           24     132

ONSITE MENTORING AND
                         0    6    6     12     6    6          12     6    6          12      6     6     0     12      48
SUPPORT PER TEACHER


PROFESSIONAL LEARNING
                              2    2     4           2    2     4      2    2          4       2     2           4       16
CLUSTERS


TOTAL HOURS PER TERM
                                         70                     88                     88                        74     320
PER TEACHER




                                                                                                                              6
Challenges in implementing the tests

 Negative perceptions about teacher testing

 Weak standardisation  because the course content covered
   during the year varies. Only difficulty level and cognitive
   demand levels were standardised

 Vacancies and attrition (27  23 teachers )




                                                                 7
FP Numeracy
Participation rate:
2010: 36 teachers (81.8%)
2011: 24 teachers (43.2%)

Overall average:
2010: 52.9%
2011: 54.1%




8
FP Numeracy continued




9
FP Numeracy
Highest scores:
Basic operations and money
Struggle: Manipulating digits to get largest difference

Middle scores:
Counting, number concept and place value
Fractions, techniques, patterns and flow diagrams, maps and grids
Struggle:     Writing ordinal number in words, number pattern involving digital time,
             Completion of number pattern up to seventh term

Lowest scores:
Shapes and objects
Time, mass, volume and length, area and perimeter
Graphs and data interpretation
Struggle: Problems where hours needs to be converted to days
            Definitions for mass, weight and capacity
            Evaluating the information in a graph critically
            Listing outcomes for a simple trial
            Reflection of a object
            Drawing a rectangle with a specific perimeter on a grid
10          using tiling to calculate area
Evaluation of the Bojanala Systemic Improvement Project

                                                         Need for cost-
 Profiles            Baselines              Follow up   benefit analysis



Investment
Decisions
                                 Progress     Impact          Sustainability   X

Teacher RBAT &
Diagnostic tests


Teacher
Standardised tests


Learner
performance



Annual OPR


                                                                                   11

More Related Content

Choose wisely - Serious Social Investing 2013

  • 1. BSSIP Monitoring and Evaluation Framework 13 March 2013
  • 2. Evaluation of the Bojanala Systemic Improvement Project Profiles Baselines Follow up Investment Decisions Progress Impact Sustainability X Teacher RBAT & Diagnostic tests Teacher Standardised tests Learner performance Annual OPR 2
  • 3. Results of the RBAT on Math Literacy 100% 90% 80% 70% 60% Score (%) RBAT: July 2010 50% Diag 1: January 2011 40% Diag 2: September 2011 30% Diag 3: October 2012 20% 10% 0% ML1 ML2 ML3 ML4 ML5 ML6 ML7 ALL Teacher Identifier 3
  • 4. Uses of the tests Designing the teacher development interventions Adjusting teacher development interventions Personal target setting by teachers Increasing the demand for teachers among teachers, e.g. on mentorships Management awareness and decisions about teacher development dynamics 4
  • 5. BSSIP FET TEACHER DEVELOPMENT DOSAGE: 2010/2011 Q1 Q2 Q3 Q4 YEAR INTERVENTION DOSAGE M1 M2 M3 TOTAL M4 M5 M6 TOTAL M7 M8 M9 TOTAL M10 M11 M12 TOTAL TOTAL CONTENT DEVELOPMENT 12 12 12 36 SELF-DIRECTED LEARNING 0 ONSITE MENTORING AND SUPPORT PER TEACHER 2 2 2 6 PROFESSIONAL LEARNING CLUSTERS 0 TOTAL HOURS PER TERM PER TEACHER 14 14 14 42 TOTAL HOURS PER YEAR PER TEACHER 84 5
  • 6. BSSIP FET TEACHER DEVELOPMENT DOSAGE: 2012 Q1 Q2 Q3 Q4 YEAR INTERVENTION DOSAGE M1 M2 M3 TOTAL M4 M5 M6 TOTAL M7 M8 M9 TOTAL M10 M11 M12 TOTAL TOTAL CONTENT DEVELOPMENT 19 19 19 19 19 19 19 19 76 SELF-DIRECTED LEARNING 12 12 12 36 12 12 12 36 12 12 12 36 12 12 24 132 ONSITE MENTORING AND 0 6 6 12 6 6 12 6 6 12 6 6 0 12 48 SUPPORT PER TEACHER PROFESSIONAL LEARNING 2 2 4 2 2 4 2 2 4 2 2 4 16 CLUSTERS TOTAL HOURS PER TERM 70 88 88 74 320 PER TEACHER 6
  • 7. Challenges in implementing the tests Negative perceptions about teacher testing Weak standardisation because the course content covered during the year varies. Only difficulty level and cognitive demand levels were standardised Vacancies and attrition (27 23 teachers ) 7
  • 8. FP Numeracy Participation rate: 2010: 36 teachers (81.8%) 2011: 24 teachers (43.2%) Overall average: 2010: 52.9% 2011: 54.1% 8
  • 10. FP Numeracy Highest scores: Basic operations and money Struggle: Manipulating digits to get largest difference Middle scores: Counting, number concept and place value Fractions, techniques, patterns and flow diagrams, maps and grids Struggle: Writing ordinal number in words, number pattern involving digital time, Completion of number pattern up to seventh term Lowest scores: Shapes and objects Time, mass, volume and length, area and perimeter Graphs and data interpretation Struggle: Problems where hours needs to be converted to days Definitions for mass, weight and capacity Evaluating the information in a graph critically Listing outcomes for a simple trial Reflection of a object Drawing a rectangle with a specific perimeter on a grid 10 using tiling to calculate area
  • 11. Evaluation of the Bojanala Systemic Improvement Project Need for cost- Profiles Baselines Follow up benefit analysis Investment Decisions Progress Impact Sustainability X Teacher RBAT & Diagnostic tests Teacher Standardised tests Learner performance Annual OPR 11