�ݺ�ߣ

Denis Smolin,
American University in Bosnia and Herzegovina
Sergey Butakov,
SolBridge International School of Business, Daejeon, South Korea

2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.

…
• First many higher education
institutions are striving to be
strategic rather than merely
reactive
• …
• Forth, universities must
operate more efficiently
overall.
• …


 Presentation in a glance
 We will be talking about using AI to analyze
syllabus quality as a driving force to improve
quality of a course and finally quality of a
program
 We will be talking about a priori and a
posteriori evaluations
 We will discuss system prototype that allows
us to automate the evaluation process


 Syllabus and its characteristics
 Definition of the “good” syllabus
 Syllabus quality management
 Syllabus metrics
 Syllabus consistency
 Student Outcomes as the Syllabus
Quality metrics


 A syllabus that follows acknowledged patterns (Davis,
1993; McKenney, 2001):

 A syllabus that promotes good outcomes (Zhao, 2007)

 Tailored or “negotiated syllabus” that fits course
requirements and student learning style (Clarke, 1991)


Syllabus Quality Depends on

Quality of the
included materials
Instructor Other, often
Motivation & Abilities
Student Unknown &
Motivation & Abilities Uncertain
Scheduling
factors
School
facilities

If a student has difficulties with
multiple choice tests in Programming I course

Correct the
Component Evaluate
Component
Quality Changes
Structure

Pros: Could improve the
Try to replace some tests with practical
understanding
coding assignments
Cons: Could be more expensive


If majority of students complain about the textbook

Correct the
Component Evaluate
Component
Quality Changes
Structure

Use another textbook Pros: Increased understanding
Cons: Revision is required for the
practical component of the course


Components

List of Other Traditional
Schedule Books Tests
topics & New Elements:
subject specific
elements, Web 2.0
apps, etc.


Overall Syllabus Quality ?

or

The quality of its Components ?
Both!

Traditional Approach:

 Student Surveys / Evaluations
 Student Outcomes (e.g. current and final grades)
 Evaluations from Accrediting Organizations

 Some others, mainly based on Expert Opinions


How many errors are in the
structure of the syllabus

 Calculate Syllabus consistency in the number of
It is valid
Real results are statistically
implementations
close to the expected ones

 Calculate Syllabus validity and reliability
lab facilities, supplimentary
materails, etc.

We called the system
 Calculate Syllabus Costs “Chopin”

The task is Complex– it requires an Intelligent tool


 COURSE SYLLABUS
FALL, 2011
INSTRUCTOR: JOHN SMITH

 A. DESCRIPTION
This course involves a careful examination of …
 B. ORGANIZATION publisher says that this
The
This is a lecture-lab course in which …
book is not available this
 C. COURSE OBJECTIVES
To introduce students to the … semester
 E. TEXT AND REQUIRED SUPPLIES
 Required text: Basic Technical Drawing, by Spencer & Dygdon


It takes a Syllabus (created with the special template), represents it
as a Graph, Detects Consistency Errors and calculates Costs

 Consistency report items:
 Check existence of required elements
 Check links between elements
 (for example, each test should have associated readings)
 Required timings and verifies time totals
 Distribution of course materials
 etc.


Relation type Typical Problems Possible Examples
(Rule Left Side) (Rule Right Side)
Refers to dead link if it doesn’t exist in the appropriate
section of the syllabus.
redundant link if it is never used in the syllabus
sections
maldistribution of links if One topic of the syllabus is
supported with much more (ratio
is more than threshold) links than
others.
Mapping Polysemy if The test for the topic covers more
than this topic or allows different
sequences of tests
non-optimal mapping if Bad correlation between the
names of the learning goals and
the names of topics.
More rules can be added by system user

Good ?

Enough? If it is Good it should
give the Required results


N of
results

F D C B A student grade

Grades for the very difficult (non-valid), very simple courses
(non-valid) and a course of required (valid) complexity

This is the Expert
System, described in this
presentation

“to-do” list: Expert Database of EDL editor:
goals, links System. Syllabi (EDL calculates
and tests scripts), Tests Consistency, V
Takes EDL
script as the (TDL scripts) alidity and
Professors, ass
input , results, logs, Reliability
istants, and
etc. administrators

Testing A set of data
Expert analysis
System. programs
Takes TDL as
its input TDL Test editor
Student


1. Syllabus quality can be objectively measured
if we look at it is at multi-dimensional
function based on the statistical data, which
takes into account the quality of syllabus
elements and the quality of its structure.
2. Defining a good syllabus as a “syllabus that
promotes good outcomes” we expand this
definition with “always”, i.e. we define the
quality as validity and reliability.


3. Accepting the definition a good syllabus as a
mathematical graph with individual
pathways, we state that the quality has to be
calculated for each branch of the graph.

4. Evaluation of the syllabus quality against validity
and reliability shall be done in two stages: pre-
processing to exclude uncertainty from the
syllabus and post-processing aimed at the analysis
of efficiency.

5. Practical implementation demonstrates efficiency
of the proposed approach and its usability.


7. The sound theoretical framework, as well as the
promising results of its implementation in an
intelligent system, opens new opportunities in
the syllabus/course quality management:

 Creation of Smart Syllabus Bases;
 Creation of efficient teaching materials for
target groups;

 Solution is based on data warehouse that
takes from LMS and Smart Syllabi DB

�ݺ�ߣ

APPLYING ARTIFICIAL INTELLIGENCE TO THE EDUCATIONAL DATA

More Related Content

APPLYING ARTIFICIAL INTELLIGENCE TO THE EDUCATIONAL DATA