Analyzing Evaluation Criteria : A Review of Student Evaluation Methods

Measuring learning is a crucial part of the learning experience. At the heart of every learning experience is the desire for students to develop certain skills and acquire knowledge. Student evaluation is usually conducted using different evaluation models based on the desired learning outcomes.

What are Student Evaluations?

This is an assessment of students’ performance after participating in a learning experience. They are systematic frameworks for investigating and analyzing the efficacy of a learning journey. The purpose of an evaluation is to ascertain whether learning has occurred, this also communicates the efficacy of a learning program.

To ensure that assessment and evaluation are accurate and reliable and that they lead to student learning development, teachers must use assessment and evaluation procedures that:

examine what students learn as well as how well they learn;
are based on both the knowledge and skill categories as well as the achievement chart;
are appropriate for the learning activities used, the purposes of instruction, and the needs and experiences of the students;
are fair and transparent to all students; are varied in nature, administered over time, and designed to provide opportunities for students to demonstrate the full range of their learning;
guarantee that each student receives clear improvement instructions;
support students' abilities to evaluate their own learning and establish specific goals;
are presented to students and parents clearly at the start of the course and at other suitable moments throughout the course

Why Use Student evaluations?

They indicate the need to modify, plan or redesign a course.
They highlight the need for different teaching strategies and learning materials.
They provide a short feedback timeline, which aids student performance.
They highlight students' needs.

Evaluation Models

There exist numerous models of evaluation. Some can be applied to specific learning scenarios, such as the CIRO model, and others can be adopted across learning scenarios, such as The Brinkerhoff Model. Here are a few models which can be applied in different learning scenarios.

Brienkerhoff`s Success Case Model

This model draws a comparison between the most successful and the least successful cases during a learning journey. With this method, we ascertain whether a particular method worked, how it can be improved, what failed, and how it can be improved upon.

Here are some questions that can be asked during this model:

How have you used the training?
How did you benefit from the training?
What difficulties did you encounter?
How did you determine you used the training correctly?

This model is adept at identifying what worked and what did not work. It is effective in helping instructional designers identify what needs to change and what to improve on.

Kaufman’s Model of Learning Evaluation

An advancement of Kirkpatrick's model, the Kaufman model, identified five stages of learning.

Level 1a - Input: here, a review of the resources and learning materials is conducted to determine if they were suitable and appropriate;
Level 1b - Process: this scrutinizes whether the training was delivered well;
Level 2 - Acquisition: at this level, evaluators measure whether new knowledge and skills were developed by virtue of the learning program;
Level 3 - Application: here, evaluators determine if learners applied their learning during the learning or in real life scenarios;
Level 4 - Organisational Results: how did the organization benefit from the learning experience is ascertained; and
Level 5 - Societal/Customer consequence: how other stakeholders benefited from the training is discussed here.

At the core of this model is the desire to evaluate learning resources separately from the delivery of learning.

The CIPP Model

CIPP stands for Context, Input, Process & Product. Created by Daniel Stufflebeam. By focusing on CIPP, there is a thorough evaluation of the planning, implementation, and assessment phases of any learning program. This model is research intensive as each part of the assessment is achieved through a study.

The Context Evaluation study is conducted when a new program is being planned to ensure the most effective evaluation methods are applied using the context as the compass for decision making.
The Input Evaluation study assesses the feasibility or cost-effectiveness of adopting alternative plans and ways of allocating relevant resources.
Process evaluation study: here an assessment of the program implementation is conducted.
The product evaluation study: here there is focus on the program outcomes.

Watch details here:

https://www.youtube.com/watch?v=qOcm5nT-o2U

The Learning Transfer Evaluation Model (LTEM)

Will Thalheimer devised the Learning Transfer Evaluation Model as an alternative to Kirkpatrick's model to determine the effectiveness of the applicable evaluation methods. LTEM comprises eight levels - the first six levels focus on learning and the other two demonstrate where learning becomes applicable and needs to be integrated.

Level 1 Attendance, as the first step it is established that it should not be a stand-alone evaluation because participation does not confirm that anything has been learned.
Level 2 Activity, this level involves three sub levels: attention, interest, and participation. Again this should not be a stand-alone evaluation because paying attention, showing interest and active participation are not indicators that learning occurred.
Level 3 Learner perceptions, more focus should be placed on measuring what the learner remembered, not just how the learner felt about the learning experience.
Level 4 Knowledge learners are tested to see what they recall from the learning experience. When this occurs immediately after the class it is called recitation and if it's a few days later it is called retention.
Level 5 Decision-making competence, here learners are tested with real-life scenarios or simulations, this is used to measure the learners’ decision-making ability.
Level 6 Task competence This is a repetition of the previous level but with more time in between the learning and testing to measure retention more accurately.
Level 7 Transfer at this stage the learner is monitored in real-life scenarios to see how the knowledge acquired is applied.
Level 8 Effects of transfer this level measures the impact of learning on the learners and third parties such as family members and colleagues.

This model is effective in step by step evaluation, it is very comprehensive and thorough. It takes into consideration the impact of time on measuring learning and emphasizes the different layers of learning, highlighting how they should be tested.

Best Evaluation Models for Instructional Designer.

Kirkpatrick’s model has stood the test of time and stands as a foundational model for evaluating any learning journey. Identifying the most effective evaluation model requires identifying the three W’s:

Who is being assessed?
What outcome is desired?
When is the evaluation feedback needed?

Answering these questions will help any instructional designer determine what model to apply in a specific scenario while using Kirkpatrick's model as a foundational evaluation.