Choosing and designing examination
Examination is a central part of a course and is one of the factors that most strongly affects how the students approach the learning in the course. The examination can, correctly designed, affect both what and how the students learn. The following advice, reflection questions and recommendations are developed to support you as an examiner in the process of thinking about examinations.
Process for selecting and designing examination
An engineer, teacher or architect who graduates from KTH needs to have demonstrated their skills and abilities in many different ways (orally, in writing, in groups and individually, in controlled and authentic environments). One of the first things you need to consider when designing an examination is what the students should be able to do after completing the course and how they are expected to show it, what the central course content is and how you can assess that the students meet the intended learning outcomes.
Steps to define what should be examined and how the examination can be designed:
- Define what the students are supposed to be able to do in order to pass the course by reflecting on what is described in the intended learning outcomes.
- Clarify what it means to be able to show the knowledge and skills described in the intended learning outcomes - that is, specify what the meaning of the intended learning outcomes are in your course.
- Define what is central to the course e.g. concepts that students MUST know and understand.
- Identify key perspectives and common misunderstandings.
- Think about concrete ways that students can demonstrate the knowledge and competencies required to pass the course. Example: "Being able to reflect on key principles in area Y" is shown by first solving a given task and then explaining the solution's limitation in relation to factor X.
- Formulate tasks
- Apply appropriate tools to the examination
To consider when choosing examination form
Choose the right form to assess the intended learning outcomes
Different intended learning outcomes can be examined successfully with different types of examination tasks / forms. There is no solution that fits all. Larger, more open questions that can be tackled in different ways provide an increased opportunity for the student to show different skills and knowledge at different levels, while smaller tasks can sometimes be automatically assessed and show procedural skills and factual knowledge. More open questions provide more opportunities for discussion in e.g. oral examination.
Keep in mind that different forms of examination can be combined to assess all the intended learning outcomes of a course.
Evaluate knowledge/abilities, not ability to seek information
The examination should be designed in such a way that it evaluates the students' knowledge and not their ability to e.g. seek information (if that is not the purpose). This is especially important when the examination is digital, since it can enable students to use various aids and resources.
Continuous examination
Examining continuously means that students are given the opportunity to show their knowledge several times spread over the course. The assessments that are carried out are recommended to be varied to give all students an equal opportunity to show their knowledge as different students perform differently in, for example, text, orally or under time pressure.
Continuous examination directs students to study continuously, which is desirable as it improves the student's work situation and makes it easier to better consolidate knowledge in memory. In order to motivate the student for the continuous examination, each assignment can be a grade for the course or they can be weighed together for the final grade, or alternatively give some form of bonus before the assessment of the final assignment.
Constructive Alignment
Constructive alignment is a model for course design that ensures intended learning outcomes, teaching activities, and assessments are carefully aligned to support student learning. The idea is that students should be given opportunities to practise and develop precisely the skills and knowledge that they will later be assessed on. When these elements are aligned, the likelihood increases that students will achieve the intended learning outcomes.
Read more about the foundations of constructive alignment (johnbiggs.com) .
The Backwash Effect
The backwash effect is a term used to describe how assessment influences how and what students choose to learn. For example, students are likely to prioritise the tasks and content areas they know will be assessed, which must be taken into account when designing assessments. If the assessment is well aligned with the course’s learning outcomes and teaching activities, as in constructive alignment, the effect is positive and supports learning. Students will then focus their efforts on what is truly important in the course. However, if there is a gap between the learning outcomes and the assessment, the backwash effect risks leading to surface learning, where students mainly focus on what earns marks on the test rather than developing deep understanding.
Factors that can be varied in examinations
- Available time: how long you let the students solve the task.
- Number of versions of each task.
- Individual task or group task
- Visual continuous monitoring or following how the task is solved through progress meetings, oral presentation when completing the task or no monitoring at all.
- Formal requirements for the solution shown, for example letting the students combine computer-written text with pictures of sketched solutions and calculations.
- Final assessment or continuous assessment and feedback, peer feedback.
Choose examination form
At each link, you will find specific information about different ways to carry out the examination form.
In the examples of examination solutions you find examples of complete examination solutions in courses at KTH. There you can find inspiration for different ways to combine examination forms.
