Evaluate knowledge of basic facts
This is an example of a specific examination conducted at KTH. Basic knowledge and mechanical skills can be one of the most difficult parts of a course to examine without a situation where the student is monitored. It is easy to search for answers in literature and on the internet and the students' answers can be identical without being plagiarized. This approach provides an example of how basic factual knowledge can be examined as a part of the complete examination.
Basic knowledge and mechanical skills
A common effective way to examine factual knowledge and procedural skills is with a proctored campus exam. However, this example of an examination approach uses an alternative method to better support the student's long-term learning. In the course, basic factual knowledge is examined through an unsupervised quiz with random questions for the students. A difficulty with the unsupervised examination is that the questions about basic knowledge are easy to search for answers to with the help of Google, for example, and it is reasonable that students answer exactly the same, which makes it hard to know if it is the student or someone else who has the knowledge to answer the questions, if they are answered at an unproctored occasion. An alternative way of monitoring to deal with this is to approach the way students view the knowledge in the course. The student who really understands that the knowledge is required to cope with the future professional role is not interested in cheating. The student understands that it only means a loss to cheat as cheating will make future studies, work and development impossible.
A quiz as part of the examination
In the course that this example comes from, the examiner and teachers work actively with the students' view of knowledge. This develops students' way of studying through an increased understanding that the knowledge from the course is needed for future professional practice. The examination of factual knowledge and procedural skills is carried out in a quiz where the student both participates and creates and answers the quiz. 10 questions are randomized based on hundreds and students receive an unlimited number of solution attempts. The quiz is compulsory and students must have passed the quiz by the end of the course to get a passing grade.
Creating the quiz
Throughout the course, it is emphasized to the students that the basic factual knowledge is needed for them to manage their future studies. Examples are given of why and how the knowledge will be needed. The message is repeated in different ways (during lectures, in writing in the quiz, etc.) for the information to get through to the students properly.
Create questions for the quiz with the help of the students
The students are given the task of creating questions. The number of questions they have to create depends on how many students participate in the course. For each question, students must state:
- The quiz question. To cover the entire course content, the teacher distributes the source material among the students.
- Three to four answer options.
- Who wrote the question. As each question states who created the question, the students are motivated to do a better job of formulating questions.
Students submit their questions in a compulsory assignment in Canvas, and the teacher cuts and pastes them to a question bank in Canvas. The work takes about 1 minute per question, expect about 2 hours to create a good quiz.
Conduct the quiz
The final quiz pulls questions from the question bank randomly. The quiz can be open throughout the entire course or have deadlines to support the student's continuous learning. If the quiz has deadlines, it can be opened up at the end of the course for a day to help any lagging students.
Peer review to deal with incorrect questions
Peer review can be used to effectively assure the quality of the questions and give students more opportunities to process the material. Add the question to the question bank after it has been reviewed.
Start using the peer review function in Canvas
Arguments for quiz created by students
The student benefits from actively participating in the design of the quiz questions. Deeper knowledge of a topic is required when formulating a good question, with answers and plausible incorrect answers, than what is required to answer a question. The student must have a larger area of knowledge in order to select what to ask, and an even deeper understanding of the subject in order to then formulate probable incorrect answers .
The method prepares the student for working life. As an engineer, it is important to design tests that show that what you create and construct works correctly. Thinking early on about how different things should be tested is thus good engineering knowledge. When performing a test or asking questions, one must always critically examine the test and the result (is the right result really the right result?). If an engineer makes a mistake in their test specifications, it will affect others (in this case the fellow students in the course in a significant way), which must not happen. One has to take responsibility and be careful for it to be right from the start.
When an engineer discovers a test that is wrong, the person must act and not let the thing pass. In this way, tests are adjusted and gradually become correct. The questions in the course should be error-free at the end of the course if everyone takes responsibility. Consider whether some form of bonus system can be introduced in the course, that similarly to a workplace benefits those who do well. Is it possible to give an advantage to those who give feedback on any errors (for example, half a bonus point to the final exam)? Students may find it uncomfortable to point out that a peer has made a mistake, so it is not recommended to give a disadvantage to those who have created an incorrect question.
Supplemental examination parts
A quiz does not work as the only examination, there needs to be a palette of different forms of examination so that each student has the opportunity to show their knowledge. In the course from which this example is taken, the supplementary examination consists of group assignments with oral presentations, a seminar where different solution methods are discussed and a final group report with individual appendices.
The quiz offers students an opportunity to demonstrate factual knowledge. When the unsupervised quiz is combined with seminars and presentations, the teacher can get an idea of whether the results of the quiz are the student's own work. The final report gives the student an opportunity to present an understanding of the entire course content. In courses with many students, where it is impossible to follow each student's development, it is recommended to orally examine the final assignment in some form, either while the students are working on it or in some sort of oral presentation when it is finished. The examination form has been conducted for several years and cheating has been very uncommon.
An arrangement with continuous examination and for example many student presentations takes a lot of time for the teacher. In order to not increase the teacher’s workload too much, a change in the examination needs to affect the teaching and administration of the course. Even if parts of the continuous examination can function as teaching, it is good to consider exchanging live lectures for recorded lectures with follow-up short question sessions, or if it is possible to distribute non-grade-based and grade-based meetings with the students among fellow teachers and assistants in an alternative way. Since examination in varying degrees can also be a learning element, more time spent on examination does not necessarily mean less teaching, it rather means teaching in a different way.