Digital and Continuous Assessment

Background and problem definition

Digital assessment can dramatically reduce costs and time for both teachers and administrators. A basic level of understanding can be assessed through multiple choice quizzes. Such quizzes can be auto-corrected in Canvas. The resource-intensive scanning process can potentially be limited and eventually abandoned at KTH. However, assessing a higher level of conceptual understanding digitally is difficult today. This project will facilitate the transition of a course to digital assessment and explore the possibilities of implementing new tools for assessing higher level conceptual understanding.

What we want to achieve

• We want to make assessment efficient, both in terms of student learning and in terms of teacher and assessor time.
• We want to develop courses and introduce more continuous assessment as an effective learning activity.
• We want to save time and resources for teachers and administrators.
• We want to digitally assess higher conceptual understanding in biotechnology and chemistry related courses using new tools.

Challenges

We faced many challenges at the beginning of the project, from difficulties in understanding and testing different settings in Canvas to logistical challenges at the actual time of the exam. Canvas is rather user unfriendly, and it is difficult to test student experience in advance.

We have selected one basic course in each study period (BB1150 Biochemistry 1, period 1; BB1200 Analysis of Biomolecules, period 2 and BB1050 Biotechnology, period 3-4). All three courses have been developed to include digital intermediate assessments using Canvas quizzes and, in addition, a digital final exam.

All assessments take place in computer rooms with KTH Digicertus exam (Safe Exam Browser), which is pre-ordered from the IT department for a certain time period.

Suggestions for improvement

The project has so far identified many bottlenecks and challenges:

• Students can access other Canvas pages other than the one specific to the assessment.
• It is practically impossible for the teachers to unpublish all relevant material prior to the exam.
• Last year's pilot was different from the current situation, and we propose to return to the situation during the pilot.
• We propose a sustainable solution where teachers request a lock down for a specific course and the admin staff take care of unpublishing and republishing after the exam.
• E-learning team should be given the resources to prioritise further improvement of KTH Digicertus Exam, with initial focus on ensuring that students do not have access to anything else than the examination material.

Results and lessons learnt

Some experiences:

• Students are swapping between other Canvas windows that (surprisingly) were accessible during the assessment on KTH Digicertus Exam.
• Students couldn’t log in or forgot their KTH id.
• The IT department opens at 9 am while the first exam started at 8 am.
• Students seem to see their results upon submission even though the settings shouldn’t allow for this.

Next step

This project will continue and in the next phase, we will explore chemical drawing software and possible implementation into Canvas. The project will also expand by including more courses.