P14 On design of our electro undergraduate courses with considerable degree of instant automatic feedback

Short paper

A proposal for a new course design of our electro undergraduate courses with several different sets of interactive questions, based on Bloom’s Taxonomy, constructed in Möbius CW and integrated into the LMS Canvas will be presented. The presented work contains a brief account of students’ first reaction in two courses in two different programs and what the respective students think of the new online tools that the course is equipped with.

1.Introduction

The presented work is the second part of ongoing project on Interactive assignments in our electro courses. The first part of the project has been presented at SoTL 2021 [1]. In this work we present a description of our proposal for course design based on Blooms Taxonomy [2].

One of four strategic areas that KTH’s operations are based on is digitization. Decision on principles for future education (V-2022-0380) [3] consist of thirteen principles. Principle number three is about active student-centered learning and to fulfill this principle we used revised version of Blooms Taxonomy [4] which is rewritten i active form of the original Blooms Taxonomy. The principle number five is about a high degree of interactive teaching elements and open experimental environments. The technical capabilities of the Möbius Courseware (CW) enabled us to fulfill principle of both interactive teaching and open experimental environments. And only the limits of our imagination set limits when constructing questions for open online experimental environments. Inspired by this document and our own multi-year proven experience, we experimented with new constructions of optional interactive engaging exercise assignments constructed in Möbius CW [5]. In two of our pilot courses listed in Table 1, we tested our proposal for new course design.

The same course design has been used and the same survey was conducted in both pilot courses. A part of analysis based on survey held on in these two courses is presented here.

2.Purpose and background

The purpose of this work is to define a course design in our Learning Management System (LMS) Canvas that helps students in our undergraduate electro courses to learn in more effective way.
One of our goals was that the new course design should make it easier for students to be more active throughout the course in accordance to [6]. The second goal is to find out a course design that would relieve teachers’ jobs around teaching and examination as a similar research reported [7] .

3.Completed work and work in progress

This attempt of practical research is an experimental design research which consists of two kinds of variables, an independent and a dependent one. The independent variables were in the form of three different constructed interactive exercises used in this case study and conssiting of questions constructed in Möbius CW. The dependent variable is in the form of the student’s results at the final exam written in a computer room and in the same environment i.e. Möbius CW and the students’ answers to the interview which students in both groups has been asked to fill in.

Table 1: Presented research has been held in these two courses

3.1 Our research questions

RQ1: In what way could we use M¨obius CW technology to offer our students interactive online environment for calculations in different case studies in our undergraduate electrical courses? RQ2: Which of these digital solutions will help our students’ learning process the most?

4.Results

Some students believe that online technology limits the communication possibilities available in face to face meetings. Limitations in the project caused by limitations of communication possibilities have been replaced with other functions and advantages of online technology such as Discussion Forum in each course module and Zoom meeting.

4.1 Limitations and advantages in Möbius CW

Basic element of Möbius CW is a Question 3 which can be defined in different ways and answers from students can be chosen from fifteen predefined answer options or question types. Defined questions can later be combined in two modes as a Lesson 4 for practicing and as an Assignment 5 for learning/training and examination. The first task what we have done was constructing enough amount questions in each course’s module. Then we constructed the Lessons with basic exercises and consisting of the ready-made questions and based on activities that Bloom’s Taxonomy describes [4]. Of the fifteen different question types found in Möbius CW, only two cannot be auto-marked. Only open questions and upload file requires manual processing by teachers. The auto-correcting feature combined with useful feedback is very practical and can save students’ time when studying and also teachers’ time when correcting exams. The student also gets instant feedback 24 hours 7 day in a week. The auto-marked questions that only require answers give the possibility that several questions can be done in the same time, thus making both the students’ and the teachers’ time more efficient.
The auto-marking function combined with useful feedback is very practical and can save students’ time when studying. This feature was most appreciated by students in both test-courses.
At the same time, teachers’ working time for marking exams is minimized. On the other hand, constructing the auto-correct questions requires experience and teacher skill. Unlike oral discussions or exams, teachers are limited with the fifteen question types and they have to think about both what they want to ask and how the question should be constructed in the Möbius CW tool.

In our courses, we mostly used the numerical answer option because our questions are often based on calculations of different quantities ocuring in electrical circuits. Math formulas and clickable pictures is also used.
In the past, paper exams in our department were constructed of four sub-questions in each exam question. We have only continued with that tradition and the sub-questions are not necessarily listed in the same order as they are to be answered.

The questions constructed using Möbius CW and used in this research have been arranged in five different ways in (optional) exercises or (mandatory) assignments. Since test courses are at the undergraduate level, given in Swedish language and thus are abbreviations in Swedish 2.

Table 2: Constructed exercises have been organised these fice ways

4.2 Observations

Traditionally, lessons at university are divided into two main branches:

• lectures with a teacher’s one-way communication, i.e. preaching and
• exercise or ”calculating lessons” (in Swedish: räknestuga) with assistants, (usually older students) who solves ”on the board” the predetermined tasks.

In both types of lessons, students are passive consumers of new knowledge. In our proposal for new course design the traditional lecture have been exchange by videos and traditional preaching exchanged by dialog and seminars during the lessons time. Exercises used in ”calculating lessons” (räknestuga) have been digitized and exchanged by mandatory INL assignments.

To replace the lack of social environment in the classrooms and the natural discussions that this environment entails, we constructed two additional types of optional exercises:

• VIL (Optional Interactive Lessons) and
• VIRS (Optionally Interactive Exercises).

4.3 Communications possibilities

In addition to interactive exercises, we opened thematic discussion forums in each module of the LMS Canvas. And these discussion forums were also used outside working hours and especially at the end of the course. In practice, this alleged deficiency is only about the traditional habit of meeting in the classroom. Discussion channels are many more in an online environment and also the number of posts online is definitely more than in courses without online forums. Tested solutions in two test courses also show a higher degree of communication between teachers and students during the course. The discussion forum in the course made contact with teachers and other students possible even outside working hours. It is common for students to send an e-mail to their teacher when they have something to ask.

If a teacher answers a question via e-mail, then the teacher has helped one student. If the same teacher has written the same text in the course Discussion Forum, the teacher would have helped the generation of students in ongoing course. If the same text had been written in the feedback of the question in Möbius MC, then it would have helped all future generations of our students who would not need to ask the same question.

4.4 Optional Interactive Exercises

Optional interactive exercises with feedback on each question were gradually introduced in our test courses according to different needs.
In the very beginning the idea was to use Möbius CW only for exam and to construct a few exercises was just to show the students what the exam will look like. We then discovered that the use of interactive exercises increased discussion, in the Discussion Forum showing a clear increase of students engagement and thus activity in the course and outside of scheduled lessons.

We hoped that interactive questions improve students learning and that led to a more organized development of different questions type and solutions based on Bloom’s Taxonomy as the Figure 1 shows. But it is clear that the new course design led to more efficient teachers’ working hours, which is confirmed in a similar survey [7].

We hoped that interactive questions improve student learning, which was also shown in the upcoming exam in one of those courses (KH1251). Before we used the new course design, the percentage of students who passed the first exam in KH1251 used to be between 40 - 50% of those who took the exam. In the last course cycle in KH1251 (exam in January 2022), 66% of students who wrote the exam passed. In the second course (HM1006), the proportion who passed the course was as expected, i.e. at the same level as previous course rounds. But one academic year is too short time to draw any conclusions and therefore we leave conclusions about results for further research.

Figure 1: Optional excercises based at different parts of Bloom's Taxonomy

4.4.1 VIL - Optional Interactive Lessons

To develop understanding of new theories we have made interactive lessons VIL. In practice, it is a mixture of old Power Point Presentation slides and simpler questions necessary for basic understanding of the lesson. VIL constructions can be used even in case when a teacher is in the classroom and have a lecture. Short questions can give interactive activities during the lecture and students can immediately check own remembering and understanding of the lecture. Questions used in VIL are short, simple and often based on theoretical text in the respective chapters of the course literature.

4.4.2 VIRS - Optional Interactive Exercises

One of the most popular course materials are questions from old exams and we usually use the old questions as practice material. M¨obius CW gave us the opportunity to comment on each question and thus help the students with the learning process. In VIRS (Optional Interactive Exercise) we only use the old exam questions with feedback on each question. Then students have started to use and analyze their knowledge and their learning comes at a deeper level according to Bloom’s Taxonomy shown in Figure 1.

Using VIRS in our courses moved our practice lessons, so-called ”r¨aknestuga” or ”calculating lesson”, to online and thus changed our need to discussions. Instead of having calculating lessons in fixed places and at fixed times, students got a flexible tool that can be done when it suits them. Discussions around exercise tasks were moved to the Discussion forum in LMS Canvas and thus became more efficient. A shorter survey among our student networks showed that a minority of students write posts in the Discussion forum, but the majority of students read the other students’ discussions. In an open discussion with students (in KH1251) students have been asked about their activities in the Discussion forum. The most common comment was that they do not need to write their own post because someone has already asked the same question and the teacher has already answered.

Our previous experience with calculating lessons was that those lessons were not well attended because attendance was optional. This change in the format of such lessons means that teachers’ working hours are made more efficient and students get help just-in-time. This opportunity and flexibility is appreciated by most of our students. Of course, there are both students and teachers who are not so inclined to change.

4.4.3 VAI - Optional Adaptive Interactive Assessment

Adaptive assignments in M¨obius CW6 are intended for assessment in which questions are grouped into groups according to degree of difficulty. We have divided our questions into three groups:

• simple theory questions
• reasonably difficult calculation questions
• old exam questions.

Each subsequent question a student receives in an adaptive assignment is based on the student’s answer. The assignment starts with an easy question and if the student answers correctly, the next question will be from a higher level of difficulty. If the student does not answer correctly, the next question from the group will be of easier difficulty. In this way, each exercise attempt can result in a different composition of questions with different degrees of difficulty. We recommended this exercise as a final exercise in each section.

4.5 Construction of questions

Formulating a question in an engineering education course that brings out what we want the student to learn is on the verge of being a state-of-art. We want students to think for themselves and to construct a question without the question being leading or misleading. In order for a course activity, for example an interactive calculation exercise, to be instructive and effective, a constant discussion between teachers and students is required. This is not only important to ensure that the questions are at the level that students can understand knowledge, but also that the feedback embedded in each question is at the right level and really helps the students. Student comments and questions about each interactive question are valuable in the development of the questions. Therefore, the Discussion Forum is an important tool not only for the students’ development, but also for the development and refinement of course content e.g. assignments, lessons and questions in Möbius CW. Our interactive questions integrated with other tools such as videos with lectures (often with lectures from other universities) and quizzes with simple questions gave us an approach to game-based learning environment. Our search for the optimal course design led us to a new area of quest-based learning described here [8].4.6 Lessons Learned

From the first day when we got LMS Canvas we tried to develop our electro courses Canvas rooms using multi-medial technical solutions. Students had access to both pdf files with respective chapters from the course book, Power Point Presentations from old lectures, video clips and quizzes with simple questions. It is surprising that students in both groups chose interactive questions to a significantly greater extent. In the beginning, we just wanted to introduce the exam environment to students so that the exam becomes a less stressful moment. But interactive questions led to more activity among students, which was reflected in more discussions.

Figure 2: Students were asked to choose the tools that they believe helped them the most.

One of questions in survey was: ”As a student, I think I got the most help from:” and students could choose multiple answers. Figure 2 shows statistics of responses to the question. On the left side is data from KH1251 and on the right side is data from HM1006. Both survey groups chose three different interactive exercises as tools that helped them the most.

5. Message

Experience from the test period of the suggested course design could lead to the development of more effective system both for students and teachers. The new technology is here to stay.

We can use the new technology wisely so that we make our engineering education more efficient, communication surfaces are expanded and students get help just-in-time when they need it. In the new technology we can get a really assistant as mentioned in interview here [5].

We live in a time when our bank offices have already moved to our computers and self-driving cars have already started rolling on our roads. In such a technical world, surrounding needs are developing, including the need for faster and more efficient dissemination of knowledge both in depth and advanced courses but also with basic courses for significantly more students. If we have confidence that our computers handle our banking affairs, we should have the same confidence that the same computer can handle the teaching of the next generation of engineers. If cars can be self-driving why not do courses in the same way? Presented proposals are perhaps one possibility that describes how the available technology could be used in teaching engineers for the future.

References

1. N. Subasic and H. Johansson, “Interactive Assignments in Electro Courses at MDA,” Stockholm, Mar. 2021. [Online]. Available: http://kth.diva-portal.org/smash/get/diva2: 1588335/FULLTEXT02.pdf.
2. B. Bloom, Taxonomy of educational objectives. 1: Cognitive domain, eng, 29. print. London: Longman, 1986, isbn: 978-0-582-28010-6.
3. S. Karlsson, Beslut om principer f¨or framtidens utbildning, Jun. 2022. [Online]. Available: https : / / intra . kth . se / styrning / framtidens - utbildning / beslut / principer - 1 .1197995.
4. D. R. Krathwohl, “A Revision of Bloom’s Taxonomy: An Overview,” en, Theory Into Practice, vol. 41, no. 4, pp. 212–218, Nov. 2002, issn: 0040-5841, 1543-0421. doi: 10 . 1207 /s15430421tip4104 _ 2. [Online]. Available: https : / / www . tandfonline . com / doi / full / 10.1207/s15430421tip4104_2 (visited on 10/22/2022).
5. Nihad Subasic, DESIGNING AN EFFECTIVE STEM COURSE, Jun. 2021. [Online]. Available: https://digitaled.com/resources/blog/designing-an-effective-stem-coursepart1.
6. C. C. Bonwell and T. E. Sutherland, “The active learning continuum: Choosing activities to engage students in the classroom,” en, New Directions for Teaching and Learning, vol. 1996, no. 67, pp. 3–16, 1996, issn: 0271-0633, 1536-0768. doi: 10.1002/tl.37219966704. [Online].
   Available: https://onlinelibrary.wiley.com/doi/10.1002/tl.37219966704 (visited on 10/11/2022).
7. O. B¨alter, R. Glassey, and M. Wiggberg, “Reduced Learning Time with Maintained Learning Outcomes,” en, in Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, Virtual Event USA: ACM, Mar. 2021, pp. 660–665, isbn: 978-1-4503-8062-1. doi: 10.1145/3408877.3432382. [Online]. Available: https://dl.acm.org/doi/10.1145/ 3408877.3432382 (visited on 10/26/2022).
8. C. Snelson, “Quest-Based Learning: A Scoping Review of the Research Literature,” en, TechTrends, vol. 66, no. 2, pp. 287–297, Mar. 2022, issn: 8756-3894, 1559-7075. doi: 10.1007/s11528-021-00674-w. [Online]. Available: https://link.springer.com/10.1007/s11528-021-00674-w (visited on 10/29/2022).

Authors

Nihad Subasic
lecturer

subasic@kth.se

Profile

Hans Johansson
associate professor

hbjo@kth.se , +4687907490

Profile