DDMV1 – Data-driven assessment of qualifications to Master's programmes
The project involves increasing the quality of the master's admissions process while reducing the workload for both PA and study administration, by automating data management to a large extent and also developing an accurate and validated model for assessment of qualifications.
Project context
KTH has many international applicants for its Master's programmes. The evaluation (of specific entry requirements and qualifications) of these applicants currently takes considerable time for both KTH's administration and for Programme Directors (PA). The procedure requires that a number of documents are manually retrieved, opened and assessed in the NyA system (the admission system of the Swedish Council for Higher Education). The efficiency of the process could be improved through automated data management. In addition, there is currently no clear, common and transparent method for how students' qualifications should be assessed to constitute a good indicator of their ability to succeed in their studies at KTH. The quality of the admissions process also has potential for improvement.
Ahead of the master's admissions work at the SCI school in 2023, a pilot project was therefore carried out during autumn 2022 with the overall goal of increasing the quality of the admissions process while reducing the workload for both PA and study administration. The pilot project succeeded in achieving both of these goals, while a number of new ideas were identified to continue towards the overall objective. There is consequently potential to further increase both the efficiency and quality of the process.
This project involves performing the work that the SCI school itself can do to move forward. Collaboration with other schools and with the university operational support (VS) has the potential to lead to further improvements.
Purpose (outcome)
The overall purpose of the project is to create better quality in the assessment for the Master's admissions in 2024 at the SCI school, while reducing the time required for the work. The impact targets are:
- Less work for both administration and PA, by automating all parts that can be automated (automated data management to quickly produce an easy-to-use decision support and a preliminary ranking of the students' qualifications). Follow-up via time comparisons.
- Better assessment for the international admissions in 2024, by creating a more accurate and validated assessment of qualifications model (using data from a larger number of former students at KTH than was the case in the pilot study). Follow-up via statistical analysis of the model.
Project results (output)
- Presentations made for other schools, with the aim of recruiting more Master's programmes and thereby increasing the possibilities for a broader implementation at KTH
- Programme-specific web form for applicants (so called “Summary sheet”) developed, tested and published (via link from antagning.se /universityadmissions.se to the web form) for all participating 27 programmes at SCI; CBH, EECS and ITM.
- Decision support to PA (the “PA view”): Excel programme to manage data from the web form, developed and tested ahead of the admission work 2024 for all participating 27 PAs.
- A statistical merit evaluation model (“MVM”) based on how previous students have performed at KTH and their background. The purpose of the model is to statistically predict (with a calculated uncertainty) how an applicant is expected to perform in their future studies at KTH (see Results below)
- Acquired data on international universities (“University List”) with a routine for handling missing universities.
- A first process description developed (VS) as well as training in process and Excel files for participating programme administrators ahead of the admissions process in 2024
- Sharp use during the 2024 admissions process of developed Excel programmes and routines with evaluation and documentation (February-April) within 28 Master's programmes.
Time plan
Start date: 2023-08-01
End date: 2024-04-30
Project documentation
If you have a KTH ID, you can read the latest project documents when logged in:
-
Project initiation document: Projektdirektiv_FrU23_2322-SCI_Masterantagning-DDMV1.pdf (Swedish)
Result
- Merit evaluation model (in Swedish – contribution to 9th Development Conference for Swedish Engineering education Nov, 2023)
In October 2023, this project Data-driven assessment of qualifications for admission to Master’s programmes (DDMV1) began collaborating with the related VS project Merit Assessment Master. The two approaches were later merged under a joint steering group and the work continues in the project DDMV2 (2406-KTH) with programme directors (PA) from SCI, ABE, CBH, EECS and ITM, the IT department and central and school-based VS at the schools concerned.