Course evaluation and analysis of doctoral courses
All doctoral courses at KI are evaluated electronically by the Evaluation Unit at LIME, at the request of the Course and Programme Committee.
Online evaluations for doctoral courses
All doctoral courses at KI are evaluated electronically once they are completed with an established set of questions in KI Survey. The Evaluation Unit at the department of LIME has been given the assignment to carry out the course evaluations (sending out the questionnaires and compiling the answers). The assignment requires cooperation with course organisers that provide the Evaluation Unit with current participant lists (see further instructions below).
The Evaluation Unit’s assignment also includes to be able to assist with consultations regarding possible additional questions, to compile evaluation reports (full version including questions with open answers and a short version that is used for direct feedback to participants as well as in the course catalogue next time the course is to be given) to course organisers and central administrative officers, as well as producing a summarising yearly report of course evaluations, in order to follow the development of the courses over time. (See below under documents, only in Swedish).
The course evaluation consists of 18 questions. Please see course evaluation questions for questions and response options. In addition to these, the course provider, in consultation with the Evaluation Unit, can add own questions when needed (after the last question in the template).
Instructions to the course organiser
When contacting the Evaluation Unit regarding your course, send an email to email@example.com with the course number and semester in the subject line (e.g. "1234 VT20), where VT stands for spring semester and HT for autumn semester.
If you want to add your own course-specific questions to the survey, those have to be sent to the Evaluation Unit no later than two weeks (14 days) before the course end date. Also send response options/response scale. If possible, adjust the questions to the response scale that is used for the mandatory questions: to a very small extent, to a small extent, to some extent, to a large extent, to a very large extent.
If you wish to get help from the Evaluation Unit to develop course-specific questions, you have to notify them at least three weeks (21 days) before the course end date.
Documentation needed for sending out course evaluations
A participant list is to be sent to the Evaluation Unit no later than the day after course start, as follows:
A list in excel with the email addresses to the course participants. Each address should be placed in a separate cell. Name the file with course code, semester (VT for spring and HT for autumn) and, if applicable, the numbering of course occasion (if the course is given more than once during a semester). Example: 1234_VT20_2 (if it is the second course occasion of the course with number 1234, spring semester 2020).
The course evaluation will be sent out the day after the course end-date, unless otherwise agreed. The participants can reply to it during about two weeks and two reminders will be sent out during this period. When the survey is closed and completed the Evaluation Unit will create two reports that will be sent as report links to you as course responsible.
In the short version report the open questions and any course specific questions have been removed. This version can be presented to the course participants and it is also this link you will put into the course occasion at the next announcement of the course in the course catalogue.
The full version report includes all questions, even the open and the course specific ones. This is to be used as a basis for your course evaluation (reflection) and as part of your report to relevant doctoral programme or research school, or to the Course and programme committee.
If you wish to make a more in-depth analysis of the result from the course evaluation, you can request that the Evaluation Unit also sends the result as an Excel file.
Course analysis - the course director’s evaluation
The Evaluation Unit has in consultation with the Course and Programme Committee developed a form for course analysis to facilitate quality assurance of doctoral courses. The form is to be used by the course director to summarise and reflect on the outcome of the participants’ course evaluation and to plan for any improvements for next course occasion. This applies to all courses; within doctoral programmes, doctoral schools as well as freestanding courses.
The course analysis form is sent to the steering committee or equivalent of the research school or doctoral programme within which the course has been given. The steering committee gives feedback to the course director, using the same form, with comments and suggestions for improvement. For courses given with funding from the Course and Programme Committee, the form should be attached to the requisition, as before.
What is the difference between course evaluation and course analysis?
Course evaluation is the course participants’ feedback given anonymously via a web based questionnaire.
The course analysis is the summary made by the course director, based on the outcome of the course evaluation, the teachers’ experiences from the course, other data sources as well as any changes that are planned as a result of these.
Purpose of course evaluations
The purpose of evaluations is to monitor the quality of the courses that are provided and to improve their quality going forward. A general questionnaire for all courses enables comparisons concerning perceived quality.
The evaluations should be distributed to a number of interested parties:
- Teachers and course administrators in charge
- Doctoral students who are taking the course, as well as future applicants
- The Course and Programme Committee
- The steering committee for the programmes that the course is part of
- The departments that provide the course
Course directors at KI have been obliged to arrange evaluations ever since June 2002 (Higher Education Ordinance, Swedish Code of Statutes 1993:100).
Suggestions on how course participants can be informed
The course participants should be given access to both the results from the previous course evaluation (when they apply for a course) and the one they themselves participated in, at least the short version, as well as the course evaluation (reflections) made by the course responsible.
Here are some suggestions on how and when this can be done:
Prior to the course:
- Enter the link to the latest course evaluation (short version) in the course occasion that shall be published in the course catalogue, so that it is available for the applicants.
- Tell students about the course evaluation at the beginning of the course. This can e.g. be done in the introductory lecture in connection to the explanation of the intended learning outcomes (ILOs) of the course, the course design, the assessment procedure and providing practical information.
- Tell students about the views of the participants of the last course and how those have been handled (where the course has been organised previously).
- Tell students what has not been changed and why the recommendations of the previous students were not followed.
Towards the end of the course:
- Tell students what kind of information is expected to be obtained with the course evaluation and how the information is intended to be used.
- Tell students when they can expect to get feedback about the survey and how.
After the course:
- Give feedback to the course participants on the outcome of the course evaluation (and the course analysis). Tell them what changes are planned for the next course occasion, as a result of their feedback as well as what will not be considered and why.
It is important that the students know that they have been listened to. Moreover, the students often say that they will get more involved if they know that their opinions are taken into account.
Course evaluation according to the Higher Education Ordinance
The Higher Education Ordinance indicates in section 14, dealing with course evaluations, that:
“Higher education institutions shall enable students who are participating in or have completed a course to express their experiences of and views on the course through a course evaluation to be organised by the higher education institution. The higher education institution shall collate the course evaluations and provide information about their results and any actions prompted by the course evaluations. The results shall be made available to the students."