Assessment and feedback using digital tools at UiA

Formative assessment at the University at Agder using Canvas in 2017-18 – Summative assessment using INSPERA

By prof. Stefanie Hillen

Background

The whole project started in 2016 and has had as objective to analyse the kinds of summative (final exams) and formative assessment processes (mandatory and non-mandatory midterm assignments) applied at the University of Agder. In specific, the change of using digital tool for assessment and feedback (habits) were traced by the newly introduced tool INSPERA and CANVAS. To gain better insights on how the teachers and students make use of these application, quantitative data were collected across the faculties. This blogg will inform you about the quantitative data collected by CANVAS and INSPERA. Results on an individual level by a pilot study conducted in the early summer this year are not reported here. This will be done in a paper (Hillen, Dalehefte, Lazareva, Schaffer & Wolcott, 2018) presented at the ECER 2018 which do include the aspect of assessment culture.

Applying CANVAS in fall 2017 and spring 2018 – general results

Above, one can find the extent in which midterm assignments were used for assessment and feedback for students work. On knows that a continually given feedback on students’ performance will enhance learning motivation and learning effectiveness.  Students who are informed about their current performance can easier work self-directed to accomplish their learning objectives. As said before data was collected across the university that is on the 5 faculties by data CANVAS can provide, using Learning Analytics on an aggregated level. The figure above informs us about that the use of CANVAS has been increased regarding the opportunities to support students learning using feed back features. In all faculties the application of midterm-assignments optional and mandatory ones, has been increased from the fall 2017 to spring 2018 (blue stack) when CANVAS was newly introduced and used by the lectures all over the University. A socalled learning effect can be concluded. The number of total lectures in the faculties or the number of teachers seems not having a strong effect on the total use of discussion themes or assignments provided by the teachers.

Another kind of data which is used and was approachable by CANVAS are activities per day or activity by category on the basis of page-views by the students and by teachers.

The following figure shows that the actual used websites and CANVAS features are applied in different degrees by the faculties. As documented the students and teacher made use of quizzes and mandatory and non-mandatory assignments. Obviously, the discipline seems to play an important role for applying or not applying these different features. The students and teachers of the faculty of arts and business have almost made no use of quizzes and grading websites during the semester related to other midterm activities in CANVAS.

Summative assessment (final exams) and the use of INSPERA

Using INSPERA was intended to support the assessment and grading process at the University for students and teachers for final examination. There are three typical and different types of examinations which can be conducted via INSPERA. The home-exam, university-exam and the group-exam.

The most popular examination form at Agder University is the school exam. Anyhow that group exams have become an established part of final examinations give indicate that the student’s competence of collaboration is seen as an important documented qualification for their later work life.

Ways of digital feedback by INSPERA

Inspera offers a variety of feedback to students if applied by the lectures. Teachers can use an annotation field, a commentary field to explain the grades, and of course the grade itself will provide information on the successful completion of the exam. The grader can too, make use of a student-hiden-annotation field to better understand how she once have assessed a specific question or task if grading is conducted together with a co-grader and needs to be explained before deciding the final grade.

Already in spring 2016 after the piloting of INSPERA the use og feedback functions has been increased. As reasons can be named, the learning effect using INSPERA and the courses offered by the administration which in sum has contributed to changes habits by the university teachers.

Inspera offers 18 question types, 11 of which are automatically evaluated, 5 requires manual evaluation and 1 that is not for evaluation purposes.

The settings available varies between the different question types, but the set-up and logic behind the various question types are the same. Clicking the question text gives four options in the panel to the right; Marks, Main illustration, PDF panel and Labels. These are general options available to all question types. Clicking the interaction area (the students response area) displays the options available to the specific question type in the panel to the right.

Summary and Conclusion

The tool Inspera and its use seems to work better as a grading tool than a feedback tool to students. Even if multiple choice approaches are popular (n =21 , m= 15) one can say that here is no strong digital multiple-choice assessment culture at the University of Agder. Most users make use of productive examination forms where students need to deliver assignment showing the students’ performance and quality of knowledge. One can say that even INSPERA offers a bunch of multiple-choice options lectures make just in a smaller extent use of it. Lectures make instead use of to combine different types of questions (n=434). The University of Agder have a partly digital assessment collaborative culture, but a strong digital supported individual active and productive assessment culture regarding final exams.