Knowledge Base Repository

In addition to research papers, the Design Society is developing several valuable resources for those interested in the study of design. These include a repository of PhD theses, a library of case studies and transcripts of design activities, and an archive of our newsletters. Please note that these resources are accessible exclusively to Design Society members.

Introduction and evaluation of the peer evaluation tool

Kallela, Timo; Lehtonen, Timo; Luostarinen, Pasi; Riitahuhta, Asko; Lanz, Minna


Type:
Year:
2009
Editor:
Clarke, A, Ion, W, McMahon, C and Hogarth, P
Author:
Section:
NEW PEDAGOGY
Page(s):
287-292
Abstract:
The aim of the present paper is to introduce an automatic peer evaluation tool for web-based examination and peer evaluation system. The peer evaluations as they are, they won't give sufficiently reliable grading, since the students performing peer evaluations are on different levels. The automated grading process tries to fix this problem through an imperative algorithm, which guides the grading procedure. The algorithm calculates credibility factors for each student. The credibility factor is based on the grades given by other students in peer evaluation. After this the algorithm 'corrects' the given grades by putting more weight for grading given by a student with high credibility value. In the earlier research conducted and reported [1] last year introduced a web-based examination application with a peer evaluation tool. The tool took some distance from traditional exams by providing time and place independency and also provided efficient peer evaluation tool. This tool was used to test automated grading algorithm. The grades created manually in the web-based exam were compared to ones calculated with the algorithm. The results seemed to correlate quite well, although there where some anomalies found when the evaluation of the system was done. Currently it is possible to isolate these problematic answers and continue the grading manually. When these problematic answers were removed, the results given by the automatic grading algorithm were very good.
Keywords:

This site uses cookies and other tracking technologies to assist with navigation and your ability to provide feedback, analyse your use of our products and services, assist with our promotional and marketing efforts, and provide content from third parties. Privacy Policy.