A relative relevance approach to refine inconsistent peer- and self-assessment scores in teamwork assessment

This study proposes an extended approach to refine inconsistent peer- and self-assessment scores in teamwork assessment. These refined scores are commonly used to estimate individual contribution factors, also referred in some literature as individual weighting factors. The individual contribution factors are then multiplied by team mark to convert team mark into individual marks, provided the scores are valid (the degree to which the scores measure the true contributions) and reliable (the extent to which the scores are consistent). However, not all peer- and self-assessment scores are valid and reliable. Although the validity is as equally, if not more, important as reliability, this study focuses on the reliability. Anecdotal and literature evidence suggests that there are several cases of inconsistencies in students’ peer- and self-assessment scores. Creative accounting scores (over-rating to self and under-rating to peers) by some minority team members are commonly encountered cases of inconsistencies, which are addressed by the proposed extension. To discuss the characteristics of the extended approach, mathematical equations and computations are presented and discussed with the help of typical inconsistent peer- and self-assessment scores. The analysis clearly shows that relative relevance approach based on standard normal probability can be a viable option in order to refine creative accounting cases of inconsistencies.