File(s) not publicly available
An extended approach to adjust inconsistent minority peer- and self-assessment scores of teamwork using assessor’s reliability
conference contribution
posted on 2016-01-01, 00:00 authored by Kali Prasad NepalTeamwork is generally assessed either solely by academic staff or by both academic staff and
students themselves confidentially as well as collaboratively. Peer- and self-assessments have been
used primarily to assess teamwork process and teacher assessment to assess teamwork product.
Peer- and self-assessments are useful to elicit team members’ contribution towards teamwork and to
convert team mark into individual marks, provided the scores are reliable (the extent to which the
scores are consistent). However, not all peer- and self-assessment scores are reliable. Anecdotal and
literature evidence suggest that there are several cases of inconsistencies in these scores. Individual
contribution scores given by teammates to an assessee (including himself/herself) can sometimes
vary significantly due to both intentional and unintentional reasons. Simply using total individual rating
scores without considering an assessor’s reliability to estimate individual contribution factors can
sometime results unfair grades and becomes hindrance to learning through teamwork.
PURPOSE
This study proposes an extended approach to adjust inconsistent and/or distorted minority peer and
self-assessment scores of teamwork using standard normal probability concept.
APPROACH
In order to adjust inconsistent and/or distorted minority peer-and self-assessment scores of teamwork,
an extended approach has been proposed. The approach uses the reliability of assessor’s scores of
an assessee using standard normal probability curve. The evaluation of the extended approach is
conducted by comparing with the existing approaches using two case examples of peer- and selfassessment
of teamwork where minority team members’ scores are inconsistent.
RESULTS
The evaluation of the extended approach shows that the proposed method is superior to the available
approaches in order to adjust inconsistent peer- and self-assessment scores for special cases where
scores of minority team members are inconsistent. The extended approach helps both to automatically
detect such scoring anomalies and to adjust the scores so that the fairer contributions to the teamwork
would be obtained and utilised.
CONCLUSIONS
The extended approach is useful in that it helps both to automatically detect scoring anomalies and to
devise the methods to adjust them. However, the approach does not address the issue of scoring
inconsistencies by majority of team members as it uses average score as a basis for identifying
inconsistencies. Moreover, the approach needs to be implemented in the real teamwork environment
in order to identify the impacts of these scoring adjustments in teamwork process and teamwork
product.
students themselves confidentially as well as collaboratively. Peer- and self-assessments have been
used primarily to assess teamwork process and teacher assessment to assess teamwork product.
Peer- and self-assessments are useful to elicit team members’ contribution towards teamwork and to
convert team mark into individual marks, provided the scores are reliable (the extent to which the
scores are consistent). However, not all peer- and self-assessment scores are reliable. Anecdotal and
literature evidence suggest that there are several cases of inconsistencies in these scores. Individual
contribution scores given by teammates to an assessee (including himself/herself) can sometimes
vary significantly due to both intentional and unintentional reasons. Simply using total individual rating
scores without considering an assessor’s reliability to estimate individual contribution factors can
sometime results unfair grades and becomes hindrance to learning through teamwork.
PURPOSE
This study proposes an extended approach to adjust inconsistent and/or distorted minority peer and
self-assessment scores of teamwork using standard normal probability concept.
APPROACH
In order to adjust inconsistent and/or distorted minority peer-and self-assessment scores of teamwork,
an extended approach has been proposed. The approach uses the reliability of assessor’s scores of
an assessee using standard normal probability curve. The evaluation of the extended approach is
conducted by comparing with the existing approaches using two case examples of peer- and selfassessment
of teamwork where minority team members’ scores are inconsistent.
RESULTS
The evaluation of the extended approach shows that the proposed method is superior to the available
approaches in order to adjust inconsistent peer- and self-assessment scores for special cases where
scores of minority team members are inconsistent. The extended approach helps both to automatically
detect such scoring anomalies and to adjust the scores so that the fairer contributions to the teamwork
would be obtained and utilised.
CONCLUSIONS
The extended approach is useful in that it helps both to automatically detect scoring anomalies and to
devise the methods to adjust them. However, the approach does not address the issue of scoring
inconsistencies by majority of team members as it uses average score as a basis for identifying
inconsistencies. Moreover, the approach needs to be implemented in the real teamwork environment
in order to identify the impacts of these scoring adjustments in teamwork process and teamwork
product.
