We consider the problem of aggregating a large number of online ratings where there may be outliers, representing biased, missing or erroneous evaluations. The penalty-based method proposed comprises both outlier detection and reallocation of weights and we focus on models dependent on the relative order of inputs, i.e. based on OWA operators, however we also define the model for weighted means.