We propose a version of Ordered Weighted Averaging (OWA) operators which are robust against inputs with outliers. Outliers may heavily bias the outputs of the standard OWA. The penalty-based method proposed here comprises both outlier detection and reallocation of weights of the OWA. At the first stage the outliers are identified based on a robust criterion that can accommodate up to half the inputs being outliers, but at the same time not removing the inputs unnecessarily. Three numerical algorithms for calculating the optimal value of this criterion are proposed. At the second stage the OWA weights are recalculated for a subset of clean data while preserving the overall character of the weighting vector. The method is numerically tested on simulated data and exemplified on aggregating a large number of online ratings where the outliers represent biased, missing or erroneous evaluations.