Macro-expression-guided micro-expression recognition: A motion similarity perspective

Micro-expression recognition (MER) is a challenging task due to the subtle and short-lived facial muscle movements involved. Macro-expressions, in contrast, are more evident and easy to recognize. Yet, both expressions share similar facial muscles to express the same emotions. We exploit this observation to propose a novel Macro-expression guidance network (MAG) that uses motion similarity to aid MER. The MAG has four key components: 1) motion vectorization, which transforms facial motion into a vector tensor representation to capture motion dynamics; 2) nonlinear amplification, which enhances the intensity of micro-expression features to make them more salient; 3) macro-micro matching, which aligns macro- and micro-expressions with the highest motion similarity to achieve a one-to-one mapping between the two modalities; and 4) guidance mechanism, which enables macro-expressions to guide the extraction of micro-expression features using convolutional operations. We perform extensive experiments on 7 datasets under 3 benchmarks and demonstrate that MAG outperforms state-of-the-art methods for MER.