Facial Expression Recognition Based on TripletLoss and Attention Mechanism

Rongqiang Gou, Qiuyan Gai, Zhijie Xu, Yuanping Xu, Chaolong Zhang, Jin Jin, Jia He, Yajing Shi

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


The paper proposes a facial expression recognition method called Triplet-Loss Attention Network (TAN), which aims to address the problem of large intra-class and inter-class distances in facial expression recognition. The method uses a self-attention mechanism to calculate the weight of triplet expression samples, which helps in determining the influence of key regions on expression recognition. The method uses images with attention weights above a certain threshold as training samples to form new hard triplets. The distance between high-weight samples in the three sets of samples is calculated using the Mahalanobis distance formula, and the difference in distance between high-weight groups of triplets is calculated using the Maharaja Loss function. The Triplet Loss is mainly used as the loss function in TAN, and the model is jointly optimized using Triplet Loss, Mahalanobis Loss, and Cross Entropy Loss functions to improve the performance of the model in facial expression recognition. Experimental results show that TAN performs well in alleviating the intra-class and inter-class distance problem and has good robustness and generalization performance. On the RAF-DB and FERPlus datasets, TAN achieves recognition accuracies of 88.40% and 88.73%, respectively, which is 1.37% and 0.18% higher than the previous state-of-the-art methods.

Original languageEnglish
Title of host publication2023 28th International Conference on Automation and Computing
Subtitle of host publicationICAC 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9798350335859
ISBN (Print)9798350335866
Publication statusPublished - 16 Oct 2023
Event28th International Conference on Automation and Computing: Digitalisation for Smart Manufacturing and Systems - Aston University, Birmingham, United Kingdom
Duration: 30 Aug 20231 Sep 2023
Conference number: 28


Conference28th International Conference on Automation and Computing
Abbreviated titleICAC 2023
Country/TerritoryUnited Kingdom
Internet address

Cite this