Deakin University
Browse

File(s) under embargo

Muti-assistant Knowledge Distillation for Lightweight Bearing Fault Diagnosis Based on Decreasing Threshold Channel Pruning

Version 2 2024-05-31, 06:33
Version 1 2024-01-11, 23:22
journal contribution
posted on 2024-05-31, 06:33 authored by Hongyu ZhongHongyu Zhong, Samson YuSamson Yu, Hieu TrinhHieu Trinh, Y Lv, R Yuan, Yanan WangYanan Wang
Bearing fault detection and classification under a diagnostics model with fewer parameters has been a challenging problem. A commonly solution is knowledge distillation (KD) using teacher-student models. Through the distillation process, the student model can acquire knowledge from the teacher model to enhance performance without introducing extra parameters. However, when using a powerful teacher model, distillation performance is not always ideal. This is because a more powerful teacher model can generate more specific classification strategies, which may result in poorer distillation performance. To this end, the multi-assistant knowledge distillation (MAKD) method is proposed, which bridges the gap between the teacher-student models by incorporating several intermediate-sized assistant models. Moreover, these teacher assistant models have the same architecture, which creates a better knowledge transfer condition at the logit layer. To further optimize the network structure to improve the distillation performance, decreasing threshold channel pruning (DTCP) is proposed to generate the required assistant models. DTCP leverages the scatter value of the decreasing function to prune the channels of the teacher model, which retains more channels close to the output layer. Finally, 4-class and 10-class classification experiments are conducted on two bearing datasets. The experimental results demonstrate that the proposed DTCP-MAKD method improves distillation performance and outperforms other state-of-the-art KD methods.

History

Journal

IEEE Sensors Journal

Volume

24

Pagination

486-494

Location

New York City, NY.

ISSN

1530-437X

eISSN

1558-1748

Language

eng

Publication classification

C1 Refereed article in a scholarly journal

Issue

1

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Usage metrics

    Research Publications

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC