Deakin University
Browse

File(s) not publicly available

SlimFL: Federated Learning With Superposition Coding Over Slimmable Neural Networks

journal contribution
posted on 2023-02-21, 04:38 authored by WJ Yun, Y Kwak, H Baek, S Jung, M Ji, M Bennis, Jihong ParkJihong Park, J Kim
Federated learning (FL) is a key enabler for efficient communication and computing, leveraging devices’ distributed computing capabilities. However, applying FL in practice is challenging due to the local devices’ heterogeneous energy, wireless channel conditions, and non-independently and identically distributed (non-IID) data distributions. To cope with these issues, this paper proposes a novel learning framework by integrating FL and width-adjustable slimmable neural networks (SNN). Integrating FL with SNNs is challenging due to time-varying channel conditions and data distributions. In addition, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, which makes SNN ill-suited for FL. Motivated by this, we propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models. By applying SC, SlimFL exchanges the superposition of multiple-width configurations decoded as many times as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations while avoiding inter-width interference during backpropagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also deals with non-IID data distributions and poor channel conditions, which is also corroborated by data-intensive simulations.

History

Journal

IEEE/ACM Transactions on Networking

Volume

PP

ISSN

1063-6692

eISSN

1558-2566

Language

English

Publication classification

C1 Refereed article in a scholarly journal

Issue

99

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC