Communication-Computation Efficient Federated Learning over Wireless Networks
Time: Fri 2023-04-21 13.00
Location: Sten Velander, Teknikringen 33, Stockholm
Language: English
Subject area: Electrical Engineering
Doctoral student: Afsaneh Mahmoudi , Elektroteknik
Opponent: Docent Nikolaos Pappas, Linköpings universitet
Supervisor: Professor Carlo Fischione, Nätverk och systemteknik; Dr. José Mairton Barros da Silva Jr., Nätverk och systemteknik
QC 20230310
Abstract
With the introduction of the Internet of Things (IoT) and 5G cellular networks, edge computing will substantially alleviate the quality of service shortcomings of cloud computing. With the advancements in edge computing, machine learning (ML) has performed a significant role in analyzing the data produced by IoT devices. Such advancements have mainly enabled ML proliferation in distributed optimization algorithms. These algorithms aim to improve training and testing performance for prediction and inference tasks, such as image classification. However, state-of-the-art ML algorithms demand massive communication and computation resources that are not readily available on wireless devices. Accordingly, a significant need is to extend ML algorithms to wireless communication scenarios to cope with the resource limitations of the devices and the networks.
Federated learning (FL) is one of the most prominent algorithms with data distributed across devices. FL reduces communication overhead by avoiding data exchange between wireless devices and the server. Instead, each wireless device executes some local computations and communicates the local parameters to the server using wireless communications. Accordingly, every communication iteration of FL experiences costs such as computation, latency, communication resource utilization, bandwidth, and energy. Since the devices' communication and computation resources are limited, it may hinder completing the training of the FL due to the resource shortage. The main goal of this thesis is to develop cost-efficient approaches to alleviate the resource constraints of devices in FL training.
In the first chapter of the thesis, we overview ML and discuss the relevant communication and computation efficient works for training FL models. Next, a comprehensive literature review of cost efficient FL methods is conducted, and the limitations of existing literature in this area are identified. We then present the central focus of our research, which is a causal approach that eliminates the need for future FL information in the design of communication and computation efficient FL. Finally, we summarize the key contributions of each paper within the thesis.
In the second chapter, the thesis presents the articles on which it is based in their original format of publication or submission. A multi-objective optimization problem, incorporating FL loss and iteration cost functions, is proposed where communication between devices and the server is regulated by the slotted-ALOHA wireless protocol. The effect of contention level in the CSMA/CA on the causal solution of the proposed optimization is also investigated. Furthermore, the multi-objective optimization problem is extended to cover general scenarios in wireless communication, including convex and non-convex loss functions. Novel results are compared with well-known communication-efficient methods, such as the lazily aggregated quantized gradients (LAQ), to further improve the communication efficiency in FL over wireless networks.