File(s) under permanent embargo
Distributed Optimization of Graph Convolutional Network Using Subgraph Variance
journal contribution
posted on 2023-03-24, 01:00 authored by Taige ZhaoTaige Zhao, Xiangyu Song, Man Li, Jianxin LiJianxin Li, Wei LuoWei Luo, Imran RazzakImran RazzakIn recent years, distributed graph convolutional networks (GCNs) training frameworks have achieved great success in learning the representation of graph-structured data with large sizes. However, existing distributed GCN training frameworks require enormous communication costs since a multitude of dependent graph data need to be transmitted from other processors. To address this issue, we propose a graph augmentation-based distributed GCN framework (GAD). In particular, GAD has two main components: GAD-Partition and GAD-Optimizer . We first propose an augmentation-based graph partition (GAD-Partition) that can divide the input graph into augmented subgraphs to reduce communication by selecting and storing as few significant vertices of other processors as possible. To further speed up distributed GCN training and improve the quality of the training result, we design a subgraph variance-based importance calculation formula and propose a novel weighted global consensus method, collectively referred to as GAD-Optimizer . This optimizer adaptively adjusts the importance of subgraphs to reduce the effect of extra variance introduced by GAD-Partition on distributed GCN training. Extensive experiments on four large-scale real-world datasets demonstrate that our framework significantly reduces the communication overhead ( $\approx$ $50\%$ ), improves the convergence speed ( $\approx $ $2$ $ \times$ ) of distributed GCN training, and obtains a slight gain in accuracy ( $\approx$ $0.45\%$ ) based on minimal redundancy compared to the state-of-the-art methods.
History
Journal
IEEE Transactions on Neural Networks and Learning SystemsVolume
PPPagination
1-12Location
Piscataway, N.J.Publisher DOI
ISSN
2162-237XeISSN
2162-2388Language
engPublication classification
C1 Refereed article in a scholarly journalIssue
99Publisher
Institute of Electrical and Electronics Engineers (IEEE)Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC