Stochastic, Distributed and Federated Optimization for Machine Learning
Explore this paper's citation graph
Summary
This work proposes novel variants of stochastic gradient descent with a variance reduction property that enables linear convergence for strongly convex objectives in distributed setting, and introduces the concept of Federated Optimization/Learning, where the main motivation comes from industry when handling user-generated data.
- Type
- preprint
- Published
- 2017-07-04
- Cited by
- 38
- References
- 206
- Access
- Open access
- OpenAlex
- https://openalex.org/W2729686365
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:20470110
Keywords
Computer science, Distributed learning, Federated learning, Distributed computing, Stochastic optimization
References
- Semi-stochastic coordinate descent
- Primal Method for ERM with Flexible Mini-batching Schemes and Non-convex Losses
- Distributed Stochastic Variance Reduced Gradient Methods
- Project Adam: Building an Efficient and Scalable Deep Learning Training System
- Semi-Stochastic Gradient Descent Methods
- Problems in decentralized decision making and computation
- On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers
- Simple Complexity Analysis of Simplified Direct Search
- Distributed Mini-Batch SDCA
- Convergence of Stochastic Proximal Gradient Algorithm
- Inexact Coordinate Descent: Complexity and Preconditioning
- DiSCO: Distributed Optimization for Self-Concordant Empirical Loss
- Hybrid Deterministic-Stochastic Methods for Data Fitting
- Minimizing finite sums with the stochastic average gradient
- SDCA without Duality
- Stochastic Dual Coordinate Ascent with Adaptive Probabilities
- Stochastic Optimization with Importance Sampling for Regularized Loss Minimization
- Distributed Box-Constrained Quadratic Optimization for Dual Linear SVM
- Perturbed Iterate Analysis for Asynchronous Stochastic Optimization
Cited by
- Securing Collaborative Deep Learning in Industrial Applications Within Adversarial Scenarios
- Data Encoding for Byzantine-Resilient Distributed Gradient Descent
- Qsparse-Local-SGD: Distributed SGD With Quantization, Sparsification, and Local Computations
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- Data Encoding for Byzantine-Resilient Distributed Optimization
- Distributed Machine Learning on Mobile Devices: A Survey
- Byzantine-Tolerant Distributed Coordinate Descent
- Data Encoding Methods for Byzantine-Resilient Distributed Optimization
- Communication Efficient Decentralized Training with Multiple Local Updates
- On the Convergence of FedAvg on Non-IID Data
- Federated Patient Hashing
- A Survey on Edge Intelligence
- Byzantine-Resilient SGD in High Dimensions on Heterogeneous Data
- Byzantine-Resilient High-Dimensional SGD with Local Iterations on Heterogeneous Data
- Byzantine-Resilient High-Dimensional Federated Learning
- On Byzantine-Resilient High-Dimensional Stochastic Gradient Descent
- Communication-Efficient Collaborative Learning of Geo-Distributed JointCloud from Heterogeneous Datasets
- Evaluating system architectures for driving range estimation and charge planning for electric vehicles
- Federated Learning with Heterogeneous Quantization
- Tradeoff between Model Accuracy and Cost for Federated Learning in the Mobile Edge Computing Systems
Related papers
- Accelerating Fair Federated Learning: Adaptive Federated Adam
- Communication-Efficient Adaptive Federated Learning
- Research on Aggregation Strategy of Federated Learning Parameters under Non-Independent and Identically Distributed Conditions
- Model-Agnostic Round-Optimal Federated Learning via Knowledge Transfer
- Communication-Efficient Agnostic Federated Averaging
- Towards Mobile Federated Learning with Unreliable Participants and Selective Aggregation
- Server Averaging for Federated Learning