Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds
Explore this paper's citation graph
Summary
An efficient coresets-based neural network compression algorithm that sparsifies the parameters of a trained fully-connected neural network in a manner that provably approximates the network's output is presented.
- Type
- preprint
- Published
- 2018-04-15
- Cited by
- 88
- References
- 66
- Access
- Open access
- OpenAlex
- https://openalex.org/W2797713747
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:4885767
Keywords
Generalization, Leverage (statistics), Artificial neural network, Computer science, Deep neural networks
References
- Fast ConvNets Using Group-Wise Brain Damage
- High dimensional probability : proceedings of the Fourth International Conference
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Compressing Deep Convolutional Networks using Vector Quantization
- Fully convolutional networks for semantic segmentation
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
- An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections
- Sparser Johnson-Lindenstrauss Transforms
- A unified framework for approximating and clustering data
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Fast sparse matrix multiplication
- Universal ε-approximators for integrals
- Feature hashing for large scale multitask learning
- A sparse Johnson: Lindenstrauss transform
- Hash Kernels for Structured Data
- Gradient-based learning applied to document recognition
- Optimal Brain Damage
- Scalable Training of Mixture Models via Coresets
- Predicting Parameters in Deep Learning
- ImageNet classification with deep convolutional neural networks
Cited by
- Wasserstein Coresets for Lipschitz Costs
- Multi-resolution Hashing for Fast Pairwise Summations
- Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data
- Revisiting hard thresholding for DNN pruning
- Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness
- On Activation Function Coresets for Network Pruning
- Core‐sets: An updated survey
- SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
- Global Capacity Measures for Deep ReLU Networks via Path Sampling
- Provable Filter Pruning for Efficient Neural Networks
- Machine, Unlearning
- Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network
- Data-Independent Neural Pruning via Coresets
- On Coresets for Support Vector Machines
- Coresets for the Nearest-Neighbor Rule
- Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
- Wasserstein Measure Coresets.
- Data-Independent Structured Pruning of Neural Networks via Coresets
- Neural Architecture Search Using Stable Rank of Convolutional Layers
- A Deeper Look at the Layerwise Sparsity of Magnitude-based Pruning
Related papers
- On Activation Function Coresets for Network Pruning
- Empirical estimation of generalization ability of neural networks
- SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
- Towards a Theoretical Understanding of Hashing-Based Neural Nets
- Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization
- Coresets Application in Channel Pruning for Fast Neural Network Slimming
- A Hidden Node Value Regulation Algorithm for Neural Network Training
- Dynamic learning with neural networks and support vector machines