Knowledge Squeezed Adversarial Network Compression
Explore this paper's citation graph
Summary
This work proposes a knowledge transfer method, involving effective intermediate supervision, under the adversarial training framework to learn the student network, and demonstrates that the proposed method achieves highly superior performances against other state-of-the-art methods.
- Type
- preprint
- Published
- 2019-04-10
- Cited by
- 12
- References
- 45
- Access
- Open access
- OpenAlex
- https://openalex.org/W2938777694
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:131773901
Keywords
Computer science, Pipeline (software), Adversarial system, Benchmark (surveying), Process (computing)
References
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Distilling the Knowledge in a Neural Network
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- Is object localization for free? - Weakly-supervised learning with convolutional neural networks
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Learning Complex, Extended Sequences Using the Principle of History Compression
- Learning Separable Filters
- Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
- Learning to combine foveal glimpses with a third-order Boltzmann machine
- Deep Fried Convnets
- Learning Where to Attend with Deep Architectures for Image Tracking
- Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous or Discrete Weights
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Convolutional neural networks with low-rank regularization
- Deep Residual Learning for Image Recognition
- Quantized Convolutional Neural Networks for Mobile Devices
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size
- Recurrent Human Pose Estimation
- Learning small-size DNN with output-distribution-based criteria
- Face Model Compression by Distilling Knowledge from Neurons
Cited by
- SDChannelNets: Extremely Small and Efficient Convolutional Neural Networks
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks
- Towards Efficient Unconstrained Palmprint Recognition via Deep Distillation Hashing
- Knowledge Distillation: A Survey
- Adversarial Deep Mutual Learning
- Rectified Binary Convolutional Networks with Generative Adversarial Learning
- Source-Free Domain Adaptation for Semantic Segmentation
- Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization
- Locality Guidance for Improving Vision Transformers on Tiny Datasets
- MKTN: Adversarial-Based Multifarious Knowledge Transfer Network from Complementary Teachers
- Class-dependent Compression of Deep Neural Networks
- Local Correlation Consistency for Knowledge Distillation