Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding

Keywords

Huffman coding, Computer science, Quantization (signal processing), Artificial neural network, Speedup

References

• On the Construction of Huffman TreesJan Van Leeuwen — 1976
• Improving the speed of neural networks on CPUsVincent Vanhoucke, Andrew Senior, M. Mao — 2011
• Memory Bounded Deep Convolutional NetworksMaxwell D. Collins, Pushmeet Kohli — 2014
• Phoneme probability estimation with dynamic sparsely connected artificial neural networksNikko Ström — 1997
• Fixed point optimization of deep convolutional neural networks for object recognitionSajid Anwar, Kyuyeon Hwang, Wonyong Sung — 2015
• Very Deep Convolutional Networks for Large-Scale Image RecognitionKaren Simonyan, Andrew Zisserman — 2014
• Compressing Deep Convolutional Networks using Vector QuantizationYunchao Gong, Liu Liu, Ming–Hsuan Yang — 2014
• The conference paperLee D. Parker — 2011
• Fixed-point feedforward deep neural network design using weights +1, 0, and −1Kyuyeon Hwang, Wonyong Sung — 2014
• Going deeper with convolutionsChristian Szegedy, Wei Liu, Yangqing Jia — 2014
• Gradient-based learning applied to document recognitionYann LeCun, Léon Bottou, Yoshua Bengio — 1998
• Optimal Brain DamageYann LeCun, John S. Denker, Sara A. Solla — 1989
• Comparing Biases for Minimal Network Construction with Back-PropagationStephen José Hanson, Lorien Pratt — 1988
• Second Order Derivatives for Network Pruning: Optimal Brain SurgeonBabak Hassibi, David G. Stork — 1992
• Deep Fried ConvnetsZichao Yang, Marcin Moczulski, Misha Denil — 2014
• Caffe: Convolutional Architecture for Fast Feature EmbeddingYangqing Jia, Evan Shelhamer, Jeff Donahue — 2014
• Predicting Parameters in Deep LearningShakibi, Babak — 2013
• ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton — 2012
• Exploiting Linear Structure Within Convolutional Networks for Efficient EvaluationEmily Denton, Wojciech Zaremba, Joan Bruna — 2014
• Compressing Neural Networks with the Hashing TrickWenlin Chen, James T. Wilson, Stephen Tyree — 2015

Cited by

• Combined Edge Loss UNet for Optimized Segmentation in Total Knee Arthroplasty Preoperative PlanningDeseada Cano Ceballos — 2023
• FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute ClustersForrest Iandola, Matthew W. Moskewicz, Khalid Ashraf — 2015
• ACDC: A Structured Efficient Linear LayerMarcin Moczulski, Misha Denil, Jeremy Appleyard — 2015
• Quantized Convolutional Neural Networks for Mobile DevicesJiaxiang Wu, Cong Leng, Yuhang Wang — 2015
• Adjustable Bounded Rectifiers: Towards Deep Binary RepresentationsZhirong Wu, Dahua Lin, Tang, Xiaoou — 2015
• Resiliency of Deep Neural Networks under QuantizationWonyong Sung, Sungho Shin, Kyuyeon Hwang — 2015
• Structured Pruning of Deep Convolutional Neural NetworksSajid Anwar, Kyuyeon Hwang, Wonyong Sung — 2015
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model sizeForrest Iandola, Song Han, Matthew W. Moskewicz — 2016
• EIE: Efficient Inference Engine on Compressed Deep Neural NetworkSong Han, Xingyu Liu, Huizi Mao — 2016
• Fixed Point Quantization of Deep Convolutional NetworksDarryl Dexu Lin, Sachin S. Talathi, V. Sreekanth Annapureddy — 2015
• Virtualizing Deep Neural Networks for Memory-Efficient Neural Network DesignMinsoo Rhu, Natalia Gimelshein, Jason Clemons — 2016
• Convolutional Neural Networks using Logarithmic Data RepresentationDaisuke Miyashita, Edward H. Lee, Boris Murmann — 2016
• Blending LSTMs into CNNsKrzysztof J. Geras, Abdelrahman Mohamed, Rich Caruana — 2015
• Proteus: Exploiting Numerical Precision Variability in Deep Neural NetworksPatrick Judd, Jorge Albericio, Tayler Hetherington — 2016
• Hardware-oriented Approximation of Convolutional Neural NetworksPhilipp Gysel, Mohammad Motamedi, Soheil Ghiasi — 2016
• Optimizing convolutional neural networks on embedded platforms with OpenCLAnton Lokhmotov, Grigori Fursin — 2016
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural NetworksPhilipp Gysel — 2016
• Path-Normalized Optimization of Recurrent Neural Networks with ReLU ActivationsBehnam Neyshabur, Yuhuai Wu, Ruslan Salakhutdinov — 2016
• Functional Hashing for Compressing Neural NetworksLei Shi, Shikun Feng, Zhifan Zhu — 2016
• Ternary Weight NetworksFengfu Li, Bin Liu, Wang, Xiaoxing — 2016

Related papers

• Learning Multiple Layers of Features from Tiny ImagesAlex Krizhevsky — 2024
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision ApplicationsAndrew Howard, Menglong Zhu, Bo Chen — 2017
• Pruning Filters for Efficient ConvNetsHao Li, Asim Kadav, Igor Đurđanović — 2016
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1Matthieu Courbariaux, Itay Hubara, Daniel Soudry — 2016
• EIE: Efficient Inference Engine on Compressed Deep Neural NetworkSong Han, Xingyu Liu, Huizi Mao — 2016
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model sizeForrest Iandola, Song Han, Matthew W. Moskewicz — 2016
• Deep Residual Learning for Image RecognitionKaiming He, Xiangyu Zhang, Shaoqing Ren — 2015