DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients

Keywords

Computer science, Convolutional neural network, Net (polyhedron), Convolution (computer science), Inference

References

• Improving the speed of neural networks on CPUsVincent Vanhoucke, Andrew Senior, M. Mao — 2011
• Training deep neural networks with low precision multiplicationsMatthieu Courbariaux, Yoshua Bengio, Jean‐Pierre David — 2014
• Compressing Deep Convolutional Networks using Vector QuantizationYunchao Gong, Liu Liu, Ming–Hsuan Yang — 2014
• Scaling up machine learning: parallel and distributed approachesRon Bekkerman, Mikhail Bilenko, John Langford — 2011
• NeuFlow: Dataflow vision processing system-on-a-chipPhi-Hung Pham, Darko Jelaca, Clément Farabet — 2012
• DaDianNao: A Machine-Learning SupercomputerYunji Chen, Tao Luo, Shaoli Liu — 2014
• ImageNet: A large-scale hierarchical image databaseJia Deng, Wei Dong, Richard Socher — 2009
• DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learningTianshi Chen, Zidong Du, Ninghui Sun — 2014
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsGeoffrey E. Hinton, Li Deng, Dong Yu — 2012
• ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton — 2012
• Quantized Convolutional Neural Networks for Mobile DevicesJiaxiang Wu, Cong Leng, Yuhang Wang — 2015
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional ComputationYoshua Bengio, Nicholas Léonard, Aaron Courville — 2013
• BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1Matthieu Courbariaux, Yoshua Bengio — 2016
• Adding Gradient Noise Improves Learning for Very Deep NetworksArvind Neelakantan, Luke Vilnis, Quoc V. Le — 2015
• Bitwise Neural NetworksMinje Kim, Paris Smaragdis — 2016
• Reading Digits in Natural Images with Unsupervised Feature LearningYuval Netzer — 2011
• 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNsFrank Seide, Hao Fu, Jasha Droppo — 2014
• Deep neural networks are robust to weight binarization and other non-linear distortionsPaul Merolla, Rathinakumar Appuswamy, John V. Arthur — 2016
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftSergey Ioffe, Christian Szegedy — 2015
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural NetworksMohammad Rastegari, Vicente Ordóñez, Joseph Redmon — 2016

Cited by

• Dissecting Xeon + FPGA: Why the integration of CPUs and FPGAs makes a power difference for the datacenter: Invited PaperHerman Schmit, Randy Huang — 2016
• Ternary neural networks for resource-efficient AI applicationsHande Alemdar, Vincent Leroy, Adrien Prost-Boucle — 2016
• Loss-aware Binarization of Deep NetworksLu Hou, Quanming Yao, James T. Kwok — 2016
• Effective Quantization Methods for Recurrent Neural NetworksQinyao He, He Wen, Shuchang Zhou — 2016
• Training Bit Fully Convolutional Network for Fast Semantic SegmentationHe Wen, Shuchang Zhou, Zhe Liang — 2016
• FINN: A Framework for Fast, Scalable Binarized Neural Network InferenceYaman Umuroglu, Nicholas J. Fraser, Giulio Gambardella — 2016
• Scaling Binarized Neural Networks on Reconfigurable LogicNicholas J. Fraser, Yaman Umuroglu, Giulio Gambardella — 2017
• Mixed Low-precision Deep Learning Inference using Dynamic Fixed PointNaveen Mellempudi, Abhisek Kundu, Dipankar Das — 2017
• Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAsRitchie Zhao, Weinan Song, Wentao Zhang — 2017
• Deep Learning with Low Precision by Half-Wave Gaussian QuantizationZhaowei Cai, Xiaodong He, Jian Sun — 2017
• A Power-Aware Digital Multilayer Perceptron Accelerator with On-Chip Training Based on Approximate ComputingDuck-Hwan Kim, Jaeha Kung, Saibal Mukhopadhyay — 2017
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision WeightsAojun Zhou, Anbang Yao, Yiwen Guo — 2017
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point ActivationsLiangzhen Lai, Naveen Suda, Vikas Chandra — 2017
• The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep LearningHantian Zhang, Li, Jerry, Kaan Kara — 2016
• Efficient Processing of Deep Neural Networks: A Tutorial and SurveyVivienne Sze, Yu‐Hsin Chen, Tien-Ju Yang — 2017
• Learning Convolutional Networks for Content-Weighted Image CompressionMu Li, Wangmeng Zuo, Shuhang Gu — 2017
• How to Train a Compact Binary Neural Network with High Accuracy?Wei Tang, Gang Hua, Liang Wang — 2017
• More is Less: A More Complicated Network with Less Inference ComplexityXuanyi Dong, Junshi Huang, Yi Yang — 2017
• Pyramid Vector Quantization for Deep LearningVincenzo Liguori — 2017
• Ternary Neural Networks with Fine-Grained QuantizationNaveen Mellempudi, Abhisek Kundu, Dheevatsa Mudigere — 2017

Related papers

• Physical design tradeoffs for ASIC technologiesJ. Banker, Anil Shanbhag, Naveed A. Sherwani — 1993
• Structured ASIC, evolution or revolution?Kun-cheng Wu, Yu-Wen Tsai — 2004
• デジタル/アナログ混載ASIC設計技術 (ASICアプリケ-ションガイド--ヒット商品を生み出すASIC開発設計ノウハウ ) -- (ASIC設計ノウハウの全て)茂 川田 — 1989
• Testing of application specific integrated circuit in LabVIEW environmentP. Maj — 2006
• Structured ASIC: Methodology and comparisonSam M. H. Ho, Steve C. L. Yuen, Hiu Ching Poon — 2010
• An Adaptable Real-Time Object Detection for Traffic Surveillance using R-CNN over CNN with Improved AccuracyG. Vinod, G. Padmapriya — 2022
• The Compatible Design Between FPGA and ASICXing Ya-nan — 2011
• Design of Convolutional Neural Network Based on Reticulated Convolution ModuleDaihui Li, Yang Lei, Zeng Shangyou — 2019