Fixed-point feedforward deep neural network design using weights +1, 0, and −1

Keywords

Backpropagation, Fixed point, Computer science, Artificial neural network, Feedforward neural network

References

• 1998 IEEE International Conference on SMCに参加して鶴岡 政子 — 1998
• Neural Network Adaptations to Hardware ImplementationsPerry D. Moerland, Emile Fiesler — 1997
• Improving neural networks by preventing co-adaptation of feature detectorsGeoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky — 2012
• X1000 real-time phoneme recognition VLSI using feed-forward deep neural networksJonghong Kim, Kyuyeon Hwang, Wonyong Sung — 2014
• Acoustic Modeling Using Deep Belief NetworksAbdelrahman Mohamed, George E. Dahl, Geoffrey E. Hinton — 2012
• Theory of the backpropagation neural networkRobert Hecht-Nielsen — 1989
• Weight discretization paradigm for optical neural networksEmile Fiesler, A. Choudry, H. John Caulfield — 1990
• Improving deep neural networks for LVCSR using rectified linear units and dropoutGeorge E. Dahl, Tara N. Sainath, Geoffrey E. Hinton — 2013
• Speaker-independent phone recognition using hidden Markov modelsK.-F. Lee, Hsiao-Wuen Hon — 1989
• The effects of quantization on multilayer neural networksGünhan Dündar, Kenneth Rose — 1995
• Finite Precision Error Analysis of Neural Network Hardware ImplementationsJ.L. Holt, Jenq–Neng Hwang — 1993
• Exploiting sparseness in deep neural networks for large vocabulary speech recognitionDong Yu, Frank Seide, Gang Li — 2012
• A Fast Learning Algorithm for Deep Belief NetsGeoffrey E. Hinton, Simon Osindero, Yee‐Whye Teh — 2006
• Multilayer feedforward neural networks with single powers-of-two weightsC.Z. Tang, Hon Keung Kwan — 1993
• Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionGeorge E. Dahl, Dong Yu, Li Deng — 2012
• Simple Method for High-Performance Digit Recognition Based on Sparse CodingKai Labusch, Erhardt Barth, Thomas Martinetz — 2008
• Comparison of classifier methods: a case study in handwritten digit recognitionLéon Bottou, Corinna Cortes, J. S. Denker — 1994
• Simulation-based word-length optimization method for fixed-point digital signal processing systemsWonyong Sung, Ki-Il Kum — 1995
• Supporting Online Material for Reducing the Dimensionality of Data with Neural NetworksGE Hinton, Russ R. Salakhutdinov — 2006
• Theory of the backpropagation neural networkHecht-Nielsen — 1989

Cited by

• Fixed point optimization of deep convolutional neural networks for object recognitionSajid Anwar, Kyuyeon Hwang, Wonyong Sung — 2015
• BinaryConnect: Training Deep Neural Networks with binary weights during propagationsMatthieu Courbariaux, Yoshua Bengio, Jean‐Pierre David — 2015
• Bridge deep learning to the physical world: An efficient method to quantize networkPei-Hen Hung, Chia‐Han Lee, Shao‐Wen Yang — 2015
• Reduced-Precision Strategies for Bounded Memory in Deep Neural NetsPatrick Judd, Jorge Albericio, Tayler Hetherington — 2015
• BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1Matthieu Courbariaux, Yoshua Bengio — 2016
• Fixed-point performance analysis of recurrent neural networksSungho Shin, Kyuyeon Hwang, Wonyong Sung — 2015
• Resiliency of Deep Neural Networks under QuantizationWonyong Sung, Sungho Shin, Kyuyeon Hwang — 2015
• Bitwise Neural NetworksMinje Kim, Paris Smaragdis — 2016
• Structured Pruning of Deep Convolutional Neural NetworksSajid Anwar, Kyuyeon Hwang, Wonyong Sung — 2015
• Fixed Point Quantization of Deep Convolutional NetworksDarryl Dexu Lin, Sachin S. Talathi, V. Sreekanth Annapureddy — 2015
• FPGA based implementation of deep neural networks using on-chip memory onlyJin-Hwan Park, Wonyong Sung — 2016
• Deep Adaptive Network: An Efficient Deep Neural Network with Sparse Binary ConnectionsXichuan Zhou, Shengli Li, A. K. Qin — 2016
• Learning separable fixed-point kernels for deep convolutional neural networksSajid Anwar, Kyuyeon Hwang, Wonyong Sung — 2016
• Improving energy efficiency and classification accuracy of neuromorphic chips by learning binary synaptic crossbarsAntonio Jimeno Yepes, Jianbin Tang — 2016
• Ternary Weight NetworksFengfu Li, Bin Liu, Wang, Xiaoxing — 2016
• On the Efficient Representation and Execution of Deep Acoustic ModelsRaziel Álvarez, Rohit Prabhavalkar, Anton Bakhtin — 2016
• Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networksSungho Shin, Wonyong Sung — 2016
• Audio computing in the wild: frameworks for big data and small computersMinje Kim — 2016
• FPGA-Based Low-Power Speech Recognition with Recurrent Neural NetworksMinjae Lee, Kyuyeon Hwang, Jin-Hwan Park — 2016
• Compact Deep Convolutional Neural Networks With Coarse PruningSajid Anwar, Wonyong Sung — 2016

Related papers

• FIR and IIR Synapses, a New Neural Network Architecture for Time Series ModelingAndrew D. Back, Ah Chung Tsoi — 1991
• OMBP: Optic Modified BackPropagation training algorithm for fast convergence of Feedforward Neural NetworkOmar Charif, Hichem Omrani, Philippe Trigano — 2011
• Backpropagation (neural) networks for fast pre‐evaluation of spectroscopic ellipsometric measurementsM. Fried, P. Masa — 1994
• A design approach for feedback-feedforward control systemsMorteza Mohammadzaheri, L Chen Member — 2010
• A Modified Backpropagation Training Algorithm for Feedforward Neural Networks*T. Kathirvalavakumar, P. Thangavel — 2006
• 신경회로망의 쟈코비안을 이용한 Feedforward/Feedback 병합제어기 설계 ( The Combined Feedforward/Feedback Controller Design Using Jacobians of Neural Network )조규상, 임제탁 — 1996
• Control-relevant neural networks for feedforward control with preview: Applied to an industrial flatbed printerLeontine Aarnoudse, Johan Kon, Wataru Ohnishi — 2024
• Fast Terminal Attractor Based Backpropagation Algorithm For Feedforward Neural NetworksBatsukh Batbayar, Xinghuo Yu — 2007