Please use this identifier to cite or link to this item: http://hdl.handle.net/10071/32038
Author(s): Susskind, Z.
Arora, A.
Miranda, I. D. S.
Villon, L. A. Q.
Katopodis, R. F.
Araújo, L. S.
Dutra, D. L. C.
Lima, P. M. V.
França, F. M. G.
Breternitz Jr., M.
John, L. K.
Editor: Andreas Kloeckner
José Moreira
Date: 2023
Title: Weightless neural networks for efficient edge inference
Book title/volume: PACT '22: Proceedings of the International Conference on Parallel Architectures and Compilation Techniques
Pages: 279 - 290
Event title: International Conference on Parallel Architectures and Compilation Techniques
Reference: Susskind, Z., Arora, A., Miranda, I. D. S., Villon, L. A. Q., Katopodis, R. F., Araújo, L. S., Dutra, D. L. C., Lima, P. M. V., França, F. M. G., Breternitz Jr., M., & John, L. K. (2023). Weightless neural networks for efficient edge inference. In A. Kloeckner, & J. Moreira (Eds.). PACT '22: Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (pp. 279 – 290). ACM - Association for Computing Machinery. https://doi.org/10.1145/3559009.3569680
ISBN: 97978-1-4503-9868-8
DOI (Digital Object Identifier): 10.1145/3559009.3569680
Keywords: Weightless Neural Networks
WNN
WiSARD
Redes neuronais -- Neural networks
Hardware acceleration
Inferência -- Inference
Edge computing
Abstract: Weightless neural networks (WNNs) are a class of machine learning model which use table lookups to perform inference, rather than the multiply-accumulate operations typical of deep neural networks (DNNs). Individual weightless neurons are capable of learning non-linear functions of their inputs, a theoretical advantage over the linear neurons in DNNs, yet state-of-the-art WNN architectures still lag behind DNNs in accuracy on common classification tasks. Additionally, many existing WNN architectures suffer from high memory requirements, hindering implementation. In this paper, we propose a novel WNN architecture, BTHOWeN, with key algorithmic and architectural improvements over prior work, namely counting Bloom filters, hardware-friendly hashing, and Gaussian-based nonlinear thermometer encodings. These enhancements improve model accuracy while reducing size and energy per inference. BTHOWeN targets the large and growing edge computing sector by providing superior latency and energy efficiency to both prior WNNs and comparable quantized DNNs. Compared to state-of-the-art WNNs across nine classification datasets, BTHOWeN on average reduces error by more than 40% and model size by more than 50%. We demonstrate the viability of a hardware implementation of BTHOWeN by presenting an FPGA-based inference accelerator, and compare its latency and resource usage against similarly accurate quantized DNN inference accelerators, including multi-layer perceptron (MLP) and convolutional models. The proposed BTHOWeN models consume almost 80% less energy than the MLP models, with nearly 85% reduction in latency. In our quest for efficient ML on the edge, WNNs are clearly deserving of additional attention.
Peerreviewed: yes
Access type: Open Access
Appears in Collections:ISTAR-CRI - Comunicações a conferências internacionais

Files in This Item:
File SizeFormat 
conferenceObject_92348.pdf1,08 MBAdobe PDFView/Open


FacebookTwitterDeliciousLinkedInDiggGoogle BookmarksMySpaceOrkut
Formato BibTex mendeley Endnote Logotipo do DeGóis Logotipo do Orcid 

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.