research-article

Optimization and deployment of CNNs at the edge: the ALOHA experience

Authors:

Gianfranco Deriu,

Andy D. Pimentel,

Todor Stefanov,

Svetlana Minakova,

Francesco Conti,

Battista Biggio,

Bernhard Moser,

Natalia Shepeleva,

Nikos Fragoulis,

Ilias Theodorakopoulos,

Francesca PalumboAuthors Info & Claims

CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

Pages 326 - 332

https://doi.org/10.1145/3310273.3323435

Published: 30 April 2019 Publication History

Abstract

Deep learning (DL) algorithms have already proved their effectiveness on a wide variety of application domains, including speech recognition, natural language processing, and image classification. To foster their pervasive adoption in applications where low latency, privacy issues and data bandwidth are paramount, the current trend is to perform inference tasks at the edge. This requires deployment of DL algorithms on low-energy and resource-constrained computing nodes, often heterogenous and parallel, that are usually more complex to program and to manage without adequate support and experience. In this paper, we present ALOHA, an integrated tool flow that tries to facilitate the design of DL applications and their porting on embedded heterogenous architectures. The proposed tool flow aims at automating different design steps and reducing development costs. ALOHA considers hardware-related variables and security, power efficiency, and adaptivity aspects during the whole development process, from pre-training hyperparameter optimization and algorithm configuration to deployment.

References

[1]

Onur Derin, Emanuele Cannella, Giuseppe Tuveri, Paolo Meloni, Todor Stefanov, Leandro Fiorin, Luigi Raffo, and Mariagiovanna Sami. 2013. A system-level approach to adaptivity and fault-tolerance in NoC-based MPSoCs: The MADNESS project. Microprocessors and Microsystems 37, 6--7 (2013), 515--529.

Digital Library

[2]

Battista Biggio et al. 2013. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Part III (LNCS), Vol. 8190. Springer Berlin Heidelberg, 387--402.

Digital Library

[3]

Christian Szegedy et al. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations.

[4]

Jungwook Choi et al. 2018. PACT: Parameterized Clipping Activation for Quantized Neural Networks. CoRR abs/1805.06085 (2018). arXiv:1805.06085 http://arxiv.org/abs/1805.06085

[5]

Xu Xiaowei et al. 2018. Scaling for edge inference of deep neural networks. Nature Electronics 1, 4 (Apr 2018), 216--222.

[6]

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations.

[7]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep Learning. Nature 521 (2015), 436--444.

[8]

Michael Masin, Lio Limonad, Aviad Sela, David Boaz, Lev Greenberg, Nir Mashkif, and Ran Rinat. 2013. Pluggable Analysis Viewpoints for Design Space Exploration. Procedia Computer Science 16 (2013), 226--235.

[9]

Paolo Meloni, Alessandro Capotondi, Gianfranco Deriu, Michele Brian, Francesco Conti, Davide Rossi, Luigi Raffo, and Luca Benini. 2018. NEURAghe: Exploiting CPU-FPGA Synergies for Efficient and Flexible CNN Inference Acceleration on Zynq SoCs. ACM Transactions on Reconfigurable Technology and Systems 11, 3 (2018), 1--24.

Digital Library

[10]

Danilo Pani, Paolo Meloni, Giuseppe Tuveri, Francesca Palumbo, Massobrio Paolo, and Luigi Raffo. 2017. An FPGA Platform for Real-Time Simulation of Spiking Neuronal Networks. Frontiers in Neuroscience 11 (2017), 90.

[11]

Andy D. Pimentel, Cagkan Erbas, and Simon Polstra. 2006. A systematic approach to exploring embedded system architectures at multiple abstraction levels. IEEE Trans. Comput. 55, 2 (Feb 2006), 99--112.

Digital Library

[12]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2014. ImageNet Large Scale Visual Recognition Challenge. CoRR abs/1409.0575 (2014). arXiv:1409.0575 http://arxiv.org/abs/1409.0575

[13]

Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge Computing: Vision and Challenges. IEEE Internet Things J. 3, 5 (2016), 637--646.

[14]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large Scale Image Recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556

[15]

Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel Emer. 2017. Efficient Processing of Deep Neural Networks: A Tutorial and Survey. Proc. IEEE 105, 12 (2017), 2295--2329.

[16]

Ilias Theodorakopoulos, V. Pothos, Dimitris Kastaniotis, and Nikos Fragoulis. 2017. Parsimonious Inference on Convolutional Neural Networks: Learning and applying on-line kernel activation rules. CoRR abs/1701.05221 (2017). http://arxiv.org/abs/1701.05221

[17]

Stylianos I. Venieris, Alexandros Kouris, and Christos-Savvas Bouganis. 2018. Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions. ACM Comput. Surv. 51, 3, Article 56 (June 2018), 39 pages.

Digital Library

Cited By

Pandey SSiddhu LPanda P(2023)NeuroCool: Dynamic Thermal Management of 3D DRAM for Deep Neural Networks through Customized PrefetchingACM Transactions on Design Automation of Electronic Systems10.1145/363001229:1(1-35)Online publication date: 18-Dec-2023
https://dl.acm.org/doi/10.1145/3630012
Diaz-de-Arcaya JTorre-Bastida AZárate GMiñón RAlmeida A(2023)A Joint Study of the Challenges, Opportunities, and Roadmap of MLOps and AIOps: A Systematic SurveyACM Computing Surveys10.1145/362528956:4(1-30)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3625289
Meindl RMoser B(2023)Measuring Overhead Costs of Federated Learning Systems by EavesdroppingDatabase and Expert Systems Applications - DEXA 2023 Workshops10.1007/978-3-031-39689-2_4(33-42)Online publication date: 21-Aug-2023
https://doi.org/10.1007/978-3-031-39689-2_4
Show More Cited By

Index Terms

Optimization and deployment of CNNs at the edge: the ALOHA experience
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems
2. Hardware
  1. Integrated circuits
    1. Reconfigurable logic and FPGAs
      1. Hardware accelerators

Recommendations

FPGA-based accelerator for object detection: a comprehensive survey
Abstract
Object detection is one of the most challenging tasks in computer vision. With the advances in semiconductor devices and chip technology, hardware accelerators have been widely used. Field-programmable gate arrays (FPGAs) are a highly flexible ...
POLYBiNN: Binary Inference Engine for Neural Networks using Decision Trees
Abstract
Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) have gained significant popularity in several classification and regression applications. The massive computation and memory requirements of DNN and CNN architectures pose ...
Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs
FPGA '19: Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

Using FPGAs to accelerate ConvNets has attracted significant attention in recent years. However, FPGA accelerator design has not leveraged the latest progress of ConvNets. As a result, the key application characteristics such as frames-per-second (FPS) ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

April 2019

414 pages

ISBN:9781450366854

DOI:10.1145/3310273

General Chairs:
Francesca Palumbo
Universit`a degli Studi di Sassari, IT
,
Michela Becchi
North Carolina State University
,
Program Chairs:
Martin Schulz
Technical University of Munich, DE
,
Kento Sato
RIKEN R-CCS, JP

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMICRO: ACM Special Interest Group on Microarchitectural Research and Processing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

H2020 LEIT Information and Communication Technologies

Conference

CF '19

Sponsor:

SIGMICRO

CF '19: Computing Frontiers Conference

April 30 - May 2, 2019

Alghero, Italy

Acceptance Rates

Overall Acceptance Rate 273 of 785 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
328
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)6

Reflects downloads up to 04 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pandey SSiddhu LPanda P(2023)NeuroCool: Dynamic Thermal Management of 3D DRAM for Deep Neural Networks through Customized PrefetchingACM Transactions on Design Automation of Electronic Systems10.1145/363001229:1(1-35)Online publication date: 18-Dec-2023
https://dl.acm.org/doi/10.1145/3630012
Diaz-de-Arcaya JTorre-Bastida AZárate GMiñón RAlmeida A(2023)A Joint Study of the Challenges, Opportunities, and Roadmap of MLOps and AIOps: A Systematic SurveyACM Computing Surveys10.1145/362528956:4(1-30)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3625289
Meindl RMoser B(2023)Measuring Overhead Costs of Federated Learning Systems by EavesdroppingDatabase and Expert Systems Applications - DEXA 2023 Workshops10.1007/978-3-031-39689-2_4(33-42)Online publication date: 21-Aug-2023
https://doi.org/10.1007/978-3-031-39689-2_4
Kum SOh SYeom JMoon J(2022)Optimization of Edge Resources for Deep Learning Application with Batch and Model ManagementSensors10.3390/s2217671722:17(6717)Online publication date: 5-Sep-2022
https://doi.org/10.3390/s22176717
Busia PDeriu GRinelli LChesta CRaffo LMeloni P(2022)Target-Aware Neural Architecture Search and Deployment for Keyword SpottingIEEE Access10.1109/ACCESS.2022.316693910(40687-40700)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3166939
Mendis HKang CHsiu P(2021)Intermittent-Aware Neural Architecture SearchACM Transactions on Embedded Computing Systems10.1145/347699520:5s(1-27)Online publication date: 17-Sep-2021
https://dl.acm.org/doi/10.1145/3476995
Theodorakopoulos IEconomou G(2021)Semantic Information in Gating Patterns of Dynamic Convolutional Neural Networks2021 12th International Conference on Information, Intelligence, Systems & Applications (IISA)10.1109/IISA52424.2021.9555567(1-8)Online publication date: 12-Jul-2021
https://doi.org/10.1109/IISA52424.2021.9555567
Buckhorst AMontavon BWolfschläger DBuchsbaum MShahidi APetruck HKunze IPennekamp JBrecher CHüsing MCorves BNitsch VWehrle KSchmitt R(2021)Holarchy for line-less mobile assembly systems operation in the context of the internet of productionProcedia CIRP10.1016/j.procir.2021.03.06499(448-453)Online publication date: 2021
https://doi.org/10.1016/j.procir.2021.03.064
Sapra DPimentel A(2021)Designing convolutional neural networks with constrained evolutionary piecemeal trainingApplied Intelligence10.1007/s10489-021-02679-752:15(17103-17117)Online publication date: 30-Jul-2021
https://doi.org/10.1007/s10489-021-02679-7
Buchgeher GCzech GRibeiro AKloihofer WMeloni PBusia PDeriu GPintor MBiggio BChesta CRinelli LSolans DPortela M(2021)Task-Specific Automation in Deep Learning ProcessesDatabase and Expert Systems Applications - DEXA 2021 Workshops10.1007/978-3-030-87101-7_16(159-169)Online publication date: 20-Sep-2021
https://doi.org/10.1007/978-3-030-87101-7_16
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents