Hardware accelerator for high accuracy sign language recognition with residual network based on FPGAs

LETTER

Dong Yang, Advanced Technology Research Institute, Beijing Institute of Technology">Jianwu Li, Guocun Hao, Qirui Chen, Xi Wei, Zirui Dai, Zixian Hou, Lei Zhang, School of Integrated Circuits and Electronics, Beijing Institute of Technology">Xiaoran Li

Author information

Dong Yang
Zhengzhou Key Laboratory of Electronic Information Functional Materials and Devices, Huanghe Science and Technology College
Jianwu Li
Zhengzhou Key Laboratory of Electronic Information Functional Materials and Devices, Huanghe Science and Technology College
Advanced Technology Research Institute, Beijing Institute of Technology
Guocun Hao
School of Integrated Circuits and Electronics, Beijing Institute of Technology
Qirui Chen
School of Integrated Circuits and Electronics, Beijing Institute of Technology
Xi Wei
School of Integrated Circuits and Electronics, Beijing Institute of Technology
Zirui Dai
School of Integrated Circuits and Electronics, Beijing Institute of Technology
Zixian Hou
School of Integrated Circuits and Electronics, Beijing Institute of Technology
Lei Zhang
School of Integrated Circuits and Electronics, Beijing Institute of Technology
Xiaoran Li
Advanced Technology Research Institute, Beijing Institute of Technology
School of Integrated Circuits and Electronics, Beijing Institute of Technology

Keywords: convolutional neural network (CNN), field programmable gate array (FPGA), residual network, high-level synthesis (HLS), hardware acceleration

JOURNAL FREE ACCESS

2024 Volume 21 Issue 4 Pages 20230579

DOI https://doi.org/10.1587/elex.21.20230579

Browse “Advance online publication” version

Details

Abstract

The ResNet series of networks has demonstrated powerful capabilities in the fields of object detection and image classification, garnering increasing attention from researchers. However, due to their deep network architectures, accelerator development based on FPGA faces challenges associated with limited on-chip resources and lengthy design cycles. This paper presents a versatile hardware acceleration system based on FPGA, achieving optimization through both hardware implementation and software inference architecture. The system reduces network complexity by employing techniques such as inter-layer fusion and dynamic quantization, while enhancing hardware resource utilization through channel parallelism and tightly-pipelined hardware design principles. By configuring and reusing computation units, the forward inference process of ResNet series networks can be rapidly deployed on FPGA, shortening the development and validation cycles. The proposed system is validated using the ResNet-18 model on a PYNQ-Z2 development board within a gesture recognition application scenario. The overall power consumption of the system is 2.136W, with hardware inference accuracy reaching 98.87%.

Corresponding author

Register with J-STAGE for free!