ACM Transactions on Embedded Computing Systems, 2023
Facial Expression Recognition (FER) task has received significant attention in the computer visio... more Facial Expression Recognition (FER) task has received significant attention in the computer vision field. However, due to low computation resources, how to deploy Deep Neural Networks (DNNs) for FER on resource-constrained edge devices remains a major issue. In this work, we start with ideas of pushing AI to edges by designing Edge-AI driven framework for the FER task. Meanwhile, edge nodes perform lowlatency and private inference, thus keeping balance between edge computing resources and FER accuracy. Moreover, to address the challenges brought by recognizing in the wild, i.e., realistic occlusion, variations in pose, illumination, and scales, we design two novel attention modules, Arbitrary-oriented Spatial Pooling (ASP) and Scalable Frequency Pooling (SFP), to extract distinctive feature representations for the FER task. The ASP module aggregates positional information into feature channels in different directions to solve the problem of pose variations, while the SFP module moves to the wavelet frequency domain for capturing unique and scalable feature changes, thus dealing with difficulties of realistic occlusion, illumination, and scale variations. Performance evaluation with several datasets, including FER2013, RaFD and SFEW, demonstrates the effectiveness of our proposed method in terms of both accuracy and computation efficiency. CCS Concepts: • Computer systems organization → Real-time systems; • Computing methodologies → Machine learning algorithms.
In an open world with a long-tail distribution of input samples, Deep Neural Networks (DNNs) may ... more In an open world with a long-tail distribution of input samples, Deep Neural Networks (DNNs) may make unpredictable mistakes for Out-of-Distribution (OOD) inputs at test time, despite high levels of accuracy obtained during model training. OOD detection can be an effective runtime assurance mechanism for safe deployment of machine learning algorithms in safety-critical applications such as medical imaging and autonomous driving. A multitude of different OOD detection algorithms have been developed, with a wide range of performance metrics in terms of accuracy and execution time. For real-time safety-critical applications, e.g., autonomous driving, timing performance is of great importance in addition to accuracy. We perform a comprehensive and systematic benchmark study of multiple OOD detection algorithms in terms of both accuracy and execution time on different hardware platforms, including a powerful workstation and a resource-constrained embedded device, equipped with both CPU and GPU. We also perform detailed analysis of different components of each algorithm to identify the performance bottlenecks and potential for GPU acceleration. This paper aims to provide a useful reference for the practical deployment of OOD detection algorithms for real-time safety-critical applications.
ACM Transactions on Embedded Computing Systems, 2023
Facial Expression Recognition (FER) task has received significant attention in the computer visio... more Facial Expression Recognition (FER) task has received significant attention in the computer vision field. However, due to low computation resources, how to deploy Deep Neural Networks (DNNs) for FER on resource-constrained edge devices remains a major issue. In this work, we start with ideas of pushing AI to edges by designing Edge-AI driven framework for the FER task. Meanwhile, edge nodes perform lowlatency and private inference, thus keeping balance between edge computing resources and FER accuracy. Moreover, to address the challenges brought by recognizing in the wild, i.e., realistic occlusion, variations in pose, illumination, and scales, we design two novel attention modules, Arbitrary-oriented Spatial Pooling (ASP) and Scalable Frequency Pooling (SFP), to extract distinctive feature representations for the FER task. The ASP module aggregates positional information into feature channels in different directions to solve the problem of pose variations, while the SFP module moves to the wavelet frequency domain for capturing unique and scalable feature changes, thus dealing with difficulties of realistic occlusion, illumination, and scale variations. Performance evaluation with several datasets, including FER2013, RaFD and SFEW, demonstrates the effectiveness of our proposed method in terms of both accuracy and computation efficiency. CCS Concepts: • Computer systems organization → Real-time systems; • Computing methodologies → Machine learning algorithms.
In an open world with a long-tail distribution of input samples, Deep Neural Networks (DNNs) may ... more In an open world with a long-tail distribution of input samples, Deep Neural Networks (DNNs) may make unpredictable mistakes for Out-of-Distribution (OOD) inputs at test time, despite high levels of accuracy obtained during model training. OOD detection can be an effective runtime assurance mechanism for safe deployment of machine learning algorithms in safety-critical applications such as medical imaging and autonomous driving. A multitude of different OOD detection algorithms have been developed, with a wide range of performance metrics in terms of accuracy and execution time. For real-time safety-critical applications, e.g., autonomous driving, timing performance is of great importance in addition to accuracy. We perform a comprehensive and systematic benchmark study of multiple OOD detection algorithms in terms of both accuracy and execution time on different hardware platforms, including a powerful workstation and a resource-constrained embedded device, equipped with both CPU and GPU. We also perform detailed analysis of different components of each algorithm to identify the performance bottlenecks and potential for GPU acceleration. This paper aims to provide a useful reference for the practical deployment of OOD detection algorithms for real-time safety-critical applications.
Uploads
Papers