The segmentation of mouth and lips is a fundamental problem in facial image analyisis. In this pa... more The segmentation of mouth and lips is a fundamental problem in facial image analyisis. In this paper we propose a method for lip segmentation based on rg-color histogram. Statistical analysis shows, using the rg-color-space is optimal for this purpose of a pure color based segmentation. Initially a rough adaptive threshold selects a histogram region, that assures that all pixels in that region are skin pixels. Based on that pixels we build a gaussian model which represents the skin pixels distribution and is utilized to obtain a refined, optimal threshold. We are not incorporating shape or edge information. In experiments we show the performance of our lip pixel segmentation method compared to the ground truth of our dataset and a conventional watershed algorithm.
This paper describes a novel method for analyzing single faces of non-cooperative persons on the ... more This paper describes a novel method for analyzing single faces of non-cooperative persons on the basis of stereoscopic color images. The challenges arise from the fact that the persons observed are non-cooperative, which in turn complicates further processing as facial feature extraction and tracking in image sequence. In our method, face detection is based on color-driven clustering of 3D points derived from stereo. A mesh model is registered with a post-processed face cluster, using a variant of the Iterative Closest Point algorithm (ICP). The pose is derived from correspondence. Then, the pose and model information are used for face normalization and facial feature localization. Automatic extraction of facial features is carried out using modified Active Shape Models (ASM). In contrast to the simple ASM, another approach is pursued in this work. It involves two modifications to the ASM, which lead to greater stability and robustness. The results show that stereo and color are pow...
In this paper we present a user independent real-time capable automatic method for recognition of... more In this paper we present a user independent real-time capable automatic method for recognition of facial expressions related to basic emotions from stereo image sequences. The method automatically detects faces in unconstraint pose based on depth and color information. In order to overcome difficulties caused by increasing change in pose, lighting transitions, or complicated background, we introduce a face normalization algorithm based on an Iterative Closest Point algorithm. In normalized face images we defined a set of physiologically motivated face regions related to a subset of facial muscles which are apt to automatically detect the six well-known basis emotions. Visual facial expression analysis takes place by an optical flow based feature extraction and a nearest neighbor classification, which uses a distance measure, i.e. the current flow vector pattern is matched against empirically determined ground truth data.
This paper demonstrates a technique of analysing the following three problems: automatic extracti... more This paper demonstrates a technique of analysing the following three problems: automatic extraction of moving objects, suppression of the remaining errors and solution of the correspondence problem for the video sequences motion analysis. Here we use a new paradigm for solving the correspondence problem and then determination of a motion trajectory based on a trisectional structure. I.e., firstly it distinguishes between real world objects, secondly extracts image features like Motion Blobs and colour-Patches and thirdly s objects like Meta-Objects that shall denote real world objects. The efficiency of the suggested technique for determination of motion trajectory of moving objects will be demonstrated in this paper on the basis of analysis of strongly disturbed real image sequences.
ABSTRACT This paper proposes a technique for analysing the automatic extraction of moving objects... more ABSTRACT This paper proposes a technique for analysing the automatic extraction of moving objects and suppression of the remaining errors under disturbed image situations from static camera. In this technique, we apply a modified difference image-based approach for the segmentation of moving objects in video sequences. The second part of the paper examines the problem of suppression of the remaining errors by means of morphological, separation and shadow detection algorithms. The efficiency of this suggested approach for moving objects segmentation will be demonstrated here on the basis of the analysis of strongly disturbed image sequences.
International Journal of Pattern Recognition and Artificial Intelligence, 2014
ABSTRACT Together with classification of facial expressions, the rating of their intensities is o... more ABSTRACT Together with classification of facial expressions, the rating of their intensities is of major interest. Classical supervised learning techniques require labeling of the intensities, which is labor intensive and requires expert knowledge, but nevertheless is not guaranteed to be objective. We propose a new approach to learn an intensity rating function which does not require expert knowledge, because it simplifies the labeling task by avoiding the difficulty of selecting an absolute intensity value and to keep the labeling consistent for the whole dataset. It is based on a novel kind of ground truth which we call Comparative Labeling. It specifies sample pairs for which the first element is desired to have a lower intensity than the second. We introduce a learning scheme to find an optimal intensity function in respect of the Comparative Labeling and propose performance measures to assess the quality of the learned function. The technique is applied to rate the intensity of facial expressions of posed pain. The evaluation results show that the learned function is well suited for determining dynamic intensity variation over time. We also assess the suitability of the rating as an inter-individual intensity measure by comparing it to the intensity ratings given by human observers.
2010 International Conference of Soft Computing and Pattern Recognition, 2010
ABSTRACT This work proposes a new approach for facial expression recognition in color image seque... more ABSTRACT This work proposes a new approach for facial expression recognition in color image sequences, based on integrated evaluation of geometric and dynamic features. For this purpose a series of methods is introduced that on the one hand achieve high recognition rates for expressive facial behavior and on the other hand address a couple of common problems in this area of research. In particular we apply physiologically motivated image regions for the detection of dynamic features by using an optical flow method. In this way dynamic features capture the variations caused by facial expression changes. Opposed, geometric features do not contain temporal information but describe spatial feature parameters. These correspond to 3-D based Euclidean distances and angles. Particularly, the hypothesis of this work is that through integrated evaluation of geometric and dynamic features, improved recognition rates can be achieved. Based on comprehensive experimental investigations we show the advantage of the suggested approach.
This work proposes new static and dynamic based methods for facial expression recognition in ster... more This work proposes new static and dynamic based methods for facial expression recognition in stereo image sequences. Computer vision 3-d techniques are applied to determine real world geometric measures and to build a static geometric feature vector. Optical flow based motion detection is also carried out which delivers the dynamic flow feature vector. Support vector machine classification is used to recognize the expression using geometric feature vector while k-nearest neighbor classification is used for flow feature vector. The proposed method achieves robust feature detection and expression classification besides covering the in/out of plane head rotations and back and forth movements. Further, a wide range of human skin color is exploited in the training and the test samples.
ABSTRACT The paper demonstrates a technique for analysing the following three problems. They are ... more ABSTRACT The paper demonstrates a technique for analysing the following three problems. They are mainly associated with the automatic extraction of moving objects, suppression of the remaining errors, and tracking analysis under disturbed image situations from static camera. For this technique, a modified difference image-based approach for the segmentation of moving objects in video sequences is applied. The second part of the paper examines the problem of suppression of the remaining errors (holes, outliers or fusion of regions) by means of morphological and separation operators. The extracted image regions represent the object candidates for the following tracking. The efficiency of this suggested technology for moving objects segmentation will be demonstrated in this paper on the basis of the analysis of strongly disturbed image sequences.
International Symposium on Computer and Information Sciences, 2007
Robust tracking of multiple interacting objects in image sequences is a challenging problem due t... more Robust tracking of multiple interacting objects in image sequences is a challenging problem due to the disturbances that occur often in the real environment. In this context, a system of independent particle filters and an adaptive motion model is used which allow the separated handling of moving objects in conflict situations. For solving the problems of the fluctuation detection and
2012 19th IEEE International Conference on Image Processing, 2012
ABSTRACT Automatic pain recognition can improve medical treatment, especially when the patient is... more ABSTRACT Automatic pain recognition can improve medical treatment, especially when the patient is not able to utter on his pain experience. Facial expressions with their intensities and dynamics contain valuable information for recognising pain. We propose a concept for distinguishing facial expressions of pain from others and assessing the pain expression intensity. It is based on a Support Vector Machine (SVM) classifier and a function model for intensity rating. The intensity model is trained using Comparative Learning, a new technique that simplifies labelling of the data. Using a database of 3D posed pain sequences we show the suitability of the concept to recognise pain expressions, distinguish different intensities and spot even slight intensity changes in its temporal context.
The most hand gestures used in Human Computer Interaction (HCI) are generated either by one hand ... more The most hand gestures used in Human Computer Interaction (HCI) are generated either by one hand or by two hands on condition that both hands do not pass each other. This constraint in the two-hands gestures is due to the difficulties in reacquiring both hands ...
2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), 2011
Abstract Hand gesture recognition plays a major role in the Human Computer Interaction (HCI), sin... more Abstract Hand gesture recognition plays a major role in the Human Computer Interaction (HCI), since it could be employed in many applications as a communication language. Nearly all hand gestures used in HCI are created using one hand; while other applications ...
ABSTRACT In his paper we describe a multi-camera based concept for facial expression recognition,... more ABSTRACT In his paper we describe a multi-camera based concept for facial expression recognition, which is of potential interest for modern man-machine-interfaces. Naturally, the recognition of facial user events is limited by the field of view of the applied cameras. However, in some application domains, such as patient state analysis it is mandatory to always get feedback. This can only be accomplished by increasing the observable field of view. Our proposed concept addresses this issue through the use of multiple cameras. For the realization of facial expression recognition we extended an existing technique. Examples are given for a three camera setup, which substantially enhances the degree of freedom for interaction and/or recognition of facial events. In this article we describe the applied components, such as the adaptation of the generic face model, multi-camera based pose estimation and relevant feature transformations to carry out machine based recognition.
ABSTRACT Feature-based matching frequently suffers from accuracy and performance due to a high nu... more ABSTRACT Feature-based matching frequently suffers from accuracy and performance due to a high number of image features that need to be matched. Instead, a hierarchical feature extraction process is used here to solve the correspondence problem in image sequences. In the first step, a modified difference image technique is applied to generate Motion-Blobs. Secondly, a color segmentation is used to determine Motion-Blob sub-segments and the respective set of properties. These properties are used to finally solve the correspondence problem. Consequently stability and accuracy of the motion analysis are increased.
The segmentation of mouth and lips is a fundamental problem in facial image analyisis. In this pa... more The segmentation of mouth and lips is a fundamental problem in facial image analyisis. In this paper we propose a method for lip segmentation based on rg-color histogram. Statistical analysis shows, using the rg-color-space is optimal for this purpose of a pure color based segmentation. Initially a rough adaptive threshold selects a histogram region, that assures that all pixels in that region are skin pixels. Based on that pixels we build a gaussian model which represents the skin pixels distribution and is utilized to obtain a refined, optimal threshold. We are not incorporating shape or edge information. In experiments we show the performance of our lip pixel segmentation method compared to the ground truth of our dataset and a conventional watershed algorithm.
This paper describes a novel method for analyzing single faces of non-cooperative persons on the ... more This paper describes a novel method for analyzing single faces of non-cooperative persons on the basis of stereoscopic color images. The challenges arise from the fact that the persons observed are non-cooperative, which in turn complicates further processing as facial feature extraction and tracking in image sequence. In our method, face detection is based on color-driven clustering of 3D points derived from stereo. A mesh model is registered with a post-processed face cluster, using a variant of the Iterative Closest Point algorithm (ICP). The pose is derived from correspondence. Then, the pose and model information are used for face normalization and facial feature localization. Automatic extraction of facial features is carried out using modified Active Shape Models (ASM). In contrast to the simple ASM, another approach is pursued in this work. It involves two modifications to the ASM, which lead to greater stability and robustness. The results show that stereo and color are pow...
In this paper we present a user independent real-time capable automatic method for recognition of... more In this paper we present a user independent real-time capable automatic method for recognition of facial expressions related to basic emotions from stereo image sequences. The method automatically detects faces in unconstraint pose based on depth and color information. In order to overcome difficulties caused by increasing change in pose, lighting transitions, or complicated background, we introduce a face normalization algorithm based on an Iterative Closest Point algorithm. In normalized face images we defined a set of physiologically motivated face regions related to a subset of facial muscles which are apt to automatically detect the six well-known basis emotions. Visual facial expression analysis takes place by an optical flow based feature extraction and a nearest neighbor classification, which uses a distance measure, i.e. the current flow vector pattern is matched against empirically determined ground truth data.
This paper demonstrates a technique of analysing the following three problems: automatic extracti... more This paper demonstrates a technique of analysing the following three problems: automatic extraction of moving objects, suppression of the remaining errors and solution of the correspondence problem for the video sequences motion analysis. Here we use a new paradigm for solving the correspondence problem and then determination of a motion trajectory based on a trisectional structure. I.e., firstly it distinguishes between real world objects, secondly extracts image features like Motion Blobs and colour-Patches and thirdly s objects like Meta-Objects that shall denote real world objects. The efficiency of the suggested technique for determination of motion trajectory of moving objects will be demonstrated in this paper on the basis of analysis of strongly disturbed real image sequences.
ABSTRACT This paper proposes a technique for analysing the automatic extraction of moving objects... more ABSTRACT This paper proposes a technique for analysing the automatic extraction of moving objects and suppression of the remaining errors under disturbed image situations from static camera. In this technique, we apply a modified difference image-based approach for the segmentation of moving objects in video sequences. The second part of the paper examines the problem of suppression of the remaining errors by means of morphological, separation and shadow detection algorithms. The efficiency of this suggested approach for moving objects segmentation will be demonstrated here on the basis of the analysis of strongly disturbed image sequences.
International Journal of Pattern Recognition and Artificial Intelligence, 2014
ABSTRACT Together with classification of facial expressions, the rating of their intensities is o... more ABSTRACT Together with classification of facial expressions, the rating of their intensities is of major interest. Classical supervised learning techniques require labeling of the intensities, which is labor intensive and requires expert knowledge, but nevertheless is not guaranteed to be objective. We propose a new approach to learn an intensity rating function which does not require expert knowledge, because it simplifies the labeling task by avoiding the difficulty of selecting an absolute intensity value and to keep the labeling consistent for the whole dataset. It is based on a novel kind of ground truth which we call Comparative Labeling. It specifies sample pairs for which the first element is desired to have a lower intensity than the second. We introduce a learning scheme to find an optimal intensity function in respect of the Comparative Labeling and propose performance measures to assess the quality of the learned function. The technique is applied to rate the intensity of facial expressions of posed pain. The evaluation results show that the learned function is well suited for determining dynamic intensity variation over time. We also assess the suitability of the rating as an inter-individual intensity measure by comparing it to the intensity ratings given by human observers.
2010 International Conference of Soft Computing and Pattern Recognition, 2010
ABSTRACT This work proposes a new approach for facial expression recognition in color image seque... more ABSTRACT This work proposes a new approach for facial expression recognition in color image sequences, based on integrated evaluation of geometric and dynamic features. For this purpose a series of methods is introduced that on the one hand achieve high recognition rates for expressive facial behavior and on the other hand address a couple of common problems in this area of research. In particular we apply physiologically motivated image regions for the detection of dynamic features by using an optical flow method. In this way dynamic features capture the variations caused by facial expression changes. Opposed, geometric features do not contain temporal information but describe spatial feature parameters. These correspond to 3-D based Euclidean distances and angles. Particularly, the hypothesis of this work is that through integrated evaluation of geometric and dynamic features, improved recognition rates can be achieved. Based on comprehensive experimental investigations we show the advantage of the suggested approach.
This work proposes new static and dynamic based methods for facial expression recognition in ster... more This work proposes new static and dynamic based methods for facial expression recognition in stereo image sequences. Computer vision 3-d techniques are applied to determine real world geometric measures and to build a static geometric feature vector. Optical flow based motion detection is also carried out which delivers the dynamic flow feature vector. Support vector machine classification is used to recognize the expression using geometric feature vector while k-nearest neighbor classification is used for flow feature vector. The proposed method achieves robust feature detection and expression classification besides covering the in/out of plane head rotations and back and forth movements. Further, a wide range of human skin color is exploited in the training and the test samples.
ABSTRACT The paper demonstrates a technique for analysing the following three problems. They are ... more ABSTRACT The paper demonstrates a technique for analysing the following three problems. They are mainly associated with the automatic extraction of moving objects, suppression of the remaining errors, and tracking analysis under disturbed image situations from static camera. For this technique, a modified difference image-based approach for the segmentation of moving objects in video sequences is applied. The second part of the paper examines the problem of suppression of the remaining errors (holes, outliers or fusion of regions) by means of morphological and separation operators. The extracted image regions represent the object candidates for the following tracking. The efficiency of this suggested technology for moving objects segmentation will be demonstrated in this paper on the basis of the analysis of strongly disturbed image sequences.
International Symposium on Computer and Information Sciences, 2007
Robust tracking of multiple interacting objects in image sequences is a challenging problem due t... more Robust tracking of multiple interacting objects in image sequences is a challenging problem due to the disturbances that occur often in the real environment. In this context, a system of independent particle filters and an adaptive motion model is used which allow the separated handling of moving objects in conflict situations. For solving the problems of the fluctuation detection and
2012 19th IEEE International Conference on Image Processing, 2012
ABSTRACT Automatic pain recognition can improve medical treatment, especially when the patient is... more ABSTRACT Automatic pain recognition can improve medical treatment, especially when the patient is not able to utter on his pain experience. Facial expressions with their intensities and dynamics contain valuable information for recognising pain. We propose a concept for distinguishing facial expressions of pain from others and assessing the pain expression intensity. It is based on a Support Vector Machine (SVM) classifier and a function model for intensity rating. The intensity model is trained using Comparative Learning, a new technique that simplifies labelling of the data. Using a database of 3D posed pain sequences we show the suitability of the concept to recognise pain expressions, distinguish different intensities and spot even slight intensity changes in its temporal context.
The most hand gestures used in Human Computer Interaction (HCI) are generated either by one hand ... more The most hand gestures used in Human Computer Interaction (HCI) are generated either by one hand or by two hands on condition that both hands do not pass each other. This constraint in the two-hands gestures is due to the difficulties in reacquiring both hands ...
2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), 2011
Abstract Hand gesture recognition plays a major role in the Human Computer Interaction (HCI), sin... more Abstract Hand gesture recognition plays a major role in the Human Computer Interaction (HCI), since it could be employed in many applications as a communication language. Nearly all hand gestures used in HCI are created using one hand; while other applications ...
ABSTRACT In his paper we describe a multi-camera based concept for facial expression recognition,... more ABSTRACT In his paper we describe a multi-camera based concept for facial expression recognition, which is of potential interest for modern man-machine-interfaces. Naturally, the recognition of facial user events is limited by the field of view of the applied cameras. However, in some application domains, such as patient state analysis it is mandatory to always get feedback. This can only be accomplished by increasing the observable field of view. Our proposed concept addresses this issue through the use of multiple cameras. For the realization of facial expression recognition we extended an existing technique. Examples are given for a three camera setup, which substantially enhances the degree of freedom for interaction and/or recognition of facial events. In this article we describe the applied components, such as the adaptation of the generic face model, multi-camera based pose estimation and relevant feature transformations to carry out machine based recognition.
ABSTRACT Feature-based matching frequently suffers from accuracy and performance due to a high nu... more ABSTRACT Feature-based matching frequently suffers from accuracy and performance due to a high number of image features that need to be matched. Instead, a hierarchical feature extraction process is used here to solve the correspondence problem in image sequences. In the first step, a modified difference image technique is applied to generate Motion-Blobs. Secondly, a color segmentation is used to determine Motion-Blob sub-segments and the respective set of properties. These properties are used to finally solve the correspondence problem. Consequently stability and accuracy of the motion analysis are increased.
Uploads
Papers