ULearn is a system that uses deep learning, computer vision and NLP to assist students with the task of web-based learning. ULearn’s goal is to detect when the student is experiencing higher levels of frustration and then present alternative meaningful alternative content. The ULearn app features a web-brower, though the intention is to have ULearn assist students in learning scenarios it is equally applicable to other web-based tasks. While the user is browsing, ULearn monitors them using the front-facing camera and when negative emotions are detected the user is presented with a set of “tips”. The first step in ULearn is to perform face detection which returns an ROI that is fed into an emotion detection system. A deep-learning CNN is used to perform the emotion detection yielding one of anger, fear, disgust, surprise, neutral, and happy. If a significate negative emotion is detected ULearn generates a set of alternative content called “tips” which are a set of links to similar content web pages to the current one being viewed. These links can be found through scraping the currently viewed web page for content that is used directly in a search or first passing this information to an NLP stage. The NLP stage gives the saliency of the most prominent entities in the current web page content. Real test results are given, and the success and challenges faced by ULearn are presented along with future avenues of work.
|