- Ritsumeikan University, Department of Human and Computer Intelligence, Graduate Studentadd
- Doctoral in Information Science and Engineeringedit
In this paper, we propose a novel task of collaborative game commentating, an artificial intelligence agent capable of collaboratively commentating with a human commentator in Live-Streaming of Esports. To this end, we propose a... more
In this paper, we propose a novel task of collaborative game commentating, an artificial intelligence agent capable of collaboratively commentating with a human commentator in Live-Streaming of Esports. To this end, we propose a collaborative game commentating system that employs a pre-trained language model trained using commentaries by professional commentators, along with metadata including title and tags. The conducted experiments show that (1) fine-tuned Text-to- Text Transfer Transformer (T5) model, a state-of-the-art generative language model, could produce more clearer and precise commentary and better recall the words from the reference commentary, as it effectively improves the scores on evaluation metrics that are widely used for concise text generation task after tuning the model. (2) The more information used for the current method fusion of information, the clearer and more precise generated commentary is. However, it performs worse to recall the words from reference commentary.
Research Interests:
To solve the existing sentence punctuation problem for collaborative commentary generation in Esports livestreaming, this paper presents two strategies for sentence punctuation for text sequences of game commentary, that is, punctuating... more
To solve the existing sentence punctuation problem for collaborative commentary generation in Esports livestreaming, this paper presents two strategies for sentence punctuation for text sequences of game commentary, that is, punctuating sentences by two or three text sequence(s) originally punctuated by Youtube to obtain a complete sentence of commentary. We conducted comparative experiments utilizing and fine-tuning a state-of-the-art pre-trained generative language model among two strategies and the baseline to generate collaborative commentary. Both objective evaluations by automatic metrics and subjective analyses showed that our strategy of punctuating sentences by two text sequences outperformed the baseline.
Research Interests:
This paper presents a commentator for providing real-time game commentary in a fighting game. The commentary takes into account highlight cues, obtained by analyzing scenes during gameplay, as input to adjust the pitch and loudness of... more
This paper presents a commentator for providing real-time game commentary in a fighting game. The commentary takes into account highlight cues, obtained by analyzing scenes during gameplay, as input to adjust the pitch and loudness of commentary to be spoken by using a Text-to-Speech (TTS) technology. We investigate different designs for pitch and loudness adjustment. The proposed AI consists of two parts: a dynamic adjuster for controlling pitch and loudness of the TTS and a real-time game commentary generator. We conduct a pilot study on a fighting game, and our result shows that by adjusting the loudness significantly according to the level of game highlight, the entertainment of the gameplay can be enhanced.
Research Interests:
This paper proposes a method for generating bullet comments for live-streaming games based on highlights (i.e., the exciting parts of video clips) extracted from the game content and evaluate the effect of mental health promotion. Game... more
This paper proposes a method for generating bullet comments for live-streaming games based on highlights (i.e., the exciting parts of video clips) extracted from the game content and evaluate the effect of mental health promotion. Game live streaming is becoming a popular theme for academic research. Compared to traditional online video sharing platforms, such as Youtube and Vimeo, video live streaming platform has the benefits of communicating with other viewers in real-time. In sports broadcasting, the commentator plays an essential role as mood maker by making matches more exciting. The enjoyment emerged while watching game live streaming also benefits the audience’s mental health. However, many e-sports live streaming channels do not have a commentator for entertaining viewers. Therefore, this paper presents a design of an AI commentator that can be embedded in live streaming games. To generate bullet comments for real-time game live streaming, the system employs highlight evaluation to detect the highlights, and generate the bullet comments. An experiment is conducted and the effectiveness of generated bullet comments in a live-streaming fighting game channel is evaluated.
Research Interests:
The study improve JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), an audience participation game with a purpose (APGWAP) designed for promoting ukiyo-e artworks and collecting descriptions about them from a group of people... more
The study improve JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), an audience participation game with a purpose (APGWAP) designed for promoting ukiyo-e artworks and collecting descriptions about them from a group of people over a game live streaming platform, by adding a musical-style TTS that narrates the best description made by the player for each image in each round. It is expected that narrating the best descriptions for each image will make the winner feel more rewarded and encouraged to make better descriptions in the upcoming rounds. The narration is provided to players in a musical fashion considering evidences that music has several therapeutic benefits, including stress relief and mental health promotion.
Research Interests:
This paper presents use of general and low-cost RGB cameras for calorie consumption assessment. We previously presented a method for assessing calorie consumption based on 3D skeleton data. The limitation of our previous method is its... more
This paper presents use of general and low-cost RGB cameras for calorie consumption assessment. We previously presented a method for assessing calorie consumption based on 3D skeleton data. The limitation of our previous method is its implementation requires a 3D depth sensor, like Kinect. In this study, we demonstrate adaptation of the implementation to make the method work with general RGB cameras. Webcams with a library called OpenPose can be used to obtain 2D skeleton data. Our results show that the adapted implementation using two low-cost web-cams yields accuracy as high as the former implementation using Kinect.
Research Interests:
This paper presents a method for adjustment of game difficulty and commentary generation based on audiences’ input to be used in Audience Participation Games(APGs) on Twitch: Background 1. We found that difficulty adjustment is one... more
This paper presents a method for adjustment of game difficulty and commentary generation based on audiences’ input to be used in Audience Participation Games(APGs) on Twitch: Background 1. We found that difficulty adjustment is one effective method for promoting player experience in APGs [1] . 2. The game commentary was reported to be effective in keeping audiences entertained and informed. 3. Additionally, rap-style commentary [2] was reported positively affect emotions and promote mental health.
Research Interests:
This paper presents two AIs that enable a fighting game to be played or live-streamed as an audience participation game. The proposed fighting game AIs imitates a social facilitation in human psychology by dynamically adjusting its... more
This paper presents two AIs that enable a fighting game to be played or live-streamed as an audience participation game. The proposed fighting game AIs imitates a social facilitation in human psychology by dynamically adjusting its strength based on audience responses during gameplay. The two AIs exploit Monte-Carlo Tree Search. They have three important mechanisms, for dynamic difficulty adjustment, humanlike behavior promotion, and social facilitation integration. We developed the two AIs with different evaluation functions. The common feature is an integrated social facilitation parameter. However, the difference is that one AI is developed by generalizing an existing AI by simply adding a parameter for setting the targeted HP difference, whereas the other was particularly designed for an social facilitation by a strict control of damage score difference. Our experiment result shows that the former one yields more human-like behavior, while the latter one yields slightly better strength adjustment.
Research Interests:
本論文では，体全身を使用するモーションゲームにおいて，身体部位の使用のバランスを促進するた めの，対戦格闘ゲームAI を提案する．本研究では，プレイヤーのゲーム支配力を調整することで，プレイ ヤーの入力に基づき，AI のアクションを制御する「プレイヤー支配調整（Player Dominance Adjustment, PDA）」と呼ばれる，新しい概念を使用する．これは，AI がプレイヤーの実行しようとしているアクショ ンを分析し，そのアクションがバランスを高める場合，AI... more
本論文では，体全身を使用するモーションゲームにおいて，身体部位の使用のバランスを促進するた めの，対戦格闘ゲームAI を提案する．本研究では，プレイヤーのゲーム支配力を調整することで，プレイ ヤーの入力に基づき，AI のアクションを制御する「プレイヤー支配調整（Player Dominance Adjustment, PDA）」と呼ばれる，新しい概念を使用する．これは，AI がプレイヤーの実行しようとしているアクショ ンを分析し，そのアクションがバランスを高める場合，AI キャラクターにもっと有利な行動を取って，プ レイヤーのゲーム支配力を上昇させる．そうでない場合，プレイヤーに対しAI が強い行動を取るように， モンテカルロ木探索（MCTS）から取得した行動を実行する．実験により，提案されたAI が既存のAI （MCTS ベースのAI）よりもバランスが取れていることが示唆される
Research Interests:
本文一部表示 文献の本文または文献内に掲載されている抄録の冒頭（最大100文字程度）を表示しています。 非表示の場合はJDreamⅢ(有料)でご覧頂けます。 J-GLOBALでは書誌登載から半年～1年程度経過後に表示されますが、医療系文献の場合はMyJ-GLOBALでのログインが必要です。... more
本文一部表示 文献の本文または文献内に掲載されている抄録の冒頭（最大100文字程度）を表示しています。 非表示の場合はJDreamⅢ(有料)でご覧頂けます。 J-GLOBALでは書誌登載から半年～1年程度経過後に表示されますが、医療系文献の場合はMyJ-GLOBALでのログインが必要です。 1.はじめに運動不足による生活習慣病の発症リスク増加が問題視されている中,体の動作によって操作を行うゲーム(モーションゲーム)をプレイすることによる健康促進が注目されている[1].モーションゲームに必要な機能として次の2つがある.体の動作を...【本文一部表示】
Research Interests: Games and Motion Gamming
2017, Yunshi Liu, Takahiro Kusano, Pujana Paliyawan, Tomohiro Harada and Ruck Thawonmas. Calorie Consumption Assessment by using Image-processing-based Method for Exerciser and Motion Gamers. ゲーム学会第16回全国大会論文集, pp. 37-38. 2017年11月3日．... more
2017, Yunshi Liu, Takahiro Kusano, Pujana Paliyawan, Tomohiro Harada and Ruck Thawonmas. Calorie Consumption Assessment by using Image-processing-based Method for Exerciser and Motion Gamers. ゲーム学会第16回全国大会論文集, pp. 37-38. 2017年11月3日． https://www.gameamusementsociety.org/staticpages/index.php?page=GAS_16th_program https://www.gameamusementsociety.org/staticpages/index.php?page=GAS_16th_program
Research Interests:
This paper proposes a Minecraft-based system for early detection of Alzheimer’s disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed... more
This paper proposes a Minecraft-based system for early detection of Alzheimer’s disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of players. Our preliminaries results show that spatial navigational patterns in both systems are highly correlated, indicating that the proposed system is likely as effective as the SHQ system for the detection task.
Research Interests:
This paper presents a new 3D game for cognitive assessment. The game is inspired by the fact that cognitive decline has become one most common diseases associated with aging nowadays, and early detection of cognitive decline is a key for... more
This paper presents a new 3D game for cognitive assessment. The game is inspired by the fact that cognitive decline has become one most common diseases associated with aging nowadays, and early detection of cognitive decline is a key for successful treatment. Common symptoms of cognitive decline can be diagnosed via cognitive processing, executive function, visual attention, immediate recognition, and delayed recognition assessment. The proposed game allows combining the advantages from previous systems, which are the presence of a 3D environment, the procedural content generation (PCG), and the universal design concept.
Research Interests:
This study introduces a virtual animated character (VTuber) to JUSTIN, an audience participation game with a purpose (APGWAP) designed for collecting descriptions for Ukiyo-e artworks. The inspiration of this study is based on the... more
This study introduces a virtual animated character (VTuber) to JUSTIN, an audience participation game with a purpose (APGWAP) designed for collecting descriptions for Ukiyo-e artworks. The inspiration of this study is based on the popularity of VTubers, which demonstrate promising results in terms of increasing positive user experience and outcomes on serious purposes. The introduced character serves as a moderator, focusing on promoting active and entertaining gameplay. It is expected that such a moderator would help participants create better descriptions and demonstrate higher involvement in the process. This moderator is also designed under generalization concept, namely, the system with this moderation can be reused on various games.
Research Interests:
This paper presents a method for adjustment of game difficulty and commentary generation based on audiences’ input to be used in Audience Participation Games on Twitch. We integrate difficulty adjustment and rap-style commentary in our... more
This paper presents a method for adjustment of game difficulty and commentary generation based on audiences’ input to be used in Audience Participation Games on Twitch. We integrate difficulty adjustment and rap-style commentary in our game system with an expectation to boost the game experience of players and audiences. The game allows the audience to influence the game difficulty in real-time by sending command messages meanwhile rap-style commentary will also be procedurally generated.
Research Interests:
This paper presents a puzzle game for early detection of dementia embedded with a singing interface for promoting mental health. In recent years, dementia has become a social problem worldwide, and early detection is regarded as a key for... more
This paper presents a puzzle game for early detection of dementia embedded with a singing interface for promoting mental health. In recent years, dementia has become a social problem worldwide, and early detection is regarded as a key for successful treatment. Moreover, spatial navigation is an essential factor for early detection. Therefore, we develop a maze-like puzzle game that the player has to navigate the maze’s exit. Furthermore, gameplay logs can then be used to analyze the cognitive state of the players and assess the risk of dementia. However, as found in existing systems, cognitive assessment could put pressure and stress on players. Therefore, we integrate a singing interface into the game as a mini-game. The mini-game will appear when the system detects that the player gets lost in the maze. After completing the mini-game, hints will be generated to help the player. This design is expected to help relieve stress and promoting mental well-being.
Research Interests:
This paper presents our work to enhance a state-ofthe-art level generator (Sketch-to-Level Generator) that generates levels for an Angry-Birds-like game from drawn sketches. To achieve this task, Cycle-Consistent Adversarial Networks... more
This paper presents our work to enhance a state-ofthe-art level generator (Sketch-to-Level Generator) that generates levels for an Angry-Birds-like game from drawn sketches. To achieve this task, Cycle-Consistent Adversarial Networks (Cycle- GAN) are used. CycleGAN is trained using two datasets: sketch drawings and typical level-structures. The former are taken from Google’s Quick, Draw! datasets, and the latter from the winning level generator at the 2017 and 2018 AIBIRDS level generation competitions. The output of the trained CycleGAN is used as the input of Sketch-to-Level Generator. Our results show that the proposed preprocessing technique using CycleGAN allows Sketch-to-Level Generator to more successfully generate levels from arbitrary sketch drawings.
Research Interests:
This paper proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player.... more
This paper proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player. Results on unseen data show the effectiveness of the proposed AI when its position is close to the settlement
Research Interests:
This paper presents JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), a game system that is designed for collecting descriptive data for artwork images (Japanese Ukiyo-e in this study) in promoting people, especially visually... more
This paper presents JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), a game system that is designed for collecting descriptive data for artwork images (Japanese Ukiyo-e in this study) in promoting people, especially visually impaired, to come closer to Ukiyo-e artwork by telling them more about it through its descriptions. The proposed game is the first “Audience Participation Game With A Purpose (APGWAP),” which combines two existing concepts: Audience Participation Games and Game With A Purpose for addressing problems by utilizing human participation on a live streaming platform. The descriptive data obtained from the game can be beneficial in humanity research and in promoting cultural heritage. However, it is noted that the data of most Ukiyo-e images is currently insufficient, sparse or poor in quality. Traditionally, the data created by a limited number of experts in the field, which is commonly very time-consuming and costly. Meanwhile, the system proves to be a powerful tool in exploiting crowd-sourcing for addressing such problems through a solid evidence provided from results of a full-scale experiment in terms of the quality of collected descriptions as well as player experiences.
Research Interests:
This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which... more
This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which allows it to be completed by only one bird shooting. By evolving the feasible population and infeasible population in CNS, our results show that the entropy of block-type frequencies is higher than the entropy by our previous generator and that two requirements to achieve playable RGM levels – the 100% stability and the perfect-shot rate – are met. The results indicate that the proposed method can generate levels with more diversity than our previous generator while maintaining their playability.
Research Interests:
This paper presents a video game system with a smile interface designed to promote the player experience. We introduce a new mechanism to an existing endless runner game, “Runner.” Such mechanism allows smiles to be taken as an input to... more
This paper presents a video game system with a smile interface designed to promote the player experience. We introduce a new mechanism to an existing endless runner game, “Runner.” Such mechanism allows smiles to be taken as an input to the game system, for triggering boosting period. To detect smiles, a deep-learning facial recognition toolkit named Affdex, by Affectiva, is employed. Evaluation on player experience was done using a shorten version of Game User Experience Satisfaction Scale. Three different game modes are compared: Standard Mode (no smile and boosting), Smile Mode (with smile for boosting), and Auto Mode (auto boosting, without smile). Our results show that the presence of the smile mechanism leads to more enjoyable gameplay.
Research Interests:
This paper presents a system for rap-style comment generation for enhancing video game live streaming. As creative storytelling can make live streaming truly entertaining, we investigate music-style comment generation in a video game.... more
This paper presents a system for rap-style comment generation for enhancing video game live streaming. As creative storytelling can make live streaming truly entertaining, we investigate music-style comment generation in a video game. Regarding that “rap” is a popular genre of music, which has also been claimed success in promoting the enjoyment and contributing to rehabilitation, we propose a system that generates rap-style comments in real-time for promoting audience experience.
Research Interests:
This paper presents a game system aimed towards promoting mental health, which implements singing as an input modality to control the bird-shots in an Angry-Birds-like game. Based on existing research, singing can act as a catalyst that... more
This paper presents a game system aimed towards promoting mental health, which implements singing as an input modality to control the bird-shots in an Angry-Birds-like game. Based on existing research, singing can act as a catalyst that medically reduces stress and thus can be used for rehabilitation. Our system works by analyzing singing by the player and adjusting the shooting performance based on the singing score. This paper focuses on how the system is developed and its functions.
Research Interests:
This paper provides a survey of existing systems and discusses the purposes that music therapy and video games serve. The effects of musical games and singing on psychological and physical well-being, learning and cognitive training are... more
This paper provides a survey of existing systems and discusses the purposes that music therapy and video games serve. The effects of musical games and singing on psychological and physical well-being, learning and cognitive training are discussed. According to previous research, singing, as a part of music therapy, has proved to impact stress hormones positively, and also commonly contributed to medical rehabilitation. This paper, therefore, aims to shed light on the prospects of previous studies inspiring future development of voice-input based games for music therapy and stress reduction.
Research Interests:
This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for... more
This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for playing games by using hand motions. This interface works by detecting hand motions through image processing, translating them into commands for keyboard press, and sending such commands to the target game application; it works with any existing games without the need to modify the game source code and only requires an off-the-self webcam. An experiment is conducted on a jumping game, where three types of game interfaces-a keyboard, a Kinect (for playing with hand raising motions), and the proposed interface (for playing with hand clenching and spreading motions)-are compared. The results show that playing games using hand clenching/spreading leads to the best memory test performance according to the N-back test, a commonly used cognitive task.
Research Interests:
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts.... more
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts. A targeted percentage of movement can be specified for each part of the body; the game will generate a level that makes the player's movement match the targeted percentages.
Research Interests:
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently... more
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently purposed concept called Player Dominance Adjustment, where the AI determines its actions based on the player's inputs so as to adjust the player's dominant power. The basic idea is to let the player dominate the game when they perform healthy movement and on the contrary to have the AI take a strong action against the player when she or he performs unhealthy movement. The AI outperforms an existing dynamic difficulty adjustment AI designed for the same propose.
Research Interests:
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments while engaging in full-body motion gaming. The proposed use a new concept called“Player Domi- nance Adjustment (PDA),” which is to control... more
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments while engaging in full-body motion gaming. The proposed use a new concept called“Player Domi- nance Adjustment (PDA),” which is to control the AI's actions based on the player's inputs in the way that adjusts the player's dominant power. Namely, the AI analyzes an action that the player is going to perform and determines whether the action will increase the balancedness or not; if it does so, the AI will determine action which may give advantage to player, to let the player dominate the game. Else it will use a strong counteraction (obtained from Monte-Carlo Tree Search) towards the player. The proposed AI was compared with a typical open-loop MCTS AI. Our results show that the proposed AI outperforms the existing AI in promoting the balancedness.
Research Interests:
This paper presents an integration of a game system and the art therapy concept for promoting the mental well-being of video game players. In the proposed game system, the player plays an Angry-Birds-like game in which levels in the game... more
This paper presents an integration of a game system and the art therapy concept for promoting the mental well-being of video game players. In the proposed game system, the player plays an Angry-Birds-like game in which levels in the game are generated based on images they draw. Upon finishing a game level, the player also receives positive feedback (praising words) toward their drawing and the generated level from an Art Therapy AI. The proposed system is composed of three major parts: (1) a drawing recognizer that identifies what object is drawn by the player (Sketcher), (2) a level generator that converts the drawing image into a pixel image, then a set of blocks representing a game level (PCG AI), and (3) the Art Therapy AI that encourages the players and improves their emotion. This paper describes an overview of the system and explains how its major components’ functions.
Research Interests:
This paper presents an idea of promoting gamerelated self-efficacy and user experience of game players by adjusting their dominant power. The assumption is that game players will gain higher self-efficacy and better gameplay experience... more
This paper presents an idea of promoting gamerelated self-efficacy and user experience of game players by adjusting their dominant power. The assumption is that game players will gain higher self-efficacy and better gameplay experience when game situations go in the way they expect. To evaluate this assumption, we conduct a pilot study using a famous social deduction game,Werewolf by letting one human participant plays this game, one at a time over the internet. Each participant thinks that he/she played with other six players, but all the other players were in fact controlled by the experimenter. Game situations are manipulated to create two cases of gameplay: (1) a case in which most of the participant’s actions impact gameplay (player-dominance games), and (2) a case in which almost none of the participant’s actions impact gameplay (nonplayer- dominance games). Based on evaluation using General Self-Efficacy (GSE) scale and Game User Experience Satisfaction Scale (GUESS), player-dominance games were found leading to higher self-efficacy, and there is a strong linear relationship between self-efficacy and enjoyment.
Research Interests:
This paper presents a design of live-streaming motion game system with audience-participation mechanisms. The design is applied on a fighting game, in which two on-screen characters fight each other. Each character in the game is mainly... more
This paper presents a design of live-streaming motion game system with audience-participation mechanisms. The design is applied on a fighting game, in which two on-screen characters fight each other. Each character in the game is mainly controlled by an artificial intelligence algorithm, but human audiences can request the AI to execute specific actions by performing hand motions. This game system is designed for promoting well-being of live-streaming audiences.
Research Interests:
This paper presents new features added to Runner, an open-source running game platform designed for games for health research. This game can be played by using body motion through a Kinect device, and the original version of the game... more
This paper presents new features added to Runner, an open-source running game platform designed for games for health research. This game can be played by using body motion through a Kinect device, and the original version of the game focused on physical health promotion. We presented a new mode that aims at enhancing mental well-being through a smile detection mechanism. An integration between motion detection and facial expression detection for a 2D-platform running game, Runner, is presented.
Research Interests:
This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one perfect shot of a bird. The proposed... more
This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one perfect shot of a bird. The proposed method generates a level by selecting predefined segments consisting of several objects arranged in a way that creates a domino effect between them. To increase the variability in level generation, the proposed method procedurally generates a random structure on the top of specified blocks of a predefined segment. We compare our RGM-based method to a baseline in terms of stability and expressivity (including frequency, linearity, density, and leniency) and further explain differences between the two methods. Finally, according to perfect-shot evaluation, the proposed method can generate levels featuring a successful domino effect with high probability.
Research Interests:
This paper proposes a motion gaming AI that encourages players to use their body parts in a well-balanced manner while promoting their enjoyment. The proposed AI uses time series forecasting to predict what actions its opponent human... more
This paper proposes a motion gaming AI that encourages players to use their body parts in a well-balanced manner while promoting their enjoyment. The proposed AI uses time series forecasting to predict what actions its opponent human player will perform with respect to a candidate action of the AI, from which result it estimates the amount of movement (momentum) to be produced on each part of the body of the human player against its action. The AI finally selects an action with the goal of making the momentum of body parts on each side of the player body equal. In this AI, a Monte-Carlo Tree Search (MCTS) is employed for candidate action selection and is embedded with a dynamic difficulty adjustment (DDA) mechanism for enhancing enjoyment of the game. Our results offer a contingent evidence that an opponent gaming AI can be used to effectively improve the human player's balance, enjoyment, engrossment, personal gratification while playing the game.
Research Interests: Video Games and game AI
This abstract provides a survey on Games With a Purpose (GWAPs) and Audience Participation Games (APGs). We also discuss potentials in combining GWAPs and APGs on live-streaming platforms. Nowadays, many deep learning and machine learning... more
This abstract provides a survey on Games With a Purpose (GWAPs) and Audience Participation Games (APGs). We also discuss potentials in combining GWAPs and APGs on live-streaming platforms. Nowadays, many deep learning and machine learning techniques have been developed, which require a large amount of high-quality data in the training process. However, for certain applications, particularly those in the humanities, which need descriptions of cultural items such as images or artworks, such descriptive data are usually sparse, insufficient or low in quality. Usually, human experts are required to create the data, but this task is tedious, demanding and costly. GWAPs have demonstrated how games can harness human intelligence to generate more quality descriptive-meaningful and relevant-data without any pain but fun and enjoyable [1]. APGs are based on an idea of game live streaming that allows audiences to not only watch but also participate in part of play. It would be, therefore, interesting to combine a GWAP and an APG to utilize their advantages for descriptive data collection in humanities research.
Research Interests:
This work presents a new g ame with a p urpose ( to collect image similarity data. Few e xisting database s contain image similarity data I n particular, there exist s no database of Japanese artwork u kiyo e images contain ing such data... more
This work presents a new g ame with a p urpose ( to collect image similarity data. Few e xisting database s contain image similarity data I n particular, there exist s no database of Japanese artwork u kiyo e images contain ing such data I nspired by existing studies on GWAP, we propose a solution that utilize s players' interest s in game s to help us obtain ukiyo e similarity data.
Research Interests: Video Games and Gwap
This paper describes the use of “Real Pixel Image Level Generator (RPILG)” along with a design of game control, called “enhancing play with smiling” to promote the players’ emotion. Our study is conducted on Science Birds, an Angry-... more
This paper describes the use of “Real Pixel Image Level Generator (RPILG)” along with a design of game control, called “enhancing play with smiling” to promote the players’ emotion. Our study is conducted on Science Birds, an Angry- Birds-like game. RPILG is a procedural content generation method that generates game levels with the appearance of a pixel image of things, such as a pumpkin. We apply RPILG in Science Birds. A facial expression tool is used to recognize smiles in the player. And in our design, the black bird, a kind of shooting object in the game having the power to explode blocks, is modified such that its explosion power can be enhanced according to the degree of smiling. The idea behind this game design is to encourage smiling. An experiment is conducted with fourteen participants where two modes of gameplay “smiling for enhancing the birds” and “baseline play” are compared. An online survey using Positive and Negative Affect Schedule is used to evaluate the players’ emotion after they have experienced both modes. Our experimental results provide an evidence that the former mode leads to statistically significantly higher positive affect and lower negative affect.
Research Interests:
This paper investigates whether and to which degree watching Angry Birds-like live streaming could affect the spec-tator's working memory (WM). We prepared Angry Birds-like gameplay on the Twitch streaming platform in which a domino... more
This paper investigates whether and to which degree watching Angry Birds-like live streaming could affect the spec-tator's working memory (WM). We prepared Angry Birds-like gameplay on the Twitch streaming platform in which a domino effect based on Rube Goldberg Machine (RGM) mechanism is featured in each generated game level. The spectators' working memory are measured using the N-back task, and our results show a possibility that watching the proposed live streaming can increase the WM performance.
Research Interests: Memory and Angry Birds
This paper proposes a method that use a Variational Autoencoder (VAE) neural network to change the style of audiences' favorite music to sleepy style one, by which our purpose of making them relaxed is achieved. Not only music can... more
This paper proposes a method that use a Variational Autoencoder (VAE) neural network to change the style of audiences' favorite music to sleepy style one, by which our purpose of making them relaxed is achieved. Not only music can influence our emotion, but also sleepiness. Some people like to listen to sleepy music to aid soothing sleep, and responding to this demand, there are currently many online channels streaming sleepy music all day. Nevertheless, the variety of music being streamed is limited. This study aims at applying style transfer techniques on music data to generate sleepy-style music from existing non-sleepy-style music. We believe our proposed technique will not only solve the above limitation but also bring music composers new ideas and inspirations.
Research Interests: Music and Deep Learning
This paper proposes a method for generating Angry Birds-like game levels, and their gameplays, that promote spectator's emotions. Each of such levels features a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms,... more
This paper proposes a method for generating Angry Birds-like game levels, and their gameplays, that promote spectator's emotions. Each of such levels features a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, allowing it to be completed by one perfect shot of a bird. We evaluate the effects on spectator's emotions by comparing two sets of gameplay videos; one with only perfect shots and the other with only imperfact shots (i.e., gameplays that were not successfully completed by one shot). We conducted an online survey using Positive and Negative Affect Schedule (PANAS) to evaluate the spectator's emotions before and after watching a video of interest. Our results show that perfect-shot videos lead to higher positive affect and lower negative affect of the spectator in the same series of generated RGM levels. In addition, the perfect-shot video has a stronger evidence in decreasing the negative affect of the spectator than the imperfect one.
Research Interests:
This paper proposes a method for creating informative descriptive sentences for Ukiyo-e images by using "Game With A Purpose", or GWAP on live-streaming platform. Informative descriptive sentences of Ukiyo-e images many benefits in Deep... more
This paper proposes a method for creating informative descriptive sentences for Ukiyo-e images by using "Game With A Purpose", or GWAP on live-streaming platform. Informative descriptive sentences of Ukiyo-e images many benefits in Deep Learning research, Image Retrieval or providing better accessibility for the visually impaired. Using computer vision techniques to automatically annotate images don't work really well, especially for artwork pictures like Ukyo-e images. Meanwhile, many existing implementations of GWAP demonstrated how game can harness the intelligence of human to label images easily. Moreover, with a huge number of users active on Twitch watching live streaming videos every day, we can exploit their time of chatting to create valuable description for Ukiyo-e images and also give them a chance to interact with the game itself through chatting.
Research Interests:
In this paper, we propose TNT Gained In optical Flows (TGIF), a new algorithm to select the most entertaining TNT explosions in an Angry Birds-like action puzzle game. We assume that spectators like a high amount of object movement... more
In this paper, we propose TNT Gained In optical Flows (TGIF), a new algorithm to select the most entertaining TNT explosions in an Angry Birds-like action puzzle game. We assume that spectators like a high amount of object movement distributed equally over both space and time and that such movement entertains spectators. We, hence, first divide a game video into multiple frames and estimate the optical flows of each frame. With these optical flows, we compute a total displacement and two kinds of Shannon entropy: spatial entropy and temporal entropy. Spatial TGIF and Temporal TGIF are computed by multiplying the total displacement and the respective entropy. We then predict the best explosion video using these two methods. Our results show that the proposed Spatial TGIF's correct rate is the highest , i.e., 95% which is much higher than 65% by our previous work.
Research Interests:
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A... more
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A regressor is then built from the collected data whose input consists of a set of features representing effects due to an explosion and output indicates the average interestingness. A user study is then conducted where participants are asked to answer a questionnaire measuring their positive and negative affect after watching each of the three gameplay videos-the explosions with the highest average interestingness, the explosions with the lowest average interestingness, the explosions with the highest interestingness predicted by the regressor-in counterbalanced order. Our results show that the first gameplay video statistically significantly increases the positive affect of the participants and that both first and third videos statistically significantly decrease the negative affect.
Research Interests:
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our... more
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our system consists of two major parts: level generator and game system. The level generator generates a level according to a given file that defines the level appearance of a pixel image. The game system includes emotion recognition and fog control; to encourage the player to smile, fog is first generated that covers the game screen, but when the player smiles it will gradually disappear. It is expected that the proposed system is useful in promoting the affect of its players, i.e., increasing the positive affect and decreasing the negative one.
Research Interests:
In this paper, we design an audience participation Angry Birds platform and evaluate its effectiveness in promoting social well-being. Existing work reported that playing online games together with other players has positive effects on... more
In this paper, we design an audience participation Angry Birds platform and evaluate its effectiveness in promoting social well-being. Existing work reported that playing online games together with other players has positive effects on the promotion of communication and improvement of social relations. With the advent of audience participation games, research on methods for cooperative operations is gaining attention. However, in games targeted in existing research, game rules are complicated, so audiences need time to learn how to play them. Therefore, we focus on the Angry Birds game with simple rules and propose a mechanism to let audiences control bird shots via chat. In addition, the maps, levels, or stages of those games in the existing studies are fixed, so such games may become boring and make the audience give up playing. Therefore, we also propose a mechanism to let audiences generate original game levels via chat, which is expected to increase fun. Our results confirm the effectiveness of both proposed mechanisms.
Research Interests:
This paper proposes two implementations for promoting men- tal well-being in Angry Birds, one of the most popular mobile games. In the first implementation, fog is introduced for hiding some parts of the game stage, and the player must... more
This paper proposes two implementations for promoting men- tal well-being in Angry Birds, one of the most popular mobile games. In the first implementation, fog is introduced for hiding some parts of the game stage, and the player must smile in order to erase fog. In the second implementation, a special TNT is introduced, which can be exploded by the player's smile. This special TNT helps the player clear the level. We evaluate each of the two implementations by analyzing e ects on the player's mental status and game satisfaction.
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A... more
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A regressor is then built from the collected data whose input consists of a set of features representing effects due to an explosion and output indicates the average interestingness. A user study is then conducted where participants are asked to answer a questionnaire measuring their positive and negative affect after watching each of the three gameplay videos – the explosions with the highest average interestingness, the explosions with the lowest average interestingness, the explosions with the highest interestingness predicted by the regressor – in counterbalanced order. Our results show that the first gameplay video statistically significantly increases the positive affect of the participants and that both first and third videos statistically significantly decrease the negative affect.
Research Interests:
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our... more
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our system consists of two major parts: level generator and game system. The level generator generates a level according to a given file that defines the level appearance of a pixel image. The game system includes emotion recognition and fog control; to encourage the player to smile, fog is first generated that covers the game screen, but when the player smiles it will gradually disappear. It is expected that the proposed system is useful in promoting the affect of its players, i.e., increasing the positive affect and decreasing the negative one.
Research Interests:
This paper proposes an image-processing-based method for per-sonalization of calorie consumption assessment during exercising. An experiment is carried out where several actions are required in an exercise called broadcast gymnastics,... more
This paper proposes an image-processing-based method for per-sonalization of calorie consumption assessment during exercising. An experiment is carried out where several actions are required in an exercise called broadcast gymnastics, especially popular in Ja-pan and China. We use Kinect, which captures body actions by separating the body into joints and segments that contain them, to monitor body movements to test the velocity of each body joint and capture the subject's image for calculating the mass of each body joint that differs for each subject. By a kinetic energy formula, we obtain the kinetic energy of each body joint, and calories consumed during exercise are calculated in this process. We evaluate the performance of our method by benchmarking it to Fitbit, a smart watch well-known for health monitoring during exercise. The experimental results in this paper show that our method outperforms a state-of-the-art calorie assessment method, which we base on and improve, in terms of the error rate from Fitbit's ground-truth values .
Research Interests:
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion... more
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion games, the user faces problems in memorization of motions or performing of repetitive movements, leading to an unbalanced use of body segments. This paper proposes a solution for these problems, and particularly we focus on the benefits of having an intelligent assistant in full-body motion games for giving instructions, training and recommendation of a variation of movements. Our development consists of two major modules: a during-gameplay motion recommender and a pre-gameplay instructor.
Research Interests:
In this paper, we describe difference between human perception and machine perception in the similarity of art images. Here the term "machine" is referred to methods using features extracted from a deep learning model. Currently,... more
In this paper, we describe difference between human perception and machine perception in the similarity of art images. Here the term "machine" is referred to methods using features extracted from a deep learning model. Currently, similarity calculated using such deep-learning features is used in various purposes such as for building image search engines and image recommendation systems. However, when humans judge similarity, they consider multiple aspects such as color, style, brightness, and scene. Therefore, we investigate such difference using ukiyo-e images in the possession of ARC and report our findings.
Research Interests:
The purpose of this study is to improve AI players in Lottery. Lottery is a card matching game designed based on the concept of Audience Participation Game With a Purpose. Namely, it lets live streaming game audiences take part in... more
The purpose of this study is to improve AI players in Lottery. Lottery is a card matching game designed based on the concept of Audience Participation Game With a Purpose. Namely, it lets live streaming game audiences take part in gameplay and collects image similarity data perceived by these audiences. The game employs two AI players that initially calculate image similarities based on deep features (deep-feature similarities). In our previous study, it was found that similarities, between a certain pair of images, perceived by machines or computers-calculated based on deep features-were different from similarities perceived by humans. This would make gameplay by the AI players unbelievable, in other words, non-human-like. This study, therefore, proposes to use a linear model, built based on pre-collected human data, for improving the deep-feature similarities. The amount of human data required to make the model stable is also discussed. Experimental results show that the linear model only requires small quantities of human data to greatly improve the deep-feature similarities. At the same time, our results also show that the game Lottery is indispensable. This is because the linear model can only make the calculated similarity closer to that of humans, but there is still discernible difference; in order to obtain accurate similarity between images, it is necessary to collect a certain amount of human-perceived similarity data. INDEX TERMS Audience participation game, game with a purpose, audience participation game with a purpose, technique for similarity data collection, card matching game, human-perceived similarity, deep feature similarity.
Research Interests:
This paper proposes a system using a Minecraft mod for early detection of Alzheimer s disease (AD). In recent years, dementia, including AD, has become a social problem worldwide, and early detection, in particular among young population,... more
This paper proposes a system using a Minecraft mod for early detection of Alzheimer s disease (AD). In recent years, dementia, including AD, has become a social problem worldwide, and early detection, in particular among young population, is regarded as an important way to prevent it, where spatial navigation is a crucial key. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of players and their play experiences. Results show that spatial navigational patterns in both systems are highly correlated, indicating that the proposed system should be as effective as the SHQ system for the detection task. In addition, play experiences in the former are better for most investigated factors.
Research Interests:
In this paper, we propose a card game system named ``Lottery'' for collecting similarity data of artwork images. It is built based on the concept of game design called ``Audience Participation Game With a Purpose (APGWAP),'' which is to... more
In this paper, we propose a card game system named ``Lottery'' for collecting similarity data of artwork images. It is built based on the concept of game design called ``Audience Participation Game With a Purpose (APGWAP),'' which is to outsource computational steps in a given task to humans via a game that allows audiences to participate in a meaningful way. The game system is streamed on Twitch. In this game system, two Arti cial Intelligence (AI) players match two cards that are most similar from a set of cards in their turn and discard them. The audiences can choose their roles between Assistant and Jury. Assistant helps the AI player of a given turn choose such a pair of cards, while Jury gives a score telling how similar a chosen pair is. In the experiments, different methods for providing rewards to audiences are investigated: equal reward, random reward, and performance-based reward. Experimental results show that the performance-based reward signi cantly promotes personal grati cation and provides a better gaming experience on several other factors. In addition, it was found in our experiment that different groups of humans provided similar similarity scores for all the pairs of images, indicating that collecting promises reliable human data that are consistent through several trials is possible using our game system.
Research Interests:
Runner is a 2D platform game designed to be used for conducting research studies on health promotion. This paper presents basic features of the game and describes how contents are procedurally generated. Physical health promotion when the... more
Runner is a 2D platform game designed to be used for conducting research studies on health promotion. This paper presents basic features of the game and describes how contents are procedurally generated. Physical health promotion when the game is played as a motion game is focused. We provide guidelines for future studies with two illustrative examples. The first example is on procedural content generation for promoting balancedness in use of body segments and increasing movement variability. The second example is dynamic difficulty adjustment (DDA) for helping the player achieve their expected amount of calories burned. This paper also presents a pilot study demonstrating a DDA method that can significantly increase the amount of calories burned and positive affects of motion game players.
Research Interests:
This paper presents an intelligent assistant for preventing injuries during motion gaming. In our previous work, we invented UKI, a middleware that allows its users to play any existing games by using body motions. The goal of this paper... more
This paper presents an intelligent assistant for preventing injuries during motion gaming. In our previous work, we invented UKI, a middleware that allows its users to play any existing games by using body motions. The goal of this paper is to add an intelligent module to UKI for promoting healthy motion gameplay. According to the literature in sports science and medicine, several exercise injuries are caused by unbalanced use of body segments and repetitive movement. To prevent such injuries, we present a module that monitors and promotes two health metrics of the player: balancedness in use of body segments and variability in body movement. The proposed module recommends motions that will improve the two health metric during gameplay. We evaluate effectiveness of the module on 45 participants with normal health condition. Two methods for motion recommendation were compared: recommendation according to health fitness functions (HFF) and recommendation by random motion selection (RMS). Our results indicate significant improvement in the two health metrics when HFF is used.
Research Interests:
This paper presents a novel approach for motion segmentation by using strategies of splitting and remerging. The presented approach, Mossar, hybridizes two existing ones to obtain their potential advantages while covering weaknesses: (1)... more
This paper presents a novel approach for motion segmentation by using strategies of splitting and remerging. The presented approach, Mossar, hybridizes two existing ones to obtain their potential advantages while covering weaknesses: (1) velocity-based, one of the most widely used approaches that has fairly low accuracy but provides computational simplicity and (2) graph-based, a state-of-the-art approach that provides outstanding accuracy, yet bears high computational complexity and a burden in setting of thresholds. An initial set of key frames is generated by a velocity-based splitting process and then fed into a graph-based remerging process for refinement. We present mechanisms that improve key-frames capturing in the velocity-based approach as well as details on how the graph-based approach is modified and later applied to remerging. The proposed approach also allows users to interactively add or reduce the number of key frames to control segmentation hierarchy without the need to change threshold values and rerun segmentation, as usually done in existing approaches. Our experimental results show that the presented hybrid approach, compared to both velocity-based and graph-based, demonstrates superior performance in terms of accuracy and in comparison to graph-based, our approach has not only less complexity but also a lesser number of thresholds, the values of which can be much more simply determined.
Research Interests:
The key of success in stock trading is to buy and sell stocks at the right time for the right price. “Buy Low, Sell High” sounds easy, but it is so difficult to carry out since the direction of stock market in the near future is almost... more
The key of success in stock trading is to buy and sell stocks at the right time for the right price. “Buy Low, Sell High” sounds easy, but it is so difficult to carry out since the direction of stock market in the near future is almost unpredictable. With the advances in data mining, it has now become possible to predict the future market direction based on historical data. In this study, different approaches are used to predict the future market direction of the Stock Exchange of Thailand (SET). Time series forecasting is conducted and a suitable span of time for the stock market data is examined. A novel approach to predict future market direction has been introduced based on chart patterns recognition by using data mining classification. Models are built through different methods including neural network, decision tree, naïve Bayes and k-nearest neighbors. Results were obtained, compared and discussed in details. Important chart patterns to support decision making in stock trading had been found out. In order to visualize the result, a visualization technique is also introduced.
Research Interests: Classification (Machine Learning), Pattern Recognition, Stock Market, Time series analysis, Stock Markets, and 9 moreTechnical Analysis, Investment, Classification, Stock Market Technical Analysis, Time Series Forecasting , Neural Network, Stock Market Prediction, Stocks, Stock Market Analysis, and Time Series Analysis and Forecasting
Universal Kinect-type-controller by ICE Lab (UKI, pronounced as “You-key”) is developed for allowing users to control any existing applications by using body motions as inputs. The middleware works by converting detected motions into... more
Universal Kinect-type-controller by ICE Lab (UKI, pronounced as “You-key”) is developed for allowing users to control any existing applications by using body motions as inputs. The middleware works by converting detected motions into keyboard and/or mouse-click events, and sending them to a target application. This paper presents the structures and designs of core modules, along with examples from real cases to illustrate how the middleware can be configured to fit a variety of applications. We present our designs of interfaces that decode all configuration details into a human-interpretable language, and these interfaces significantly promote user experience and eliminate user divide in, for example, programming skill. The performance of the middleware is evaluated on fighting-game motion data, and we make these data publicly available so that they can be used in other researches. UKI welcomes its use to everyone in unlimited purpose; for instance, it can be used to promote healthy life through a means of gaming and/or used to conduct serious researches on motion systems. The middleware serves as a shortcut in the development of motion applications – coding of an application to detect motions can be replaced with simple clicks on UKI.
Research Interests:
Many office workers today sit and work at computers for extended periods of time, which can result in a group of symptoms called “Office Workers Syndrome”. To help prevent these symptoms, a novel system for monitoring computer users is... more
Many office workers today sit and work at computers for extended periods of time, which can result in a group of symptoms called “Office Workers Syndrome”. To help prevent these symptoms, a novel system for monitoring computer users is proposed by using a Kinect camera. Firstly, data mining classification is applied for detection of prolonged sitting, while mathematics as well as threshold models are applied to detect unhealthy postures. Secondly, the system gives an alert to the user when unhealthy habits are detected, via simple pop-up/voice messages or via an alerting device developed by using a microcontroller. Moreover, this study also focuses on enhancing the user experience, various user-interfaces and data visualization techniques for generating useful summary reports are provided.
Research Interests:
A motion controller is a type of device capable of capturing human movement and allowing gestural interaction. There exist three issues limiting its potential. First, to build a system for gestural interaction, coding must be done for... more
A motion controller is a type of device capable of capturing human
movement and allowing gestural interaction. There exist three issues
limiting its potential. First, to build a system for gestural interaction,
coding must be done for detecting motions, according to a motion
controller in use, which costs manpower and time. Second, limitations
reside in widely-used velocity-based and state-of-the-art graph-based
approaches for motion segmentation, i.e., the former approach cannot
accurately segment motions while the latter one can but with high
computational complexity. Third, the risk of injuries initiated from
motion gaming, a major application, is commonly overlooked.
Three solutions are proposed for solving the above issues. The first
solution is UKI, middleware that detects motions, converts them into
keyboard and/or mouse-click events, and sends them as input to a
targeted application. As the second solution, a novel
motion-segmentation approach is proposed that hybridizes the
aforementioned two approaches to obtain their potential advantages
while covering weaknesses. For the third solution, to prevent
motion-gaming-related injuries, two intelligent mechanisms are proposed
that monitor and promote the player’s health metrics during gameplay:
balancedness in use of body parts and variability in body movement. The
first mechanism is a fighting-game AI that determines its next actions
against the player based on usage log of the player’s body parts assessed
through UKI, and the other is a UKI module called intelligent assistant
(IA) for recommending players what motions they should perform.
Through UKI, a gestural interaction system can be built without the
need to modify the source code of targeted applications and to do
complex coding. The proposed hybrid approach for motion segmentation
demonstrates superior performance in terms of segmentation accuracy
compared to both velocity-based and graph-based, and has much less
complexity than graph-based. Experimental results demonstrate that AI
and IA significantly improve the health metric.
movement and allowing gestural interaction. There exist three issues
limiting its potential. First, to build a system for gestural interaction,
coding must be done for detecting motions, according to a motion
controller in use, which costs manpower and time. Second, limitations
reside in widely-used velocity-based and state-of-the-art graph-based
approaches for motion segmentation, i.e., the former approach cannot
accurately segment motions while the latter one can but with high
computational complexity. Third, the risk of injuries initiated from
motion gaming, a major application, is commonly overlooked.
Three solutions are proposed for solving the above issues. The first
solution is UKI, middleware that detects motions, converts them into
keyboard and/or mouse-click events, and sends them as input to a
targeted application. As the second solution, a novel
motion-segmentation approach is proposed that hybridizes the
aforementioned two approaches to obtain their potential advantages
while covering weaknesses. For the third solution, to prevent
motion-gaming-related injuries, two intelligent mechanisms are proposed
that monitor and promote the player’s health metrics during gameplay:
balancedness in use of body parts and variability in body movement. The
first mechanism is a fighting-game AI that determines its next actions
against the player based on usage log of the player’s body parts assessed
through UKI, and the other is a UKI module called intelligent assistant
(IA) for recommending players what motions they should perform.
Through UKI, a gestural interaction system can be built without the
need to modify the source code of targeted applications and to do
complex coding. The proposed hybrid approach for motion segmentation
demonstrates superior performance in terms of segmentation accuracy
compared to both velocity-based and graph-based, and has much less
complexity than graph-based. Experimental results demonstrate that AI
and IA significantly improve the health metric.
Research Interests: Information Science, Artificial Intelligence, Health Promotion, Data Analysis, Game Design, and 15 moreVideo Games, Video Game Design, Physical Activity, Well-Being, Motion control, Motion Analysis, Health, Wellbeing, Games, Gaming, Kinect, Motion Detection, User interfaces, Exercise, and Motion Capture
Design and Development of an Online Game. The game can be downloaded at https://drive.google.com/drive/folders/0B_hTYHLVy_HbUWxRYy1Mcmd5RGc
Research Interests:
In this paper, we propose a system for sitting monitoring an office worker to prevent office workers syndrome. This research has focused on detection of prolonged sitting of office workers by performing data mining classification on the... more
In this paper, we propose a system for sitting monitoring an office worker to prevent office workers syndrome. This research has focused on detection of prolonged sitting of office workers by performing data mining classification on the real-time skeleton data stream captured by a single Kinect camera set up in an office worker’s work station area. The system classifies the input stream into sequences of stills or moves. The performance of several classification methods such as decision tree, neural network, naive Bayes, and k-Nearest Neighbors are compared in order to acquire the optimal classifier. The proposed system can effectively monitor the user’s postures with 97% accuracy and give the user real-time feedback based on the three levels of healthy in ergonomics. In addition, the proposed system detects unhealthy sitting posture using mathematic theorems which are spherical coordinate system and geometry for translating human body angle translation. And this research also includes development of an alerting device using a microcontroller, and provision of data visualization for a useful health summary report.
Research Interests:
Research Interests:
This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed... more
This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of players. Our preliminary results show that spatial navigational patterns in both systems are highly correlated, indicating that the proposed system is likely as effective as the SHQ system for the detection task.
Research Interests: Computer Science, Serious Games, Disease, MoD, Hero, and 3 moreEarly Detection, Minecraft, and arXiv
Research Interests:
This paper proposes a motion-gaming AI for health promotion that can adapt to the player's behavior change in an effective manner. Through modeling of the player's behavior and predicting of their counteraction, this AI learns how... more
This paper proposes a motion-gaming AI for health promotion that can adapt to the player's behavior change in an effective manner. Through modeling of the player's behavior and predicting of their counteraction, this AI learns how its actions can induce its opponent player to move. The proposed AI aims at suppressing health risks associated with motion gaming, by improving balancedness in use of body segments, as well as at increasing the level of calories consumption.
Research Interests:
This paper describes analysis of relationship between the player's behavior change and the effectiveness of a Health Promotion AI (HP-AI). We measured player's behavior change by Jensen-Shannon divergence (JSD). Our results show... more
This paper describes analysis of relationship between the player's behavior change and the effectiveness of a Health Promotion AI (HP-AI). We measured player's behavior change by Jensen-Shannon divergence (JSD). Our results show that the effectiveness of HP-AI becomes higher when a player has low JSD.
Research Interests:
This paper presents our work to enhance a state-of-the-art level generator (Sketch-to-Level Generator) that generates levels for an Angry-Birds-like game from drawn sketches. To achieve this task, Cycle-Consistent Adversarial Networks... more
This paper presents our work to enhance a state-of-the-art level generator (Sketch-to-Level Generator) that generates levels for an Angry-Birds-like game from drawn sketches. To achieve this task, Cycle-Consistent Adversarial Networks (Cycle-GAN) are used. CycleGAN is trained using two datasets: sketch drawings and typical level-structures. The former are taken from Google’s Quick, Draw! datasets, and the latter from the winning level generator at the 2017 and 2018 AIBIRDS level generation competitions. The output of the trained CycleGAN is used as the input of Sketch-to-Level Generator. Our results show that the proposed preprocessing technique using CycleGAN allows Sketch-to-Level Generator to more successfully generate levels from arbitrary sketch drawings.
Research Interests:
Research Interests:
Research Interests:
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion... more
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion games, the user faces problems in memorization of motions or performing of repetitive movements, leading to an unbalanced use of body segments. This paper proposes a solution for these problems, and particularly we focus on the benefits of having an intelligent assistant in full-body motion games for giving instructions, training and recommendation of a variation of movements. Our development consists of two major modules: a during-gameplay motion recommender and a pre-gameplay instructor.
Research Interests:
Research Interests:
This paper provides a survey of existing systems and discusses the purposes that music therapy and video games serve. The effects of musical games and singing on psychological and physical well-being, learning and cognitive training are... more
This paper provides a survey of existing systems and discusses the purposes that music therapy and video games serve. The effects of musical games and singing on psychological and physical well-being, learning and cognitive training are discussed. According to previous research, singing, as a part of music therapy, has proved to impact stress hormones positively, and also commonly contributed to medical rehabilitation. This paper, therefore, aims to shed light on the prospects of previous studies inspiring future development of voice-input based games for music therapy and stress reduction.
Research Interests:
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently... more
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently purposed concept called Player Dominance Adjustment, where the AI determines its actions based on the player’s inputs so as to adjust the player’s dominant power. The basic idea is to let the player dominate the game when they perform healthy movement and on the contrary to have the AI take a strong action against the player when she or he performs unhealthy movement. The AI outperforms an existing dynamic difficulty adjustment AI designed for the same propose.
Research Interests:
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts.... more
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts. A targeted percentage of movement can be specified for each part of the body; the game will generate a level that makes the player’s movement match the targeted percentages.
Research Interests:
This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for... more
This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for playing games by using hand motions. It works by detecting hand motions through image processing, translating them into commands for keyboard press, and sending such commands to the target game application; it works with any existing games without the need to modify the game source code and only requires an off-the-self webcam. An experiment is conducted on a jumping game, where three types of game interfaces: a keyboard, a Kinect, and the proposed interface are compared. The results show that playing games using hand clenching/spreading leads to the best memory test performance according to the N-back test, a commonly used cognitive task for the assessment of working memory.
Research Interests:
This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one shot of a bird. The proposed method... more
This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one shot of a bird. The proposed method generates a level by selecting predefined segments consisting of several objects arranged in a way that creates a domino effect among them. To increase the variability of generated levels, the proposed method procedurally generates a varying structure on the top of certain blocks in a predefined segment. Our results show that the proposed RGM generator is comparable to two existing generators, including the winner of the 2018 AIBIRDS Level Generation Competition, in terms of stability while it outperforms both baseline generators with respect to running time and an expressivity metric called "dynamic" which is introduced in this work to measure the time period where moving objects, including a shooting bird, reside in a given level. In additio...
Research Interests:
Research Interests:
This study improves JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), an audience participation game with a purpose (APGWAP) designed for promoting ukiyo-e artworks and collecting descriptions about them from a group of people... more
This study improves JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), an audience participation game with a purpose (APGWAP) designed for promoting ukiyo-e artworks and collecting descriptions about them from a group of people over a game live streaming platform, by adding a musical-style TTS that narrates the best description made by the player for each image in each round. It is expected that narrating the best descriptions for each image will make the winner feel more rewarded and encouraged to make better descriptions in the upcoming rounds. The narration is provided to players in a musical fashion considering evidences that music has several therapeutic benefits, including stress relief and mental health promotion.
Research Interests:
This study introduces a virtual animated character (VTuber) to JUSTIN, an audience participation game with a purpose (APGWAP) designed for collecting descriptions for Ukiyo-e artworks. The inspiration of this study is based on the... more
This study introduces a virtual animated character (VTuber) to JUSTIN, an audience participation game with a purpose (APGWAP) designed for collecting descriptions for Ukiyo-e artworks. The inspiration of this study is based on the popularity of VTubers, which demonstrate promising results in terms of increasing positive user experience and outcomes on serious purposes. The introduced character serves as a moderator, focusing on promoting active and entertaining gameplay. It is expected that such a moderator would help participants create better descriptions and demonstrate higher involvement in the process. This moderator is also designed under generalization concept, namely, the system with this moderation can be reused on various games.
Research Interests:
This paper proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player.... more
This paper proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player. Results on unseen data show the effectiveness of the proposed AI when its position is close to the settlement center.
Research Interests:
This paper proposes a method for creating informative descriptive sentences for artwork images by using “Games With a Purpose,” or GWAP on a live streaming platform. In existing studies, automatic annotation of images did not perform... more
This paper proposes a method for creating informative descriptive sentences for artwork images by using “Games With a Purpose,” or GWAP on a live streaming platform. In existing studies, automatic annotation of images did not perform well, in particular, for artwork images such as Ukiyo-e. On the other hand, existing studies on GWAP, the concept of using games for human to address problems that computer cannot solve, demonstrated how games can exploit human intelligence in labeling images. With a huge number of audiences watching games live streaming on Twitch and other similar services nowadays, we propose a solution that makes chatting of audiences become an effective means for generating valuable descriptions for artwork images, Ukiyo-e images in this study, as well as promoting the interaction between the audiences and the game.
Research Interests:
This paper proposes a method that use a Variational Autoencoder (VAE) neural network to change the style of audiences’ favorite music to sleepy style one, by which our purpose of making them relaxed is achieved. Not only music can... more
This paper proposes a method that use a Variational Autoencoder (VAE) neural network to change the style of audiences’ favorite music to sleepy style one, by which our purpose of making them relaxed is achieved. Not only music can influence our emotion, but also sleepiness. Some people like to listen to sleepy music to aid soothing sleep, and responding to this demand, there are currently many online channels streaming sleepy music all day. Nevertheless, the variety of music being streamed is limited. This study aims at applying style transfer techniques on music data to generate sleepy-style music from existing non-sleepy-style music. We believe our proposed technique will not only solve the above limitation but also bring music composers new ideas and inspirations.
Research Interests:
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A... more
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A regressor is then built from the collected data whose input consists of a set of features representing effects due to an explosion and output indicates the average interestingness. A user study is then conducted where participants are asked to answer a questionnaire measuring their positive and negative affect after watching each of the three gameplay videos - the explosions with the highest average interestingness, the explosions with the lowest average interestingness, the explosions with the highest interestingness predicted by the regressor - in counterbalanced order. Our results show that the first gameplay video statistically significantly increases the positive affect of the participants and that both first and third videos statistically signi...
Research Interests:
This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which... more
This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which allows it to be completed by only one bird shooting. By evolving the feasible population and infeasible population in CNS, our results show that the entropy of block-type frequencies is higher than the entropy by our previous generator and that two requirements to achieve playable RGM levels – the 100% stability and the perfect-shot rate – are met. The results indicate that the proposed method can generate levels with more diversity than our previous generator while maintaining their playability.
Research Interests:
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our... more
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our system consists of two major parts: level generator and game system. The level generator generates a level according to a given file that defines the level appearance of a pixel image. The game system includes emotion recognition and fog control; to encourage the player to smile, fog is first generated that covers the game screen, but when the player smiles it will gradually disappear. It is expected that the proposed system is useful in promoting the affect of its players, i.e., increasing the positive affect and decreasing the negative one.
Research Interests:
Research Interests:
Research Interests:
This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed... more
This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of players. Our preliminary results show that spatial navigational patterns in both systems are highly correlated, indicating that the proposed system is likely as effective as the SHQ system for the detection task.
Research Interests: Computer Science, Serious Games, Disease, MoD, Hero, and 3 moreEarly Detection, Minecraft, and arXiv
Research Interests:
This paper proposes a motion-gaming AI for health promotion that can adapt to the player's behavior change in an effective manner. Through modeling of the player's behavior and predicting of their counteraction, this AI learns how... more
This paper proposes a motion-gaming AI for health promotion that can adapt to the player's behavior change in an effective manner. Through modeling of the player's behavior and predicting of their counteraction, this AI learns how its actions can induce its opponent player to move. The proposed AI aims at suppressing health risks associated with motion gaming, by improving balancedness in use of body segments, as well as at increasing the level of calories consumption.
Research Interests:
This paper describes analysis of relationship between the player's behavior change and the effectiveness of a Health Promotion AI (HP-AI). We measured player's behavior change by Jensen-Shannon divergence (JSD). Our results show... more
This paper describes analysis of relationship between the player's behavior change and the effectiveness of a Health Promotion AI (HP-AI). We measured player's behavior change by Jensen-Shannon divergence (JSD). Our results show that the effectiveness of HP-AI becomes higher when a player has low JSD.
Research Interests:
This paper presents our work to enhance a state-of-the-art level generator (Sketch-to-Level Generator) that generates levels for an Angry-Birds-like game from drawn sketches. To achieve this task, Cycle-Consistent Adversarial Networks... more
This paper presents our work to enhance a state-of-the-art level generator (Sketch-to-Level Generator) that generates levels for an Angry-Birds-like game from drawn sketches. To achieve this task, Cycle-Consistent Adversarial Networks (Cycle-GAN) are used. CycleGAN is trained using two datasets: sketch drawings and typical level-structures. The former are taken from Google’s Quick, Draw! datasets, and the latter from the winning level generator at the 2017 and 2018 AIBIRDS level generation competitions. The output of the trained CycleGAN is used as the input of Sketch-to-Level Generator. Our results show that the proposed preprocessing technique using CycleGAN allows Sketch-to-Level Generator to more successfully generate levels from arbitrary sketch drawings.
Research Interests:
Research Interests:
Research Interests:
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion... more
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion games, the user faces problems in memorization of motions or performing of repetitive movements, leading to an unbalanced use of body segments. This paper proposes a solution for these problems, and particularly we focus on the benefits of having an intelligent assistant in full-body motion games for giving instructions, training and recommendation of a variation of movements. Our development consists of two major modules: a during-gameplay motion recommender and a pre-gameplay instructor.
Research Interests:
Research Interests:
This paper provides a survey of existing systems and discusses the purposes that music therapy and video games serve. The effects of musical games and singing on psychological and physical well-being, learning and cognitive training are... more
This paper provides a survey of existing systems and discusses the purposes that music therapy and video games serve. The effects of musical games and singing on psychological and physical well-being, learning and cognitive training are discussed. According to previous research, singing, as a part of music therapy, has proved to impact stress hormones positively, and also commonly contributed to medical rehabilitation. This paper, therefore, aims to shed light on the prospects of previous studies inspiring future development of voice-input based games for music therapy and stress reduction.
Research Interests:
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently... more
This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently purposed concept called Player Dominance Adjustment, where the AI determines its actions based on the player’s inputs so as to adjust the player’s dominant power. The basic idea is to let the player dominate the game when they perform healthy movement and on the contrary to have the AI take a strong action against the player when she or he performs unhealthy movement. The AI outperforms an existing dynamic difficulty adjustment AI designed for the same propose.
Research Interests:
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts.... more
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts. A targeted percentage of movement can be specified for each part of the body; the game will generate a level that makes the player’s movement match the targeted percentages.
Research Interests:
This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for... more
This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for playing games by using hand motions. It works by detecting hand motions through image processing, translating them into commands for keyboard press, and sending such commands to the target game application; it works with any existing games without the need to modify the game source code and only requires an off-the-self webcam. An experiment is conducted on a jumping game, where three types of game interfaces: a keyboard, a Kinect, and the proposed interface are compared. The results show that playing games using hand clenching/spreading leads to the best memory test performance according to the N-back test, a commonly used cognitive task for the assessment of working memory.
Research Interests:
This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one shot of a bird. The proposed method... more
This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one shot of a bird. The proposed method generates a level by selecting predefined segments consisting of several objects arranged in a way that creates a domino effect among them. To increase the variability of generated levels, the proposed method procedurally generates a varying structure on the top of certain blocks in a predefined segment. Our results show that the proposed RGM generator is comparable to two existing generators, including the winner of the 2018 AIBIRDS Level Generation Competition, in terms of stability while it outperforms both baseline generators with respect to running time and an expressivity metric called "dynamic" which is introduced in this work to measure the time period where moving objects, including a shooting bird, reside in a given level. In additio...
Research Interests:
Research Interests:
This study improves JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), an audience participation game with a purpose (APGWAP) designed for promoting ukiyo-e artworks and collecting descriptions about them from a group of people... more
This study improves JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), an audience participation game with a purpose (APGWAP) designed for promoting ukiyo-e artworks and collecting descriptions about them from a group of people over a game live streaming platform, by adding a musical-style TTS that narrates the best description made by the player for each image in each round. It is expected that narrating the best descriptions for each image will make the winner feel more rewarded and encouraged to make better descriptions in the upcoming rounds. The narration is provided to players in a musical fashion considering evidences that music has several therapeutic benefits, including stress relief and mental health promotion.
Research Interests:
This study introduces a virtual animated character (VTuber) to JUSTIN, an audience participation game with a purpose (APGWAP) designed for collecting descriptions for Ukiyo-e artworks. The inspiration of this study is based on the... more
This study introduces a virtual animated character (VTuber) to JUSTIN, an audience participation game with a purpose (APGWAP) designed for collecting descriptions for Ukiyo-e artworks. The inspiration of this study is based on the popularity of VTubers, which demonstrate promising results in terms of increasing positive user experience and outcomes on serious purposes. The introduced character serves as a moderator, focusing on promoting active and entertaining gameplay. It is expected that such a moderator would help participants create better descriptions and demonstrate higher involvement in the process. This moderator is also designed under generalization concept, namely, the system with this moderation can be reused on various games.
Research Interests:
This paper proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player.... more
This paper proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player. Results on unseen data show the effectiveness of the proposed AI when its position is close to the settlement center.
Research Interests:
This paper proposes a method for creating informative descriptive sentences for artwork images by using “Games With a Purpose,” or GWAP on a live streaming platform. In existing studies, automatic annotation of images did not perform... more
This paper proposes a method for creating informative descriptive sentences for artwork images by using “Games With a Purpose,” or GWAP on a live streaming platform. In existing studies, automatic annotation of images did not perform well, in particular, for artwork images such as Ukiyo-e. On the other hand, existing studies on GWAP, the concept of using games for human to address problems that computer cannot solve, demonstrated how games can exploit human intelligence in labeling images. With a huge number of audiences watching games live streaming on Twitch and other similar services nowadays, we propose a solution that makes chatting of audiences become an effective means for generating valuable descriptions for artwork images, Ukiyo-e images in this study, as well as promoting the interaction between the audiences and the game.
Research Interests:
This paper proposes a method that use a Variational Autoencoder (VAE) neural network to change the style of audiences’ favorite music to sleepy style one, by which our purpose of making them relaxed is achieved. Not only music can... more
This paper proposes a method that use a Variational Autoencoder (VAE) neural network to change the style of audiences’ favorite music to sleepy style one, by which our purpose of making them relaxed is achieved. Not only music can influence our emotion, but also sleepiness. Some people like to listen to sleepy music to aid soothing sleep, and responding to this demand, there are currently many online channels streaming sleepy music all day. Nevertheless, the variety of music being streamed is limited. This study aims at applying style transfer techniques on music data to generate sleepy-style music from existing non-sleepy-style music. We believe our proposed technique will not only solve the above limitation but also bring music composers new ideas and inspirations.
Research Interests:
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A... more
This paper proposes a TNT-generation system in Science Birds, an Angry Birds clone, which is aimed to help spectators maintain healthy emotion. Interestingness-rating data on TNT explosions are collected from a number of participants. A regressor is then built from the collected data whose input consists of a set of features representing effects due to an explosion and output indicates the average interestingness. A user study is then conducted where participants are asked to answer a questionnaire measuring their positive and negative affect after watching each of the three gameplay videos - the explosions with the highest average interestingness, the explosions with the lowest average interestingness, the explosions with the highest interestingness predicted by the regressor - in counterbalanced order. Our results show that the first gameplay video statistically significantly increases the positive affect of the participants and that both first and third videos statistically signi...
Research Interests:
This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which... more
This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which allows it to be completed by only one bird shooting. By evolving the feasible population and infeasible population in CNS, our results show that the entropy of block-type frequencies is higher than the entropy by our previous generator and that two requirements to achieve playable RGM levels – the 100% stability and the perfect-shot rate – are met. The results indicate that the proposed method can generate levels with more diversity than our previous generator while maintaining their playability.
Research Interests:
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our... more
This paper presents a system aimed at encouraging players to smile when playing an Angry Birds-like game by generating levels whose designs look like pixel images and explicitly requiring the players to smile to see such levels. Our system consists of two major parts: level generator and game system. The level generator generates a level according to a given file that defines the level appearance of a pixel image. The game system includes emotion recognition and fog control; to encourage the player to smile, fog is first generated that covers the game screen, but when the player smiles it will gradually disappear. It is expected that the proposed system is useful in promoting the affect of its players, i.e., increasing the positive affect and decreasing the negative one.
Research Interests:
This paper presents a method for adjustment of game difficulty and commentary generation based on audiences’ input to be used in Audience Participation Games(APGs) on Twitch Background We found that difficulty adjustment is one... more
This paper presents a method for adjustment of game difficulty and commentary generation based on audiences’ input to be used in Audience Participation Games(APGs) on Twitch Background We found that difficulty adjustment is one effective method for promoting player experience in APGs [1] . The game commentary was reported to be effective in keeping audiences entertained and informed. Additionally, rap style commentary [2] was reported positively affect emotions and promote mental health.
Research Interests:
This paper provides a survey on Games With a Purpose (GWAP) and Audience Participation Games (APG). Possible potentials from combining GWAP’s and APG’s mechanisms on live streaming platforms for creating informative descriptive data for... more
This paper provides a survey on Games With a Purpose (GWAP) and Audience Participation Games (APG). Possible potentials from combining GWAP’s and APG’s mechanisms on live streaming platforms for creating informative descriptive data for artwork is further discussed.
Research Interests:
This paper proposes a method of using Variational Autoencoder (VAE) neural network to change the style of people’s favorite music to sleepy style, thereby achieving the purpose of making audiences relax. Not only music can influence our... more
This paper proposes a method of using Variational Autoencoder (VAE) neural network to change the style of people’s favorite music to sleepy style, thereby achieving the purpose of making audiences relax. Not only music can influence our emotion, but also sleepiness. Some people like to listen to sleepy music to aid soothing sleep, and responding to this demand, we can see many online channels streaming sleepy music all day. Nevertheless, resources for such streaming are limited. This study aims at applying style transfer techniques on music data to generate sleepy-style music from existing non-sleepy-style ones. We believe our proposed technique will not only solve the above resource problem but also lit music composers new ideas and inspiration.
Research Interests:
In 2020, Nguyen et al. introduced JUSTIN, audience participation game for collecting Ukiyo e artworks[1] • This work aims at improving JUSTIN by adding a virtual character as a moderator • Avatar explains rules of the game as it goes •... more
In 2020, Nguyen et al. introduced JUSTIN, audience participation game for collecting Ukiyo e artworks[1] • This work aims at improving JUSTIN by adding a virtual character as a moderator • Avatar explains rules of the game as it goes • She aims at improving the experience of players by entertaining them through the game • Improved experience may result in a higher quality of artwork descriptions
Research Interests:
This paper presents a new tool for detecting the cognitive decline by using the neuropsychological screening test, as described in [1] that for detection of cognitive decline, five BrainCheck’s [2] assessments can be used.
Research Interests:
This paper proposes Player Dominance Adjustment Health Promotion AI (PDAHP AI) that encourages players to use their body parts in a well balanced manner and outperform s our previous DDAHP AI[1].
Research Interests:
This paper presents an idea of promoting gamerelated self-efficacy and user experience of game players by adjusting their dominant power. The assumption is that game players will gain higher self-efficacy and better gameplay experience... more
This paper presents an idea of promoting gamerelated self-efficacy and user experience of game players by adjusting their dominant power. The assumption is that game players will gain higher self-efficacy and better gameplay experience when game situations go in the way they expect. To evaluate this assumption, we conduct a pilot study using a famous social deduction game,Werewolf by letting one human participant plays this game, one at a time over the internet. Each participant thinks that he/she played with other six players, but all the other players were in fact controlled by the experimenter. Game situations are manipulated to create two cases of gameplay: (1) a case in which most of the participant’s actions impact gameplay (player-dominance games), and (2) a case in which almost none of the participant’s actions impact gameplay (nonplayer- dominance games). Based on evaluation using General Self-Efficacy (GSE) scale and Game User Experience Satisfaction Scale (GUESS), player-dominance games were found leading to higher self-efficacy, and there is a strong linear relationship between self-efficacy and enjoyment.
Research Interests:
This work proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player.... more
This work proposes an aerial cinematographer AI for shooting videos of settlements in Minecraft worlds. A supervised learning approach is adopted to train the AI using a decision tree with movement behavior data of a target player. Results on unseen data show the effectiveness of the proposed AI when its position is close to the settlement center.
Research Interests:
This paper proposes a maze game for cognitive assessment [1]. It has a singing mini-game to help relieving stress when players get lost. The navigation will show to help the player reach the finish point when they sing.
Research Interests:
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts.... more
This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts. A targeted percentage of movement can be specified for each part of the body; the game will generate a level that makes the player’s movement match the targeted percentages.
Research Interests:
Motion gaming, the style of game playing by using motion sensors to detect movement of the player’s whole body, encourages players to exercise while playing game. However, to play motion games bears risk of muscle damage that may occur if... more
Motion gaming, the style of game playing by using motion sensors to detect movement of the player’s whole body, encourages players to exercise while playing game. However, to play motion games bears risk of muscle damage that may occur if some parts of the body move too excessively. Therefore, recently, we have proposed a Health Promoting AI (HP-AI) for suppressing health risks from motion gaming through the means of balancing use of body segments [1]. In this paper we further analyze relationship between change in the player’s behavior to health promotion results.
Research Interests:
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion... more
This paper presents a system that encourages players to use their body segments in a balance fashion and perform more various motions. To achieve a healthy status, the balance in the use of body segments is imperative. In general motion games, the user faces problems in memorization of motions or performing of repetitive movements, leading to an unbalanced use of body segments. This paper proposes a solution for these problems, and particularly we focus on the benefits of having an intelligent assistant in full-body motion games for giving instructions, training and recommendation of a variation of movements. Our development consists of two major modules: a during-gameplay motion recommender and a pre-gameplay instructor.
Research Interests:
This paper proposes a method for creating informative descriptive sentences for artwork images by using GWAP on a live streaming platform.
Research Interests:
This work explores other possibilities of AR in music learning by proposing an application that lets users interact with a virtual character. The application is expected to present a fun way to learn music and keep students’ motivation.
Research Interests:
This paper proposes Dynamic Difficulty Adjustment Health Promotion AI (DDAHP AI) that encourages players to use their body parts in a well balanced manner while promoting their experience
Research Interests:
Descriptions are needed to help the user understand the content of images, make more accurate search engines, and assist visually impaired people.
JUSTIN is an APGWAP for collecting descriptions
JUSTIN is an APGWAP for collecting descriptions
Research Interests:
This paper presents a design of live-streaming motion game system with audience-participation mechanisms. The design is applied on a fighting game, in which two on-screen characters fight each other. Each character in the game is mainly... more
This paper presents a design of live-streaming motion game system with audience-participation mechanisms. The design is applied on a fighting game, in which two on-screen characters fight each other. Each character in the game is mainly controlled by an artificial intelligence algorithm, but human audiences can request the AI to execute specific actions by performing hand motions. This game system is designed for promoting well-being of live-streaming audiences.
Research Interests:
Uncut version of "Encourage Players to Smile While Playing Games Bring More Enjoyment" published in CoG2019 Proceedings [ABSTRACT] This paper proposes a video game system that encourages the player to smile to increases player experience.... more
Uncut version of "Encourage Players to Smile While Playing Games Bring More Enjoyment" published in CoG2019 Proceedings [ABSTRACT] This paper proposes a video game system that encourages the player to smile to increases player experience. We enhance an existing endless runner game called "Runner" with a mechanism, defined as a smile mechanism, that allows smiles to be detected as an input to the game system. We assess player experience using an existing player experience scale called Game User Experience Satisfaction Scale. Our results show that the introduction of the smile mechanism to Runner leads to better player experience, especially in terms of enjoyment. This paper also reports our findings in the relationship between player experience and player performance.
Research Interests:
This report proposes an improved version of JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), a game introduced to promote ukiyo-e artworks and collect description about them, by adding rap-style TTS to rap the descriptions and... more
This report proposes an improved version of JUSTIN (Japanese Ukiyo-e Streaming That Improves Narrative), a game introduced to promote ukiyo-e artworks and collect description about them, by adding rap-style TTS to rap the descriptions and votes made by the player. Rap-style TTS implemented to JUSTIN is a system introduced to rap the comments generated by AI in fighting games. By doing this we believe that the system will be able to enhance the player’s experience, and the result showed that using TTS, either normal or rap-style TTS, can relatively increase the player’s experience.
Research Interests:
This research presents an improved and modified version of Angry-Birds-like game that uses singing as input. While we feature increased variability of generated levels, we aim towards promotion of mental well-being and improved mood using... more
This research presents an improved and modified version of Angry-Birds-like game that uses singing as input. While we feature increased variability of generated levels, we aim towards promotion of mental well-being and improved mood using the game system Science Birds and singing therapy. In this paper, not only did we achieve diverse and non-linear levels, but also, randomized collapse flow of the objects and shooting directions of the birds in the levels. We used singing as an input modality to play the game as research has shown singing therapy can effectively reduces stress and is used as a rehabilitation tool for patients.
Research Interests:
Runner is a 2D platformer game that can be used as a tool to conduct experiments on the promotion of mental and physical health through video games. The levels of the game will be created procedurally. As part of the promotion of mental... more
Runner is a 2D platformer game that can be used as a tool to conduct experiments on the promotion of mental and physical health through video games. The levels of the game will be created procedurally. As part of the promotion of mental health, it allows to adapt the contents of the game according to the emotion of the player, and to watch the behavior of the player following these changes.