KR20250079559A - Apparatus and method for ai matching, recommendation, feedback coaching based on digital phenotyping - Google Patents

Abstract

The present invention relates to an AI matching, recommendation, and feedback coaching device and method based on digital phenotyping, wherein the device comprises: a data collection unit for collecting data on behavioral patterns of users in the process of using digital content; a data analysis unit for preprocessing and analyzing the data to produce at least one digital indicator on the behavioral pattern; a coaching feedback generation unit for generating coaching feedback on the behavioral pattern of each user using an AI coaching model built on the basis of a dataset on the digital indicator; a coaching report generation unit for providing the coaching feedback to an administrator terminal to generate a coaching report including the signature of the administrator; and a coaching report provision unit for providing the coaching report to each user through a user interface.

Description

{APPARATUS AND METHOD FOR AI MATCHING, RECOMMENDATION, FEEDBACK COACHING BASED ON DIGITAL PHENOTYPING}

The present invention relates to AI coaching service technology, and more specifically, to an AI matching, recommendation, and feedback coaching device and method based on digital phenotyping that collects various digital signals while a user uses a smart device, analyzes the user's current emotional state or mental health state from a set of collected digital signals, and provides information generated by AI.

Digital phenotyping can mean continuously quantitatively evaluating users' cognitive, emotional, behavioral, physiological, social, and environmental indicators by collecting them at any moment in their daily lives without visiting a hospital using personal digital devices such as smartphones. Digital phenotyping is attracting attention as a new mental health measurement tool because it can provide multidimensional and large-scale objective information, and it is expected to be used as a tool for personalized medicine (preventive medicine, predictive medicine, participatory medicine, and precision medicine) that seeks to improve the diagnostic accuracy of psychiatry by reflecting individual patient data or establishing a new dimensional diagnostic system of RDoC.

Recently, there have been attempts to collect and analyze users' biometric indicators, mobile phone usage patterns, and behavioral characteristics through mobile devices and wearable devices, but there are limitations in that the scope of utilization is very limited in collecting and analyzing massive amounts of data.

In particular, in public education settings such as schools, there may be many restrictions on data collection due to the high sensitivity to personal information and opposition from parents. Therefore, digital phenotyping technology is required that can non-invasively collect and utilize information such as touch, stroke, and stylus pen usage behavior other than sensitive personal information such as face, facial expression, body behavior, word or sentence expression in writing or speech, and social media posts.

Korean Publication No. 10-2021-0076462 (2021.06.24)

One embodiment of the present invention provides an AI matching, recommendation, and feedback coaching device and method based on digital phenotyping that collects various digital signals while a user uses a smart device, analyzes the user's current attention, learning, emotional state, or mental health state from a set of collected digital signals, and provides AI-generated information.

Among the embodiments, the AI coaching device includes a data collection unit that collects data on behavioral patterns of users in the process of using digital content; a data analysis unit that preprocesses and analyzes the data to produce at least one digital indicator on the behavioral pattern; a coaching feedback generation unit that generates coaching feedback on the behavioral pattern of each user using an AI coaching model built on the basis of a dataset on the digital indicator; a coaching report generation unit that provides the coaching feedback to an administrator terminal to generate a coaching report including the signature of the administrator; and a coaching report provision unit that provides the coaching report to each user through a user interface.

The above data collection unit collects multimodal data on user behavior through an application executed on a user terminal during a digital learning process on the user terminal, and the multimodal data may include operation indicators due to the user behavior on a display screen of the user terminal.

The above data collection unit can collect keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process in the digital learning process as the multimodal data.

The data analysis unit can determine at least one digital indicator from among a plurality of indicators including stroke acceleration, stroke speed, vertical distance, number of blank spaces, and time interval.

The above coaching feedback generation unit can provide feedback and recommendations reflecting the user's attention, emotions, and mental health based on at least one digital indicator of the user's behavioral pattern collected during the process of using digital content.

The above coaching feedback generation unit can generate the coaching feedback including visualized information regarding the at least one digital indicator and an evaluation text of a behavioral pattern associated with the digital indicator.

The above coaching report generation unit can calculate a correlation index between the visualized information and the evaluation text for each digital indicator, and independently receive the signature of the corresponding manager for the corresponding evaluation text for a digital indicator whose correlation index is below a preset threshold value on the manager terminal.

Among the embodiments, the AI coaching method includes: a step of collecting data on behavioral patterns of users in the process of using digital content through a data collection unit; a step of preprocessing and analyzing the data through a data analysis unit to produce at least one digital indicator on the behavioral pattern; a step of generating coaching feedback on the behavioral pattern of each user using an AI coaching model built based on a dataset on the digital indicator through a coaching feedback generation unit; a step of providing the coaching feedback to an administrator terminal through a coaching report generation unit to generate a coaching report including the signature of the administrator; and a step of providing the coaching report to each user through a user interface through a coaching report provision unit.

The disclosed technology may have the following effects. However, this does not mean that a specific embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

A digital phenotyping-based AI matching, recommendation, and feedback coaching device and method according to one embodiment of the present invention can collect various digital signals while a user uses a smart device, and analyze the user's current attention, learning, emotional state, or mental health state from a set of collected digital signals to provide AI-generated information.

The AI matching, recommendation, and feedback coaching device and method based on digital phenotyping according to one embodiment of the present invention can be expanded from the market for elementary, middle, and high schools, students, parents, and teachers, and from public education institutions to the private education market, and can improve and supplement the inconveniences of the existing system and provide various data to the digital mental health-related market that has been growing since COVID-19.

Figure 1 is a drawing explaining an AI coaching system according to the present invention.
Figure 2 is a drawing explaining the functional configuration of the user terminal of Figure 1.
Figure 3 is a drawing explaining the system configuration of the AI coaching device of Figure 1.
Figure 4 is a drawing explaining the functional configuration of the AI coaching device of Figure 1.
Figure 5 is a flowchart explaining an AI coaching method based on digital phenotyping according to the present invention.
Figure 6 is a diagram explaining the correlation and multiple regression results between the IGDS and the indicator according to the present invention.
FIG. 7 is a diagram illustrating one embodiment of an AI coaching process according to the present invention.

The description of the present invention is only an embodiment for structural and functional explanation, so the scope of the rights of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be variously modified and can have various forms, the scope of the rights of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purpose or effect presented in the present invention does not mean that a specific embodiment must include all of them or only such effects, so the scope of the rights of the present invention should not be understood as being limited thereby.

Meanwhile, the meanings of the terms described in this application should be understood as follows.

Terms such as "first", "second", etc. are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

When it is said that a component is "connected" to another component, it should be understood that it may be directly connected to that other component, but there may also be other components in between. On the other hand, when it is said that a component is "directly connected" to another component, it should be understood that there are no other components in between. Meanwhile, other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

A singular expression should be understood to include the plural expression unless the context clearly indicates otherwise, and terms such as "comprises" or "have" should be understood to specify the presence of a feature, number, step, operation, component, part, or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

In each step, the identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of each step, and each step may occur in a different order than stated unless the context clearly indicates a specific order. That is, each step may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

The present invention can be implemented as a computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all kinds of recording devices that store data that can be read by a computer system. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. In addition, the computer-readable recording medium can be distributed over network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner.

All terms used herein, unless otherwise defined, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the contextual meaning of the relevant art, and shall not be interpreted as having an ideal or overly formal meaning unless explicitly defined in this application.

Figure 1 is a drawing explaining an AI coaching system according to the present invention.

Referring to FIG. 1, the AI coaching system (100) may include a user terminal (110), an AI coaching device (130), and a database (150).

The user terminal (110) may correspond to a terminal device operated by a user. In the embodiment of the present invention, a user may be understood as one or more users, and multiple users may be classified into one or more user groups. Each of the one or more users may correspond to one or more user terminals (110). That is, a first user may correspond to a first user terminal, a second user may correspond to a second user terminal, ..., an n-th user (where n is a natural number) may correspond to an n-th user terminal.

In addition, the user terminal (110) may correspond to a computing device that can participate in the process of using digital content by linking with the AI coaching device (130) as a device that constitutes the AI coaching system (100). Here, the digital content is content that exists in the form of digital data, and may correspond to content such as text, voice, sound, image, and video that is produced or processed in a digital format by digitizing existing analog content. For example, the digital content may include games, animations, mobile content, digital images, electronic learning, electronic books, and digital audio. In particular, electronic learning (or digital learning) may correspond to a learning process in which online participation is made in an educational process conducted in an Internet space, such as non-face-to-face education and online education.

In addition, the user terminal (110) may be implemented as a smartphone, laptop, or computer that can be connected to and operated by the AI coaching device (130), but is not necessarily limited thereto and may also be implemented as various devices including tablet PCs, etc. In particular, the user terminal (110) may install and execute a dedicated program or application for linking with the AI coaching device (130).

For example, the user terminal (110) can participate in a dedicated online learning space provided by the AI coaching device (130) to perform digital learning as one of the digital contents, and can participate in a problem-solving process in which an evaluation is performed by entering the correct answer to a given problem during the digital learning process. In this case, the user terminal (110) may correspond to a terminal of a learner (or student) participating in online learning. As another example, the user terminal (110) may participate in the online learning space as an administrator conducting an online learning process. In this case, the user terminal (110) may correspond to a terminal of a teacher (or instructor) conducting online learning.

In addition, the user terminal (110) can provide an interface for using digital content through a dedicated program or application. That is, when using digital learning, the user can participate online in an educational course conducted in an Internet space, such as non-face-to-face education or online education, through the interface, and the user terminal (110) can collect pattern data on the user's behavior through interaction with the interface. Meanwhile, the user terminal (110) can be connected to the AI coaching device (130) through a network, and a plurality of user terminals (110) can be connected to the AI coaching device (130) simultaneously.

The AI coaching device (130) may be implemented as a server corresponding to a computer or program that performs the AI coaching method based on digital phenotyping according to the present invention. In addition, the AI coaching device (130) may be connected to the user terminal (110) via a wired network or a wireless network such as Bluetooth, WiFi, LTE, etc., and may transmit and receive data with the user terminal (110) via the network.

In addition, the AI coaching device (130) may be implemented to operate in connection with an independent external system (not shown in FIG. 1) in order to perform the AI coaching method based on digital phenotyping according to the present invention. For example, the AI coaching device (130) may operate in connection with a learning system that provides online learning, a learning management system that manages learning history, an artificial intelligence system that builds an AI model, etc.

Meanwhile, for convenience of explanation, the user terminal (110) and the AI coaching device (130) are expressed as independent devices here, but this is not necessarily limited to this, and it goes without saying that one device may be implemented by being included in the other device.

The database (150) may correspond to a storage device that stores various information required during the operation of the AI coaching device (130). For example, the database (150) may store information on content data for using digital content or data on behavioral patterns of users collected during the use of digital content. However, the database (150) is not necessarily limited thereto, and may store information collected or processed in various forms during the process of the AI coaching device (130) performing the AI coaching method based on digital phenotyping according to the present invention.

In addition, in FIG. 1, the database (150) is depicted as a device independent of the AI coaching device (130), but it is not necessarily limited thereto and can of course be implemented as a logical storage device included in the AI coaching device (130).

Figure 2 is a drawing explaining the functional configuration of the user terminal of Figure 1.

Referring to FIG. 2, the user terminal (110) can execute a dedicated program or application for using digital content. For example, the user terminal (110) can execute an online learning program for digital learning or a non-face-to-face meeting application, and perform an operation for collecting various multimodal data related to user behavior during a digital learning process on a dedicated interface.

To this end, the user terminal (110) may be implemented including independent modules for monitoring the user's behavior and collecting data. Specifically, the user terminal (110) may include a multimodal data collection module (210), a foreground process monitoring module (230), and a web browser monitoring module (250). At this time, the embodiment of the present invention does not have to include all of the above configurations at the same time, and some of the above configurations may be omitted or selectively included in some or all of the above configurations according to each embodiment.

The multimodal data collection module (210) can collect various passive sensor data, keyboard input data, and multimodal data. To this end, the multimodal data collection module (210) can operate in conjunction with various sensors included in the user terminal (110). For example, the multimodal data collection module (210) can periodically collect data information from passive sensors such as angular velocity, acceleration, and illuminance sensors. The multimodal data collection module (210) can collect data such as keyboard values and time when a user inputs a keyboard, and can collect data when an event such as a user using a stylus or touching with a finger occurs. The data collected by the multimodal data collection module (210) can be stored and kept in an internal memory, and can be periodically transmitted to the AI coaching device (130) by the user terminal (110).

The foreground process monitoring module (230) can perform an operation to detect the execution of a process, and for this purpose, can include a command to check the package name of the foreground process. The foreground process monitoring module (230) can perform process monitoring as a periodic task, and can collect and record information on the foreground process using the display screen of the user terminal (110) at preset intervals.

The web browser monitoring module (250) can perform an operation of examining information of a web page that a user accesses through a web browser on a user terminal (110). The web browser monitoring module (250) can operate in an event-driven manner in which a processing function is executed whenever an event occurs, and can collect and record text information whenever a user interacts with a text input window (e.g., a search window, an address window, etc.) of a web browser.

Figure 3 is a drawing explaining the system configuration of the AI coaching device of Figure 1.

Referring to FIG. 3, the AI coaching device (130) may include a processor (310), a memory (330), a user input/output unit (350), and a network input/output unit (370).

The processor (310) can execute an AI coaching procedure based on digital phenotyping according to an embodiment of the present invention, manage a memory (330) that is read or written in the process, and schedule a synchronization time between a volatile memory and a non-volatile memory in the memory (330). The processor (310) can control the overall operation of the AI coaching device (130), and is electrically connected to the memory (330), the user input/output unit (350), and the network input/output unit (370) to control data flow therebetween. The processor (310) can be implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the AI coaching device (130).

The memory (330) may include an auxiliary memory device implemented as a non-volatile memory such as an SSD (Solid State Disk) or an HDD (Hard Disk Drive) and used to store all data required for the AI coaching device (130), and may include a main memory device implemented as a volatile memory such as a RAM (Random Access Memory). In addition, the memory (330) may store a set of commands that execute the AI coaching method based on digital phenotyping according to the present invention by being executed by an electrically connected processor (310).

The user input/output unit (350) includes an environment for receiving user input and an environment for outputting specific information to the user, and may include, for example, an input device including an adapter such as a touch pad, a touch screen, a visual keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input/output unit (350) may correspond to a computing device connected via a remote connection, and in such a case, the AI coaching device (130) may be performed as an independent server.

The network input/output unit (370) provides a communication environment for connecting to a user terminal (110) via a network, and may include, for example, an adapter for communication such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), and a Value Added Network (VAN). In addition, the network input/output unit (370) may be implemented to provide a short-distance communication function such as WiFi or Bluetooth, or a wireless communication function of 4G or higher for wireless transmission of data.

Fig. 4 is a drawing explaining the functional configuration of the AI coaching device of Fig. 1.

Referring to FIG. 4, the AI coaching device (130) can perform an AI coating method based on digital phenotyping according to the present invention. To this end, the AI coaching device (130) can include a data collection unit (410), a data analysis unit (430), a coaching feedback generation unit (450), a coaching report generation unit (470), a coaching report provision unit (490), and a control unit (not shown in FIG. 4).

At this time, the embodiments of the present invention do not have to include all of the above configurations at the same time, and some of the above configurations may be omitted or selectively included depending on each embodiment. Hereinafter, each configuration will be described in more detail.

The data collection unit (410) can collect data on the behavioral patterns of users during the process of using digital content. To this end, the data collection unit (410) can be implemented to operate in conjunction with the user terminal (110). For example, the data collection unit (410) can receive various data collected through a dedicated program or application running on the user terminal (110) from the dedicated program or application and store them in the database (150).

In one embodiment, the data collection unit (410) may collect multimodal data on user behavior through an application executed on the user terminal (110) during a digital learning process on the user terminal (110). At this time, the multimodal data may include action indicators by the user behavior on the display screen of the user terminal (110). For example, the multimodal data may include data collected from a touch or stroke action by the user on a touch screen. In particular, the data collection unit (410) may collect various behavioral information occurring during the process of the user using a dedicated application for a specific purpose as multimodal data.

In one embodiment, the data collection unit (410) may collect, as multimodal data, keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and the touchscreen sensor of the user terminal (110) during the problem-solving process in the digital learning process. Here, the problem-solving process is one of the digital learning processes, and may correspond to a process in which a problem is provided through the display screen of the user terminal (110) and the user reads the problem and inputs the correct answer. At this time, the user may input the correct answer by touching the display screen or using the keyboard displayed on the screen, and the data collection unit (410) may collect, as a motion index, multimodal data collected through the display screen among the user actions occurring during the problem-solving process or the correct answer input process.

In one embodiment, when the data collection unit (410) collects data on a user's behavioral pattern during a problem-solving process in a digital learning process, the data collection unit (410) may select and provide customized problems and collect multimodal data corresponding thereto. That is, the data collection unit (410) may provide the user with customized problems composed of problems at a level suitable for the user by considering the user's gender, age, academic ability, etc., and may collect keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal (110) while solving the customized problems as multimodal data. The data collection unit (410) may classify the collected multimodal data by difficulty (or level) of the customized problems and store them in the database (150).

The data analysis unit (430) can preprocess and analyze data to produce at least one digital indicator regarding a behavioral pattern. The user's behavioral pattern may correspond to a regularity of a series of actions repeated in the user's behavioral process. For example, if the user has symptoms such as Internet Gaming Disorder (IGD), a specific pattern may exist in the user's behaviors. The data analysis unit (430) can extract digital indicators from the user's behavioral data and extract a pattern regarding the user's behavior based on the digital indicators.

In addition, the data analysis unit (430) may perform preprocessing operations on data collected from users in order to more accurately calculate various digital indicators. For example, the data analysis unit (430) may perform preprocessing operations such as missing data processing and denoising on the collected data.

In one embodiment, the data analysis unit (430) can determine at least one digital indicator from among a plurality of indicators including stroke acceleration, stroke speed, vertical distance, number of blank spaces, and time interval. Specifically, the stroke acceleration may correspond to a rate of change in speed while making a stroke on the touchscreen, the stroke speed may correspond to a number of pixels per second or a distance, and the vertical distance may correspond to a distance from the top of the screen to a starting position during a stroke. In addition, the number of blank spaces may correspond to the number of times the spacebar is pressed between letters, words, and sentences, and the time interval may correspond to the time (ms) between touching one key and touching another key.

For example, in the correlation between Internet Game Addiction (IGD) and digital indicators, the IGDS score, which is a measure of game addiction, can show a positive correlation with indicators of stroke acceleration, stroke speed, and vertical distance, and can show a negative correlation with indicators of the number of blanks and time interval. In this way, the data analysis unit (430) can selectively extract digital indicators that have a high correlation with Internet game addiction from among 83 digital indicators collected during the digital learning process. As another example, the data analysis unit (430) can selectively extract digital indicators that have a high correlation with the user's emotional state or mental health state from among digital indicators collected during the problem-solving process during the digital learning process.

That is, the data analysis unit (430) can selectively extract digital indicators that have a high correlation with the digital content usage process and user behavior patterns among various digital indicators through data analysis, and the selected digital indicators can then be utilized in the coaching feedback generation stage.

The coaching feedback generation unit (450) can generate coaching feedback on the behavioral pattern of each user using an AI coaching model built based on a dataset on digital indicators. Here, the AI coaching model may correspond to an artificial intelligence model that generates coaching feedback for the user as output based on input data on the user's behavioral pattern. At this time, the coaching feedback may correspond to a report generated by analyzing data collected during the user's use of digital content, and may include analysis results on the user's behavioral pattern, mental health status, and academic achievement. The dataset used for training the AI coaching model may include digital indicators on the user's behavioral pattern collected during the use of content, and the user's behavioral pattern, mental health status, and academic achievement that match the digital indicators.

In addition, the AI coaching model can be implemented by including one or more sub-models. For example, the AI coaching model can include multiple sub-models that independently generate analysis results according to behavioral patterns, mental health conditions, and academic achievements based on inputs composed of digital indicators. In other words, the AI coaching model can ultimately generate coaching feedback for the user as a result of integrating the outputs of each sub-model. In addition, the AI coaching model can be designed according to various structures, including sequential connections, parallel connections, etc. between one or more sub-models.

In one embodiment, the coaching feedback generation unit (450) can generate coaching feedback including visualized information about at least one digital indicator and evaluation text of a behavioral pattern associated with the digital indicator. The AI coaching model can analyze a correlation between digital indicators received as input and then output the analysis result for the corresponding user as evaluation text. The coaching feedback generation unit (450) can convert digital indicators about the user's behavioral pattern into visualized information and integrate them with the evaluation text output by the AI coaching model to generate coaching feedback for the corresponding user.

In one embodiment, the coaching feedback generation unit (450) may select digital indicators highly related to the evaluation text generated by the AI coaching model, and may generate visualized information only for the selected digital indicators and add them to the coaching feedback. For example, when the coaching feedback generation unit (450) generates an evaluation text regarding the degree and presence of a user's Internet gaming disorder in a digital learning process through the AI coaching model, the coaching feedback generation unit (450) may select stroke acceleration or stroke speed as a digital indicator highly related to Internet gaming disorder and then generate visualized information. That is, a high IGDS (Internet Gaming Disorder Score) score may be highly correlated with an increase in stroke acceleration and a decrease in stroke speed. In addition, a higher IGDS score may tend to cause strokes to start near the bottom of the screen, and may be correlated with fast typing and omission of spaces between letters, words, or sentences.

In one embodiment, the coaching feedback generation unit (450) may generate coaching feedback on the user's behavioral pattern by using data on the user's behavioral pattern collected during the process of using digital content and at least one digital indicator extracted therefrom. In this case, the coaching feedback may be implemented as feedback and recommendations that reflect not only the user's learning achievements but also his/her attention, emotions, and mental health status.

For example, the coaching feedback generation unit (450) may provide coaching feedback to a student whose behavior pattern is determined to be impulsive to slow down the problem-solving speed or to read the problem text carefully and solve the problem even if the correct answer has been entered during the problem-solving process.

As another example, when recommending the next problem to a user, the coaching feedback generation unit (450) may provide coaching feedback recommending an easier problem than the previous one or recommending a problem of a different subject/genre to a student whose behavioral pattern in solving the previous problem showed a bad feeling.

In addition, the coaching feedback generation unit (450) can predict the learning status and emotional state of a user (e.g., a student) because data on the user's behavioral patterns collected during the process of using digital content corresponds to multimodal time series sensor data, and if the prediction result is negative, it can selectively provide a path, environment, or problem where the same situation does not occur.

The coaching report generation unit (470) can provide coaching feedback to an administrator terminal to generate a coaching report including the signature of the administrator. Here, the administrator terminal may correspond to a terminal operated by an administrator who manages users using online content. For example, the administrator may correspond to a teacher or an instructor who manages learners participating in online learning, and the administrator terminal may correspond to a terminal of the teacher or instructor. When the coaching report generation unit (470) receives a signature as the administrator's approval for the coaching feedback from the administrator terminal, the coaching report generation unit (470) can generate a coaching report including the final evaluation contents. That is, the coaching report may correspond to a report in which the evaluation contents of the coaching feedback generated by the AI coaching model are finally approved by the administrator. In addition, the coaching report generated by the coaching report generation unit (470) may be stored in the database (150) by classifying it by user.

In one embodiment, the coaching report generation unit (470) can calculate a correlation index between visualized information and evaluation text for each digital indicator and independently receive a signature of the corresponding manager for the corresponding evaluation text for a digital indicator whose correlation index is lower than a preset threshold value on the manager terminal. That is, the coaching report generation unit (470) can operate in conjunction with the manager terminal and provide coaching feedback on a learner participating in an online learning course to the manager terminal. The learner's coaching feedback can be provided through an interface on the manager terminal, and the teacher can check the detailed information of the coaching feedback and input a signature through the interface if he or she agrees to the evaluation of the coaching feedback.

In particular, the coaching report generation unit (470) can calculate an association index between visualized information and evaluation text for each digital indicator while providing coaching feedback to the administrator terminal. The association index can be calculated higher as the similarity between the visualized information and the evaluation text is higher. That is, if the association index is below a preset threshold, it can be estimated that the information expressed by the corresponding digital indicator is not accurately reflected in the evaluation content of the evaluation text, and the coaching report generation unit (470) can provide a separate interface that requests the administrator's signature input for digital indicators or evaluation texts whose association index is below the threshold while providing coaching feedback through the interface of the administrator terminal.

At this time, the administrator can modify the wording of the corresponding assessment text and update the coaching feedback by entering a signature for the modified assessment. In addition, the administrator's signature entry can be performed independently for each digital indicator or assessment text.

In one embodiment, the coaching report generation unit (470) may provide coaching feedback to the manager model to generate a coaching report including a signature of the manager model. Instead of providing the coaching feedback to the manager terminal, the coaching report generation unit (470) may provide the coaching feedback as input to the previously built manager model. That is, the coaching report generation unit (470) may generate a coaching report based on the coaching feedback modified by the manager model, and may add a signature generated based on the identification information of the manager model to the coaching report. Here, the manager model may correspond to an artificial intelligence model that generates an output of the result of modifying the coaching feedback based on input data regarding at least one digital indicator and the coaching feedback. In addition, the manager model may modify an evaluation text having the lowest similarity with each digital indicator among the evaluation texts included in the coaching feedback and output the modified coaching feedback.

In one embodiment, the coaching report generation unit (470) can generate a coaching report by first modifying coaching feedback through the manager model and then providing the modified coaching feedback to the manager terminal to secondarily receive the manager's signature. In this case, the manager can confirm the coaching feedback modified by the manager model through the interface of the manager terminal and then enter a signature to approve the coaching feedback. If the coaching report generation unit (470) receives the manager's signature, it can generate a coaching report based on the modified coaching feedback and the manager's signature.

The coaching report providing unit (490) can provide the coaching report generated by the coaching report generating unit (470) to each user through a user interface. That is, the coaching report can be displayed on the user terminal (110) through various forms of templates. For example, the coaching report can be generated according to various templates such as a mobile version, a PC version, a tablet version, etc. and provided to the user.

In one embodiment, the coaching report providing unit (490) may receive user feedback on the coaching report from the user terminal (110). To this end, an interface for inputting user feedback may be provided on the user terminal (110). The user feedback may be input for each digital indicator or evaluation text of the coaching report. In addition, the user feedback may be input in the form of evaluation points, comments, and ratings. The user feedback information input from the user terminal (110) may be used as data for updating the AI coaching model in a subsequent process.

The control unit (not shown in FIG. 4) controls the overall operation of the AI coaching device (130) and can manage the control flow or data flow between the data collection unit (410), the data analysis unit (430), the coaching feedback generation unit (450), the coaching report generation unit (470), and the coaching report provision unit (490).

Figure 5 is a flowchart illustrating an AI coaching method based on digital phenotyping according to the present invention.

Referring to FIG. 5, the AI coaching device (130) can collect data on users' behavioral patterns during the digital content usage process through the data collection unit (410) (step S510). The AI coaching device (130) can preprocess and analyze the data through the data analysis unit (430) to produce at least one digital indicator on the behavioral pattern (step S530).

In addition, the AI coaching device (130) can generate coaching feedback on the behavioral pattern of each user by using the AI coaching model built based on the dataset on the digital indicator through the coaching feedback generation unit (450) (step S550). The AI coaching device (130) can provide the coaching feedback to the administrator terminal through the coaching report generation unit (470) to generate a coaching report including the signature of the administrator (step S570). The AI coaching device (130) can provide the coaching report to each user by using the user interface through the coaching report provision unit (490) (step S590).

Figure 6 is a diagram explaining the correlation between the IGDS and the indicator and the multiple regression results according to the present invention.

Referring to FIG. 6, the AI coaching device (130) can determine digital indicators that are correlated with the IGDS in order to detect a user's Internet game disorder during a digital learning process, which is one of the digital contents. To this end, the AI coaching device (130) can extract meaningful indicators from multimodal data associated with the user's behavioral pattern collected during the digital learning process.

In the case of Fig. 6, multiple regression results can be presented for five digital indicators highly correlated with IGDS. Specifically, F(5, 921) = 18.54, the significance probability is p. < 0.001, and the indicators of IGDS can account for 10.0% of the adjusted coefficient of determination.

Accordingly, the AI coaching device (130) can collect digital indicators regarding the user's behavior during the digital learning process, and can generate coaching feedback regarding the user's behavior pattern from the digital indicators through a pre-learned AI coaching model. The AI coaching device (130) can generate a coaching report evaluating the user's level of Internet game disorder based on the coaching feedback and provide it to the user.

FIG. 7 is a diagram illustrating one embodiment of an AI coaching process according to the present invention.

Referring to FIG. 7, the AI coaching process performed in the AI coaching device (130) may proceed by collecting data on the user's behavioral pattern during the process of using digital content (Data Collection), analyzing the collected data (Data Analysis), and generating information output by the AI coaching model (Recommendation Engine) regarding the user's current emotional state or mental health state as coaching feedback for the user.

In particular, the AI coaching process can be provided to users as a coaching report through an interface (User Interface) after being updated through the manager's review, modification, and signature of the coaching feedback, and can proceed as a process (RLHF) to optimize the AI coaching model through reinforcement learning based on the user's feedback.

The AI coaching device (130) can analyze the behavior of a user using digital content and generate and provide feedback on the same, and can provide various information on the user's emotional state, mental health state, etc. through continuous updates reflecting the user's opinions, so that the user's digital content usage process can be effectively managed.

Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

100: AI Coaching System
110: User terminal 130: AI coaching device
150: Database
210: Multimodal Data Collection Module
230: Foreground Process Monitoring Module
250: Web Browser Monitoring Module
310: Processor 330: Memory
350: User I/O section 370: Network I/O section
410: Data Collection Department 430: Data Analysis Department
450: Coaching Feedback Generation Section 470: Coaching Report Generation Section
490: Coaching Report Provider

Claims (8)

A data collection unit that collects data on users' behavioral patterns while using digital content;
A data analysis unit that preprocesses and analyzes the above data to produce at least one digital indicator regarding the behavioral pattern;
A coaching feedback generation unit that generates coaching feedback on the behavioral patterns of each user using an AI coaching model built based on a dataset on the above digital indicators;
A coaching report generation unit that provides the above coaching feedback to the administrator terminal and generates a coaching report including the signature of the administrator; and
An AI coaching device including a coaching report providing unit that provides the coaching report to each user through a user interface.
In the first paragraph, the data collection unit
During the digital learning process on the user terminal, multimodal data on user behavior is collected through the application executed on the user terminal,
An AI coaching device, characterized in that the multimodal data includes action indicators based on the user's actions on the display screen of the user terminal.
In the first paragraph, the data collection unit
An AI coaching device characterized in that it collects keyboard-related actions and stroke actions of each user collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process in the digital learning process as the multimodal data.
In the first paragraph, the data analysis unit
An AI coaching device characterized by determining at least one digital indicator from among a plurality of indicators including stroke acceleration, stroke speed, vertical distance, number of blank spaces, and time interval.
In the first paragraph, the coaching feedback generation unit
An AI coaching device characterized by providing feedback and recommendations reflecting the user's attention, emotions, and mental health based on at least one digital indicator of the user's behavioral pattern collected during the use of digital content.
In the first paragraph, the coaching feedback generation unit
An AI coaching device characterized in that it generates the coaching feedback including visualized information about at least one digital indicator and an evaluation text of a behavioral pattern associated with the digital indicator.
In the 6th paragraph, the coaching report generation unit
An AI coaching device characterized in that it calculates a correlation index between the visualized information and the evaluation text for each digital indicator and independently receives the signature of the corresponding manager for the corresponding evaluation text for digital indicators whose correlation index is below a preset threshold value on the manager terminal.
In an AI coaching method performed in an AI coaching device,
A step of collecting data on users' behavioral patterns during the process of using digital content through the data collection department;
A step of preprocessing and analyzing the data through a data analysis unit to produce at least one digital indicator regarding the behavioral pattern;
A step of generating coaching feedback on the behavioral pattern of each user using an AI coaching model built based on a dataset on the digital indicator through a coaching feedback generation unit;
Step of providing the coaching feedback to the manager terminal through the coaching report generation unit to generate a coaching report including the manager's signature; and
An AI coaching method comprising: providing the coaching report to each user through a user interface via a coaching report providing unit;