KR102707406B1 - Method and computer device for providing analytical information related to sleep - Google Patents

Abstract

The present invention relates to a method for providing sleep analysis information performed in a processor of a computing device, comprising: a step of obtaining sleep sensing data of a user; and a step of outputting sleep analysis information by using the obtained sleep sensing data as an input of a sleep analysis model; wherein the sleep sensing data is time series data obtained during a sleep period of the user, and is composed of a combination of one or more sequence data divided by unit time. The present invention relates to a method for providing sleep analysis information performed in a processor of a computing device, comprising:

Description

METHOD AND COMPUTER DEVICE FOR PROVIDING ANALYTICAL INFORMATION RELATED TO SLEEP}

The present disclosure provides information on sleep status based on sleep data of a user acquired in a sleep environment, and more specifically, provides analysis information corresponding to the sleep status of a user by utilizing an artificial neural network.

There are various ways to maintain and improve health, such as exercise and diet, but it is most important to manage sleep, which takes up about 30% of the day. However, despite the simple replacement of labor by machines and the leisure of life, modern people are unable to get good sleep due to irregular eating and living habits and stress, and suffer from sleep disorders such as insomnia, hypersomnia, sleep apnea syndrome, nightmares, night terrors, and sleepwalking.

According to the National Health Insurance Corporation, the number of patients with sleep disorders in Korea increased by an average of about 8% per year from 2014 to 2018, and in 2018, the number of patients receiving treatment for sleep disorders in Korea reached about 570,000. In particular, sleep apnea, the most common and dangerous sleep disorder, affects 1 in 6 adults in Korea. Sleep apnea not only interferes with deep sleep, but is also considered a major cause of heart disease, mental illness, and brain disease. Polysomnography can be performed to diagnose these sleep disorders.

Polysomnography, which is used as a standard method for analyzing sleep stages and diagnosing sleep disorders, collects various biosignals from patients while they are sleeping. These biosignals show different patterns depending on the sleep stage and the presence or absence of sleep disorders. Since the same sleep stage can have various patterns depending on the patient's characteristics and the time of measurement, a sleep technician or specialist must directly report and analyze the biosignals for sleep stage analysis and sleep disorder diagnosis. Usually, since the sleep stage is set to 30 seconds, it takes about 2 to 3 hours to analyze the entire sleep of a patient, which can last several hours, and the labor cost for the analysis can also be a burden.

Although biometric data analysis using artificial intelligence is already widely used in the field of imaging medicine, research on biometric data analysis consisting of time series data is insufficient. In particular, research on techniques that enable specialists to interpret, trust, and use predictions from artificial intelligence models is insufficient.

In reality, the reason why it is difficult for AI model predictions to replace expert judgment is not because the average accuracy of the model is low, but because the risk due to the uncertainty of AI models that work well in some situations and not in others is large.

Accordingly, in order to minimize the risk due to the uncertainty of artificial intelligence, there may be a demand for research and development of a reliable and practically helpful sleep stage prediction assistance model by systematically analyzing cases where the prediction of the artificial intelligence model is correct and cases where it is not correct through analysis of the correlation between sleep disorders and performance as well as performance by sleep stage.

Republic of Korea Publication Patent No. 2003-0032529

The present disclosure has been made in response to the aforementioned background technology, and is intended to provide information related to sleep status based on data measured in the sleep environment of each user.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In one embodiment of the present disclosure for solving the above-described problem, a method for providing analysis information related to sleep performed in a processor of a computing device is disclosed. The method may include a step of acquiring sleep sensing data of a user, a step of processing the sleep sensing data as an input of a sleep analysis model to output sleep analysis information, and a step of generating characteristic map information based on the sleep analysis information.

In an alternative embodiment, the sleep sensing data may include one or more sequence data related to a user's biosignal acquired in time series through one or more channels in relation to a sleep environment of the user, the sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of the one or more sleep stages, and the sleep analysis model may be characterized by taking the sleep sensing data as input and outputting sleep analysis information corresponding to each of the one or more sequence data.

In an alternative embodiment, the sleep analysis model may include a first sub-model and a second sub-model, wherein the first sub-model may be characterized by taking each of the one or more sequence data included in the sleep sensing data as input, extracting one or more features corresponding to each of the one or more channels, and integrating the extracted one or more features to generate one or more integrated features corresponding to each of the sequence data, and the second sub-model may be characterized by taking each of the one or more integrated features as input and outputting the sleep analysis information.

In an alternative embodiment, the sleep analysis model may further include one or more attention modules, wherein the one or more attention modules may be characterized by calculating attention weights for sleep analysis information corresponding to each of the one or more integrated features.

In an alternative embodiment, the step of generating feature map information based on the sleep analysis information is characterized in that the feature map information is generated by visualizing attention weights for sleep analysis information according to each integrated feature through the one or more attention modules, and the feature map information may be information that visualizes data that have influenced prediction for each sequence data output by the sleep analysis model.

In an alternative embodiment, the sleep analysis model is characterized in that transfer learning is performed so that an algorithm trained in a source domain can be applied in another target domain, and the transfer learning may include performing learning in a target domain related to sleep disorder based on an algorithm trained in a source domain related to normality.

In an alternative embodiment, the sleep analysis information includes prediction confidence information corresponding to each of one or more sequence data, and the method may further include a step of performing a correction on the prediction confidence information output by the sleep analysis model through temperature scaling.

In an alternative embodiment, the method may include the steps of generating relationship information between the prediction confidence information and the sleep stage information and updating the sleep analysis information based on the relationship information.

In another embodiment of the present disclosure, a computing device for providing analysis information related to sleep is disclosed. The computing device includes a processor including one or more cores, a memory storing program codes executable in the processor, and a network unit transmitting and receiving data with a user terminal, wherein the processor can obtain sleep sensing data of a user, process the sleep sensing data as an input of a sleep analysis model to output sleep analysis information, and generate characteristic map information based on the sleep analysis information.

In another embodiment of the present disclosure, a computer program stored in a computer-readable storage medium is disclosed. The computer program, when executed on one or more processors, causes the one or more processors to perform the following operations for providing analysis information related to sleep, wherein the operations may include: acquiring sleep sensing data of a user, processing the sleep sensing data as an input of a sleep analysis model to output sleep analysis information, and generating feature map information based on the sleep analysis information.
In another embodiment of the present disclosure, a method for providing sleep analysis information performed in a processor of a computing device includes the steps of: acquiring sleep sensing data of a user; and outputting sleep analysis information by using the acquired sleep sensing data as an input of a sleep analysis model; wherein the sleep sensing data is time series data acquired during a sleep period of the user, and may include a method for providing sleep analysis information performed in a processor of a computing device, the method comprising a combination of one or more sequence data divided by unit time.
In another embodiment of the present disclosure, a method for providing sleep analysis information performed in a processor of a computing device may be included, wherein the unit time is 30 seconds.
In another embodiment of the present disclosure, the sequence data may include a method for providing sleep analysis information performed in a processor of a computing device, characterized in that the sequence data is user's biometric information acquired through one or more channels based on the unit time.
In another embodiment of the present disclosure, a method for providing sleep analysis information performed in a processor of a computing device may be included, wherein the sleep analysis information includes sleep stage information, wherein the sleep stage information is information indicating that a user's sleep at a specific point in time corresponds to at least one of one or more sleep stages.
In another embodiment of the present disclosure, the sleep analysis information may include a method for providing sleep analysis information performed on a processor of a computing device, wherein the sleep analysis information further includes prediction confidence information regarding information indicating that the sleep of the user at a particular point in time corresponds to at least one of one or more sleep stages.
In another embodiment of the present disclosure, a method for providing sleep analysis information performed in a processor of a computing device may be included, wherein the sleep analysis model is characterized in that it is transfer learned, and the transfer learning includes performing learning in a target domain based on an algorithm trained in a source domain, wherein the source domain is a domain for users who do not suffer from a sleep disorder, and the target domain is a domain for users who suffer from a sleep disorder.
In another embodiment of the present disclosure, a method for providing sleep analysis information performed in a processor of a computing device may be included, further comprising: performing a correction on the prediction confidence information; and updating the sleep analysis information based on the prediction confidence information and the sleep stage information.
In another embodiment of the present disclosure, a computing device providing sleep analysis information comprises: a network unit for transmitting and receiving data with another electronic device; a memory for storing a computer program; and a processor for reading the stored computer program; wherein the processor obtains sleep sensing data of a user, and outputs sleep analysis information by using the obtained sleep sensing data as an input of a sleep analysis model, wherein the sleep sensing data is time series data obtained during a sleep period of the user, and is composed of a combination of one or more sequence data divided by unit time. The computing device providing sleep analysis information may include:
In another embodiment of the present disclosure, a computing device providing sleep analysis information may be included, wherein the unit time is 30 seconds.
In another embodiment of the present disclosure, the sequence data may include a computing device providing sleep analysis information, characterized in that it is a user's biometric information obtained through one or more channels based on the unit time.
In another embodiment of the present disclosure, the sleep analysis information may include a computing device providing sleep analysis information, wherein the sleep stage information is information indicating that the user's sleep at a specific point in time corresponds to at least one of one or more sleep stages.
In another embodiment of the present disclosure, the sleep analysis information may include a computing device providing sleep analysis information, wherein the sleep analysis information further includes prediction confidence information regarding information indicating that the sleep of the user at a particular point in time corresponds to at least one of one or more sleep stages.
In another embodiment of the present disclosure, the sleep analysis model may include a computing device providing sleep analysis information, wherein the sleep analysis model is characterized in that it is transfer learned, and the transfer learning includes performing learning in a target domain based on an algorithm trained in a source domain, wherein the source domain is a domain for users who do not suffer from a sleep disorder, and the target domain is a domain for users who suffer from a sleep disorder.
In another embodiment of the present disclosure, the processor may include a computing device providing sleep analysis information, characterized in that it performs a correction on the prediction confidence information and updates the sleep analysis information based on the prediction confidence information and the sleep stage information.

delete

delete

The present disclosure has been devised in response to the aforementioned background technology, and can provide information related to sleep disorders based on data measured in the sleep environment of each user.

Additionally, by systematically analyzing cases where the predictions of the artificial intelligence model are correct and cases where they are not, we can provide a reliable and practically helpful sleep stage prediction auxiliary model.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

Various aspects are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements generally. In the following examples, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It will be apparent, however, that such aspects may be practiced without these specific details.
FIG. 1 is a conceptual diagram illustrating a system in which various aspects of a computing device for providing sleep-related analysis information related to one embodiment of the present disclosure may be implemented.
FIG. 2 illustrates a block diagram of a computing device for providing sleep-related analysis information related to one embodiment of the present disclosure.
FIG. 3 is a diagram illustrating sleep sensing data according to an embodiment of the present disclosure.
FIG. 4 is a diagram exemplarily illustrating a process in which sleep sensing data related to one embodiment of the present disclosure is input into a sleep analysis model.
FIG. 5 is a diagram exemplarily illustrating an output process of a sleep analysis model related to one embodiment of the present disclosure.
FIG. 6 illustrates a flowchart exemplarily illustrating a method for providing sleep-related analysis information related to one embodiment of the present disclosure.
FIG. 7 is a schematic diagram illustrating one or more network functions related to one embodiment of the present disclosure.

The advantages and features of the present disclosure, and the methods for achieving them, will become apparent by referring to the embodiments described in detail below together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, and these embodiments are provided only to make the disclosure of the present disclosure complete and to fully inform a person skilled in the art of the scope of the present disclosure, and the present disclosure is defined only by the scope of the claims.

The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present disclosure. As used herein, the singular includes the plural unless the context clearly dictates otherwise. The terms “comprises” and/or “comprising,” as used herein, do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the specification, and “and/or” includes each and every combination of one or more of the mentioned components. Although “first,” “second,” etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present disclosure.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with the meaning commonly understood by those skilled in the art to which this disclosure belongs. In addition, terms defined in commonly used dictionaries shall not be ideally or excessively interpreted unless explicitly specifically defined.

The term “part” or “module” as used in this specification means a software or hardware component such as an FPGA or ASIC, and the “part” or “module” performs certain functions. However, the “part” or “module” is not limited to software or hardware. The “part” or “module” may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, by way of example, the “part” or “module” includes components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided in the components and “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or further separated into additional components and “parts” or “modules.”

In this specification, a computer means any kind of hardware device including at least one processor, and may be understood to encompass software configurations operating on the hardware device according to an embodiment. For example, a computer may be understood to encompass, but is not limited to, a smartphone, a tablet PC, a desktop, a laptop, and all user clients and applications running on each device.

Those skilled in the art should additionally recognize that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as combinations of electronic hardware, computer software, or both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

Although each step described in this specification is described as being performed by a computer, the subject of each step is not limited thereto, and at least some of each step may be performed by different devices depending on the embodiment.

FIG. 1 is a conceptual diagram illustrating a system in which various aspects of a computing device for providing sleep-related analysis information related to one embodiment of the present disclosure may be implemented.

A system according to embodiments of the present disclosure may include a computing device (100), a user terminal (10), an external server (20), and a network. The computing device (100), the user terminal (10), and the external server (20) according to embodiments of the present disclosure may mutually transmit and receive data for a system according to embodiments of the present disclosure through a network.

A network according to embodiments of the present disclosure may use various wired communication systems such as a public switched telephone network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a local area network (LAN).

Additionally, the network presented herein can use various wireless communication systems such as Code Division Multi Access (CDMA), Time Division Multi Access (TDMA), Frequency Division Multi Access (FDMA), Orthogonal Frequency Division Multi Access (OFDMA), Single Carrier-FDMA (SC-FDMA), and other systems.

The network according to the embodiments of the present disclosure can be configured regardless of its communication mode, such as wired or wireless, and can be configured with various communication networks, such as a personal area network (PAN) and a wide area network (WAN). In addition, the network can be the well-known World Wide Web (WWW), and can also use a wireless transmission technology used for short-distance communication, such as infrared (IrDA: Infrared Data Association) or Bluetooth. The technologies described in this specification can be used not only in the networks mentioned above, but also in other networks.

According to one embodiment of the present disclosure, the user terminal (10) is a terminal that can receive information related to the user's sleep through information exchange with the computing device (100), and may refer to a terminal possessed by the user. For example, the user terminal (10) may be a terminal related to a user who wants to improve his/her health through information related to his/her sleep habits. In addition, the user terminal (10) may be a terminal related to an examiner (e.g., a specialist) who provides examination results to a user (e.g., a examinee). If the user terminal (10) is a terminal related to an examiner who provides examination results (e.g., examination results for a polysomnography test) to the examinee, the user terminal (10) may be utilized as a medical auxiliary terminal for the examiner's interpretation of the examination results through analysis information acquired from the computing device (100).

The user can receive sleep stage information corresponding to each point in time in the sleep environment, prediction confidence information corresponding to each sleep stage information for each point in time, etc. through the user terminal (10). This user terminal (10) is equipped with a display, and can receive user input and provide output of any form to the user.

These user terminals (10) may be referred to as, but are not limited to, any device capable of using a wireless mechanism, such as a customer terminal (UE), a mobile, a PC capable of wireless communication, a mobile phone, a kiosk, a cellular phone, a cellular, a cellular terminal, a subscriber unit, a subscriber station, a mobile station, a terminal, a remote station, a PDA, a remote terminal, an access terminal, a user agent, a cellular telephone, a cordless telephone, a Session Initiation Protocol (SIP) telephone, a wireless local loop (WLL) station, a portable device having a wireless access function, a wireless model, etc. In addition, the user terminal (10) may be referred to as, but are not limited to, any device capable of using a wired connection mechanism, such as a wired fax machine, a PC having a wired model, a wired telephone, a terminal capable of wired communication, etc.

According to one embodiment of the present disclosure, the external server (20) may be a server that stores health checkup information or sleep checkup information for a plurality of users. For example, the external server (20) may be at least one of a hospital server and an information server, and may be a server that stores information on a plurality of polysomnography records, electronic health records, electronic medical records, etc. For example, the polysomnography records may include information on brain waves, eye movements, muscle movements, respiration, electrocardiograms, etc. during sleep of a sleep examination subject, and information on sleep diagnosis results (e.g., sleep stages, sleep apnea, sleep disorders, etc.) corresponding to the information. The information stored in the external server (20) may be utilized as learning data, verification data, and test data for training the neural network of the present disclosure. That is, the external server (20) may be a server that stores a data set for training the sleep analysis model of the present disclosure.

The computing device (100) of the present disclosure can receive health checkup information or sleep checkup information, etc. from an external server (20), and construct a learning data set based on the information. The computing device (100) can generate a sleep analysis model for producing sleep analysis information corresponding to the user's sleep sensing data by training a sleep analysis model including one or more network functions through the learning data set. A specific description of a configuration for constructing a learning data set for neural network learning of the present disclosure and a learning method utilizing the learning data set will be described later with reference to FIG. 2.

The external server (20) may be a digital device, such as a laptop computer, a notebook computer, a desktop computer, a web pad, or a mobile phone, which may be equipped with a processor and a computing capability equipped with memory. The external server (20) may be a web server that processes a service. The types of servers described above are merely examples, and the present disclosure is not limited thereto.

According to one embodiment of the present disclosure, the computing device (100) may obtain sleep sensing data related to the user's sleep environment, and generate sleep analysis information related to the user's sleep state based on the sleep sensing data. Specifically, the computing device (100) may generate a sleep analysis model that learns one or more network functions using a labeled learning data set to output sleep analysis information based on the sleep sensing data. That is, the computing device (100) may obtain sleep sensing data related to the user's sleep environment, and process the sleep sensing data as input to a sleep analysis model, which is a learned neural network model, to generate sleep analysis information. In this case, the sleep analysis information provided by the computing device (100) may include prediction information regarding the sleep stage during the user's sleep. For example, the sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages at a specific point in time. Here, the one or more sleep stages may include, for example, wake (non-sleep state), N1 (Non REM 1), N2, N3, and REM sleep. For a specific example, the sleep analysis information may include information that the user's sleep stage at a first point in time corresponds to REM sleep.

Additionally, the sleep analysis information may include prediction certainty information for each time point related to the user's sleep environment. The prediction certainty information may be information regarding the prediction accuracy of sleep stage information output (or predicted) through the sleep analysis model corresponding to a specific time point. For example, the sleep analysis information may include prediction information that the user's sleep stage at the first time point corresponds to REM sleep and information regarding the reliability of the prediction certainty information of '80' for the prediction information. For example, the greater the prediction certainty information, the higher the accuracy (or reliability) of the sleep stage predicted through the artificial neural network. That is, the computing device (100) may provide prediction information for sleep stages at each time point during the user's sleep and prediction certainty information corresponding to the prediction information for each sleep stage. In this case, the prediction certainty information may be utilized as an indicator of how reliable the output of the artificial neural network (i.e., the sleep analysis model) (i.e., the prediction information for the sleep stage) is. In other words, by providing prediction confidence information related to sleep stage estimation at each time point together with sleep stage information, reliable use in medical environments can be made possible.

According to one embodiment of the present disclosure, the computing device (100) can generate characteristic map information based on sleep analysis information. The characteristic map information may be information that visualizes data that has an influence in the process of generating sleep analysis information output by the sleep analysis model. Specifically, the computing device (100) can obtain attention weights for sleep analysis information according to sleep sensing data related to a specific point in time through one or more attention modules. In this case, one or more attention modules may be modules that learn a matching relationship between the input and output of the sleep analysis model. One or more attention modules may emphasize elements to be focused on between the input data and the output data by assigning attention weights to each element of input data (i.e., sleep sensing data) related to a time series during the learning process of the neural network. In other words, one or more attention modules may be modules that generate information about which input element the output value of the sleep analysis model is most highly correlated with.

For a specific example, the computing device (100) may generate prediction information that the user's sleep stage corresponds to the N3 sleep stage for the 1 minute based on the user's sleep sensing data acquired in response to the first time point (e.g., the first minute of the user's sleep after going to bed). In addition, the computing device (100) may obtain attention weights of each input element for sleep analysis information related to the first time point (i.e., prediction information that the user's sleep corresponds to the N3 sleep stage at the time point) from one or more attention modules. The computing device (100) may generate feature map information by visualizing the acquired attention weights for each element.

That is, the computing device (100) can visualize the basis for judgment in the process of generating prediction information for input data (i.e., sleep sensing data) related to a time series by utilizing one or more attention modules. In other words, it is possible to provide a visualization of which signal (i.e., which sleep sensing data) played an important role in the reading of sleep stages at each time point through attention weight values. For example, feature map information can be generated to provide meaningful visualization information by differently displaying each pixel based on signals or information that are noted through attention weights.

In general, it is difficult to understand the internal algorithm of an artificial neural network model. In other words, it is difficult to understand the basis on which the output related to the input data is generated. This may have the problem of making it difficult to use in an actual medical environment as it acts as a risk due to the uncertainty of the artificial neural network model.

The computing device (100) of the present disclosure provides a visualization of the pattern of signals that played an important role in predicting sleep stages at each time point, thereby visualizing the basis on which the neural network model made predictions or judgments. Accordingly, the validity of the prediction information output by the sleep analysis model of the present disclosure can be precisely verified, thereby maximizing the synergy of collaboration with a user (e.g., a specialist).

The specific configuration of the computing device (100) of the present disclosure and the effects according to the configuration will be described in detail later with reference to FIG. 2.

In addition, the computing device (100) may be a terminal or a server, and may include any type of device. The computing device (100) may be a digital device, such as a laptop computer, a notebook computer, a desktop computer, a web pad, or a mobile phone, which has a computing ability equipped with a processor and a memory. The computing device (100) may be a web server that processes a service. The types of servers described above are merely examples, and the present disclosure is not limited thereto.

According to one embodiment of the present disclosure, the computing device (100) may be a server that provides a cloud computing service. More specifically, the computing device (100) may be a server that provides a cloud computing service, which is a type of Internet-based computing that processes information with another computer connected to the Internet rather than the user's computer. The cloud computing service may be a service that stores data on the Internet and allows the user to use the data or programs required by the user anytime and anywhere through an Internet connection without having to install them on their computer, and allows the data stored on the Internet to be easily shared and transmitted with simple manipulation and clicks. In addition, the cloud computing service may be a service that not only simply stores data on a server on the Internet, but also performs a desired task by utilizing the functions of an application program provided on the web without having to install a separate program, and allows multiple people to simultaneously share documents and proceed with work. In addition, the cloud computing service may be implemented in the form of at least one of an IaaS (Infrastructure as a Service), a PaaS (Platform as a Service), a SaaS (Software as a Service), a virtual machine-based cloud server, and a container-based cloud server. That is, the computing device (100) of the present disclosure may be implemented in the form of at least one of the above-described cloud computing services. The specific description of the above-described cloud computing services is only an example, and may include any platform that constructs the cloud computing environment of the present disclosure.

Hereinafter, a method for a computing device (100) to provide sleep analysis information based on sleep sensing data will be described in more detail with reference to FIG. 2.

FIG. 2 illustrates a block diagram of a computing device for providing sleep-related analysis information related to one embodiment of the present disclosure.

As illustrated in FIG. 2, the computing device (100) may include a network unit (110), a memory (120), and a processor (130). The components included in the computing device (100) described above are exemplary, and the scope of the present disclosure is not limited to the components described above. That is, additional components may be included or some of the components described above may be omitted depending on the implementation aspect of the embodiments of the present disclosure.

According to one embodiment of the present disclosure, the computing device (100) may include a network unit (110) that transmits and receives data with a user terminal (10) and an external server (20). The network unit (110) may transmit and receive data, etc. for performing a method for providing sleep analysis information based on sleep environment sensing data according to one embodiment of the present disclosure, with other computing devices, servers, etc. That is, the network unit (110) may provide a communication function between the computing device (100), the user terminal (10), and the external server (20). For example, the network unit (110) may receive sleep examination records and electronic health records for a plurality of users from a hospital server. Additionally, the network unit (110) may allow information transmission between the computing device (100), the user terminal (10), and the external server (20) by calling a procedure with the computing device (100).

The network unit (110) according to one embodiment of the present disclosure can use various wired communication systems such as a public switched telephone network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a local area network (LAN).

In addition, the network unit (110) presented in this specification can use various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA) and other systems.

In the present disclosure, the network unit (110) may be configured regardless of the communication mode, such as wired or wireless, and may be configured with various communication networks, such as a short-range communication network (PAN: Personal Area Network) and a wide area communication network (WAN: Wide Area Network). In addition, the network may be the well-known World Wide Web (WWW: World Wide Web), and may also utilize a wireless transmission technology used for short-range communication, such as infrared (IrDA: Infrared Data Association) or Bluetooth. The technologies described in this specification may be used not only in the networks mentioned above, but also in other networks.

According to one embodiment of the present disclosure, the memory (120) can store a computer program for performing a method for providing sleep analysis information according to one embodiment of the present disclosure, and the stored computer program can be read and operated by the processor (130). In addition, the memory (120) can store any form of information generated or determined by the processor (130) and any form of information received by the network unit (110). In addition, the memory (120) can store data related to the user's sleep. For example, the memory (120) can temporarily or permanently store input/output data (for example, sleep sensing data related to the user's sleep environment, i.e., sensing data related to a polysomnographic examination, etc.).

According to one embodiment of the present disclosure, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, an SD or XD memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only Memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may also operate in relation to web storage that performs the storage function of the memory (120) on the internet. The description of the above-described memory is merely an example, and the present disclosure is not limited thereto.

According to one embodiment of the present disclosure, the processor (130) may be composed of one or more cores and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device.

The processor (130) can read a computer program stored in the memory (120) and perform data processing for machine learning according to one embodiment of the present disclosure. According to one embodiment of the present disclosure, the processor (130) can perform operations for learning a neural network. The processor (130) can perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating weights of a neural network using backpropagation.

In addition, at least one of the CPU, GPGPU, and TPU of the processor (130) can process learning of a network function. For example, the CPU and the GPGPU can process learning of a network function and data classification using the network function together. In addition, in one embodiment of the present disclosure, processors of a plurality of computing devices can be used together to process learning of a network function and data classification using the network function. In addition, a computer program executed in a computing device according to one embodiment of the present disclosure can be a CPU, GPGPU, or TPU executable program.

In this specification, the network function may be used interchangeably with artificial neural network, neural network. In this specification, the network function may include one or more neural networks, in which case the output of the network function may be an ensemble of outputs of one or more neural networks.

In this specification, a model may include a network function. The model may include one or more network functions, in which case the output of the model may be an ensemble of outputs of one or more network functions.

The processor (130) can read a computer program stored in the memory (120) to provide a sleep analysis model according to one embodiment of the present disclosure. According to one embodiment of the present disclosure, the processor (130) can perform calculations to derive sleep analysis information based on sleep sensing data. According to one embodiment of the present disclosure, the processor (130) can perform calculations to train the sleep analysis model.

According to one embodiment of the present disclosure, the processor (130) can typically process the overall operation of the computing device (100). The processor (130) can process signals, data, information, etc. input or output through the components described above, or can operate an application program stored in the memory (120) to provide or process appropriate information or functions to the user terminal.

According to one embodiment of the present disclosure, the processor (130) can obtain sleep sensing data of the user. According to one embodiment of the present disclosure, obtaining the sleep sensing data may be receiving or loading sleep sensing data stored in the memory (120). Obtaining the sleep sensing data may be receiving or loading sleep sensing data from another storage medium, another computing device, or a separate processing module within the same computing device based on a wired/wireless communication means.

The sleep sensing data may be a biosignal acquired during the user's sleep. For example, the sleep sensing data may be a biosignal measured from the user in relation to a polysomnographic examination. The biosignal associated with the polysomnographic examination may include information on brain waves, acuity, chin electromyography, respiration, electrocardiogram, arterial blood oxygen saturation, peroneal muscle electromyography, and other physiological and physical variables. According to a further embodiment, the biosignal associated with the polysomnographic examination may further include information on the user's respiration and movement during sleep. Such sleep sensing data may be acquired through one or more channels. Each of the one or more channels may be associated with each electrode pair configured to acquire a biosignal.

For example, sleep sensing data may be a biosignal measured in relation to a polysomnographic examination, and may include information on electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG). In this case, the information on the electroencephalography, the electrooculography, and the electromyography may each be obtained through different channels. For a specific example, as illustrated in FIG. 3, the information on the electroencephalography may be obtained through six channels (i.e., F3-A2, F4-A1, C3-A2, C4-A1, O1-A2, and O1-A1), the information on the stability may be obtained through two channels (LEFT, RIGHT), and the information on the electromyography may be obtained through one channel. That is, the sleep sensing data of the present disclosure refers to biosignals related to electroencephalography, stability, and electromyography that are measured from the user's body during a sleep period for a polysomnographic examination, and the biosignals can be obtained through six channels related to electroencephalography, two channels related to stability, and one channel related to electromyography, that is, a total of nine channels. Since the sleep sensing data, which is the basis for generating the sleep analysis information of the present disclosure, includes various biosignals obtained through a plurality of channels, as described above, the output accuracy of the artificial neural network can be improved through the data.

In addition, the sleep sensing data of the present disclosure may be time series data acquired during the user's sleep period. In this case, the sleep sensing data may be a combination of one or more sequence data separated by a predetermined unit time. The predetermined unit time may be, for example, 30 seconds, which is a standard for detecting a sleep change. That is, the sleep sensing data may include one or more sequence data related to the user's biosignal acquired in a time series manner through one or more channels in relation to the user's sleep environment. For example, the one or more sequence data may include information on electroencephalogram, safety, and electromyogram acquired through nine channels based on a predetermined unit time (e.g., 30 seconds).

For example, referring to FIG. 3, one or more sequence data included in the sleep sensing data may include first sequence data (201) and second sequence data (202). Here, the first sequence data (201) may be a biosignal related to electroencephalography, stability, and electromyography acquired through nine channels for 0 to 30 seconds. The second sequence data (202) may be a biosignal related to electroencephalography, stability, and electromyography acquired through nine channels for 31 to 60 seconds after the first sequence data (201). The description described above with reference to FIG. 3 is only an example to help understanding of the present disclosure, and the one or more sequence data included in the sleep sensing data of the present disclosure is not limited to the first and second sequence data. That is, since the sleep sensing data of the present disclosure is data related to a biosignal measured during a user's sleep period (e.g., for 8 hours), it will be apparent to a person skilled in the art that the sleep sensing data includes a larger number of sequence data.

According to one embodiment of the present disclosure, the processor (130) may perform preprocessing on sleep sensing data. Preprocessing on sleep sensing data according to one embodiment may include performing down sampling on sleep sensing data. For example, sleep sensing data acquired through multiple channels (e.g., 9 channels) in relation to polysomnography may be acquired at different sampling rates, but may have a sampling rate that is unnecessarily high for sleep stage measurement. Accordingly, the processor (130) may down sample the data acquired through each channel to a sampling rate suitable for the neural network model of the present disclosure. For a specific example, the data acquired through each channel may be down sampled to a sampling rate of 25 Hz to reduce the period of each signal. This may provide an effect of reducing distortion of data acquired in time series and preventing aliasing. Accordingly, the distinction between data acquired through each channel may become clear.

In an additional embodiment, the preprocessing for the sleep sensing data may include preprocessing for removing noise from each of the data acquired through each channel. Specifically, the processor (130) may standardize the magnitude of the signal included in each of the data acquired through each channel based on a comparison between the magnitude of the signal included in each of the data acquired through each channel and the magnitude of a predetermined reference signal. For example, if the magnitude of the signal included in each of the sleep sensing data acquired through a plurality of channels is less than the predetermined reference signal, the processor (130) may adjust the magnitude of the corresponding signal to be large, and if the magnitude of the signal included in each of the sleep sensing data acquired through each channel is greater than the predetermined reference signal, the processor (130) may adjust the magnitude of the corresponding signal to be small (i.e., not to be clipped). The specific description of the preprocessing for the sleep sensing data described above is merely an example, and the present disclosure is not limited thereto.

According to one embodiment of the present disclosure, the processor (130) may process sleep sensing data as input to a sleep analysis model to output sleep analysis information. The sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages. In other words, the sleep analysis information may include prediction information about the sleep stage during the user's sleep. For example, the sleep analysis information may include sleep stage information indicating that the user's sleep corresponds to at least one of one or more sleep stages with respect to a specific point in time. Here, the one or more sleep stages may include, for example, wake (non-sleep state), N1 (Non REM 1), N2, N3, and REM sleep. As a specific example, the sleep analysis information may include information indicating that the user's sleep stage at a first point in time corresponds to REM sleep. The specific description of the sleep analysis information described above is merely an example, and the present disclosure is not limited thereto.

According to one embodiment of the present disclosure, the processor (130) may perform learning for one or more network functions through a learning data set. To this end, the processor (130) may receive a learning data set from an external server (20). Here, the external server (20) may be a server related to at least one of a hospital server and a government server, and may receive examination data of each of a plurality of users related to a sleep polygraph from at least one of an electronic health record, an electronic medical record, and a health examination DB of each server. The processor (130) may construct a learning data set including a plurality of learning data based on the examination data of each of a plurality of users received from the external server (20). The learning data set may include a learning input data set related to an input of a neural network and a learning output data set compared to an output.

More specifically, the learning data set may include a learning input data set related to biosignals measured from the user's body through a polysomnography in relation to the sleep environment of each of the plurality of users, and a learning output data set related to sleep stage determination information at each time point of each biosignal. The processor (130) may perform learning on one or more network functions through the learning data set to generate the sleep analysis model of the present disclosure.

The sleep analysis model may include a first sub-model (310) and a second sub-model (330). Hereinafter, the first sub-model (310) is described as a dimension reduction sub-model (310), and the second sub-model (330) is described as a dimension restoration sub-model (330), but is not limited thereto. The processor (130) may train the dimension restoration sub-model (330) to output learning data associated with the label of the corresponding learning input data by using learning input data as input to the dimension reduction sub-model (310).

The dimension reduction sub-model (310) may be a model that extracts features (i.e., embeddings) by inputting learning input data related to bio-signals acquired in a time-series manner during the user's sleep. That is, the dimension reduction sub-model (310) may receive learning input data from the processor (130) and designate a feature vector column of the learning input data as an output to learn an intermediate process in which the input data is converted into a feature.

In addition, the processor (130) can transfer the embedding (i.e., feature) related to the output of the dimension reduction sub-model (310) to the dimension restoration sub-model (330). The dimension reduction sub-model (310) can take a feature as input and output sleep analysis information related to the feature. The processor (130) can compare the sleep analysis information, which is the output of the dimension restoration sub-model, with the learning output data to derive an error, and adjust the weights of each model using the backpropagation method based on the derived error.

The processor (130) can adjust the weights of one or more network functions so that the sleep analysis information, which is the output of the dimension restoration sub-model (330), becomes closer to the learning output data based on the operation result of the dimension restoration sub-model (330) for the learning input data and the error of the learning output data.

In other words, the dimensionality reduction sub-model can be trained to extract features for learning input data, and the dimensionality restoration sub-model can be trained to output analysis information corresponding to the extracted features.

In addition, the sleep analysis model may further include one or more attention modules. The one or more attention modules may calculate attention weights for analysis information corresponding to features. Specifically, the processor (130) may cause the one or more attention modules to learn a matching relationship between inputs (i.e., learning input data) and outputs (i.e., learning output data) through learning input data and learning output data. Accordingly, the one or more attention modules (320) may generate association information related to associations between time steps of the feature and dimension restoration sub-models (330). In other words, the one or more attention modules may emphasize elements to be focused on between the input data and the output data by assigning attention weights to each element of input data related to the time series during the learning process of the neural network. In other words, the one or more attention modules may be modules that are learned to assign attention weights related to associations between the output values and input values of the sleep analysis model.

For a specific example, the dimension restoration sub-model (330) may include one or more RNNs, and may be a model that inputs prediction information of a first time stamp through one or more RNNs and outputs prediction information of a second time stamp. In this case, the first time stamp may be a time point earlier than the second time stamp. Specifically, the prediction information predicted through the first RNN of the dimension restoration sub-model may be transferred to the second RNN of the next time stamp, and the prediction information predicted through the second RNN may be transferred to the third RNN of the next time stamp. In this process, one or more attention modules may determine the time point of a feature to be focused on through the correlation information between time steps. That is, the dimension restoration sub-model (330) may be trained to output prediction information on the correlation between the time points of various factors by determining the time point of a change factor to be focused on through one or more attention modules in the process of repeatedly predicting prediction information through one or more RNNs.

In other words, the processor (130) may use learning input data as input to a dimensionality detection sub-model to output features corresponding to the learning input data, and may process the features as inputs to a dimensionality restoration sub-model (330) through one or more attention modules. In this case, the dimensionality restoration sub-model (330) may use the features as inputs to output sleep analysis information, and the processor (130) may compare the output sleep analysis information with learning output data, which is a label of the learning input data, to derive an error, and may adjust the weights of each model based on the derived error. Through the learning process described above, the processor (130) may perform learning on one or more network functions to generate a sleep analysis model.

Accordingly, the sleep analysis model can input sleep sensing data and output sleep analysis information.

To be more specific, the sleep analysis model can take sleep sensing data including one or more sequence data as input, and output sleep analysis information corresponding to each of the one or more sequence data. That is, the sleep analysis model can be characterized by taking sleep sensing data as input and outputting sleep analysis information corresponding to each of the one or more sequence data. This sleep analysis model can include a dimensionality reduction sub-model (e.g., an encoder) and a dimensionality restoration sub-model (e.g., a decoder). The dimensionality reduction sub-model and the dimensionality restoration sub-model described below may refer to models learned through the aforementioned learning process. A specific description of a process in which the sleep analysis model takes sleep sensing data including one or more sequence data as input and outputs sleep analysis information corresponding to each sequence data will be described below with reference to FIGS. 4 and 5.

The dimensionality reduction sub-model (310) may be characterized by taking as input one or more sequence data included in the sleep sensing data, extracting one or more features corresponding to each of one or more channels, and integrating the extracted one or more features to generate one or more integrated features corresponding to each sequence data.

More specifically, referring to FIG. 4, the learning input sleep sensing data of the present disclosure may be related to a user's bio-signal acquired in time series through one or more channels during the user's sleep. The sleep sensing data may include one or more sequence data. The one or more sequence data may include first sequence data (201) and second sequence data (202). Here, the first sequence data (201) may be a bio-signal related to an electroencephalogram (EEG), a safety level, and an electromyogram (EMG) acquired through nine channels for 0 to 30 seconds. The second sequence data (202) may be a bio-signal related to an EEG, a safety level, and an EMG acquired through nine channels for 31 to 60 seconds after the first sequence data (201). Furthermore, the sleep sensing data used for analysis may include, but is not limited to, 10 sequence data per 30 seconds.

In this case, the dimensionality reduction sub-model (310) can extract one or more features corresponding to each channel by inputting the first sequence data (201). Specifically, the dimensionality reduction sub-model can generate six features corresponding to each of six channels related to EEG for 0 to 30 seconds, two features corresponding to each of two channels related to safety during the corresponding time period, and one feature corresponding to one channel related to EMG during the corresponding time period. That is, when the dimensionality reduction sub-model (310) inputs the first sequence data (201) included in the sleep sensing data, it can generate nine features corresponding to each channel. In addition, the dimensionality reduction sub-model (310) can integrate one or more features, that is, nine features, to generate an integrated feature. An integrated feature may mean a feature that represents a sleep environment related to a specific point in time by integrating various bio-signals acquired through multiple channels during the same time. In other words, one integrated feature may mean a feature (i.e., embedding) corresponding to one sequence data. In addition, the dimensionality reduction sub-model (310) may generate an integrated feature corresponding to the second sequence data (202) by inputting second sequence data (202) related to a sleep environment at a subsequent point in time of the first sequence data (201).

For example, in the dimension reduction sub-model (310), a model that extracts features for each channel may be Res1DCNN, and a model that extracts integrated features by integrating features for each channel may also be Res1DCNN. That is, the dimension reduction sub-model (310) may be MultiRes1DCNN, but is not limited thereto.

In one embodiment, the dimensionality reduction sub-model of the present disclosure may be provided in multiple numbers to perform integrated feature extraction operations corresponding to one or more sequence data in parallel. When the dimensionality reduction sub-model is provided in multiple numbers to perform one or more integrated feature extraction operations in parallel by taking each sequence data as input, the processing speed can be further improved.

As described above, the dimension restoration sub-model (330) can generate one or more integrated features corresponding to each sequence data when inputting sleep sensing data including one or more sequence data separated by a predetermined unit time. For example, when performing analysis on 10 sequences, 10 integrated features can be obtained.

In addition, the dimension restoration sub-model (330) may be characterized by taking an integrated feature as input and outputting sleep analysis information.

For a specific example, the dimension restoration sub-model (330) may include an RNN, and may input a plurality of integrated features obtained as analysis results for a plurality of sequences, analyze the integrated features for the plurality of sequences together, and output sleep analysis information for each of the plurality of sequences. The output sleep analysis information may include, but is not limited to, sleep stage information and confidence information for each sequence.

For example, when obtaining sleep analysis information using integrated features extracted from 10 sequences, integrated features extracted from 10 MultiRes1DCNN operations can be input into an RNN, and sleep stage information and confidence information for each of the 10 sequences can be obtained from the output thereof, but is not limited thereto.

As another example, the dimension restoration sub-model (330) may include one or more RNNs, and may be a model that inputs prediction information of a first time stamp through one or more RNNs and outputs prediction information of a second time stamp. In this case, the first time stamp may be a time point earlier than the second time stamp. Specifically, the prediction information predicted through the first RNN of the dimension restoration sub-model may be transferred to the second RNN of the next time stamp, and the prediction information predicted through the second RNN may be transferred to the third RNN of the next time stamp. In this process, one or more attention modules may determine the time point of a feature to focus on through the correlation information between time steps. That is, the dimension restoration sub-model (330) may output prediction information on the correlation between the time points of various factors by determining the time point of a change factor to focus on through one or more attention modules (320) in the process of repeatedly predicting prediction information through one or more RNNs.

That is, the integrated feature output through the dimension reduction sub-model (310) can be transferred to one or more attention modules (320). In this case, one or more attention modules (320) can calculate attention weights for sleep analysis information corresponding to each integrated feature. One or more attention modules (320) can assign attention weights to each feature of input data (i.e., sleep sensing data) related to the time series to emphasize elements to be focused on between the input data and the output data.

That is, the sleep analysis model including the dimension reduction sub-model (310), one or more attention modules (320), and the dimension restoration sub-model (330) of the present disclosure can output sleep analysis information (i.e., prediction information related to sleep stages) by reflecting sequence data at each point in time, rather than simply outputting sleep analysis information (i.e., prediction information related to sleep stages) by considering a point in time related to one sequence data.

For example, when a specialist or sleep technician actually interprets the results of a polysomnography test, they may interpret the results by considering not only a single sequence corresponding to a single 30-second period, but also the previous sequence. In other words, if the prediction information about the sleep stage is derived based on a single sequence without considering the relationship with the previous sequence, there is a concern that the accuracy may be lacking.

The sleep analysis model of the present disclosure can predict sleep stages by considering the characteristics of the preceding and following sequence data in the process of calculating sleep analysis information corresponding to one sequence data through the aforementioned process. Accordingly, the accuracy and reliability of the calculated sleep analysis information can be guaranteed.

According to one embodiment of the present disclosure, the sleep analysis model may be characterized by transfer learning. For example, the accuracy of sleep stage prediction for patients suffering from sleep disorders may be lower than that of normal people. For example, when performing sleep stage prediction using a neural network based on sleep sensing data acquired through patients suffering from sleep apnea, the accuracy of sleep stage prediction for users who do not suffer from sleep disorders may be about 10 to 15% lower.

In general, it is difficult to clearly classify whether a user has a sleep disorder before performing a polysomnographic test. Accordingly, in order to create a sleep analysis model that provides sleep analysis information corresponding to sleep sensing data of users with sleep disorders and users without sleep disorders, sleep sensing data related to users with sleep disorders may be additionally required. In other words, multiple sleep sensing data of multiple users with sleep disorders must be constructed as learning data for training a sleep analysis model (i.e., a neural network model), and this may incur a lot of time and cost for training a neural network model through a data set.

Accordingly, the sleep analysis model of the present disclosure may be characterized by being transferred learned to output sleep analysis information corresponding to sleep sensing data corresponding to a different domain (e.g., users suffering from a sleep disorder) through transfer learning. Specifically, the sleep analysis model may be characterized by being transferred learned to enable application of an algorithm trained in a source domain to a different target domain. Here, the source domain may mean something related to sleep sensing data related to a user who does not suffer from a sleep disorder, and the target domain may mean something related to sleep sensing data related to a user suffering from a sleep disorder. Transfer learning may mean, for example, recalibrating the weights of a model trained in the source domain to suit the target domain and using them. According to an additional embodiment, the sleep analysis model of the present disclosure may also perform learning on data in a situation where there is no label related to the correct answer through Domain Adaptation. That is, through transfer learning and domain adaptation, sleep analysis models can improve learning speed and performance while using a small amount of learning data for various domains (e.g., users with or without sleep disorders).

Accordingly, the sleep analysis model of the present disclosure can work well not only for users who do not suffer from sleep disorders but also for patients with sleep disorders. That is, the sleep analysis model can output more robust sleep analysis information in response to sleep sensing data of users suffering from sleep disorders. In other words, it can provide robust and automated sleep analysis information for users with sleep disorders.

According to one embodiment of the present disclosure, sleep analysis information output by the sleep analysis model may include prediction confidence information. The sleep analysis information may include prediction confidence information corresponding to each of one or more sequence data.

Specifically, the sleep analysis model can input sleep sensing data including one or more sequence data, and output sleep stage information corresponding to each sequence data. In this case, the sleep analysis model can input one sequence data, calculate a score (i.e., softmax) corresponding to each of one or more sleep stages, and generate sleep stage information based on the score calculated corresponding to each sleep stage. When the processor (130) calculates prediction information for a sleep stage through the sleep analysis model, it can generate prediction confidence information through the score value that contributed to calculating the prediction information.

For example, when the sleep analysis model inputs the first sequence data (201) acquired through 0 channels for 0 to 30 seconds in relation to the user's sleep, the sleep analysis model can calculate a score corresponding to each of one or more sleep stages corresponding to the first sequence data (201). For a specific example, the sleep analysis model can calculate wake - 2 points, N1 - 10 points, N2 - 80 points, N3 - 7 points, and REM - 1 point corresponding to the first sequence data (201), and determine 'N2' as the sleep stage information corresponding to the first sequence data (201) based on the largest score value. That is, the user for 0 to 30 seconds can calculate prediction information corresponding to the N2 sleep stage. In this case, the processor (130) can determine the prediction certainty information as '80' based on the score that contributed to the prediction of the N2 sleep stage. For example, the greater the prediction confidence information, the higher the accuracy (or reliability) of the sleep stage predicted by the artificial neural network. The specific description of the above-described sequence data, score value, sleep stage, and prediction confidence information is only an example, and the present disclosure is not limited thereto.

That is, the processor (130) can provide prediction information on sleep stages at each time point during the user's sleep and prediction confidence information corresponding to the prediction information for each sleep stage. In this case, the prediction confidence information can be used as an indicator of how reliable the output of the artificial neural network (i.e., the sleep analysis model) (i.e., the prediction information on sleep stages) is. In other words, by providing prediction confidence information related to sleep stage estimation at each time point together with the sleep stage information, reliable use in a medical environment can be enabled.

According to one embodiment of the present disclosure, the processor (130) may perform correction on the prediction certainty information output by the sleep analysis model through temperature scaling. For example, the use of an artificial neural network in a medical environment may have a risk due to uncertainty in operation that operates well in some situations but not in other situations, not because of low average accuracy. Accordingly, when prediction certainty information is generated by utilizing only the softmax value, which is the output of the final output layer of the artificial neural network, it may be difficult to ensure the reliability of the prediction certainty information. For example, if the neural network is overconfident in its own prediction, there is a concern that inaccurate information may be transmitted to a user who refers to the prediction certainty information.

Accordingly, the processor (130) can perform a task of adjusting the level of confidence of the model in the prediction and the actual accuracy to the same level through the temperature scaling process. Specifically, when the logit vector Z is given using a single scalar parameter T, the confidence prediction can be as follows.

As T increases, the probability value can be close to 1/K, which represents the maximum uncertainty. That is, the present disclosure can perform correction for the prediction confidence through temperature scaling that optimizes to minimize the NLL (Negative log likelihood) for the validation set. In this case, temperature scaling does not affect the accuracy of the model, but can only affect the calibration.

That is, by correcting the confidence level using temperature scaling, it is possible to secure the reliability of the prediction confidence level information itself, which is used as a user's judgment indicator.

According to one embodiment of the present disclosure, the processor (130) can generate relationship information between prediction confidence information and sleep stage information. In addition, the processor (130) can update sleep analysis information based on the relationship information.

To explain in detail, sleep analysis information including sleep stage information and prediction certainty information may be characterized by being generated for each time point. For a specific example, as illustrated in FIG. 5, sleep analysis information (400) may include sleep stage information and prediction certainty information generated corresponding to each time point. For example, first sleep analysis information (401) may be generated corresponding to first sequence data (201) related to biosignals acquired through nine channels during 0 to 30 seconds of sleep. The first sleep analysis information (401) may include information that the user's sleep corresponds to N1 at the time point corresponding to the first sequence data and that the prediction certainty information is 90%.

In this case, the processor (130) can generate relationship information by considering the prediction certainty information and the sleep stage information during the entire sleep period. For example, if the overall average of the prediction certainty information is calculated to be relatively low and the change in the sleep stage information during the entire sleep is more than a predetermined change threshold, the processor (130) can generate relationship information indicating that the change in the sleep stage is frequent when the prediction certainty is low. For another example, if the user is interpreted to have a disease related to sleep apnea through the sleep stage ratio during the user's sleep and the overall average of the accumulated prediction certainty information is relatively low, the processor (130) can generate relationship information indicating that sleep apnea is related to low prediction certainty information. In other words, relationship information indicating that a user with a low prediction certainty is likely to have a sleep disorder can be generated. The specific description of the relationship information described above is merely an example, and the present disclosure is not limited thereto.

That is, the processor (130) can generate relationship information between the prediction confidence information and the sleep stage information, and update the sleep analysis information based on the generated relationship information. This can enable the provision of sleep analysis information through new relationship information between the sleep pattern according to the change in the sleep stage and the prediction accuracy. Accordingly, the processor (130) can provide insight into new correlations other than sleep analysis of generally known patterns to the user (e.g., a specialist).

According to one embodiment of the present disclosure, the processor (130) can generate feature map information based on sleep analysis information. Specifically, the processor (130) can generate feature map information by visualizing attention weights for sleep analysis information according to each integrated feature through one or more attention modules.

For a specific example, the computing device (100) may generate prediction information that the user's sleep stage corresponds to the N3 sleep stage for the 1 minute based on the user's sleep sensing data acquired in response to the first time point (e.g., the first minute of the user's sleep after going to bed). In addition, the computing device (100) may obtain attention weights of each input element for sleep analysis information related to the first time point (i.e., prediction information that the user's sleep corresponds to the N3 sleep stage at the time point) from one or more attention modules. The computing device (100) may generate feature map information by visualizing the acquired attention weights for each element.

That is, the computing device (100) can visualize the basis for judgment in the process of generating prediction information for input data (i.e., sleep sensing data) related to a time series by utilizing one or more attention modules. In other words, it is possible to provide a visualization of which signal (i.e., which sleep sensing data) at which time point played an important role in the reading of sleep stages at each time point through attention weight values. For example, feature map information can be generated to provide meaningful visualization information by differently displaying each pixel based on signals or information that are noted through attention weights.

In general, it is difficult to understand the internal algorithm of an artificial neural network model. In other words, it is difficult to understand the basis on which the output related to the input data is generated. This may have the problem of making it difficult to use in an actual medical environment as it acts as a risk due to the uncertainty of the artificial neural network model.

The processor (130) of the present disclosure provides a visualization of the pattern of signals that played an important role in predicting sleep stages at each time point, thereby visualizing the basis on which the neural network model made predictions or judgments. Accordingly, the validity of the prediction information output by the sleep analysis model of the present disclosure can be precisely verified, thereby maximizing the synergy of collaboration with users (e.g., specialists).

FIG. 6 illustrates a flowchart exemplarily illustrating a method for providing sleep-related analysis information related to one embodiment of the present disclosure.

According to one embodiment of the present disclosure, the method may include a step (510) of obtaining sleep sensing data of a user.

According to one embodiment of the present disclosure, the method may include a step (520) of processing sleep sensing data as input to a sleep analysis model to output sleep analysis information.

According to one embodiment of the present disclosure, the method may include a step (530) of generating feature map information based on sleep analysis information.

The steps illustrated in the above-described FIG. 6 may be changed in order as needed, and at least one step may be omitted or added. That is, the above-described steps are merely an embodiment of the present disclosure, and the scope of the rights of the present disclosure is not limited thereto.

FIG. 7 is a schematic diagram illustrating one or more network functions related to one embodiment of the present disclosure.

Throughout this specification, the terms computational model, neural network, network function, and neural network may be used interchangeably. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as “nodes.” These “nodes” may also be referred to as “neurons.” A neural network is composed of at least one or more nodes. The nodes (or neurons) constituting the neural networks may be interconnected by one or more “links.”

In a neural network, one or more nodes connected through a link can form a relationship of input node and output node relatively. The concept of input node and output node is relative, and any node in an output node relationship to one node can be in an input node relationship in a relationship to another node, and vice versa. As described above, the input node-to-output node relationship can be created based on a link. One or more output nodes can be connected to one input node through a link, and vice versa.

In a relationship between input nodes and output nodes connected through a single link, the value of the output node can be determined based on the data input to the input node. Here, the node that interconnects the input node and the output node can have a weight. The weight can be variable and can be varied by a user or an algorithm so that the neural network can perform a desired function. For example, when one or more input nodes are interconnected to one output node through each link, the output node can determine the output node value based on the values input to the input nodes connected to the output node and the weight set for the link corresponding to each input node.

As described above, a neural network is a network in which one or more nodes are interconnected through one or more links to form input node and output node relationships within the neural network. Depending on the number of nodes and links within the neural network, the relationship between the nodes and links, and the weight values assigned to each link, the characteristics of the neural network can be determined. For example, if there are two neural networks with the same number of nodes and links and different weight values between the links, the two neural networks can be recognized as being different from each other.

A neural network can be composed of one or more nodes. Some of the nodes constituting the neural network can form a layer based on the distances from the initial input node. For example, a set of nodes whose distance is n from the initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed through to reach the node from the initial input node. However, this definition of a layer is arbitrary for the purpose of explanation, and the order of a layer in a neural network can be defined in a different way than described above. For example, a layer of nodes can be defined by the distance from the final output node.

The initial input node may refer to one or more nodes in the neural network to which data is directly input without going through a link in a relationship with other nodes. Or, in a neural network, in a relationship between nodes based on a link, it may refer to nodes that do not have other input nodes connected by a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in a relationship with other nodes in the neural network. In addition, the hidden node may refer to nodes that constitute the neural network other than the initial input node and the final output node. The neural network according to one embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be less than the number of nodes in the output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer. In addition, a neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be greater than the number of nodes in an output layer, and the number of nodes increases as it progresses from the input layer to the hidden layer. A neural network according to another embodiment of the present disclosure may be a neural network in the form of a combination of the neural networks described above.

A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. Using a deep neural network, latent structures of data can be identified. That is, latent structures of photos, text, videos, voices, and music (for example, what objects are in photos, what the content and emotion of text are, what the content and emotion of voice are, etc.) can be identified. A deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an auto encoder, a generative adversarial network (GAN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, etc. The description of the above-described deep neural network is only an example, and the present disclosure is not limited thereto.

A neural network can be trained by at least one of supervised learning, unsupervised learning, and semi-supervised learning. The training of a neural network is to minimize the error of the output. In the training of a neural network, training data is repeatedly input into a neural network, the output of the neural network and the target error for the training data are calculated, and the error of the neural network is backpropagated from the output layer of the neural network to the input layer in the direction of reducing the error, thereby updating the weights of each node of the neural network. In the case of supervised learning, training data in which the correct answer is labeled for each training data is used (i.e., labeled training data), and in the case of unsupervised learning, the correct answer may not be labeled for each training data. That is, for example, in the case of supervised learning for data classification, the training data may be data in which each training data category is labeled. Labeled training data is input to the neural network, and the error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of unsupervised learning for data classification, the error can be calculated by comparing the input training data with the output of the neural network. The calculated error is backpropagated in the neural network in the reverse direction (i.e., from the output layer to the input layer), and the connection weights of each node of each layer of the neural network can be updated according to the backpropagation. The amount of change in the connection weights of each node to be updated can be determined according to the learning rate. The calculation of the neural network for the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of repetitions of the learning cycle of the neural network. For example, in the early stage of learning of the neural network, a high learning rate can be used so that the neural network can quickly acquire a certain level of performance, thereby increasing efficiency, and in the later stage of learning, a low learning rate can be used to increase accuracy.

In learning neural networks, learning data can generally be a subset of real data (i.e., data to be processed using the learned neural network), and therefore, there can be a learning cycle in which errors for learning data decrease but errors for real data increase. Overfitting is a phenomenon in which excessive learning on learning data increases errors for real data. For example, a neural network that learned cats by showing yellow cats cannot recognize cats when it sees cats that are not yellow, which can be a type of overfitting. Overfitting can act as a cause of increasing errors in machine learning algorithms. Various optimization methods can be used to prevent overfitting. To prevent overfitting, methods such as increasing learning data, regularization, and dropout, which omits some nodes of the network during the learning process, can be applied.

The steps of a method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination of these. The software module may reside in a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present disclosure pertains.

The components of the present disclosure may be implemented as a program (or application) to be executed by combining with a computer as hardware and stored on a medium. The components of the present disclosure may be implemented as software programming or software elements, and similarly, the embodiments may be implemented in a programming or scripting language such as C, C++, Java, assembler, etc., including various algorithms implemented as a combination of data structures, processes, routines, or other programming elements. Functional aspects may be implemented as algorithms that are executed on one or more processors.

Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein for convenience as “software”), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" includes a computer program, a carrier, or a medium that is accessible from any computer-readable device. For example, computer-readable media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). In addition, the various storage media presented herein include one or more devices and/or other machine-readable media for storing information. The term "machine-readable media" includes, but is not limited to, wireless channels and various other media that can store, retain, and/or transmit instructions(s) and/or data.

It is to be understood that the specific order or hierarchy of steps in the processes presented is an example of exemplary approaches. It is to be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Thus, the disclosure is not intended to be limited to the embodiments disclosed herein, but is to be construed in the widest scope consistent with the principles and novel features disclosed herein.

Claims (14)

A method for providing sleep analysis information performed on a processor of a computing device,
Step of acquiring user's sleep sensing data; and
A step of using the acquired sleep sensing data as input to a sleep analysis model, wherein the sleep analysis model includes one or more attention modules, and outputting sleep analysis information;
The above sleep sensing data is time series data acquired during the user's sleep period.
Consisting of a combination of one or more sequence data separated by a unit of time,
A method for providing sleep analysis information performed on a processor of a computing device. In paragraph 1,
The above unit time is characterized by being 30 seconds.
A method for providing sleep analysis information performed on a processor of a computing device. In the second paragraph,
The above sequence data is,
Characterized in that it is a user's biometric information obtained through one or more channels based on the above unit time.
A method for providing sleep analysis information performed on a processor of a computing device. In paragraph 1,
The above sleep analysis information
Including sleep stage information,
The above sleep stage information is information indicating that the user's sleep at a specific point in time corresponds to at least one of one or more sleep stages.
A method for providing sleep analysis information performed on a processor of a computing device. In paragraph 4,
The above sleep analysis information
Further comprising prediction confidence information regarding information indicating that the sleep of the user at a particular point in time corresponds to at least one of one or more sleep stages;
A method for providing sleep analysis information performed on a processor of a computing device. In the first paragraph,
The above sleep analysis model is,
It is characterized by transfer learning,
The above transfer learning is,
Including performing learning in the target domain based on an algorithm trained in the source domain,
The above source domain is for users who do not suffer from sleep disorders,
The above target domain is characterized as being a domain for users suffering from sleep disorders.
A method for providing sleep analysis information performed on a processor of a computing device. In paragraph 5,
A step of performing correction on the above prediction confidence information; and
A step of updating the sleep analysis information based on the above prediction confidence information and the sleep stage information; further comprising;
A method for providing sleep analysis information performed on a processor of a computing device. In a computing device providing sleep analysis information,
A network section that transmits and receives data with other electronic devices;
memory for storing computer programs; and
A processor for reading the stored computer program; including:
The above processor,
Obtaining a user's sleep sensing data, and using the obtained sleep sensing data as an input to a sleep analysis model - the sleep analysis model including one or more attention modules - outputting sleep analysis information.
The above sleep sensing data is time series data acquired during the user's sleep period.
Consisting of a combination of one or more sequence data separated by a unit of time,
A computing device that provides sleep analysis information. In Article 8,
The above unit time is characterized by being 30 seconds.
A computing device that provides sleep analysis information. In Article 9,
The above sequence data is,
Characterized in that it is a user's biometric information obtained through one or more channels based on the above unit time.
A computing device that provides sleep analysis information. In Article 8,
The above sleep analysis information
Including sleep stage information,
The above sleep stage information is information indicating that the user's sleep at a specific point in time corresponds to at least one of one or more sleep stages.
A computing device that provides sleep analysis information. In Article 11,
The above sleep analysis information
Further comprising prediction confidence information regarding information indicating that the sleep of the user at a particular point in time corresponds to at least one of one or more sleep stages;
A computing device that provides sleep analysis information. In Article 8,
The above sleep analysis model is,
It is characterized by transfer learning,
The above transfer learning is,
Including performing learning in the target domain based on an algorithm trained in the source domain,
The above source domain is for users who do not suffer from sleep disorders,
The above target domain is characterized as being a domain for users suffering from sleep disorders.
A computing device that provides sleep analysis information. In Article 12,
The above processor
Perform corrections to the above prediction confidence information,
Characterized in that the sleep analysis information is updated based on the above prediction confidence information and the sleep stage information.
A computing device that provides sleep analysis information.