KR20240153464A - Ai-health care robot appatus and self-exam method using the same - Google Patents

Abstract

The present invention relates to an AI-healthcare robot device capable of storing and managing medical data measuring a user's health status in a storage device on a robot by a hand biosensor and a retina image sensor installed on the robot in daily life, and automatically analyzing the medical data by artificial intelligence to identify risk factors of the user's health. In particular, the AI-healthcare robot device and a self-testing method using the same can simultaneously measure the user's health status and authenticate the user's status by the hand biosensor and the retina image sensor installed on the robot during body contact between a person and the robot to check eye diseases, dementia, body fever, blood pressure, blood sugar, electrocardiogram, heart rate, oxygen saturation, cholesterol, body fat, and the like, which are major indicators of health management, thereby preventing the user's disease in advance in daily life.

Description

AI-HEALTH CARE ROBOT APPATUS AND SELF-EXAM METHOD USING THE SAME

This invention relates to an AI-healthcare robot device that measures a user's health status by a hand biosensor and a retinal image sensor installed on a robot in daily life, stores and manages medical data in a storage device on the robot, and automatically analyzes the data by artificial intelligence to identify risk factors for the user's health. In particular, the invention provides an AI-healthcare robot device that can simultaneously measure the user's health status and authenticate the user's identity by a hand biosensor and a retinal image sensor installed on the robot during physical contact between a person and the robot to check for major indicators of health management such as eye disease, dementia, body temperature, blood pressure, blood sugar, electrocardiogram, heart rate, oxygen saturation, cholesterol, and body fat, thereby enabling the user to prevent diseases in advance in daily life, and a self-examination method using the same.

With the recent advancement of medical equipment technology and the expansion of hospital infrastructure, many people are receiving abundant medical benefits, and with the extension of life expectancy and improvement in quality of life, people are seeking more happiness than ever before.

However, in modern society, with the improvement of eating habits, lack of exercise, and rapid progress toward an aging society, the need for regular health checkups is becoming increasingly important.

In addition, because it is inconvenient to have to set aside extra time to get a health checkup, most people visit the hospital late when they have health problems.

Therefore, if we can regularly check the health status of our body in our daily lives, we can prepare in advance for many situations that threaten our health.

In particular, eye diseases, dementia, body temperature, blood pressure, blood sugar, heart rate, electrocardiogram, oxygen saturation, cholesterol, and body fat are major indicators of health management in daily life that tell us well about our body condition. If we manage these well on a regular basis, we can not only prevent many diseases in advance, but also live a healthier life.

Nowadays, many medical devices that allow patients to check their body fat, blood sugar, and blood pressure anytime and anywhere without going to the hospital are widely available, and patients use them at home or in the office. However, these medical devices are scattered in different places, so it is very inconvenient for the general public to consistently monitor key indicators of health management in their daily lives. For example, patients have to take out each medical device from its storage location and use it, which is inconvenient, but they also have to authenticate themselves for each medical device every time they use it.

Today's digital medical devices not only allow multiple people (e.g. family members) to use them together, but also the data measured from each patient is individually stored and managed for continuous patient monitoring, and for this purpose, user registration and authentication procedures must be followed before using the medical device. However, in this case, users have to authenticate themselves for each medical device every time they use it, which is inconvenient.

In addition, since existing digital medical devices use different products for each measurement item, it is difficult to obtain synchronized bio-signals (e.g., electrocardiogram, PPG signal, oxygen saturation) measured at the same time. Considering that these bio-signals change frequently depending on the patient's body condition and time, diagnosing using bio-signals measured at different times not only reduces reliability but also leads to inaccurate results.

Korean Patent Publication, No. 1278552 (Registered on June 19, 2013)

In order to solve the problems of the prior art described above, the present invention provides an AI-healthcare robot device and a self-examination method using the same, which not only allows user authentication without a separate authentication procedure, but also allows easy measurement of major indicators of health management, such as eye disease, dementia, body temperature, blood pressure, blood sugar, heart rate, electrocardiogram, oxygen saturation, cholesterol, and body fat, during physical contact with the robot, and provides a remote medical diagnosis service through an Internet communication connection with a doctor terminal when abnormal symptoms are found.

In addition, the present invention aims to provide an AI-healthcare robot device equipped with an artificial intelligence neural network capable of measuring blood pressure, cholesterol, blood sugar, and heart disease by measuring synchronized bio-signals from a user's body at the same time, and a self-examination method using the same.

However, the technical tasks to be achieved by the embodiments of the present invention are not limited to the technical tasks described above, and other technical tasks may exist.

As a technical means for achieving the above technical task, an AI-healthcare robot device according to one embodiment of the present invention may include a hand print as a body contact part that is installed on a chest panel of the robot and allows contact with a hand of a person (subject, user), a hand biosensor embedded in the hand print for collecting a biosignal of the person while in contact with the hand of the person, a retinal image sensor installed in an eye part of the robot for collecting a retinal image of the subject, an eye navigator for performing optical axis alignment between the retina of the subject and the retinal image sensor, a focal length adjustment driving unit for performing focal length adjustment between the retina of the subject and the retinal image sensor, and a wireless communication connection means for analyzing biosignals measured by the hand biosensor and the retinal image sensor through an artificial intelligence neural network and providing a remote medical diagnosis service through an Internet communication connection with a doctor terminal.

The hand print of the present invention provides a contact surface with a user's hand, so that it is possible to collect a user's biosignal while the hand is in contact with the hand print, and at the same time perform user authentication.

The biosignal of the present invention can be measured by a hand biosensor embedded in or connected to a hand print.

The eye navigator of the present invention can request the subject to come into close contact with the ocular portion (the portion that touches the eyeball) of the retinal image measuring unit by a voice message command.

According to one embodiment of the present invention, the AI-healthcare robot device includes: a hand print that is installed on the chest panel of the robot and allows contact with a human hand; a hand biosensor embedded in the hand print that performs user authentication and simultaneously measures a human biosignal during contact with the human hand; a retinal image measuring unit that includes a retinal image sensor installed in the eye area of the robot to collect a retinal image of a subject; a biosensing unit that amplifies or converts a biosignal obtained from the hand biosensor into a digital signal; a biosignal collecting unit that collects the retinal image and the digitalized biosignal; a biosignal validity determination means that filters out only a valid biosignal portion among biosignal components for which user authentication has been completed among the biosignals collected by the biosignal collecting unit; a vision camera installed on the head area of the robot to obtain an image of a body part of the subject; a short-range wireless communication connection unit that receives medical data measured from an external medical device and transmits it to the biosignal collecting unit; a medical data storage unit that stores the received medical data or the valid biosignal; and the medical data stored in the medical data storage unit is transmitted to a server through the Internet, and thereafter, an artificial intelligence neural network on the server. The medical data may include a wireless communication connection means for receiving feedback on the analysis results of the medical data by an expert system, and a control unit for controlling the bio-sensing unit, the retinal image measuring unit, the bio-signal collecting unit, the medical data storage unit, and the wireless communication connection means and providing the analysis results of the medical data to the user through a screen or text/voice message.

The above hand biosensor may be configured with at least one selected from among an SpO2 sensor for measuring oxygen saturation and a photoplethysmography (PPG) signal, a plurality of hand electrodes, a body fat sensor for measuring body fat, a fingerprint sensor, an ECG sensor for measuring heart rate or electrocardiogram, and a blood sugar sensor for obtaining glucose spectroscopy (impedance frequency response).

The above retinal image measuring unit may include a retinal image sensor for obtaining a retinal image, a pressure sensor for sensing contact pressure between the eyepiece and the eye of the subject, an eye navigator, a focal distance adjustment driving unit for adjusting the focal distance between the retina of the subject and the retinal image sensor, and a temperature sensor for measuring body heat of an eye area in contact with the eyepiece.

The above SpO2 sensor is preferably installed at the fingertip of the hand print, and a reflective vascular optical sensor can be used.

For example, when the reflective vascular optical sensor is brought into contact with a human finger (e.g., thumb, index finger, or middle finger), the oxygen saturation and PPG signal (photoplethysmographic signal) can be measured from an artery of the human finger by the optical detector.

The above fingerprint sensor is preferably installed at the fingertip of the hand print, and fingerprint authentication can be performed when the fingerprint area of a person's finger (e.g., thumb, index finger, or middle finger) comes into contact with the fingerprint sensor.

In addition, it is preferred that the hand print includes a hand electrode 1 that is simultaneously contacted with the fingerprint sensor when a person's finger is in contact with the fingerprint sensor. In this case, when the fingerprint portion of the person's finger is in contact with the fingerprint sensor, the hand electrode 1 is also simultaneously contacted, so that while the fingerprint sensor performs fingerprint authentication, the hand electrode 1 can operate as a drive electrode for body fat measurement or as a ground electrode of the ECG sensor.

Additionally, electrocardiogram measurement is performed by an ECG sensor, and the voltage (potential difference) signal formed between the ECG sensor and the ground electrode can be amplified and measured.

In the electrocardiogram measurement of the present invention, it is preferred to measure the PQRST wave for Lead Ⅰ of a standard limb lead using an ECG sensor and a ground electrode.

Another aspect of the electrocardiogram measurement of the present invention is that body fat and glucose spectroscopy can also be measured simultaneously by using hand electrodes embedded in the fingertips of the right hand print as the right-hand contact surface and the left hand print as the left-hand contact surface during electrocardiogram measurement using the ECG sensor.

The artificial intelligence neural network of the present invention uses bio-signals and personal body information as input, and may include at least one selected from an artificial intelligence neural network for measuring blood pressure, an artificial intelligence neural network for measuring cholesterol, an artificial intelligence neural network for measuring blood sugar, an artificial intelligence neural network for measuring heart disease, and an artificial intelligence neural network for analyzing the retina.

The above personal physical information includes standard blood pressure values.

The standard blood pressure value of the present invention is the blood pressure of a statistically known normal person, and the standard blood pressure value can be calculated by applying [Mathematical Formula 1].

[Mathematical Formula 1]

To calculate the blood pressure value in the above [Mathematical Formula 1], it is preferred to use d=0.6×height and h=0.3×height.

Additionally, the above coefficients σ and β are parameters that can be set by statistical information.

Additionally, the input of the artificial intelligence neural network can use a synchronized biosignal based on the R point of the ECG signal.

In addition, the biosignal of the present invention may include at least one selected from body temperature, an ECG signal, a PPG signal, SpO2, glucose spectroscopy, body fat, PTT (Pulse Transit Time), Pulse Arrival Time (PAT), PWV (Pulse Wave Velocity), HRV (Heart Rate Variability), retinal images, and medical data received from an external medical device.

In addition, the blood sugar sensor is preferably configured using a plurality of hand electrodes installed at the fingertips of the hand print, and at this time, the blood sugar sensor can measure the impedance frequency response for the subject's body, thereby obtaining glucose spectroscopy that can estimate the subject's blood sugar level without blood collection.

Additionally, the hand print may be equipped with a contact sensing means to detect or visually display whether there is close contact between the hand print and the palm of the subject's hand.

In addition, the control unit may include a health tracking management unit for observing changes in the bio-signals and medical data to inform the user of health risks, inform the user of items requiring intensive care examination, or inform the user of the next examination schedule.

In addition, the health tracking management unit can discover health care items that require intensive care and control the robot to automatically perform the corresponding bio-signal examination for the discovered intensive care items on the patient. In this case, the eye navigator recognizes the image of the patient's face from the vision camera and searches for the patient that requires a bio-signal examination.

In addition, the control unit is characterized by controlling the speaker unit and the screen (display device) to provide the patient with health management instructions, remote medical care, and guidelines on how to use medical devices as voice and video services based on the patient's health condition based on the medical data analysis results. The screen is preferably installed on the chest panel of the robot.

It is preferred that the above screen display the results of analysis of medical data collected from the biosignal collection unit or display additional information provided by the health tracking management unit.

In addition, the control unit may include a body temperature diagnosis unit for determining the patient as a subject suspected of having abnormal body temperature if the patient's temperature collected by the bio-signal collection unit is an abnormal body temperature.

The above-described short-range communication connection unit can provide a short-range wireless communication connection (e.g., a Near Field Communication (NFC) interface, Bluetooth, the Internet of Things, or an infrared communication connection) with external medical devices to receive medical data measured from a patient by the medical devices and transmit it to a biosignal collection unit.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the solution to the aforementioned problem of the present invention, not only is user authentication possible without a separate user authentication process, but also, by measuring the user's bio-signals by the hand bio-sensor installed in the hand print of the robot and the retinal image sensor installed in the eye area of the robot during contact with the robot in daily life, the user's disease can be prevented in advance by managing major indicators of health such as eye disease, dementia, body temperature, blood pressure, blood sugar, heart disease, heart pulse, oxygen saturation, cholesterol, and body fat.

In addition, the AI healthcare robot device of the present invention can use synchronized bio-signals measured from the user's body at the same time as input signals for an artificial intelligence neural network, thereby enabling accurate measurement of blood pressure, cholesterol, blood sugar, and heart disease.

In addition, the AI healthcare robot device of the present invention can measure eye diseases and dementia of a patient by analyzing the patient's retinal image.

However, the effects that can be obtained from this invention are not limited to the effects described above, and other effects may exist.

Figure 1 is an example of an AI-healthcare robot device according to the present invention.
FIG. 2 is an example of a digital biosignal analysis module of an AI-healthcare robot device according to the present invention.
Figure 3a is an example in which a hand biosensor, which collects a subject's biosignal through contact with the subject's hand, is placed and embedded in the fingertips or palms of a right hand print and a left hand print.
FIGS. 3b and 3c illustrate several embodiments of a contact sensing means for informing a contactee of whether there is close contact between the hand of the subject and the hand print.
FIG. 3d is another embodiment of a contact sensing means using a touch screen.
FIGS. 3e and 3f show an embodiment in which the hand print of the present invention is installed on the cover or smart mirror of a notebook.
FIG. 4 is an example of a retinal image measuring unit installed in the eye area of an AI healthcare robot device according to the present invention.
FIG. 5 is an example of a body fat measurement unit that measures body fat using hand electrodes and a glucose measurement unit that measures glucose spectroscopy, while fingerprint authentication is performed by a fingerprint sensor on a hand print installed on an AI healthcare robot device.
FIG. 6 is an example of an electrocardiogram measuring unit that measures an ECG signal or electrocardiogram when the bottom surface of a subject's hand comes into contact with an ECG sensor and hand electrode 1 installed on a hand print of an AI-healthcare robot device according to the present invention.
FIG. 7a and FIG. 7b are examples of an oxygen saturation and PPG signal measurement unit using a reflective SpO2 sensor on a hand print of an AI-healthcare robot device according to the present invention.
FIG. 8a and FIG. 8b are examples of predicting blood pressure, cholesterol, blood sugar, and heart disease using biosignals obtained from a hand biosensor installed on a hand print of an AI-healthcare robot device.

Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those with ordinary skill in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are assigned similar drawing reference numerals throughout the specification.

Throughout this specification, when a part is said to be “connected” to another part, this includes not only the case where it is “directly connected,” but also the case where it is “electrically connected” or “indirectly connected” with another element in between.

Throughout this specification, when it is said that an element is located “on,” “above,” “below,” “under,” or “below” another element, this includes not only cases where an element is in contact with another element, but also cases where another element exists between the two elements.

Throughout this specification, whenever a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

Hereinafter, preferred embodiments of the present invention will be described in detail using the attached drawings.

Hereinafter, the artificial intelligence neural network in the present invention includes an expert system.

The expert system of the present invention may be a computer application that makes decisions or solves problems by utilizing knowledge accumulated through intellectual activities and experience of medical experts and inference rules defined by medical experts.

Hereinafter, medical data in the present invention includes medical data measured by external medical devices, and biosignals measured by a hand biosensor and a retinal image sensor.

Hereinafter, the blood pressure measurement unit of the present invention may be used in conjunction with an artificial intelligence neural network for blood pressure measurement.

Hereinafter, the cholesterol measurement unit of the present invention may be used interchangeably with an artificial intelligence neural network for cholesterol measurement.

Hereinafter, the blood sugar measurement unit of the present invention may be used in combination with an artificial intelligence neural network for blood sugar measurement.

Hereinafter, the heart disease measurement unit of the present invention may be used in conjunction with an artificial intelligence neural network for heart disease measurement.

Hereinafter, the retinal disease measurement unit of the present invention may be used in conjunction with an artificial intelligence neural network for retinal analysis.

The artificial intelligence neural network of the present invention may be an artificial intelligence neural network for measuring blood pressure, an artificial intelligence neural network for measuring cholesterol, an artificial intelligence neural network for measuring blood sugar, an artificial intelligence neural network for measuring heart disease, or an artificial intelligence neural network for analyzing the retina.

The biosignal in the present invention may include at least one selected from retinal images, ECG signals, PPG signals, oxygen saturation, PTT, PAT, PWV, HRV, glucose spectroscopy (impedance frequency response), and body fat.

In the present invention, glucose spectroscopy may be a spectrogram or a scalogram.

The ECG signal in the present invention can be used for electrocardiogram measurement.

In the present invention, the robot may be a personal computer (PC), a computer input device such as a mouse, a mouse pad, a laptop, a mobile phone, a tablet PC, a touchscreen device, or a smart mirror including a hand print according to the present invention.

Hereinafter, in the present invention, patient, person, subject or user may be used interchangeably.

Hereinafter, personal body information is characterized as information that includes at least one of body temperature, blood type, sex, age, body height, weight, and standard blood pressure.

According to one embodiment of the present invention, a medical device is a device that provides medical data measured from a patient to a bio-signal collection unit of a robot via short-range wireless communication (e.g., an NFC interface, an Internet connection, a Bluetooth communication connection, or an infrared communication unit), and may include at least one medical device selected from among a mobile phone having a healthcare app, a smartwatch, a glasses-type smart device, an ultrasound scanner, a thermal imaging camera, a stool tester installed in a toilet, a urine tester installed in a toilet, a blood pressure measuring device, a blood sugar measuring device, a body fat measuring device, a scale, a stethoscope, a thermometer, an image sensor showing a sore throat or dental condition, a blood analyzer, a DNA amplification testing device, a virus diagnostic kit device that diagnoses using a specific antigen of a virus, a rapid test device using a biomarker, a dementia testing device, a wearable device, a cancer diagnostic device, a POCT (Point of Care Testing) device, and a healthcare device.

The wearable device is preferably a patch-type wearable device that is attached to the patient's skin, and it is preferred that data obtained by non-invasively measuring electrocardiogram, blood pressure, cholesterol or blood sugar from the patient's skin surface by a sensor built into the patch be wirelessly transmitted to the robot's bio-signal collection unit via a Bluetooth communication connection.

The above patch-type wearable device may be a device that measures blood sugar levels by supplying power to the patch with a reader built into the robot hand via an NFC interface when a patch with a blood sugar sensor is taped to the skin and a robot hand is brought near the patch, thereby measuring the ratio of water and glucose in the blood and providing the blood sugar level to the robot's biosignal collection unit.

Another aspect of the above patch-type wearable device may be a continuous glucose monitoring (CGM) device that measures the glucose concentration of interstitial fluid (the liquid component that fills between cells) at periodic intervals through sensors attached to subcutaneous fat such as the abdomen, arms, and buttocks, and transmits the measurement results to the robot's biosignal collection unit via a short-range wireless communication connection.

FIG. 1 is a diagram showing an embodiment of an AI-healthcare robot device (100) according to the present invention. The AI-healthcare robot device (100) includes a right-hand hand print (70R) and a left-hand hand print (70L) which are installed on a chest panel (10) of a robot (100a) and allow contact with a human hand (72R, 72L), a hand biosensor which is embedded in the hand print (70R, 70L) and performs measurement of a human biosignal during contact with the human hand (72R, 72L) and user authentication during the measurement of the biosignal, a retinal image measuring unit (79) which is installed on an eye area of the robot (100a) and includes a retinal image sensor for collecting a retinal image of the human, a vision camera (22) which is installed on a head area of the robot (100a) and obtains a body image of a subject, and a digital biosignal analysis module (400) which analyzes biosignals obtained from the hand biosensor and the retinal image sensor.

The above hand biosensor and retinal image sensor can obtain a patient's biosignals by physical contact with the patient.

In addition, the robot (100a) is equipped with two lasers (14a, 14b) to measure the distance to the subject of measurement, and the control unit (53) can provide the result of analyzing the bio-signal to the user as a voice message through the speaker unit (38a) or drive the speaker unit (38a) and the microphone unit (38b) to interactively request the user to measure the bio-signal.

The beams injected from the two lasers (14a, 14b) are installed at a predetermined angle with respect to each other so that they intersect each other after a predetermined distance.

Accordingly, the beams injected from the two lasers intersect each other after a certain distance, and the straight-line distance between the two laser beam spots focused on the body of the subject can be measured to calculate the distance between the subject and the robot.

The eye navigator (79a) of the present invention recognizes the face and eyes of the subject by the vision camera (22) and controls the position of the robot so that the robot face faces the front of the subject's face.

In addition, the eye navigator (79a) obtains an image that can measure the distance between two laser beam spots focused on the body of the subject by using the vision camera (22), calculates the distance between the robot (100a) and the subject, and with reference to the calculated distance, controls the movement of the robot (100a) so that it approaches the front of the subject's face at an appropriate distance. Through this, the optical path can be aligned between the subject's eye (62) and the retinal image sensor (42).

Another aspect of the eye navigator (79a) of the present invention is that it can request the subject to come into close contact with the ocular portion of the retinal image measuring unit (79) by a voice message command.

Another aspect of the eye navigator (79a) of the present invention is that when the pressure of the eye part of the subject being measured in contact with the ocular portion (31) is judged to be insufficient by the pressure sensor (40) or the temperature sensor (25), the subject can be requested to make close contact by a voice message command.

Another aspect of the eye navigator (79a) of the present invention is that it can request the subject to make contact with his or her hand (fingers, palm) on the right hand print (70R) and the left hand print (70L) by voice message command.

Another aspect of the eye navigator (79a) of the present invention is that when it is determined that the contact location and contact range of the subject's hand in contact with the surface of the hand print (70L, 70R) are insufficient, it can notify the subject of this by a voice message command.

The hand print (70R, 70L) of the present invention includes a hand-shaped engraved portion that allows contact with a hand (72R, 72L) of a subject, and a hand biosensor that simultaneously acquires a subject's biosignal and authenticates a user during contact with the subject's hand (72R, 72L) can be embedded on the surface of the engraved portion.

Another aspect of the hand print (70R, 70L) of the present invention may include a touch screen, a virtual finger pattern (34) displayed on the touch screen, and a hand biosensor (24a, 24b, 24c, 24d, 26, 35). At this time, when the subject of measurement aligns his/her finger to the virtual finger pattern (34) and places it on the touch screen (11), the hand biosensor can measure the subject's biosignal and perform user authentication at the same time.

The above hand biosensors (24a, 24b, 24c, 24d, 26, 35) are arranged so as to be aligned with each finger on the virtual finger pattern (34), and the ECG sensor (48) is preferably arranged so as to be exposed to the palm portion of the virtual finger pattern (34).

FIG. 2 is an example of a digital biosignal analysis module (400) of an AI-healthcare robot device according to the present invention. The digital biosignal analysis module (400) of the AI-healthcare robot device includes a biosensing unit (440) that amplifies or converts a biosignal obtained from a hand biosensor into a digital signal, a biosignal collection unit (420) for collecting a digitalized biosignal obtained from the biosensing unit (440), a retinal image obtained from a retinal image measurement unit (79), or medical data measured by external medical devices (430) by a short-range wireless communication connection unit (8), a short-range wireless communication connection unit (8) for receiving medical data measured from an external medical device (430) and transmitting it to the biosignal collection unit (420), a biosignal validity determination means (420a) for filtering out only a valid biosignal portion among the biosignal components for which user authentication has been completed among the biosignals collected by the biosignal collection unit (420), and a medical storage unit for storing the biosignal collected by the biosignal collection unit (420). It may include a data storage unit (15), a wireless communication connection means (37) for transmitting medical data stored in the medical data storage unit (15) to a server (13) via the Internet (202), and thereafter receiving feedback on the analysis results of the medical data by an artificial intelligence neural network (16) or an expert system (18) on the server (13); and a control unit (53) for controlling the biosensing unit (440), the biosignal collection unit (420), the medical data storage unit (15), and the wireless communication connection means (37) and providing the analysis results of the medical data to the user via a screen or text/voice message.

The above biosignal collection unit (420) is equipped with a biosignal validity determination means (420a) to filter out only the biosignals (retinal images, ECG signals, PPG signals, oxygen saturation, glucose spectroscopy, body fat, body temperature) that have been verified by the user and are valid among the biosignals collected by the biosignal collection unit (420) and store them in the medical data storage unit (15).

In addition, biosignals such as Pulse Transit Time (PTT), Pulse Wave Velocity (PWV), Pulse Arrival Time (PAT), and Heart Rate Variability (HRV) can be additionally obtained from the correlation between the ECG signal and the PPG signal.

The medical data stored in these medical data storage units (15) are used to analyze and manage the health of the subject by an artificial intelligence neural network (16), an expert system (18), and a health tracking management unit (39).

In other words, the bio-signal validity determination means (420a) filters out only the valid bio-signal portions among the bio-signal components for which user authentication has been completed among the bio-signals collected from the bio-sensing unit (440) and the retinal image measurement unit (79).

It is preferred that the above bio-signal validity determination means (420a) use at least one technique selected from among a 1-D (Dimension) artificial intelligence neural network, LSTM (Long short time Memory), auto-correlation coefficient, a validity range check method, and a cross-correlation coefficient between a feature vector of a standard (reference) bio-signal and a feature vector of a collected bio-signal.

Preferably, the cross-correlation coefficient is preferably calculated to measure the degree of similarity or correlation between two components by using one of the Sum of Squared Difference (SSD), Sum of Absolute Difference (SAD), Euclidean Distance, KNN (K-nearest neighbor algorithm), and NCC (Normalized Cross Correlation).

For example, if the cross-correlation coefficient between the feature vector of the standard (reference) biosignal and the feature vector of the biosignal collected through the biosensing unit (440) is greater than a set threshold, the biosignal collected through the biosensing unit (440) is determined to be a valid biosignal.

A standard (reference) bio-signal is a bio-signal that statistically shows the characteristics and values of a normal person's bio-signal. The feature vector of a standard (reference) bio-signal can be the average values of the feature vectors for a normal person's bio-signal, and feature vectors of multiple standard (reference) bio-signals can be used.

The feature vector of the above biosignal can be extracted by a deep learning neural network or spectrum analysis in the frequency domain.

For example, the above deep learning neural network can be a 1-D CNN (Convolutional Neural Network).

The feature vector by spectral analysis of the above frequency domain may be extracted by frequency component analysis, Short time Fourier Transform (STFT), Wavelet transform, Mel-Frequency Cepstral Coefficient (MFCC), Linear Prediction Codes (LPC), Mel-Frequency Cepstral Coefficient (MFCC), Mel-Coefficient, Perceptual Linear Prediction (PLP), or Bark Frequency Cepstral Coefficients (BFCC).

The above 1-D artificial intelligence neural network or LSTM-based bio-signal validity determination means (420a) can determine the validity of a subject's bio-signal by using a 1-D artificial intelligence neural network learned using bio-signals labeled as valid or invalid bio-signals.

In the case of body temperature, the effective range is preferably 10 to 40 degrees, and it is desirable that a body temperature within this range be recognized as a valid body temperature signal if it remains at the same temperature for more than 1 second.

The above user authentication can be performed by a fingerprint authentication unit (75) by collecting the user's fingerprint from a fingerprint sensor (26).

The above biosensing unit (440) includes a fingerprint authentication unit (75), a body temperature measurement unit (74), an electrocardiogram measurement unit (71), an oxygen saturation and PPG signal measurement unit (72), a body fat measurement unit (77), and a glucose measurement unit (78). The fingerprint authentication unit (75) is electrically connected to a fingerprint sensor (26) for user authentication, the electrocardiogram measurement unit (71) is electrically connected to an ECG sensor (48) for measuring an electrocardiogram, the oxygen saturation and PPG signal measurement unit (72) is electrically connected to an SpO2 sensor (35) for measuring oxygen saturation and a photoelectric volume pulse signal, the body fat measurement unit (77) is electrically connected to a plurality of hand electrodes (24a, 24b, 24c, 24d) for measuring body fat, and the glucose measurement unit (78) is electrically connected to a blood sugar sensor for obtaining glucose spectroscopy.

In the present invention, it is preferred that the blood sugar sensor be configured using hand electrodes (24a, 24b, 24c, 24d).

In addition, the retinal disease measurement unit (20e) can analyze the retinal image of the subject obtained by the retinal image sensor (42) of the retinal image measurement unit (79) to identify risk factors for eye diseases and dementia of the subject.

Additionally, the control unit (53) can transmit a voice message to the user requesting biosignal measurement.

For example, a speaker section (38a) for voice reproduction and a microphone section (38b) for voice recognition can be used to interactively request a user to measure biosignals.

The voice reproduction unit (28a) reproduces voice, and the voice recognition unit (27a) recognizes the user's voice.

In addition, the control unit (53) may include a health tracking management unit (39) that observes changes in bio-signals and medical data to inform the user of the level of risk, inform the user of items requiring intensive care examination, or inform the user of the next examination schedule.

In addition, the health tracking management department (39) schedules examination schedules for each patient.

The above eye navigator can automatically perform a bio-signal examination on a patient in need of intensive care identified by the health tracking management department, or can control the robot to move to visit a patient in need of a bio-signal examination by recognizing the patient's face by the vision camera to automatically perform a bio-signal examination according to the examination schedule of each patient.

In addition, the control unit (53) may include a body temperature diagnosis unit (19) for determining a patient as suspected of having abnormal body temperature if the patient's temperature collected by the bio-signal collection unit (420) through the body temperature measurement unit (74) is abnormal.

The body temperature diagnosis unit (19) can presume (decide) that the user is suspected of having abnormal body temperature if the user's body temperature is 37.5 degrees or higher.

In addition, the retinal image measuring unit (79) includes an eye navigator (79a) that recognizes the position of the eye of the subject using an image captured from a vision camera (22) and controls the movement of a robot to achieve optical axis (optical path) alignment between the eye (62) of the subject and the retinal image sensor (42), and a focal distance adjustment driving unit (79b) for adjusting the focal distance between the retinal image sensor (42) and the retina (60) of the subject.

The above control unit (53) can control the eye navigator (79a) and the focal length adjustment driving unit (79b) to maintain an optimized focal length and optical alignment between the subject's retina and the retinal image sensor.

Another aspect of the digital bio-signal analysis module (400) of the AI-healthcare robot device following the embodiment of FIG. 2 is characterized in that, instead of using an artificial intelligence neural network (16), which is a service application program built on a server (13), the digital bio-signal analysis module (400) of the AI-healthcare robot device or an artificial intelligence neural network app (12) that resides on a mobile phone and is installed as an application (application software) analyzes medical data measured by a hand bio-sensor and a retinal image sensor or medical devices and provides the results to the user.

In this case, the artificial intelligence neural network app (12) provides the advantage of continuously improving the performance of the AI-robot medical hand device according to version upgrades.

FIG. 3a is an example in which hand biosensors for collecting bio-signals of a subject through contact with the subject's hand are installed and arranged on the fingertips or palms of a right hand print (70R) and a left hand print (70L). It is preferred that the right hand print (70R) and the left hand print (70L) be installed on the chest panel (10) of a robot (100a).

In this case, when the subject touches the left hand print (70L) and the right hand print (70R) with each of the two hands (72L, 72R), the subject's fingerprint is recognized by the fingerprint sensor (26) from the left hand print (70L), and the oxygen saturation and photoelectric pulse signal are measured by the SpO2 sensor (35), while the subject's body fat, glucose spectroscopy (impedance frequency response), and electrocardiogram can be measured at the same time from the right hand print (70L).

Considering that biosignals change frequently depending on the subject's body condition and time, it is important to acquire biosignals at the same time in order to accurately measure the subject's condition.

For example, body fat and glucose spectroscopy (impedance frequency response) can be measured at the same time using hand electrodes (24a, 24b, 24c, 24d), and at the same time, an electrocardiogram can be measured using an ECG sensor (48) and hand electrode 1 (24a).

In this embodiment, the fingertips of the left hand print (70L) include a hand biosensor including a fingerprint sensor (26) and an SpO2 sensor (35).

In contrast, the palm of the right hand print (70R) includes a hand biosensor including an ECG sensor (48).

In addition, this example includes hand electrode 1 (24a) and hand electrode 2 (24b) at the tips of the thumb and index finger, respectively, of the left hand print (70L), and hand electrode 3 (24c) and hand electrode 4 (24d) at the tips of the thumb and index finger, respectively, of the right hand print (70R).

The above hand electrode 1 (24a) is composed of a metal conductor that surrounds the outer periphery of the fingerprint sensor (26) and the SpO2 sensor (35) placed on the thumb of the left hand print (70L). Therefore, it has a structure in which the thumb fingerprint area of the left hand finger of the subject is simultaneously contacted with the hand electrode 1 (24a) and the SpO2 sensor (35) during fingerprint authentication. The hand electrode 1 (24a) can be used as a driving electrode when measuring body fat, and as a ground electrode of the ECG sensor (48) when measuring electrocardiogram.

The above SpO2 sensor (35) is a reflective vascular optical sensor composed of a red light LED (51a), an infrared LED (51b), and a light detector (50). When the tip of the left thumb of the subject comes into contact with the tip of the thumb of the left hand print (70L), the oxygen saturation from the peripheral blood vessel artery of the subject's finger is measured by the light detector (50).

The above hand electrode 2 (24b) is configured to wrap the tip of the index finger of the left hand print (70L) with a metal conductor, so that it can be used as a driving electrode together with the hand electrode 1 (24a) by coming into contact with the left index finger of the subject when measuring body fat.

The above hand electrode 3 (24c) is configured in a form that wraps the tip of the thumb of the right hand print (70R) with a metal conductor, and can be used as a detection electrode by coming into contact with the right thumb of the subject when measuring body fat.

The above hand electrode 4 (24d) is configured in a form that wraps the tip of the index finger of the right hand print (70R) with a metal conductor, and can be used as a detection electrode together with the hand electrode 3 (24c) by coming into contact with the index finger of the right hand of the subject when measuring body fat.

In addition, glucose spectroscopy can be measured to estimate the blood sugar level of the subject by making contact with the subject's finger using a hand electrode (24a, 24b, 24c, 24d).

FIGS. 3b and 3c show several embodiments of a contact sensing means for informing a contactee of whether there is close contact between the hand of the subject and the hand print (70L, 70R).

In order to accurately obtain bio-signals from the subject's hand, the bottom surface of the subject's hand must be in close contact with the hand biosensor on the hand print (70L, 70R).

Fig. 3b shows an example in which semicircular LEDs (52a, 52b, 52c, 52d, 52e, 52f) are arranged around the edge of a hand print (70L, 70R) and utilized as a contact sensing means. For example, LEDs corresponding to areas where close contact is not made can be blinked or turned off so that the subject can easily recognize which part (finger, palm) is not properly in contact.

In addition, Fig. 3c also includes an embodiment in which the LED (29) is positioned along the outer edge of the hand print (70L, 70R) and utilized as a contact sensing means. For example, the portion of the LED (29) corresponding to the part that is not in close contact may be displayed darkly so that the subject can easily recognize which part of the hand is not in close contact.

FIG. 3d is another embodiment of a contact sensing means using a touch screen (11), in which the chest panel (10) of the robot (100a) may include a touch screen (11), a virtual finger pattern (34) displayed on the touch screen, and a hand biosensor (24a, 24b, 24c, 24d, 26, 35, 48). At this time, when the subject of measurement places his/her finger on the touch screen (11) while aligning it with the virtual finger pattern (34), the hand biosensor can perform user authentication while simultaneously acquiring the subject's biosignal.

The above hand biosensors (24a, 24b, 24c, 24d, 26, 35) are arranged so as to be aligned with each finger on the virtual finger pattern (34), and the ECG sensor (48) is preferably arranged so as to be exposed on the palm portion of the virtual finger pattern (34).

That is, the virtual finger pattern (34) is displayed on the touch screen (11), and the hand biosensors (24a, 24b, 24c, 24d, 26, 35, 48) are embedded on the chest panel (10), and at this time, the virtual finger pattern (34) and the hand biosensors (24a, 24b, 24c, 24d, 26, 35, 48) are connected to each other to form a hand print shape, thereby providing a right hand print (70R) and a right hand print (70L).

Additionally, the touch screen (11) may include a hand print on/off button (9) for turning the display of a hand print on the touch screen (11) on/off.

In addition, the touch screen (11) applied to this embodiment detects or visually displays whether there is close contact between the hand print (70L, 70R) surface and the palm of the subject's hand.

It can also be used as a contact sensing means.

For example, the control unit (53) can recognize the hand position and contact range that came into contact with the surface of the hand print (70L, 70R) while scanning the touch screen coordinates, and visually inform the subject of the part that is not in close contact.

For example, the image pixel area of the virtual finger pattern (34) corresponding to the part that is not in close contact on the touch screen (11) can be blinked, darkened, or colored.

The touch screen of the present invention is characterized by using one selected from among capacitive, resistive, ultrasonic, and infrared beam methods.

Figures 3e and 3f show an example in which the hand print (70R, 70L) of the present invention is installed on the cover (85) or smart mirror (300) of a notebook (200). In this case, when the subject places his/her finger on the hand print (70R, 70L) while aligning it, the subject's biosignal can be measured by the hand biosensor built into the cover (85) or smart mirror (300) of the notebook (200) and user authentication can be performed by the fingerprint sensor (26) at the same time.

If user authentication is successful by the above fingerprint sensor (26), the user can use the notebook (200) by activating the notebook (200) without a separate authentication process.

The above notebook (200) and smart mirror (300) may be equipped with a digital bio-signal analysis module (400) and may include a speaker unit (38a) and a microphone unit (38b) for providing a voice message to the user or requesting bio-signal measurement interactively.

Another aspect of the present embodiment may include a touch screen (11) on the cover (85) or smart mirror (300) of the notebook (200), a virtual finger pattern (34) that displays the shape of a hand print (70R, 70L) on the touch screen (11), and a hand biosensor (24a, 24b, 24c, 24d, 26, 35).

As can be seen from this embodiment, the robot in the present invention can be replaced by a personal computer (PC), a computer input device such as a mouse, a mouse pad, a laptop, a mobile phone, a smart mirror, a tablet PC, or a touchscreen device including a hand print according to the present invention.

FIG. 4 is an embodiment of a retinal image measuring unit (79) installed in the eye area of an AI-healthcare robot device (100) according to the present invention, which captures a retinal image of a subject obtained by a retinal image sensor (42). The retinal image measuring unit (79) includes a light source (43) that generates a beam, a first polarizer (36a) that changes the polarity of a beam generated from the light source (43) to generate a first beam (B1), a beam splitter (45) that refracts the first beam (B1) toward the retina (60) of the subject (86), a lens unit (46) that causes the first beam to be incident on the retina (60) or collects an unpolarized beam reflected from the retina (60) to generate a second beam (B2); It includes a second polarizer (36b) for minimizing the cornea reflection component from the second beam (B2) passing through the beam splitter (45); and a retinal image sensor (42) for receiving the second beam (B2) passing through the second polarizer (36b) to form a retinal image.

Drawing symbol 23 is a light absorber for absorbing the first beam (B1) that has passed through the beam splitter (45).

Preferably, the first polarizer (36a) and the second polarizer (36b) are cross-polarized to have a phase difference of 90° to reduce interference and corneal reflection components between the first beam (B1) and the second beam (B2).

The above retinal image measuring unit (79) is preferably installed on the left and right eye areas of the AI healthcare robot device (100) so that the retinas of the left and right eyes of the subject can be observed.

In addition, the retinal image acquired from the retinal image sensor (42) is analyzed by an artificial intelligence neural network for retinal analysis (16) or an ophthalmology expert system (18), and the presence or absence of a disease related to the patient's retina and eye and the risk of dementia can be determined and the patient can be informed of this by voice or text message.

In the present invention, an artificial intelligence neural network (16) or an ophthalmology expert system (18) can be built on a server (13), a digital biosignal analysis module (400), or a mobile phone app.

It is preferred that the artificial intelligence neural network (16) for retinal analysis is preliminarily trained through deep learning using retinal images (learning retinal images) labeled by type of retinal disease, and then automatically determines whether or not the patient has a disease related to the retina and the level of risk by analyzing the patient's retinal image acquired from the retinal image sensor (42).

The retinal images labeled by the above retinal disease type are preferably retinal images including retinal diseases such as retinal microvascular damage, cataracts, glaucoma, retinal detachment, hypertensive retinopathy, retinal vascular occlusion, diabetic retinopathy, macular degeneration, etc., and the retinal analysis artificial intelligence neural network (16) is characterized in that it is learned in advance by retinal images (learning retinal images) labeled by grade according to the severity of the retinal disease.

In addition, by training an artificial intelligence neural network (16) for retinal analysis using retinal images labeled with a dementia risk level, the artificial intelligence neural network (16) for retinal analysis can automatically determine the risk of dementia from a user's new retinal image.

In addition, by training an artificial intelligence neural network (16) for retinal analysis using retinal images labeled with myocardial infarction and stroke, the artificial intelligence neural network (16) for retinal analysis can automatically determine the risk of myocardial infarction or stroke from a user's new retinal image.

In addition, by training an artificial intelligence neural network (16) for retinal analysis using retinal images labeled with cholesterol grade, the artificial intelligence neural network (16) for retinal analysis can automatically determine the risk of cholesterol from a user's new retinal image.

In addition, by training an artificial intelligence neural network (16) for retinal analysis using retinal images labeled with the presence or absence of virus infection, the artificial intelligence neural network (16) for retinal analysis can automatically determine whether a user has a virus infection from a new retinal image.

Drawing symbol 30 is a lens barrel that mechanically supports the lens unit (46).

The focal distance adjustment driving unit (79b) of the retinal image measuring unit (79) adjusts the focal distance between the retinal image sensor (42) and the subject's retina (60) by moving the lens barrel (30) forward and backward along the optical path of the lens unit (46) according to the rotation of the step motor (32).

To this end, the focal length adjustment driving unit (79b) includes a step motor (32), a lens barrel (30) that mechanically supports the lens unit (46), a worm (30a) connected to and installed at the tip shaft of the step motor (32), and a worm gear (30b) that is designed on the outer surface of the lens barrel (30) and mechanically meshed with the worm (30a).

Accordingly, when the step motor (32) rotates, the worm (30a) rotates simultaneously with the rotation of the step motor (32), and the worm gear (30b) meshed with and connected to the worm (30a) moves forward and backward along the optical path of the lens portion (46), thereby adjusting the focal distance between the retina image sensor (42) and the retina (60) of the subject.

Preferably, the eyepiece (31) of the lens barrel (30) includes a contact structure including at least one pressure sensor (40) and at least one temperature sensor (25) formed to correspond to the shape of the area around the eye of the subject, so that the eyepiece (31) of the lens barrel (30) and the area around the eye of the subject (86) can be brought into close contact with each other.

While the lens barrel (30) moves forward and backward in accordance with the rotation of the step motor (32), it is preferable to measure how well the lens barrel (30) is fitted around the eye of the subject by the pressure sensor (40), and if it is fitted properly, adjust the optical focal distance between the retina image sensor (42) and the retina (60) of the subject.

Additionally, the pressure sensor (40) and temperature sensor (25) can be used together to determine how well the subject's eye area is in contact with the eyepiece (31).

In addition, when it is confirmed that the eyepiece (31) of the lens barrel (30) is in close contact with the subject's eye area by the pressure sensor (40), the body temperature can be measured by measuring the temperature around the subject's eye by the temperature sensor (25).

Another aspect of the pressure sensor (40) is that the pressure sensor (40) can be replaced by a temperature sensor (25). That is, when body temperature is measured as a valid body temperature signal through temperature measurement around the eye of the subject by the temperature sensor (25), the body temperature of the subject can be measured by determining that the eyepiece (31) of the lens barrel (30) is in close contact with the eye area of the subject.

The lens portion (46) of the present invention can be replaced with a liquid lens.

The above liquid lens can change the focal length by changing the thickness of the liquid by utilizing the electrowetting phenomenon in which the surface tension of the liquid changes according to electrical control.

In addition, the focal length adjustment driving unit (79b) of the present invention may further include a fine automatic focal length adjustment means that can automatically and finely adjust the focal length after the focal length between the retinal image sensor (42) and the retina (60) of the subject is adjusted according to the rotation of the step motor (32).

The above-described fine automatic focal length adjustment means can automatically adjust the focal length by having the control unit (53) extract focus adjustment image information from the retinal image obtained by the retinal image sensor (42) and precisely control the step motor (32) of the retinal image measuring unit (79).

Another aspect of the above fine auto focus distance adjustment means may be a Wiener filter or an artificial intelligence for image correction.

It is preferred that the above focus adjustment image information be obtained using a phase difference detection method or a contrast detection method.

FIG. 5 shows an example of a body fat measurement unit (77) that measures body fat by hand electrodes (24a, 24b, 24c, 24d) and a glucose measurement unit (78) that measures glucose spectroscopy, while fingerprint authentication is performed by a fingerprint sensor (26) on a hand print installed on an AI-healthcare robot device (100).

In the present invention, it is preferred to measure body fat using the BIA (Bioelectrical impedance analysis) method known in the art.

For example, the BIA method views the human body as a type of resistor because the electrical resistance varies depending on the amount of body water, and passes a weak alternating current (about 400 microamperes) to the measurement area of the body, detects the potential difference at this time, and calculates the resistance value. Then, the detected resistance value and the subject's personal body information (height, weight, age, and gender) and other variables are used in a predetermined algorithm to calculate biometric information such as body fat mass, muscle mass, and body water content.

According to one embodiment of the present invention, the body fat measurement unit (77) operates by being connected to a driving electrode that directly contacts the left finger of the subject and applies current to the body of the subject, and a detection electrode that directly contacts the right finger of the subject and measures the electrical resistance value of the body of the subject to obtain a body fat measurement signal.

In this embodiment, hand electrodes (24a, 24b, 24c, 24d) are used as driving electrodes and detection electrodes for body fat measurement. For example, when hand electrodes (24a, 24b) are used as driving electrodes, the remaining hand electrodes (24c, 24d) are used as detection electrodes. Hereinafter, a body fat measurement method using driving electrodes (24a, 24b) and detection electrodes (24c, 24d) will be described.

During body fat measurement, the patient can measure body fat by touching the fingers of his/her left hand (72L) to the driving electrodes (24a, 24b) of the left hand print (70L), and at the same time, touch the fingers of his/her right hand (72R) to the detection electrodes (24c, 24d) of the right hand print (70R), thereby performing fingerprint authentication from the patient's fingers touching the fingerprint sensor (26).

The current that is passed through the fingers of the patient's left hand (72L) that come into contact with the driving electrodes (24a, 24b) passes through the patient's body and is passed to the detection electrodes (24c, 24d), and the impedance value of the current that is passed into the patient's body is obtained from this passed current.

For example, the body fat measurement unit (77) preferably obtains body fat information including body fat percentage, body mass index (BMI), and muscle mass by sending a small current (e.g., about 500 μA) through the drive electrodes (24a, 24b) into the body while the fingers of the left hand (72L) and the fingers of the right hand (72R) of the subject are in contact with the drive electrodes (24a, 24b) and the detection electrodes (24c, 24d) of the hand print, and measuring the electrical resistance of the body of the subject through the detection electrodes (24c, 24d), and transmitting the same to the biosignal collection unit (420).

A body fat measurement unit (77) according to an example of the present invention is composed of a driving electrode (24a, 24b) that applies a driving signal for measuring a bio-impedance of a subject, a sine wave generator (49) that generates a sine wave driving signal (e.g., 10 to 50 KHz) to be applied to the driving electrodes (24a, 24b), a detection electrode (24c, 24d) that detects a signal received through a human body, a spectrum calculation unit (41) that converts a signal received through the detection electrodes (24c, 24d) into a frequency component, and an AD converter (46) that converts the result of the spectrum calculation unit (41) into a digital signal.

The above spectrum calculation unit (41) may include a band pass filter for extracting only the frequency components of the driving signal from among the signals received through the FFT (Fast Fourier Transform) and detection electrodes (24c, 24d).

The FFT (Fast Fourier Transform) is STFT (Short Time Fourier Transform)

It can be replaced by wavelet transform.

In addition, the biosignal validity determination means (420a) of the biosignal collection unit (420) determines whether the body fat information converted by the AD converter (46) has completed user authentication and is a valid biosignal, and the control unit (53) controls to store only the user-authenticated valid biosignal information in the medical data storage unit (15).

For example, in the case of the present embodiment, the thumb and index finger of the left hand (72L) of the subject are respectively contacted with the hand electrode 1 (24a) and hand electrode 2 (24b) of the left hand print (70L), and the thumb and index finger of the right hand (72R) of the subject are respectively contacted with the hand electrode 3 (24c) and hand electrode 4 (24d) of the right hand print (70R), so that fingerprint authentication and body fat measurement are performed.

In addition, by configuring the hand electrodes (24a, 24b, 24c, 24d) as blood sugar sensors, glucose spectroscopy can be measured simultaneously. For example, as in body fat measurement, the thumb and index finger of the left hand (72L) of the subject are respectively contacted with the hand electrode 1 (24a) and hand electrode 2 (24b) of the left hand print (70L), and the thumb and index finger of the right hand (72R) of the subject are respectively contacted with the hand electrode 3 (24c) and hand electrode 4 (24d) of the right hand print (70R), and by obtaining the impedance frequency response of the body of the subject placed between the drive electrodes (24a, 24b) and the detection electrodes (24c, 24d) using the glucose measurement unit (78), glucose spectroscopy can be measured.

In order to obtain glucose spectroscopy, the glucose measuring unit (78) preferably uses a frequency band of 50 Hz to 350 KHz for the oscillation frequency of the sine wave oscillator (49), and by performing a frequency scan using this frequency band, the impedance of the subject's body placed between the driving electrodes (24a, 24b) and the detection electrodes (24c, 24d) is sequentially measured by frequency to obtain an impedance frequency response, thereby enabling glucose spectroscopy to be measured.

Simultaneous measurement of body fat and glucose spectroscopy is preferably performed by alternately changing the driving signal of the driving electrode.

FIG. 6 shows an example of an electrocardiogram measuring unit (71) that measures an ECG signal (Electrocardiogram) or electrocardiogram when the bottom surface of a subject's hand comes into contact with an ECG sensor (48) and hand electrode 1 (24a) installed on a hand print (70L, 70R) of an AI-healthcare robot device (100) according to the present invention.

The heart is a muscular organ that acts as a pump to circulate blood, and its muscular movements are controlled by minute electrical currents. As the heart contracts and relaxes, an ECG signal is generated that shows minute electrical fluctuations from the myocardial cells that make up the heart, and this signal can be read by an ECG sensor that detects it on the surface of the skin.

The ECG sensor (48) of the present invention is composed of an insulating film (48a) that allows capacitive coupling with a human skin surface (47) and an ECG electrode surface (48b), and senses an ECG signal showing a minute electrical fluctuation from a myocardial cell by capacitive coupling with the human skin surface (47). The human skin surface (47) is preferably the palm area of a patient. The ECG electrode surface (48b) is preferably a conductive copper thin film, and is input to a differential amplifier (61) through an ECG output electrode (48c).

In addition, the ECG electrode surface (48b) is coated with an insulating film (48a) and can sense an ECG signal from the human skin surface (47) through capacitive coupling with the human skin surface (47).

The electrocardiogram measuring unit (71) removes noise from an ECG signal obtained from an ECG sensor (48), amplifies the signal, and transmits it to a biosignal collection unit (420). Specifically, it is composed of a differential amplifier (61) equipped with a common mode rejection ratio (CMRR) circuit for removing ripple noise mixed in the power supply, a high pass filter (HPF, High Pass Filter, 62) for allowing the ECG signal passing through the differential amplifier (61) to pass a signal of 0.1 Hz or higher, a band rejection filter (BRF, Band Rejection Filter, 63) for removing a power component of 50 Hz or 60 Hz, and an AD converter (64) for converting the output of the ECG signal passing through the high pass filter (62) and the band rejection filter (63) into a digital signal.

In addition, the biosignal collection unit (420) determines whether the ECG signal digitized by the AD converter (64) has completed user authentication and is a valid ECG signal, and the control unit (53) stores only the valid ECG signals for which user authentication has been completed in the medical data storage unit (15).

FIG. 7a and FIG. 7b show an example of an oxygen saturation and PPG signal measuring unit (72) using a reflective SpO2 sensor (35) on a hand print of an AI-healthcare robot device (100) according to the present invention.

One of the important roles of blood is to supply oxygen to every part of the body.

To this end, hemoglobin in red blood cells combines with oxygen to supply oxygen to each part of the body, and the amount of oxygen supplied to the blood is measured as oxygen saturation (Saturation of partial pressure oxygen, SpO2).

That is, oxygen saturation is defined as the ratio of the concentration of hemoglobin containing oxygen to the concentration of total hemoglobin. It measures the light absorbance when hemoglobin in the blood is bound to oxygen and the light absorbance when it is not bound, and uses the ratio to measure blood oxygen saturation without directly collecting blood.

Another aspect of measuring blood oxygen saturation is a method of acquiring a signal by utilizing the change in blood volume according to the contraction and relaxation of the heart and the change in the relative ratio between oxyhemoglobin and reduced hemoglobin. Based on the property that hemoglobin (HbO2) combined with oxygen absorbs infrared light (e.g., 940 nm) well and reduced hemoglobin (Hb) not combined with oxygen absorbs red light (e.g., 660 nm) better, the amount of light detected according to the expansion and contraction of the pulsatile arterial blood is quantified using an SpO2 sensor (35) composed of a light-emitting element (red light LED (51a), infrared LED (51b)) and a light detector (50), and the oxygen saturation of arterial blood is non-invasively measured by quantifying the amount of light.

For example, if there is sufficient oxygen in the blood, the amount of hemoglobin combined with oxygen increases relatively, and most of the infrared light emitted by the infrared LED (51b), which is an infrared light-emitting element, is absorbed as it passes through the blood vessels, so that the infrared light is not detected by the photodetector (50).

In healthy people, blood oxygen saturation is usually observed to be between 98 and 99%, which is close to 100%.

In general, when the oxygen saturation level falls below 95%, the supply of oxygen to the brain and organs is insufficient, so attention and concentration decrease and you feel tired easily.

In the present invention, it is preferred that the PPG signal be obtained using an SpO2 sensor (35).

For example, the PPG signal is a pulse signal measured in peripheral blood vessels when blood ejected during ventricular systole is delivered to the peripheral blood vessels, and can be measured by collecting a signal reflected from a finger by a photodetector (50) during irradiation with a red light LED (51a) of an SpO2 sensor (35).

In addition, the PPG signal is a biosignal that converts the change in blood flow, which is the volume change of peripheral blood vessels, into a change in light quantity at locations where peripheral blood vessels are distributed, such as the fingertip arteries, according to the human heartbeat. Not only can this be used to measure the pulse of the subject, but by comparing the correlation with the electrocardiogram (ECG) signal, the pulse transit time (PTT), pulse arrival time (PAT), and pulse wave velocity (PWV) can be measured, which can be used to diagnose cardiovascular diseases such as the condition of blood vessels, arteriosclerosis, and peripheral circulatory disorders.

The SpO2 sensor (35) of the present invention includes an infrared LED (51b) that irradiates infrared light (hereinafter referred to as “IR light”) including light in an infrared wavelength range (e.g., 900 nm to 950 nm) to a finger portion of a subject; a red light LED (51a) that irradiates red light (hereinafter referred to as “RED light”) including light in a red wavelength range (e.g., 640 nm to 670 nm) to the finger portion; and a light detector (50) for receiving the IR light and RED light reflected from the finger portion, but it is preferred to calculate the oxygen saturation based on the ratio of components between the received IR light and RED light.

According to one embodiment of the present invention, the SpO2 sensor (35) is installed at the fingertip of the left hand print (70L), and the on-off of the LED is controlled by the light-emitting diode driver (80) according to the command of the control unit (53), and the red light LED (51a) and the infrared LED (51b) are sequentially irradiated onto the peripheral blood vessel area of the patient's finger (800) with one cycle at a time difference, and then the amount of light reflected from the blood vessel area where the irradiated light flows is detected by the light detector (50), and the amount of oxygenated hemoglobin and reduced hemoglobin is calculated from the electrical signal changed into current according to the intensity of the light, and the oxygen saturation is calculated therefrom.

The differential amplifier (84) amplifies the electrical signal output from the photodetector (50) after converting it to current-voltage, removes noise by a filter (81), and then converts the analog signal to digital by an AD converter (82).

The light emitting diode driver (80) is a circuit that controls the driving current of the red LED (51a) and the infrared LED (51b) so that an electrical signal of sufficient intensity is output from the light detector (50) after receiving feedback on the output value of the light detector (50). This is to control the size of the amount of light applied to the finger skin because the absorption rate of light is different for each person's finger skin.

It is preferred that the above control unit (53) adjusts the driving intensity of infrared light and red light using the DC level read through the AD converter (82) to drive the light-emitting diode driver (80) so that the red light LED (51a) and the infrared LED (51b) have the same DC level.

In addition, in the oxygen saturation calculation means (not shown), it is preferred to calculate the oxygen saturation value by inverting the polarity of the digitized IR light component (infrared component) and RED light component (red light component), removing the direct current component and using only the alternating current component, and then storing the calculation result in the medical data storage unit (15).

In addition, the PPG signal can be obtained using only the RED light component among the above AC components, and it is preferred to store the calculation result in the medical data storage unit (15).

That is, while oxygen saturation (SpO2) is measured by sequentially irradiating the peripheral arterial blood vessel (55) area of a finger (800) with red light LED (51a) and infrared LED (51b) alternately and sequentially in one cycle, and then receiving the reflected IR light component and RED light component with a light detector (50) and calculating the ratio of the components between them, the PPG signal can be measured using only the RED light component.

The biosignal collection unit (420) determines whether the IR light component or RED light component digitized by the AD converter (82) has completed user authentication and is a valid biosignal, and the control unit (53) stores only the user-authenticated valid biosignal information (oxygen saturation or PPG signal) in the medical data storage unit (15).

Here, the alternating current (AC) component represents the amount of light absorbed by blood passing through peripheral blood vessels, while the direct current (DC) component represents the amount of light absorbed by tissues such as capillaries, muscles, skin, and bones. That is, blood changes due to dilatation and contraction of pulsatile arterial blood vessels are represented by the AC component. In contrast, the remaining elements (i.e., skin and tissues) except for blood are represented by the DC component, so only the AC component is extracted and used to obtain the final oxygen saturation and PPG signal.

The above oxygen saturation calculation means (not shown) preferably calculates the oxyhemoglobin (HbO2) concentration and the deoxyhemoglobin (Hb) concentration by the intensity of the signal received by the photodetector (50), and at this time, the oxygen saturation (SpO2) can be calculated by a formula such as HbO2 concentration/(HbO2 concentration + Hb concentration).

Here, the oxyhemoglobin (HbO2) concentration is the intensity (red light component) of the signal received by the photodetector (50) when the red light LED (51a) is turned on, whereas the deoxyhemoglobin (Hb) concentration is preferably the intensity (IR light component) of the signal received by the photodetector (50) when the infrared LED is turned on. At this time, it is preferred to calculate the oxyhemoglobin (HbO2) concentration and the deoxyhemoglobin (Hb) concentration by excluding the DC component from the intensity of the signal received by the photodetector (50) and using only the AC component.

Another aspect of the above oxygen saturation calculation means (not shown) is that the Red2IR ratio is first calculated as in the mathematical expression 2 below, and then the Red2IR ratio value is converted into the corresponding oxygen saturation value by a loop up table based on the Beer-Lambert law to obtain the oxygen saturation.

[Mathematical formula 2]

Here, RED _DC is the average value of the DC components (DC) of red light detected by the photodetector (50) when the red light LED (51a) is On, and IR _DC is the average value of the DC components (DC) of infrared light detected by the photodetector (50) when the infrared LED (51b) is On.

Additionally, RED _AC is the average value of the AC components (AC) of red light detected by the photodetector (50) when the red light LED (51a) is On, and IR _AC is the average value of the AC components (DC) of infrared light detected by the photodetector (50) when the infrared LED (51b) is On.

In the present invention, it is preferred that the photodetector use an image sensor, a photo diode or a photo transistor.

Also, Fig. 7b shows an example in which a patient's fingerprint is authenticated by a fingerprint sensor (26) simultaneously while measuring oxygen saturation and photoelectric volume pulse signals by an SpO2 sensor (35) of a left hand print (70L).

FIGS. 8a and 8b illustrate an embodiment of predicting blood pressure, cholesterol, blood sugar, and heart disease using bio-signals obtained from a hand bio-sensor installed on a hand print (70R, 70L) of an AI-healthcare robot device (100), in which a blood pressure measurement unit (20a) for predicting blood pressure, a cholesterol measurement unit (20b) for predicting cholesterol, a blood sugar measurement unit (20c) for predicting blood sugar, and a heart disease measurement unit (20d) for predicting heart disease are implemented by an artificial intelligence neural network (16).

Simultaneously with personal fingerprint authentication, synchronized bio-signals based on ECG or PPG signals are bio-signals measured from the user's body at the same time, so more accurate blood pressure measurement, cholesterol measurement, blood sugar measurement, and heart disease measurement are possible than using non-synchronized bio-signals (bio-signals acquired at different times) as input signals for the artificial intelligence neural network (16). Since the patient's bio-signals change frequently depending on the patient's body condition and time, bio-signals collected at the same time or at synchronized times must be used as input signals for the artificial intelligence neural network (16) in order to obtain accurate diagnosis results.

The biosignals input to the artificial intelligence neural network (16) of the present invention may include at least one selected from among an ECG signal (69a), a PPG signal (68), oxygen saturation (69c), PTT (69d), PAT, PWV, HRV, glucose spectroscopy (69e), body fat (69f), retinal image, and medical data received from an external medical device.

For example, while the left hand print (70L) surface and the bottom surface of the left hand (72L) of the subject are placed in contact with each other, and the right hand print (70R) surface and the bottom surface of the right hand (72R) of the subject are placed in contact with each other so that the subject's fingerprint is recognized, and ECG signals, oxygen saturation, PPG signals, body fat, and glucose spectroscopy can be measured at the same time.

That is, on the left hand print (70L), the fingerprint of the subject can be recognized by the fingerprint sensor (26), and the oxygen saturation and PPG signal (68) by the SpO2 sensor (35) can be measured at the same time.

Additionally, the ECG signal (69a) can be measured simultaneously using the ECG sensor (48) and hand electrode 1 (24a) on the right hand print (70R).

Additionally, body fat and glucose spectroscopy can be simultaneously measured using hand electrodes (24a, 24b, 24c, 24d) on the hand prints (70R, 70L) of both hands.

PTT(69d) can be defined as the time difference between an ECG peak (e.g., R peak in the PQRST wave) and the peak point of a PPG signal, or the time difference between an ECG peak and a peak of a Second Derivative Photoplethysmographic (SDPPG).

In addition, by comparing the correlation between the ECG signal (69a) and the PPG signal (68), in addition to the pulse transit time (PTT), the pulse arrival time (PAT) and the pulse wave velocity (PWV) can be measured.

Since the ventricle begins to contract and push blood into the artery at the R peak point on the ECG signal (69a), the R peak can be used as the starting point for PTT measurement, and the peak of the PPG signal or the peak of the SDPPG can be used as the end point for PTT measurement.

The heart pulse measurement of the present invention can be calculated as the number of beats per minute (beats/min) using the peak (R point) of the ECG signal or the PPG signal. In addition, HRV (Heart Rate Variability) refers to the rate of change in the heart pulse, and can be calculated by measuring the amount of change in the heart pulse rate.

The artificial intelligence neural network (16) of the present invention may be an artificial intelligence neural network for measuring blood pressure, an artificial intelligence neural network for measuring cholesterol, an artificial intelligence neural network for measuring blood sugar, an artificial intelligence neural network for measuring heart disease, or an artificial intelligence neural network for analyzing retina.

It is preferred that the blood pressure measuring unit (20a) of the present invention predicts the blood pressure of the subject by using an artificial intelligence neural network for blood pressure measurement learned using labeled bio-signals and personal body information (69g) according to various blood pressure values.

Blood pressure is the pressure exerted by the blood sent from the heart on the walls of the arteries. When the ventricles of the heart contract, the amount of blood flowing increases, so the pressure increases. Conversely, when the ventricles expand, the amount of blood flowing decreases, so the pressure decreases.

In the present invention, blood pressure measurement is expressed as the size of systolic pressure and diastolic pressure.

Systolic pressure refers to the maximum blood pressure when the ventricles contract, and diastolic pressure refers to the minimum blood pressure when the ventricles expand.

In addition, it is preferred that the cholesterol measurement unit (20b) predicts the cholesterol of the subject by using an artificial intelligence neural network for cholesterol measurement learned using labeled bio-signals and personal body information (69g) according to various cholesterol levels.

In this case, the cholesterol measurement unit (20b) can measure cholesterol more accurately using combined information of biosignals such as ECG signal (69a) and PPG (68).

For example, when cholesterol accumulates in normal blood vessels, the elasticity of the blood vessels decreases and the elastic coefficient (the degree of resistance to pressure) decreases. The cholesterol level can be estimated using the ECG signal (69a) and the PPG signal (68) that show the driving power of the heart. When cholesterol accumulates, the elasticity of the blood vessels decreases and the volume change of the peripheral blood vessels decreases compared to the reference, so the size change of the PPG signal (68) decreases or the PTT (69d) becomes longer.

Another aspect of the cholesterol measurement unit (20b) is that it is preferable to predict a cholesterol level selected from among the triglyceride (TG), high-density lipoprotein-cholesterol (HDL-C), and low-density lipoprotein-L cholesterol (LDL-C) values of the subject by using a learned artificial intelligence neural network for cholesterol measurement using the bio-signals and personal body information (69g) labeled according to one or more values selected from among the triglyceride (TG), high-density lipoprotein-cholesterol (HDL-C), and low-density lipoprotein-L cholesterol (LDL-C).

In addition, it is preferred that the blood sugar measurement unit (20c) predicts the blood sugar level of the subject by using an artificial intelligence neural network for blood sugar measurement learned using labeled bio-signals and personal body information (69g) according to various blood sugar levels. It is preferred that the blood sugar level use postprandial blood sugar or HbA1c (glycated hemoglobin) levels.

In addition, it is preferred that the heart disease measurement unit (20d) predicts the heart disease of the subject by using an artificial intelligence neural network for heart disease measurement learned using labeled bio-signals and personal body information (69g) according to the type and grade of heart disease.

In this case, the heart disease measurement unit (20d) can diagnose the prognosis of heart disease more accurately using combined information of biosignals such as the ECG signal (69a) and the PPG signal (68).

That is, by comparing the correlation between the ECG signal (69a) and the PPG signal (68), the pulse transit time (PTT), the pulse arrival time (PAT), and the pulse wave velocity (PWV) are calculated, and at this time, the heart disease measurement unit (20d) can be used to diagnose cardiovascular diseases such as the condition of blood vessels, arteriosclerosis, and peripheral circulatory disorders by using the calculated information.

Another aspect of the above heart disease measurement unit (20d) is heart failure, The artificial intelligence neural network for measuring heart disease learned using bio-signals and personal body information (69g) labeled according to one or more values selected from arteriosclerosis, myocardial infarction, and angina pectoris, is used to measure the subject's heart failure, Preference is given to predicting selected cardiac diseases among arteriosclerosis, myocardial infarction, and angina pectoris.

It is preferred that the above artificial intelligence neural network (16) uses synchronized bio-signals and personal body information (69g) based on two R points of the ECG signal (69a) as inputs, that is, it is preferred that the bio-signals generated at each R-to-R section be used as input signals of the artificial intelligence neural network (16).

Since the above synchronized bio-signals are bio-signals measured from the user's body at the same point in time, an artificial intelligence neural network (16) that uses these synchronized bio-signals as input can measure blood pressure, cholesterol, blood sugar, and heart disease more accurately.

Another aspect of the above artificial intelligence neural network (16) is that it is preferred to use synchronized bio-signals and personal body information (69g) based on the peak point of the above PPG signal (68).

Figures 8a and 8b are examples of artificial intelligence neural networks (16) that measure blood pressure, cholesterol, blood sugar, and heart disease by a classifier (21a) using an ECG signal (69a), a PPG signal (68), oxygen saturation (69c), PTT (69d), PAT, PWV, HRV, glucose spectroscopy (69e), body fat (69f), and personal body information (69g).

In addition, in the case of the examples of Figs. 8a and 8b, a time sequence neural network can be connected to the output of the classifier (21a) and used. In this case, blood pressure, cholesterol, blood sugar, and heart disease can be estimated by analyzing the time series information of the biosignal.

In Fig. 8a, feature vector extraction unit 1 (33a) extracts a feature vector from an ECG signal (69a), and feature vector extraction unit 2 (33b) extracts a feature vector from a PPG signal (68).

The feature vectors of the above ECG signal (69a) and PPG signal (68) can be extracted by a deep learning neural network or spectrum analysis in the frequency domain.

For example, the above deep learning neural network can be a 1-D CNN (Convolutional Neural Network).

In addition, feature vector extraction by spectral analysis of the frequency domain can be performed by Linear Prediction Codes (LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient (MFCC), Cepstral Coefficient, Perceptual Linear Prediction (PLP) or Bark Frequency Cepstral Coefficients (BFCC).

For example, bio-signals and personal body information (69g) can be provided to a server (13), and then diagnostic results for blood pressure, cholesterol, blood sugar, and heart disease can be fed back from an artificial intelligence neural network (16) on the server (13).

Fig. 8b is another embodiment of an artificial intelligence neural network (16) for estimating blood pressure, cholesterol, blood sugar, and heart disease. A spectrogram produced by Linear Prediction Codes (LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient (MFCC), Cepstral Coefficient, Perceptual Linear Prediction (PLP), Short Time Fourier Transform (STFT), or Bark Frequency Cepstral Coefficients (BFCC) can be used as an input to CNN (57).

Feature vector extraction unit 4 (33d) outputs a spectrogram of an ECG signal from an ECG signal (69a), and feature vector extraction unit 5 (33e) outputs a spectrogram of a PPG signal from a PPG signal (68).

These spectrograms are input to a CNN (Convolutional Neural Network, 57) to output a feature vector. In addition, the CNN (57) can be used by applying a separate, independent CNN to each of the ECG signal (69a) and the PPG signal (68).

These feature vectors are input into a classifier (21a) together with oxygen saturation (69c), PTT (69d), PAT, PWV, HRV, glucose spectroscopy (69e), body fat (69f), and personal body information (69g) to estimate blood pressure, cholesterol, blood sugar, and heart disease.

The classifier (21a) of the present invention may be a fully connected network (FCN) or a support vector machine (SVM).

An AI-healthcare robot device according to one embodiment of the present invention may include a medical data storage unit (15) for receiving and storing medical data measured from an external medical device (430) by a short-range communication connection unit (8), and an artificial neural network (16) or an expert system (18) that has been previously trained by deep learning using medical data for learning, and the artificial neural network (16) or the expert system (18) that has been trained by deep learning may analyze the medical data received by the short-range communication connection unit (8) to automatically determine whether a patient has a disease and the risk of the disease. In addition, when an abnormal symptom is found, a remote medical diagnosis service may be provided through an Internet communication connection with a doctor terminal.

Another aspect of the artificial intelligence neural network of the present invention is that the AI-healthcare robot device can transmit measured or acquired medical data to a mobile phone, and then provide the subject with the analysis results of the medical data obtained by the artificial intelligence neural network app installed as an application software on the mobile phone.

Below, based on the detailed description above, we will briefly review the operation flow of the self-examination processing method using the AI-healthcare robot device of this hospital.

In step S100, the user's hand is placed on the hand print so that the user's biosignal can be collected by physical contact between the hand biosensor and the bottom surface of the user's hand.

In step S101, a user's biosignal can be collected by physical contact with the user using a retinal image sensor.

In step S102, a valid biosignal for which user authentication has been completed can be determined among the collected biosignals using a biosignal validity determination means.

In step S103, medical data or biosignals measured by external medical devices can be analyzed by an artificial intelligence neural network and an expert system.

In step S104, if a sign suspected of a disease is found in the analysis result in step S103, a remote medical diagnosis service can be provided through an Internet communication connection with a doctor terminal.

In the above description, steps S100 to S104 may be further divided into additional steps or combined into fewer steps, depending on the implementation example of the present invention. In addition, some steps may be omitted as needed, and the order between the steps may be changed.

A self-examination processing method using an AI-healthcare robot device according to one embodiment of the present invention may be implemented in the form of a program command that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention or may be those known to and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The above hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, the self-examination processing method using the aforementioned AI-healthcare robot device can also be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The above description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner.

The scope of the present application is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present application.

8: Short-range communication connection
39: Health Tracking Management Department
13: Server
15: Medical Data Storage
16: Artificial Intelligence Neural Network
18: Expert System
19: Body temperature diagnostic section
20a: Blood pressure measurement section
20b: Cholesterol measurement section
20c: Blood sugar measurement section
20d: Heart disease measurement department
20e: Retinal Disease Measurement Section
22: Vision Camera
26: Fingerprint sensor
24a: Hand electrode 1 24b: Hand electrode 2
24c: Hand electrode 3 24d: Hand electrode 4
27a: Voice recognition unit
28a: Voice reproduction section
37: Wireless communication connection means
38a: Speaker section 38b: Microphone section
40: Pressure sensor
46: AD converter
47: Human skin surface
48: ECG sensor
48a: Insulating film 48b: ECG electrode surface
48c: ECG output electrode
49: Sine wave generator
50: Photodetector
51a: Red LED 51b: Infrared LED
53: Control Unit
61: Differential Amplifier
62: HPF
63: BRF
69e: Glucose Spectroscopy 69f: Body Fat 69g: Personal Body Information
70: Body fat measurement section
71: Electrocardiogram measurement unit
72: Oxygen saturation and PPG signal measurement unit
74: Body temperature measurement unit
79: Retinal image measurement unit
79a: Snow Navigator
79b: Focal length adjustment drive
81: Filter
84: Differential Amplifier
202: Internet
420: Biosignal collection unit
420a: Means for determining the validity of biosignals

Claims (23)

A hand print providing a contact surface with the subject's hand;
A hand bio sensor that simultaneously measures the subject's vital signals and authenticates the subject while the subject's hand is in contact with the hand print; and
An AI healthcare robot device including an artificial intelligence neural network for analyzing the above biosignals. A hand print that is not only installed on the robot but also provides a contact surface with the subject's hand;
A hand biosensor that simultaneously measures the subject's vital signals and authenticates the subject while the subject's hand is in contact with the hand print;
A vision camera installed on the above robot to obtain a body image of the subject;
An eye navigator that recognizes the face and eyes of the subject by the vision camera and controls the position of the robot so that the robot face faces the front of the subject's face;
An artificial intelligence neural network that analyzes biosignals obtained from the hand biosensor; and
An AI healthcare robot device including a control unit that operates and controls a speaker unit and a microphone unit to provide the results of analyzing the above biosignals to a subject as a voice message or to interactively request biosignal measurement. An AI healthcare robot device according to claim 1 or 2, wherein the hand print further includes a contact sensing means for detecting or visually indicating whether there is close contact between the hand print and the palm surface of the subject's hand. An AI healthcare robot device in accordance with claim 3, wherein the contact sensing means is an LED or a touch screen arranged around the edge of the hand print. An AI healthcare robot device in claim 1 or 2, wherein the hand print includes a hand-shaped engraved portion that allows contact with the hand of the subject, and the hand biosensor is embedded in the surface of the engraved portion. An AI healthcare robot device according to claim 1 or 2, wherein the hand print includes a touch screen that allows contact with a hand of a subject, and a virtual finger pattern displayed on the touch screen. An AI healthcare robot device, further comprising a means for determining validity of a biosignal for filtering out only a valid biosignal portion among biosignal components for which subject authentication has been completed among the biosignals in claim 1 or 2. An AI healthcare robot device according to claim 1 or 2, wherein the hand biosensor comprises at least one selected from the group consisting of an SpO2 sensor for measuring oxygen saturation and a photoplethysmography (PPG) signal, a plurality of hand electrodes, a body fat sensor for measuring body fat, a fingerprint sensor for authenticating the subject, an ECG sensor for measuring heart rate or electrocardiogram, and a blood sugar sensor for obtaining glucose spectroscopy. In the 8th paragraph, the plurality of hand electrodes,
An AI healthcare robot device characterized in that it obtains biosignals of body fat or glucose spectroscopy by being used as a driving electrode and a detection electrode. In the 8th paragraph, the plurality of hand electrodes,
An AI healthcare robot device that operates with one selected electrode among an electrode for measuring body fat, a ground electrode of an ECG sensor, and an ECG input electrode. An AI healthcare robot device further comprising a health tracking management unit for observing changes in the bio-signal data in claim 1 or 2 to inform the subject of health risks, inform the subject of items requiring intensive care examination, or inform the subject of the next examination schedule. An AI healthcare robot device according to claim 1 or 2, wherein the artificial intelligence neural network comprises one selected from the group consisting of an artificial intelligence neural network for measuring blood pressure, an artificial intelligence neural network for measuring cholesterol, an artificial intelligence neural network for measuring blood sugar, an artificial intelligence neural network for measuring heart disease, an artificial intelligence neural network for analyzing retinas, and combinations thereof, and wherein the artificial intelligence neural network takes the bio-signals and personal body information as inputs. In the 12th paragraph, the input of the artificial intelligence neural network is a biosignal synchronized based on the R point of the ECG signal or the peak point of the PPG signal,
An AI healthcare robot device, wherein the above bio-signals include at least one selected from body temperature, ECG signal, PPG signal, PCG signal, oxygen saturation, PTT, glucose spectroscopy, body fat, heart rate, PTT, PAT, PWV, HRV, retinal image, and medical data received from an external medical device. In the second paragraph,
A retinal image measurement unit including a retinal image sensor installed in the eye area of the robot to collect a retinal image of a subject;
A biosensing unit that amplifies or converts a biosignal obtained from the hand biosensor into a digital signal;
A contact sensing means for visually indicating or detecting whether there is close contact between the hand print and the palm surface of the subject's hand;
A biosignal collection unit for collecting the above retinal image and the above digitalized biosignal; and
An AI healthcare robot device further comprising an artificial intelligence neural network or ophthalmology expert system for analyzing the retinal image to determine the presence or absence of diseases related to the patient's retina and eye and the risk of dementia. In the 14th paragraph, a wireless communication connection means for analyzing bio-signals measured by the hand bio-sensor and retinal image sensor through an artificial intelligence neural network and providing a remote medical diagnosis service through an Internet communication connection with a doctor terminal AI healthcare robotic devices including: In the second paragraph, the robot further includes two lasers for measuring the distance to the subject,
An AI healthcare robot device in which the two laser beams are installed at a predetermined angle to each other so that they intersect each other after a predetermined distance, and the distance between the two laser beam spots focused on the subject is measured to calculate the distance between the subject and the robot. In Article 14,
An AI healthcare robot device further comprising: an eye navigator for controlling the movement of the robot to achieve optical axis alignment between the eye of the subject and the retinal image sensor; and a focal distance adjustment driving unit for adjusting the focal distance between the retinal image sensor and the retina of the subject. In the 17th paragraph, the eye navigator obtains an image capable of measuring the distance between two laser beam spots focused on the body of the subject by using the vision camera, calculates the distance between the robot and the subject, and controls the movement of the robot so that the robot approaches the front of the subject's face at an appropriate distance, thereby aligning the optical path between the subject's eye and the retinal image sensor. In claim 17, the eyepiece of the retinal image measuring unit includes a temperature sensor for measuring body heat of the eye area in contact or a pressure sensor for sensing contact pressure with the eye of the subject in contact with the eyepiece. In claim 19, the eye navigator is an AI healthcare robot device that, if it is determined by a pressure sensor or a temperature sensor that the contact pressure on the subject's eye area in contact with the eyepiece is insufficient, requests the subject to press against the eyepiece by a voice message command. In Article 19,
The above eye navigator is an AI healthcare robot device that notifies the subject of the measurement by a voice message command when the contact location and contact range of the subject's hand in contact with the surface of the hand print are determined to be insufficient by the contact sensing means. In the 17th paragraph, a health tracking management unit is further included to observe the change trend of the bio-signal data to identify patients requiring intensive care or to schedule examination schedules for each patient.
The above-mentioned eye navigator is an AI healthcare robot device that controls the movement of the robot to search and visit patients in need of a bio-signal examination by the vision camera to automatically conduct a bio-signal examination on patients in need of intensive care identified by the health tracking management department or to automatically conduct a bio-signal examination according to the examination schedule of each patient. This relates to a method for processing self-examination using an AI healthcare robot device.
A step of collecting a subject's biosignals by placing both hands on a two-hand hand print and making physical contact between the hand biosensor and the palm surface of the subject's hands;
A step of determining a valid biosignal for which subject authentication has been completed among the biosignals using a biosignal validity judgment means;
A step of analyzing the above biosignal by an artificial intelligence neural network and an expert system; and
A method for processing self-examination using an AI healthcare robot device, comprising: a step of providing a remote medical diagnosis service through an Internet communication connection with a doctor terminal when a suspected disease sign is found in the analysis results;