JPH1199140A - Device for detecting sleeping condition abnormality - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect abnormality of sleeping conditions while using an existing bed and a thick bedquilt without imparting a pressured feeling and restriction on movement, without requiring a special apparatus and an environment, and without being influenced by an illuminance change, a rotary movement and a parallel movement of a person. SOLUTION: In the device for detecting sleeping condition abnormality, an image inputted from an image input means 1 is accumulated in an image accumulating means 2, and an operation area of a sleeping person contained in the image is detected by an operation area detecting means 3, and a head part area is extracted by a head part area extracting means 4 from a detecting result from this operation area detecting means 3, and a turning-over condition is detected by a turning-over detecting means, and abnormality of sleeping conditions is detected by a abnormality detecting means 6 from a detected result from the operation area detecting means 3, and abnormality of the turning-over condition and the sleeping conditions are informed to an informing means 7.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sleeping state abnormality detecting apparatus for detecting abnormalities in a sleeping state of a hospital patient, an elderly person, an infant and the like.

[0002]

2. Description of the Related Art It is automatically detected that a hospital patient, an elderly person, an infant or the like suffers for some reason while sleeping, that the patient is lying prone for a long time, or that he or she is wandering around. As a means of notifying family members or family members by checking the condition of a person with a weight sensor or pressure sensor built into the bed, or by placing a nurse call with a touch sensor that can be easily touched with a simple shake of the neck. Some are smelling.

There are also examples of using image processing using a CCD camera or an infrared sensor. However, there are few examples using image processing that is specially devised. Often accompanied by special equipment such as.

[0004] However, when a patient or an elderly person wants to call someone, a nurse call that calls using a touch sensor or the like cannot give a notification unless the sensor is touched, so that the location can be confirmed when it is really difficult. Call may not be possible. Also, in order to make it easy to touch, it must be installed in a place where it is usually in the way, or it gives a sense of oppression.

In addition, if a weight sensor, a pressure sensor, or the like is incorporated in the bed itself, the bed must be replaced and the bed cannot be used with a normal futon.

Although there are examples using a CCD camera or an infrared sensor, special equipment is required, such as illuminating a grid-like spot and observing its displacement, and it is susceptible to illuminance changes and temperature changes in the surrounding environment. There was a problem that light from spots and the like caused discomfort to the person.

[0007]

As described above, when it is necessary to know the sleeping state of a hospital patient, an elderly person, an infant, etc., a feeling of oppression or movement restriction is given to the hospital patient, the elderly person, an infant, etc. It requires special equipment and environment such as spotting, is affected by changes in illuminance, rotation and translation of people, and can not use existing beds and futons, There has been a problem that it is not possible to detect an abnormal state such as a prone state of prolonged interrogation, a state of struggling to turn over, or a state of wandering.

Therefore, the present invention does not give a feeling of oppression or restriction of operation, does not require special equipment or an environment, is less susceptible to changes in illuminance, rotational or parallel movement of a person, and uses existing beds and the like. An object of the present invention is to provide a sleeping state abnormality detection device that can use a futon and detect an abnormality in a sleeping state.

[0009]

A sleeping state abnormality detecting apparatus according to the present invention comprises an image inputting means for inputting an image of a sleeping person, and an abnormality based on the frequency of movement of the person in the upper body image input by the image inputting means. And detecting means for detecting

A sleeping state abnormality detecting device according to the present invention comprises: image input means for inputting an image of a sleeping person; analysis means for analyzing the motion and state of the person in the upper body image input by the image input means; And a detecting means for detecting an abnormality based on the frequency of movement based on the analysis by the analyzing means.

According to the present invention, there is provided a sleeping state abnormality detecting apparatus comprising: an image input unit for inputting an image of a sleeping person; an analyzing unit for analyzing a motion and a state of the person in the upper body image input by the image input unit; The detecting means is configured to detect an abnormal state based on the frequency of movement based on the analysis by the analyzing means, and a notifying means connected to the detecting means for notifying the abnormal state detected by the detecting means.

A sleeping state abnormality detecting device according to the present invention is provided vertically downward, and has an image input means for inputting an image of a sleeping person, and an image storing means for temporarily storing the image input by the image input means. Means, an operation area detecting means for detecting a motion area, a motion amount, and a motion direction of a person from an image stored in the image storage means; and an operation area detected by the motion area detection means, or the image storage means. A head region extracting unit that extracts only the head region from the accumulated image and tracks the head, information of the head region extracted by the head region extracting unit, and head region information registered in advance. Turnover detection means for detecting that the person is lying prone by comparing the amount of movement detected by the operation area detection means, abnormal operation from the operation direction information and abnormal sleep state Detection During the sleep state abnormality detecting means, an abnormality of sleep state in this sleeping state abnormality detection means has been detected that,
Alternatively, it is constituted by a notifying means for notifying detection information when the lying-down detecting means detects the prone sleeping.

[0013]

An embodiment of the present invention will be described below with reference to the drawings. The sleeping state abnormality detection device of the present invention recognizes the state of a sleeping person included in an image input from a camera or a sensor by an image, and detects a temperature or a temperature while not giving a feeling of oppression or discomfort to the target person. In this case, a notification is automatically made in the case of an abnormal state while being not affected by environmental changes such as illuminance.

FIG. 1 is a schematic configuration of a sleeping state abnormality detecting apparatus according to the present invention. This apparatus is a monochrome or color CCD camera 11 vertically looking down from above the upper body of a person sleeping on a bed or a futon, a distance sensor, an infrared sensor, a personal computer (PC) 12 for main processing, or It comprises a workstation and a notifying device 13 for notifying a person at a remote place.

FIG. 2 shows the configuration of the processing means of the sleeping state abnormality detecting device shown in FIG. The configuration of the processing means of the sleeping state abnormality detecting device is such that the CCD camera 11 is the image input means 1, and the PC 12 is the image storage means 2, the operation area detecting means 3, the head area extracting means 4, the turning detecting means 5,
A sleep state abnormality detecting means 6;
Is the notification means 7.

An input image is obtained from a camera or a sensor looked down from above by the image input means 1, and is temporarily stored in the image storage means 2. By applying the motion area detection means 3 to the input image, the motion direction, motion area, position, and motion amount of the sleeping person are extracted as parameters.

Further, only the head region is extracted by the head region extracting means 4, and the histogram of the gradient direction of the edge and the higher-order local self-registration are used so as not to be affected by the lighting conditions, the temperature fluctuation, and the rotation of the person. The head is tracked by template matching based on the correlation parameters.

Further, the turning-over detecting means 5 detects the turning of the person, and if the target person is in the prone state for a long time, it is detected as abnormal,
The notification is output to the notification unit 7. In addition, the sleeping state abnormality detecting means 6 detects abnormal operation such as suffering or wandering while distinguishing the target person, and if the operation is continued for a certain period of time or more, it is detected as abnormal, The notification is output to the notification unit 7.

The notifying means 7 uses sound or light to notify an abnormal state in order to call a nurse or a family member. next,
The processing of the image input means 1 will be described.

An image from a camera or a sensor such as an infrared sensor or a distance sensor installed so as to look down vertically from a vertically upper portion of a bed or a futon is input. In some cases, the object person does not want to be photographed by a camera or a sensor. Therefore, it is possible to use a small camera, a magic mirror, or the like to take a picture without being noticed.

The input image size is about 320.times.240 mm, which is the gray scale or color of the CCD camera 11 of 256 gradations for each of R, G, and B, or the input gradation and the input resolution of the sensor in the case of a sensor. Since the optical CCD camera cannot be used in a dark state, it is possible to take measures such as using an infrared sensor together. The video should show the upper body of the person, including the pillow. FIG. 3 shows an example of an input image in the image input means 1.

Next, the processing of the image storage means 2 will be described. The image fetched from the image input means 1 is stored as it is in the memory as the image storage means 2, and the immediately preceding (one frame before) image is stored in another area. This is because, when performing the continuous difference image processing, it is possible to use a means for storing the data by switching the image data for each frame by double buffering.

Next, the processing of the operation area detecting means 3 will be described. First, the processing inside the operation area detecting means 3 will be described. A continuous difference image is obtained by calculating a difference image between the immediately preceding image and the current image among the images stored in the image storage unit 2. As a result, the motion area of the person in the screen is detected. By performing differentiation on the difference image, the direction of motion of the person can be determined by checking the direction of differentiation. Also, by using optical flow, the amount and direction of movement of the entire image,
It is also possible to check the position of a person.

The motion amount of the person is obtained by adding the differential gradient amount and the size of the flow vector over the entire screen. Then, as shown in FIG. 4, the left and right ends of an area showing a movement equal to or more than a certain threshold can be taken as the left and right ends of the person, and the positions of the left and right ends of the operation area are always recorded. Even if the target person is stationary, the existence position can be determined.

In order to see the tendency of the fluctuation direction over the entire screen instead of the local feature, the histogram is represented here by using the angle of the motion direction at the pixel of the entire screen as an axis. In the case of performing difference image processing using continuous images, the gradient direction of differentiation is obtained, and in the case of optical flow, the direction of the flow vector is obtained for each pixel to form a histogram. This histogram also includes information on the amount of motion by integrating all values.

FIG. 5 shows this processing. The motion direction of the person is defined as an angle as shown in FIG.
As shown in FIG. next,
The processing of the head region extraction means 4 will be described. That is, a means for performing template matching based on the feature amount of the head region information and tracking the head region will be described. The circumscribed rectangular area of the pillow in the input image is learned in advance by an operator of the apparatus artificially giving an instruction with a mouse or the like. Further, as shown in FIGS. 6A to 6D, the head region images in the four directions of upward, lateral (left and right), and downward are registered as the head region information in the initial state. deep.

First, the motion region extracted by the motion region extracting means 3 is checked, and if the motion region is on a pillow, it may be used as an initial head region as it is. If not found, an initial head position is obtained by template matching on the entire input image.

From the next processing unit time, template matching is performed on the region around the head position in the immediately preceding frame using the head feature amounts in four directions, and the head is located at a position considered to be closest to the head. Tracking is performed by updating the unit position.

When the feature amount in the template matching used here is traced by a gradation pattern such as density, a rotational movement, a change in brightness (in the case of a visible light camera), and a change in temperature (in the case of an infrared sensor). Therefore, matching is performed using the gradient direction histogram pattern of the differential image. In addition, higher-order local autocorrelation (Nobuyuki Otsu, Takio Kurita, Iwao Sekida: "Pattern recognition", Asakura Shoten, 165-1)
81, 1996) can also be used.

As the evaluation value used in matching, similarity can be used in addition to the Euclidean distance. In order to eliminate the influence of the rotational movement of the head, the histogram is also shifted in the angle axis direction, and the evaluation is performed with the best score. That is, when a directional component histogram from 0 degrees to 360 degrees is taken, a graph of a certain angle x is represented by x +
It shifts on the axis of the histogram as 1, x + 2, and for those exceeding 360 degrees, 36
Subtracting 0 also takes the value from 0 degrees.

This processing is shown in FIGS. 7A and 7B.
Since the peak C shown in FIG. 7A is indicated by the peak C shown in FIG. 7B, it can be seen that the peak C has moved to the 0 degree side.

Next, the processing of the turning detecting means 5 will be described. This means uses the image of the head region obtained by the head region extracting means 4. By tracking the head, the influence of wrinkles and patterns on clothes and futons of the person can be reduced, and the motion of the person can be reliably obtained.

Here, as shown in FIG. 8, the characteristic amount obtained from the head region image shown in FIG.
(B) to (e), which are registered in advance, are compared with the head dictionary image feature amounts pointing in four directions to determine which state is closest.

[0034] As the feature amount, the head region extracting means 4
Similarly to the above, the gradient direction histogram of the differential image is used to reduce the influence of the illuminance change. This is realized by using the Euclidean distance or similarity as the evaluation value indicating the proximity to the dictionary.

Also, here, in order to eliminate the influence on the rotation of the human image, the histogram is shifted in the direction of the angle axis similarly to the head area extracting means 4, and the best evaluation value is used. That is, a 360-degree shift is performed for the one-way dictionary, and the shift is similarly performed for the four types of dictionaries, and the face that gives the best evaluation value among the 360 × 4 = 1440 evaluation values is obtained. The direction is determined to be the direction in which the person is currently facing.

In addition to using the feature amount in the head region, a method using the feature amount in the entire image can be considered. This is a technique in which a gradient direction histogram on the entire differential screen is used as a feature amount representing a state of a person. Another method uses a 25-dimensional parameter of the higher-order local autocorrelation feature. These have a feature that almost the same feature amount is obtained even if the target object moves in parallel within the screen to obtain a histogram. In this case, in the initial state, the feature amounts of the whole image oriented in four directions are registered as a dictionary, and it is determined which dictionary is closest to the above-mentioned method.

If it is determined from the above processing that the prone face is prone for a certain time or more (for a certain processing time unit or more), it is determined that it is dangerous and a message is sent to the notifying means 7. Next, the processing of the sleeping state abnormality detecting means 6 will be described.

In the present means, the size is treated as the reference motion amount S of the person by using the motion area area extracted by the motion area detection means 3. The direction of the flow in the optical flow and the gradient direction of the edge of the difference image are represented as a histogram, and the average m and the variance V of the motion amount in the past fixed frame T are obtained.

The value obtained by adding m to the value obtained by multiplying the square root of V by the constant k is defined as the reference motion amount. If the motion amount is smaller than the reference motion amount, it is determined that the motion is a breathing motion of a person or a motion as small as a heartbeat. Can be determined. This condition is as shown by the following equation.

S <(k × ΔV) + m If an operation larger than the reference operation amount S shown here continues for a certain period of time, it is determined that the state is abnormal. Even if it continues for a certain period of time, it may continue intermittently.
It is also possible to output a warning if an abnormal operation occurs at a certain ratio or more in (t <T).

The operation state histogram obtained by the operation area detecting means 3 is used for the determination of the abnormal state. Then, as shown in FIG. 9, the variance of the histogram is obtained and compared. However, in FIGS. 9A and 9B, when moving in the direction of 1 degree and in moving in the direction of 359 degrees, the variance becomes extremely large in practice, although the directions are almost the same. There is a disadvantage that it will. Therefore, the variance is calculated by shifting the histogram by one degree in the angle axis direction.

As the variance value to be the final evaluation value, the minimum value or the average value shifted by 360 degrees is used. If this value is smaller than the predetermined threshold value, it means that the motion is continuing in a certain direction, and if the subject is about to leave the futon or returns to the bed or futon. I can judge.

If information is given in advance that a person is likely to wander, it is determined that the person is about to wander, and a warning can be made. When the variance of the motion direction histogram is larger than a predetermined threshold,
It is shown that the motion direction is not one-way but scattered everywhere, and it is determined that the target person is performing an operation such as suffering and struggling, and is determined to be an abnormal operation.

In such a configuration, the processing of the sleeping state abnormality detecting means 6 will be described with reference to the flowchart of FIG. First, the variance of the operation area S in the past unit time T is obtained (ST1). Subsequently, it is checked whether or not the expression of S <(k × ΔV) + m has been satisfied for a certain ratio or more at a certain time t in the past (ST2).

In step ST2, if the ratio is equal to or more than the predetermined ratio, it is determined that the operation is abnormal, in which the operation is larger than in the resting state, and the variance of the operation direction histogram is calculated (ST3).

Subsequently, it is checked whether or not the variance of the operation direction histogram is larger than a certain threshold (ST).
4) If it is large, it is determined that it is struggling and struggling, and if it is small, it is determined that it is leaving (or wandering) from the bed.

Here, the type of operation (wandering,
(Go away, suffering) and send the results. Next, the processing of the notification unit 7 will be described.

In order to call a nurse, a family member, or the like, an abnormal condition result is notified by connecting to a notifying device having a display device for generating a sound or outputting an image by using a wireless or network. At the same time, information about a state (prone, wandering, struggling) causing a warning given from the turnover detecting means 5 or the sleeping state abnormality detecting means 6 is also notified.

As described above, according to the embodiment of the present invention, the illuminance change does not give a feeling of oppression or movement to a patient or an elderly person, does not require a special device such as a spot, or an environment. Less susceptible to rotational and translational movements of people and people, can use existing beds and futons, and is prone to prolonged intermission during bedtime, It is possible to automatically detect a state of wandering or wandering, and notify a nurse or family by video or audio.

[0050]

As described in detail above, according to the present invention,
Does not apply pressure or movement restrictions, does not require special equipment or environment, is less susceptible to changes in illuminance, rotation and translation of people, and can use existing beds and futons to sleep It is possible to provide a sleeping state abnormality detecting device capable of detecting an abnormality of the sleeping state.

[Brief description of the drawings]

FIG. 1 is a diagram showing a schematic configuration of a sleeping state abnormality detection device according to the present invention.

FIG. 2 is a block diagram showing a configuration of a processing unit of the sleeping state abnormality detection device in FIG. 1;

FIG. 3 is a diagram showing an example of an input image.

FIG. 4 is a diagram for explaining a person motion amount;

FIG. 5 is a diagram for explaining a motion direction histogram of a person.

FIG. 6 is a view for explaining registration of head area information.

FIG. 7 is a view for explaining a histogram shift process;

FIG. 8 is a diagram for explaining a feature amount obtained from a head region image.

FIG. 9 is a diagram for explaining abnormal state determination.

FIG. 10 is a flowchart illustrating an operation of determining an abnormal operation.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Image input means 2 ... Image storage means 3 ... Operation area detection means 4 ... Head area extraction means 5 ... Turnover detection means 6 ... Sleeping state abnormality detection means 7 ... Notification means 11 ... CCD camera 12 ... Personal computer 13 ... Notification apparatus

Claims (11)

[Claims] 1. An image input means for inputting an image of a sleeping person, and a detecting means for detecting an abnormality from the frequency of movement of the person in the upper body image input by the image input means. Sleeping state abnormality detection device. 2. An image input means for inputting an image of a sleeping person, an analyzing means for analyzing a motion and a state of a person in an upper body image input by the image input means, and a frequency of movement by the analysis of the analyzing means. A sleep state abnormality detection device, comprising: a detection unit that detects an abnormality from a sleep state. 3. An image input means for inputting an image of a sleeping person, an analysis means for analyzing a motion or a state of a person in an upper body image input by the image input means, and a frequency of movement by the analysis of the analysis means. A sleep state abnormality detection device, comprising: a detection unit configured to detect an abnormal state from the detection unit; and a notification unit connected to the detection unit and configured to notify the state of the abnormality detected by the detection unit. 4. The sleeping state abnormality detecting device according to claim 3, wherein said image input means uses an image obtained by a monochrome or color CCD camera. 5. The sleeping state abnormality detecting device according to claim 3, wherein said image input means uses an image obtained by a distance sensor. 6. The sleeping state abnormality detecting device according to claim 3, wherein said image input means uses an image obtained by an infrared sensor to be less affected by a temperature change. 7. An image input means which is provided vertically downward and inputs an image of a sleeping person, an image storage means for temporarily storing an image input by the image input means, Motion region detection means for detecting a motion region, a motion amount, and a motion direction of a person from the stored image; and a motion region detected by the motion region detection device, or a head region from the image stored in the image storage device. Head area extracting means for extracting only the head and tracking the head, information on the head area extracted by the head area extracting means,
Turnover detection means for comparing the head area information registered in advance to detect that the person is lying prone; abnormal movement based on the movement amount and movement direction information detected by the movement area detection means; A sleeping state abnormality detecting means for distinguishing the sleeping state and detecting the sleeping state abnormality; and detecting information when the sleeping state abnormality is detected by the sleeping state abnormality detecting means, or when the prone sleeping state is detected by the turnover detecting means. A sleep state abnormality detection device, comprising: 8. The head region extracting means registers a head by template matching based on an image of the person registered in advance and lying on his / her back, prone, rightward, and leftward, using a gradient direction histogram of differentiation as a parameter. The sleeping state abnormality detecting device according to claim 7, wherein the device is used for tracking. 9. The rollover detecting means obtains a gradient direction histogram of a differential image as a feature amount for the head area image extracted by the head area extracting means, and stores a supine state and a prone state registered in advance. An error is calculated by comparing the same feature value obtained from four rightward and leftward images while shifting the value in the angle axis direction in the histogram, and the closest head direction is obtained. The sleeping state abnormality detecting device according to claim 7, wherein the device detects that the user is lying on his / her stomach without being affected by the rotational movement or the change in illumination. 10. The sleeping state abnormality detecting means obtains a variance of the amount of movement in a past fixed frame T based on the information on the amount of movement and the direction of movement detected by the movement region detecting means, and calculates a constant as the constant. Multiplied value and past constant frame T
Is detected as abnormal when an operation larger than the reference operation amount obtained by adding the average operation amount in the above operation continues for a predetermined time t (t <T), or when the operation is larger than the reference operation amount for a predetermined ratio or more within the predetermined time. Determining whether or not the minimum or average variance value among a plurality of motion direction histograms obtained while shifting in the angle axis direction is larger than a predetermined threshold value. The sleeping state abnormality detection device according to claim 7, wherein the abnormality is detected while distinguishing the painful operation. 11. The sleeping state abnormality detection according to claim 7, wherein the notifying unit notifies the abnormal state using an image or a sound when the operation or the sleeping state of the person is abnormal. apparatus.