CN103136919B - Remote control and display system - Google Patents

Abstract

A remote controller comprises a plurality of control keys, an optical finger mouse and a transmission interface. The control key is used for triggering a control signal. The optical finger mouse is used for detecting physiological characteristics and displacement. The transmission interface is used for outputting the control signal, the physiological characteristic and the displacement to a display device. The invention also provides a display system.

Description

Remote control and display system

technical field

The invention relates to a man-machine interface system, in particular to a remote controller and a display system with a user's physiological feature detection function.

Background technique

An optical finger mouse (OFM) is usually suitable for matching with other electronic devices due to its small size. The known optical finger mouse is used to detect the light intensity change of the reflected light on the surface of the user's finger, so as to judge the contact state of the finger and the displacement of the finger relative to the touch surface. However, with the development of industrialization, users spend more and more time using various electronic devices, even exceeding physical load without realizing it. Therefore, if the electronic device has the function of detecting the physiological characteristics of the user and can display the physiological condition at that time, the situation of overuse can be avoided.

It is known that the pulse oximeter uses a non-invasive method to detect the blood oxygen concentration and pulse rate of the user, which can generate red light beams (wavelength of about 660 nanometers) and infrared light beams (wavelength of about 910 nanometers) Penetrate the part to be measured, and use the characteristics of different absorption rates of oxygenated hemoglobin (oxyhemoglobin) and deoxyhemoglobin (Deoxyheamo-globin) for specific spectra to detect the change of light intensity of the transmitted light, for example, refer to US Patent No. 7,072,701 No., titled Method for Spectro-photometric Blood Oxygenation Monitoring. After detecting the light intensity changes of the two wavelengths of transmitted light, the blood oxygen concentration is calculated by the following formula

Blood oxygen concentration=100%×[HbO ₂ ]/([HbO ₂ ]+[Hb]);

Wherein, [HbO ₂ ] represents the concentration of oxygenated hemoglobin; [Hb] represents the concentration of deoxygenated hemoglobin.

The light intensity of the two wavelengths of penetrating light detected by the general oximeter will change as shown in Figure 1 with the heartbeat. The amount of blood passing through changes, which in turn changes the proportion of light energy that is absorbed. In this way, according to the constantly changing light intensity information, the absorption rate of blood to different spectra can be calculated to calculate physiological information such as oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration, and finally the blood oxygen concentration formula can be used to calculate blood oxygen concentration. concentration.

However, because the blood oxygen saturation meter detects the light intensity change of the penetrating light, it will detect different light intensity signals with different parts to be tested; in addition, when the part to be tested detected by the blood oxygen saturation meter moves , will detect violently changing chaotic waveforms and cannot correctly calculate physiological information, so it is not suitable for electronic devices operating on the move.

In view of this, the present invention proposes a remote controller and a display system capable of detecting physiological characteristics of a user, wherein the remote controller can effectively eliminate signal interference caused by finger movement when detecting physiological characteristics.

Contents of the invention

The purpose of the present invention is to provide a remote controller and a display system. The remote controller detects finger displacement, contact state and user physiological characteristics by analyzing the reflected light signal of the finger, and generates control according to the operating state of at least one control button. The signal is used to control the display device to perform corresponding actions.

Another object of the present invention is to provide a remote controller control chip, which detects a finger displacement, a contact state, and the user's physiological characteristics by analyzing the reflected light signal of the finger, and generates a signal according to an operation state of at least one control button A control signal, whereby the encoded, sequenced and/or compressed finger information, physiological information and control signal information are output.

Another object of the present invention is to provide a remote controller and a display system, which can detect finger displacement, contact state and user's physiological characteristics, and have a mechanism to eliminate the influence of ambient light sources.

Another object of the present invention is to provide a remote controller and a display system, which can detect finger displacement, contact state and user's physiological characteristics, and have a mechanism to reduce interference.

Another object of the present invention is to provide a remote controller and a display system, which can detect finger displacement, contact state and user's physiological characteristics, and enter into a sleep mode after being idle for a preset time.

Another object of the present invention is to provide a remote controller and a display system, which can detect finger displacement, contact status and physiological characteristics of the user, and can discard the physiological characteristics when the finger displacement is too large.

To achieve the above purpose, the present invention provides a remote controller for detecting and outputting physiological characteristics and control signals of fingers. The remote controller includes a plurality of control keys, a first light source, a second light source, a light source control unit, at least one image sensor and a processing unit. The control buttons are used to trigger control signals. The first light source emits light of a first wavelength to the finger. The second light source emits light of a second wavelength to the finger. The light source control unit controls the first light source and the second light source to emit light. The image sensor receives reflected light from the finger at a sampling frequency to generate a plurality of first image frames illuminated relative to the first light source and a plurality of second image frames illuminated relative to the second light source . The processing unit calculates the physiological characteristics according to the first image frame and the second image frame and generates the control signal according to the operation state of the control button.

According to another feature of the present invention, the present invention also provides a remote control for the user to control. The remote controller includes a plurality of control buttons, an optical finger mouse and a transmission interface. The control buttons are used to trigger control signals. The optical finger mouse is used to detect the user's physiological characteristics and finger displacement. The transmission interface is used to output the control signal, the physiological characteristics and the finger displacement.

According to another feature of the present invention, the present invention also provides an image system, including a display device and a remote controller. The display device is used for displaying images. The remote controller outputs control signals and physiological characteristics to the display device, so as to control the display device to update the displayed image and display the physiological characteristics according to the control signals.

In the embodiment of the present invention, each first image frame is divided into at least two parts and the average brightness of each part is calculated, and the independent component analysis method or blind signal source separation method is used to analyze the first image frame. The average brightness of each part is used to obtain the first brightness change; each second image frame is divided into at least two parts and the average brightness of each part is calculated, and the independent component analysis method or blind signal source separation method is used for analysis. obtaining the average brightness of each part of the second image frame to obtain a second brightness change; and calculating physiological characteristics according to the first brightness change and the second brightness change.

In the remote controller and image system of the present invention, the physiological characteristics include blood oxygen concentration and pulse rate. The present invention can eliminate signal interference caused by finger movement by using an independent component analysis method or a blind signal source separation method to separate movement information and physiological information.

Description of drawings

FIG. 1 is a schematic diagram of light intensity changes of transmitted light detected by an oximeter.

FIG. 2A is a schematic diagram of a display system according to an embodiment of the present invention.

FIG. 2B is a schematic diagram of a remote controller according to an embodiment of the present invention.

FIG. 2C is a block diagram of a remote controller according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of an image frame captured by an image sensor of a remote controller according to an embodiment of the present invention.

FIG. 4 is an image sensor of a remote controller according to an embodiment of the present invention, wherein a filter is disposed in front of part of the sensing surface.

FIG. 5 is a schematic diagram of image acquisition and light emitting from a light source in the remote controller according to the embodiment of the present invention.

Fig. 6 is a schematic diagram of separating movement information and physiological information by the processing unit of the remote controller according to the embodiment of the present invention.

Fig. 7 is a flowchart of a physiological feature detection method according to an embodiment of the present invention.

Fig. 8 is a schematic diagram of a remote controller according to another embodiment of the present invention.

Explanation of reference signs

1 remote control 10 control buttons

111, 112 light sources 12 light guides

13 touch controls 13S touch surface

14 image sensor 14f filter

14S sensing surface 15 processing units

151 Finger detection unit 152 Display control unit

16 light source control unit 17 memory unit

18 transmission interface 2 display devices

21 cursor 22 control unit

23 means unit 9 fingers

9S Finger surface I ₁ -I _2N image frame

B ₁ -B _2N , B ₁ ′-B _2N ′ average brightness S ₁₁ -S ₁₅ steps

Detailed ways

In order to make the above and other objects, features and advantages of the present invention more apparent, a detailed description will be given below with reference to the accompanying drawings. In the description of the present invention, the same components are denoted by the same symbols, and will be described first.

FIG. 2A is a schematic diagram of an image system according to an embodiment of the present invention, including a remote controller 1 and a display device 2 . The display device 2 can be, for example, a TV, a projection screen, a game machine screen, a computer screen or other display devices for displaying images. With respect to the display device 2, the remote controller 1 can be a TV remote controller, a projection screen remote controller, a game console remote controller, a computer remote controller and other remote controllers, and is used to control the display device 2 to update display content or images and Displaying physiological characteristics; wherein, the remote controller 1 can be coupled to the display device 2 by wire or wirelessly. The remote controller 1 is used to detect and output physiological characteristics, contact state, finger displacement and control signals to the display device 2 .

The remote control 1 includes a plurality of control buttons 10, an optical finger mouse and a transmission interface 18, wherein the optical finger mouse includes a touch element 13 for finger manipulation, and the touch element 13 can be combined with the control buttons 10 either or independently of them. The control buttons 10 are used to trigger control signals according to the user's pressing, for example, by pressing at least one control button to generate control signals related to different impedance values, voltage values, and oscillation frequencies. The optical finger mouse is used to detect the contact state and displacement of the user's finger and the physiological characteristics of the user, wherein the physiological characteristics include, for example, blood oxygen concentration and pulse rate. In this embodiment, when the optical finger mouse determines that the contact state is a contact state, the detection of the displacement amount and the physiological feature starts. The transmission interface 18 transmits the control signal, contact state, displacement and physiological characteristics to the display device 2 via wired or wireless, so that the display device 2 updates the display image according to the control signal and displays the The above displacement and physiological characteristics. It can be understood that, although the touch control 13 is arranged on the upper surface of the remote control 1 in FIG. There is no specific limitation on the position touched by the finger.

2B is a schematic diagram of a remote control 1 according to an embodiment of the present invention, which includes a plurality of control buttons 10, two light sources 111 and 112, a light guide 12 (the number of light guides here is only exemplary), and a touch control 13. , an image sensor 14 , a processing unit 15 , a light source control unit 16 and a transmission interface 18 . It must be noted that the spatial relationship of the various components in FIG. 2B is only illustrative and not intended to limit the present invention. The control button 10 has functions such as, but not limited to, channel selection, parameter adjustment, device selection, etc., and triggers a control signal according to the operation state (press state) of the finger 9 . The light sources 111 and 112 can be, for example, light-emitting diodes or laser diodes, which are used to generate light of different wavelengths to the finger surface 9S respectively. light and infrared light around 905, 910 or 940 nanometers. It can be understood that the wavelength mentioned here refers to the central wavelength of the emission spectrum of the light source.

The light guide 12 is used to guide the light emitted by the light sources 111 and 112 to the touch control 13; wherein, the light guide 12 only needs to be able to guide the light to the touch control 13, and its structure , the number and the light guiding method are not particularly limited. In other implementation manners, if the light emitted by the light sources 111 and 112 can be incident on the touch element 13 , the light guide element 12 may not be implemented.

The touch element 13 has a touch surface 13S for the finger 9 to operate on, and the touch element 13 is preferably transparent to the light emitted by the light sources 111 and 112 . When the finger 9 approaches or touches the touch surface 13S of the touch element 13 , the light emitted by the light sources 111 and 112 is reflected. It can be understood that the area of the touch surface 13S may be larger or smaller than the finger surface 9S, and there is no specific limitation.

The image sensor 14 receives the reflected light from the touch element 13 (the finger surface 9S) with sampling parameters to generate a plurality of image frames (the image frames have, for example, 16×16 pixels); wherein the sampling parameters are, for example, Including exposure time, image gain (can be analog gain or digital gain), etc. The image sensor 14 is preferably an active array image sensor, such as a CMOS image sensor.

The processing unit 15 generates a corresponding control signal according to the operation state of the control button 10, and detects the contact state of the finger 9 relative to the touch surface 13S according to the multiple image frames output by the image sensor 14. , the amount of displacement and the physiological characteristics of the user. The control signal, contact state, displacement and physiological characteristics obtained by the processing unit 15 can be sent to a display device having at least one display unit for display or corresponding control, for example, wired or wirelessly; The above-mentioned display unit can be, for example, a display, a light sign, a seven-character display and/or a sound device. The display device can be a portable electronic device or a home electronic device.

The light source control unit 16 is coupled to the processing unit 15 to control the light sources 111 and 112 to emit light according to the image frame acquisition of the image sensor 14 , and its implementation will be described in detail later.

The transmission interface 18 transmits the control signal, contact state, displacement and physiological characteristics to the display device 2 by wire or wirelessly.

In this embodiment, the light sources 111 and 112, the image sensor 14, the processing unit 15 and the light source control unit 16 are used as an optical finger mouse to detect the contact state, displacement and physiological state of the finger 9. feature.

Referring to FIGS. 2A to 2C, FIG. 2C shows a block diagram of a remote controller 1 according to an embodiment of the present invention, which includes a plurality of control buttons 10, a first light source 111, a second light source 112, the image sensor 14, and the processing unit 15 , the light source control unit 16 , the memory unit 17 and the transmission interface 18 , wherein the remote controller 1 , the control unit 22 and the display unit 23 form a display system. Since the processing unit 15 performs multifunctional calculations, it may include a finger detection unit 151 for detecting the contact state, finger displacement, and physiological characteristics of the finger 9 relative to the touch surface 13S, and includes a display control unit 152 is used to generate and output the control signal according to the operation state (pressed state) of the control button 10; that is, the processing unit 15 can be a single element or divided into two units.

The first light source 111, for example, emits red light with a wavelength of about 660 nanometers to the finger 9; the second light source 112, for example, emits infrared light with a wavelength of about 905, 910 or 940 nanometers to the finger 9; in a broad sense In other words, the first light source 111 and the second light source 112 can respectively emit light of two wavelengths used by general oximeters. The light source control unit 16 controls the first light source 111 and the second light source 112 to emit light. The image sensor 14 receives reflected light from the first light source 111 and the second light source 112 from the finger surface 9S. The memory unit 17 is used to store the control signal obtained by the processing unit 15, contact state, displacement, physiological characteristics and various parameter information required in the calculation process; wherein, the control signal may not stored in the memory unit 17 and directly transmitted by the transmission unit 18 . The transmission interface 18 is used to transmit the control signal, contact state, displacement and physiological characteristics stored in the memory unit 17 to the external control unit 22 through wired or wireless transmission; wherein, wired and wireless The transmission technology is known, so it will not be repeated here. The control unit 22 may be included in the display device 2 having at least one display unit 23, for controlling the display device 2 to display and/or respond to the received control signal, contact State, displacement and physiological characteristics.

The remote controller 1 according to the embodiment of the present invention can be equipped with a display device 2 having a display unit 23, so that the user can control the cursor displayed on the display unit 23 or the software executed by the remote controller 1, and in the physiological characteristics Alerting the user when fatigue or excitement is displayed (depending on the value of the physiological characteristics); wherein, the way of expressing the physiological characteristics and the warning can be achieved, for example, by using software to execute screen display, light signal display or sound display, There are no specific restrictions. The display device 2 can, for example, switch screens, update images, adjust volume, adjust display parameters, etc. according to the control signal; control the action of the cursor according to the displacement; display the physiological characteristics and when the physiological characteristics exceed the preset A warning state is generated when the value is set, such as darkening of the screen, image insertion, sound prompt, etc., but not limited thereto.

In one embodiment, the remote controller 1 can also use two image sensors to detect light of different wavelengths generated by the light sources 111 and 112 respectively (that is, the image sensor 14 is replaced by two image sensors), one of which is Or two image sensors can be set with a bandpass filter (bandpass filter) to select the spectrum to be received.

Since the manner in which the processing unit 15 generates the control signal according to the control button 10 is known, it will not be repeated here. In the following, only the manner in which the processing unit 15 calculates the contact state, finger displacement and physiological characteristics will be described in detail; that is, only the light sources 111 and 112, the image sensor 14, the processing The operation of the optical finger mouse composed of unit 15 (finger detection unit 151 ) and the light source control unit 16 will be described.

sampling mechanism

The optical finger mouse of this embodiment uses two light sources 111 and 112 and performs two functions at the same time; wherein, the detection function of contact state and displacement is not limited to use an image frame of a specific wavelength, while the detection of physiological feature function must correspond to Image frames for different wavelengths are calculated separately. First, the sampling mechanism of the image frame will be described below.

In one embodiment, the light source control unit 16 controls the first light source 111 and the second light source 112 to emit light in turn, and the image sensor 14 synchronizes the The lighting of the first light source 111 or the second light source 112 acquires an image frame, and outputs a plurality of image frames I ₁ -I ₆ . . . to the processing unit 15 (finger detection unit 151) as shown in FIG. 3 , Wherein the image frames I ₁ -I ₆ ... include the first image frames I ₁ , I ₃ , I ₅ ..., which are for example illuminated relative to the first light source 111, and the second image frames I ₂ , I ₄ , I _{6 .} . . which, for example, relative to all

The processing unit 15 can judge the contact state and calculate the displacement according to the first image frame and the second image frame I ₁ -I ₆ ..., for example, according to the first image frame and the second image frame The brightness of the image frame is compared with at least one threshold to determine whether the finger 9 is close to or in contact with the touch surface 13S, wherein when the brightness of the image frame is greater than or less than the at least one threshold, it is judged to enter Contact state; after entering the contact state, the processing unit 15 can be based on the correlation between two first image frames, a first image frame and a second image frame, or two second image frames to calculate the displacement. It must be noted that, in this embodiment, image frames corresponding to two different wavelengths of reflected light must be used to determine the contact state and calculate the displacement, which is different from traditional navigation devices.

The processing unit 15 calculates the brightness change of the first image frame according to the first image frame I ₁ , I ₃ , I ₅ ..., and calculates the brightness change of the first image frame according to the second image frame I ₂ , I ₄ , I ₆ ... Calculate the brightness change of the second image frame (details will be described later), and accordingly calculate the absorbed ratio of the two spectra to obtain the concentration of oxygenated hemoglobin HbO ₂ and deoxygenated hemoglobin Finally, the blood oxygen concentration is calculated using the blood oxygen concentration formula; and a pulse rate is calculated by comparing the brightness change of the first image frame and/or the second image frame with at least one threshold value.

In yet another embodiment, the light source control unit 16 controls the first light source 111 and the second light source 112 to emit light simultaneously with the image frame acquisition of the image sensor 14; that is, at this time, the image sensor 14 will receive reflected light of two wavelengths at the same time. Therefore, in this embodiment, an optical filter 14f (as shown in FIG. 4 ) is preferably provided in front of a part of the sensing surface 14S of the image sensor 14, wherein the optical filter 14f can be a bandpass filter to The part of the sensing surface 14S behind the filter 14f can only sense the spectrum of the first light source 111 or the spectrum of the second light source 112, so that the processing unit 15 can distinguish the first image frame (a partial image frame relative to the first light source 111) and a second image frame (a partial image frame relative to the second light source 112). It can be understood that the location and area of the optical filter 14f in this embodiment are not limited to those shown in FIG. 4 .

In this way, the processing unit 15 can also calculate the contact state and displacement according to the first image frame and the second image frame I ₁ -I ₆ . . . ₁ , I ₃ , I ₅ ... calculate the brightness change of the first image frame and calculate the brightness change of the second image frame according to the second image frame I ₂ , I ₄ , I ₆ ... , and calculate at least one of blood oxygen concentration and pulse rate according to the relationship between the two brightness changes.

It can be understood that since the image sensor 14 may have different photosensitive efficiencies for light of different wavelengths, or the luminance of the first light source 111 and the second light source 112 are not completely the same, it can be used in the remote control Before the device 1 leaves the factory, it is preferable to adjust the brightness of the image frame detected by the image sensor 14 (such as adjusting the exposure time, image gain and other sampling parameters relative to the image frame of different wavelengths), so that the image sensor 14 acquired The initial image frames have approximately the same brightness to eliminate the possibility of misjudgment.

The spirit of this embodiment is to use the light source control unit 16 to control the first light source 111 and the second light source 112 to emit light, so that the image sensor 14 receives the reflected light from the finger 9 at a sampling frequency to Generate a plurality of first image frames illuminated relative to the first light source and a plurality of second image frames illuminated relative to the second light source; the processing unit 15 then according to the first image frame and The second image frame calculates the contact state, finger displacement and physiological characteristics.

Eliminate ambient light mechanism

In FIG. 2B , since the contact 13 is transparent and the finger 9 is transparent, the ambient light outside the remote controller 1 will be received by the image sensor 14 through the finger 9 and the contact 13 and be affected. The image quality of the image frame it fetches. In this embodiment, the light source control unit 16 can control the first light source 111 and the second light source 112 to not emit light during a part of the period.

Fig. 5 is the image acquisition of the image sensor 14 and the lighting situation of the first light source 111 and the second light source 112; wherein, the solid line arrow indicates that the light source is lit (or lighted with the first brightness) and the dotted line arrow Indicates that the light source is turned off (or turned on with a second brightness); wherein the second brightness is smaller than the first brightness. FIG. 5A shows that the image sensor 14 continuously acquires image frames at a fixed sampling frequency. Fig. 5B shows that the first light source 111 and the second light source 112 are turned on and off at the same time, so the image sensor 14 can acquire bright image frames in turn (the light source is turned on or with the first brightness on) and dark image frame (light source off or on with said second brightness). FIG. 5C is a situation where the first light source 111 and the second light source 112 are turned on at the same time every second image frame, which generally has a relatively low displacement relative to the finger 9 . As mentioned above, when the first light source 111 and the second light source 112 are turned on at the same time (FIGS. 5B and 5C), the image sensor 14 includes a filter 14f to spatially separate image frames of different light sources. so that a part of the image sensor 14 can sense the reflected light of the first light source 111 and another part can sense the reflected light of the second light source 112 .

When the finger 9 touches or approaches the touch surface 13S, the bright image frame obtained when the light source is turned on contains (finger reflected light+stray light+environmental light), and when the light source is not turned on The obtained dark image frame only contains (ambient light), so if the bright image frame is subtracted from the dark image frame, the influence of ambient light can be effectively eliminated. The processing unit 15 can calculate the contact state, finger displacement and physiological characteristics according to the difference image frame of the light and dark image frames.

FIG. 5D is an embodiment in which the first light source 111 and the second light source 112 are turned on in turn. In this embodiment, since the image sensor 14 needs to acquire a dark image frame, the light source control unit 16 controls the first light source 111 and the second light source 112 to be separated by one image frame to take turns. On, for example, at time _td in FIG. 5D, neither light source is on. In this way, the processing unit 15 can calculate the difference between the first image (bright first image frame-dark image frame) and the difference between the second image (bright second image frame-dark image frame), and according to the The contact state, finger displacement and physiological characteristics are calculated from the difference image. As mentioned above, when the first light source 111 and the second light source 112 are turned on in turn, the image sensor 14 separates different image frames relative to the light sources at time intervals.

The spirit of this embodiment is to enable the light source control unit 16 to control the first light source 111 and the second light source 112 to emit light at the same time or alternately, and to enable the image sensor 14 to acquire the The dark image is framed, and the influence of ambient light is eliminated by calculating the difference image of the light and dark images. Therefore, the lighting conditions of the light sources shown in FIG. 5 are only exemplary, and are not intended to limit the present invention.

noise reduction mechanism

Since there will be interference in the image frame acquired by the image sensor 14, and the interference is usually randomly distributed in the acquired image frame, so this embodiment can further calculate the sum of M image frames to improve Signal-to-noise ratio (SNR) to increase the accuracy of calculating physiological characteristics; for example, every 10 image frames are added, and the 10 image frames added by two groups can be partially repeated or not repeated at all. It can be understood that when the first light source 111 and the second light source 112 are turned on in turn, the sum of the image frames in this embodiment is respectively the first image frame (for example, I ₁ + I ₃ +I ₅ . . . ) and the second image frame (for example, I ₂ +I ₄ +I ₆ . However, when the first light source 111 and the second light source 112 are turned on at the same time, the sum of the image frames in this embodiment is a continuous image frame (such as I ₁ +I ₂ +I ₃ + I ₄ +I ₅ +I ₆ ...), and distinguish two groups of light intensity changes in a spatially separated manner through post-processing. In addition, when the above-mentioned mechanism for eliminating ambient light is used, the sum of the image frames in this embodiment is the sum of the difference image frames; that is, the noise reduction process is performed after the ambient light elimination processing is performed. In other implementation manners, only one of the ambient light elimination processing and the noise reduction processing may be performed.

As mentioned above, the image sensor 14 may acquire images with different sampling parameters under different conditions. Sampling parameters such as exposure time and image gain to make the first image frame and the second image frame have approximately the same brightness, so that post-processing can be performed correctly according to the image frame, that is, relative to the second image frame The sampling parameters of one image frame and the second image frame may be different. In order to exclude the influence of different sampling parameters, the sum or average of each image frame or M image frames can be divided by the sampling parameters for normalization, for example (the sum/sampling parameters of M image frames) or (average/sampling parameters of M image frames); wherein, M is a positive integer.

Physiological characteristic calculation

When different light sources are turned on, the image frame acquired by the image sensor 14 contains physiological information and finger movement information at the same time. Therefore, in this embodiment, the processing unit 15 (or the finger detection unit 151) first needs to separate the two kinds of information before it can correctly calculate the physiological characteristics; (Independent Component Analysis, ICA) or blind source separation (Blind Source Separation, BSS) to separate the two kinds of information.

3 and 6, taking the first image frames I ₁ , I ₃ , I ₅ ... in FIG. mechanism and/or the first image frame processed by the noise reduction mechanism) or a plurality of first image frames and (M original image frames and M sheets processed by the ambient light elimination mechanism and/or the noise reduction mechanism Each image frame or image frame of the first image frame and) is divided into at least two parts and obtains the average brightness respectively, for example, the image frame _I1 is divided into two parts with average brightness _B1 and _B1 ';The image frame I ₃ is divided into two parts whose average brightness is B ₃ and B ₃ ′; ...; the image frame I _2N-1 is divided into two parts whose average brightness is B _2N-1 and B _2N-1 ′ (other There may be more than two parts in an embodiment). Next, the first movement information and the first physiological information (as shown in FIG. 6 ) are separated by using the independent component analysis method or the blind signal source separation method, both of which are displayed as brightness change lines. In this embodiment, the movement information is discarded and the physiological characteristics are calculated by using the brightness change line of the physiological information. It can be understood that, since the sampling frequency of the image sensor 14 is far greater than the pulse rate, the separated physiological information can show a line pattern (similar to FIG. 1 ) in which the light intensity varies with the pulse; the separated movement information distribution and It is not limited to those shown in FIG. 6 . In addition, the two parts of the image frame division are not limited to upper and lower parts. In addition, since it is necessary to calculate the relative two ₁ , I ₃ , I ₅ ... (corresponding to the first light source being turned on) and the second image frame I ₂ , I ₄ , I ₆ ... (corresponding to the second light source being turned on) to conduct. The second image frame (I ₂ , I ₄ , I ₆ ...) is also separated into brightness changes such as second motion information and second physiological information, wherein the second motion information is discarded and the brightness change of the second physiological information is used . It must be noted that when the image frame sum or average is used for information separation, each of I ₁ -I _2N-1 and I ₂ -I _2N in Fig. 6 represents the sum, average or its normalized result.

It must be emphasized that the contact state and displacement of the finger 9 are obtained directly by the processing unit 15 based on the first image frame and the second image frame, and there is no need to use the separated first and second image frames. Second mobile information. The independent component analysis method or the blind signal source separation method is mainly used to separate the mixed signal, and after the separated mobile information is discarded, the signal interference caused by finger movement can be eliminated.

In this embodiment, the processing unit 15 further calculates a pulse rate according to a comparison result of at least one threshold value and the first brightness change and/or the second brightness change.

sleep mode

The remote controller 1 according to the embodiment of the present invention can enter the sleep mode after being idle for a preset time. For example, when the processing unit 15 determines that the finger 9 is not approaching or touching the touch surface 13S at a preset time, it can enter the sleep mode.

Physiological trait discarding mechanism

The processing unit 15 of the remote controller 1 according to the embodiment of the present invention can calculate the displacement and the physiological characteristics at the same time, but the accurate calculation of the physiological characteristics is preferably in the case of a low displacement. Therefore, this embodiment can judge in advance whether the finger displacement is greater than a preset value, and if the finger displacement is greater than the preset value, the image frame acquired by the image sensor 14 is only used for calculating the displacement or judging The contact state is not used to calculate the physiological feature, or even if the physiological feature is calculated, it is not transmitted through the transmission interface 18 and is directly discarded from the memory unit 17 . The preset value is determined according to the actual application, for example, but not limited to, it may be determined according to the size of the sensing surface 13S and/or the search box.

The method for the remote controller 1 to detect physiological characteristics according to the reflected light of the finger surface 9S includes the following steps: providing the light of the first wavelength and the second wavelength to the finger surface (step S ₁₁ ); acquiring the light of the first wavelength Reflecting light to generate a plurality of first image frames and acquiring reflected light of light of the second wavelength to generate a plurality of second image frames (step S ₁₂ ); combining each of the first image frames and each The second image frame is divided into at least two parts and the average brightness of each part is obtained (step S ₁₃ ); the each part of the first image frame is analyzed by an independent component analysis method or a blind signal source separation method. said average luminance of a portion to obtain a first luminance change and analyze said average luminance of said each portion of said second image frame to obtain a second luminance change (step _S14 ); and according to said first The first brightness change and the second brightness change are used to obtain physiological characteristics (step S ₁₅ ). The implementation manners of each step in this implementation manner have been described in detail above, so details are not repeated here.

In another embodiment, some or all of the light sources 111 and 112, the image sensor 14, the processing unit 15, the light source control unit 16, the memory unit 17 and the transmission interface 18 It can also be made as a control chip or package, as shown in Figure 8. The control chip is used to detect the operating state of the control button 10 and the contact state, displacement and physiological characteristics of the finger 9, and output the encoded, sorted and/or compressed contact state, displacement, physiological characteristics Features and control signals (for example, encoded, sorted and/or compressed by the transmission interface or also include a communication protocol unit to perform these procedures); wherein the methods of calculating the contact state, displacement and physiological characteristics are as described above, so No longer. In other words, in one embodiment, the optical finger mouse and the remote controller control unit can be packaged into a control chip or a package. In addition, it can be understood that the arrangement of components in the remote controller 1 in FIG. 8 is only exemplary, and is not intended to limit the present invention. In other implementation manners, the compression processing may also be performed by an external compression unit.

To sum up, it is known that the remote control cannot detect the user's physiological characteristics, and the way the oximeter calculates the blood oxygen concentration cannot be compatible with the remote control due to factors such as the inability to determine the moving part to be measured. Therefore, the present invention also provides a remote controller (Figures 2B and 8) and a display system (Figure 2A), wherein the remote controller can simultaneously detect finger information and image control information, and control the display device according to at least the image control information The displayed content is updated and the finger information is displayed. The remote controller in each embodiment of the present invention can detect the user’s physiological characteristics while detecting the finger displacement, and can effectively eliminate the signal interference caused by finger movement and the influence of environmental light sources, and has a sleep mode and a mechanism for discarding physiological information .

Although the present invention has been disclosed in the foregoing embodiments, it is not intended to limit the present invention. Any person skilled in the art to which the present invention belongs can make various changes without departing from the spirit and scope of the present invention. with modification. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (17)

1. A remote controller for detecting and outputting physiological characteristics and control signals of fingers, the remote controller comprising: Multiple control buttons for triggering control signals; a first light source emitting light of a first wavelength to the finger; a second light source emitting light of a second wavelength to the finger; a light source control unit, controlling the first light source and the second light source to emit light; at least one image sensor receiving reflected light from said finger at a sampling frequency to generate a plurality of first image frames illuminated relative to said first light source and a plurality of second image frames illuminated relative to said second light source frame; and a processing unit that divides each of the first image frames into at least two parts and calculates the average brightness of each part, and analyzes the average brightness of each part of the first image frame to separate the first moving information and first physiological information; dividing each second image frame into at least two parts and calculating the average brightness of each part, and analyzing the average brightness of each part of the second image frame to separate Calculate the physiological characteristics according to the first brightness change of the first physiological information and the second brightness change of the second physiological information and discard the first movement information and the second physiological information The second movement information is used to eliminate the interference caused by finger movement, and the control signal is generated according to the operation state of the control button. 2. The remote controller according to claim 1, wherein the physiological characteristics include blood oxygen concentration and/or pulse rate. 3. The remote controller of claim 1, wherein the processing unit further calculates a pulse rate based on a comparison result of at least one threshold with at least one of the first brightness change and the second brightness change. 4. The remote controller according to claim 1, wherein the processing unit further compares the brightness of the first image frame and the second image frame with at least one threshold to determine a contact state. 5. The remote controller according to claim 1, wherein the processing unit is further based on two frames of the first image, one frame of the first image and one frame of the second image, and Calculate the displacement of the two frames of the second image. 6. The remote controller according to claim 1, wherein the light source control unit: controls the first light source and the second light source to turn on in turn so that the image sensor receives the first light source and the second light source in turn The reflected light of the second light source; or control the simultaneous lighting of the first light source and the second light source so that the image sensor simultaneously receives the reflected light of the first light source and the second light source, and the image The sensor includes a filter covering a portion of the sensing surface of the image sensor. 7. The remote controller according to claim 1, wherein the first light source, the second light source, the light source control unit, the at least one image sensor and the processing unit are packaged into a control chip to output coded The physiological characteristics and the control signals are processed by at least one of programming, sorting and compressing. 8. The remote controller according to claim 1, further comprising a touch element, which is manipulated by the finger; wherein the touch element can be one of the control buttons or be independent of the control buttons. other than the control buttons described above. 9. A remote controller controlled by a user, the remote controller comprising: Multiple control buttons for triggering control signals; An optical finger mouse is used to detect the user's physiological characteristics and finger displacement, and the optical finger mouse includes: a first light source emitting light of a first wavelength to the user's finger; a second light source emitting light of a second wavelength to the finger; at least one image sensor receiving reflected light from the finger to generate a plurality of first image frames illuminated relative to the first light source and a plurality of second image frames illuminated relative to the second light source; and A processing unit, calculating and outputting the finger displacement according to the first image frame and the second image frame; dividing each first image frame into at least two parts and calculating the average brightness of each part , analyzing the average brightness of each part of the first image frame to separate the first movement information and the first physiological information; dividing each second image frame into at least two parts and calculating each an average brightness of a portion, analyzing the average brightness of each portion of the second image frame to separate second motion information and second physiological information; and according to the first brightness change and the first physiological information of the first physiological information The second brightness change of the second physiological information calculates the physiological characteristics and discards the first movement information and the second movement information to eliminate interference caused by finger movement; and a transmission interface, configured to output the control signal, the physiological characteristic and the finger displacement. 10. The remote control of claim 9, wherein the optical finger mouse further comprises: A light source control unit, configured to control the first light source and the second light source to emit light. 11. The remote controller of claim 9, wherein the processing unit further calculates a pulse rate based on a comparison result of at least one threshold with at least one of the first brightness change and the second brightness change. 12. The remote controller according to claim 9, wherein the processing unit further compares the brightness of the first image frame and the second image frame with at least one threshold to determine the contact state. 13. A display system comprising: a display device for displaying images; and A remote controller that outputs control signals and physiological characteristics to the display device to control the display device to update the displayed image and display the physiological characteristics according to the control signal, the remote controller includes: a first light source emitting light of a first wavelength to the user's finger; a second light source emitting light of a second wavelength to the finger; at least one image sensor receiving reflected light from the finger to generate a plurality of first image frames illuminated relative to the first light source and a plurality of second image frames illuminated relative to the second light source; and a processing unit that divides each of the first image frames into at least two parts and calculates the average brightness of each part, and analyzes the average brightness of each part of the first image frame to separate the first moving information and first physiological information; dividing each second image frame into at least two parts and calculating the average brightness of each part, and analyzing the average brightness of each part of the second image frame to separate Calculate the physiological characteristics according to the first brightness change of the first physiological information and the second brightness change of the second physiological information and discard the first movement information and the second physiological information The second movement information is used to eliminate interference caused by finger movement. 14. The display system according to claim 13, wherein when the physiological characteristic exceeds a preset value, the display device generates an alarm state. 15. The display system according to claim 13, wherein the remote controller further comprises: a light source control unit, controlling the first light source and the second light source to emit light, Wherein, the processing unit also generates the control signal according to the operation states of a plurality of control keys. 16. The display system according to claim 13, wherein the processing unit is further based on two frames of the first image, one frame of the first image and one frame of the second image, or The displacement of the two second image frames is calculated, and the remote controller outputs the displacement to the display device to relatively control the cursor displayed on the display device. 17. The display system of claim 13, wherein the processing unit further calculates a pulse rate based on a comparison of at least one threshold with at least one of the first brightness change and the second brightness change.