CN110353648A - A kind of real-time heart rate detection method based on dual camera - Google Patents
A kind of real-time heart rate detection method based on dual camera Download PDFInfo
- Publication number
- CN110353648A CN110353648A CN201910802268.XA CN201910802268A CN110353648A CN 110353648 A CN110353648 A CN 110353648A CN 201910802268 A CN201910802268 A CN 201910802268A CN 110353648 A CN110353648 A CN 110353648A
- Authority
- CN
- China
- Prior art keywords
- ref
- frame
- interest
- camera
- heart rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/024—Measuring pulse rate or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient; User input means
- A61B5/7475—User input or interface means, e.g. keyboard, pointing device, joystick
- A61B5/748—Selection of a region of interest, e.g. using a graphics tablet
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/20—Workers
- A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Surgery (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Public Health (AREA)
- Molecular Biology (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Cardiology (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Image Analysis (AREA)
Abstract
本发明公开一种基于双摄像头的实时心率检测方法,包括以下步骤:打开主摄像头和参考摄像头,以相同的帧率交替拍摄包含人脸的主视频和参考视频;创建两个人脸检测器,分别获取主摄像头感兴趣区域和参考摄像头感兴趣区域;截取主摄像头背景区域和参考摄像头背景区域;根据参考摄像头感兴趣区域和背景区域提取的噪声信息,计算每一帧感兴趣区域和背景区域的特征值,得到时间轴上的一维特征序列;采用低通滤波器对一维特征序列进行滤波,得到心率信号;采用峰值计数法计算心率值;实时采集数据更新信号序列,重复上述步骤,计算实时心率值。本发明中提出的方法降低了由于环境光干扰引起的信号失真,大大提高了非接触式心率检测中的准确度和稳定性。
The invention discloses a real-time heart rate detection method based on dual cameras, comprising the following steps: turn on the main camera and the reference camera, alternately shoot the main video and the reference video containing human faces at the same frame rate; create two human face detectors, respectively Obtain the ROI of the main camera and the ROI of the reference camera; intercept the background area of the main camera and the background area of the reference camera; calculate the characteristics of the ROI and the background area of each frame based on the noise information extracted from the ROI and the background area of the reference camera value, get the one-dimensional feature sequence on the time axis; filter the one-dimensional feature sequence with a low-pass filter to get the heart rate signal; use the peak counting method to calculate the heart rate value; collect data in real time to update the signal sequence, repeat the above steps, and calculate heart rate value. The method proposed in the invention reduces the signal distortion caused by ambient light interference, and greatly improves the accuracy and stability in non-contact heart rate detection.
Description
技术领域technical field
本发明涉及数字图像处理技术领域,特别涉及一种基于双摄像头的实时心率检测方法。The invention relates to the technical field of digital image processing, in particular to a real-time heart rate detection method based on dual cameras.
背景技术Background technique
根据世界卫生组织的统计,心血管病死亡率居首位,明显高于肿瘤及其他疾病。然而很多人对心脏病的认识往往不足,导致错过最佳寻医问诊时机。According to the statistics of the World Health Organization, the mortality rate of cardiovascular disease ranks first, significantly higher than that of tumors and other diseases. However, many people often have insufficient understanding of heart disease, resulting in missing the best time to seek medical advice.
研究表明,大部分的早期心脏病发作和中风都是可以预防的。除了保证良好的生活习惯,最关键的就是早发现早治疗。心慌、胸闷、胸痛、头昏或晕厥这些看似不起眼的症状,都是心脑血管疾病的信号,而大部分人都因为大意错过了最佳治疗时机。Research shows that most early heart attacks and strokes are preventable. In addition to ensuring good living habits, the most important thing is early detection and early treatment. Palpitation, chest tightness, chest pain, dizziness or fainting, these seemingly insignificant symptoms, are all signals of cardiovascular and cerebrovascular diseases, and most people miss the best time for treatment because of carelessness.
当前医院最普遍的检查方法是采用心电图机(ECG)检查患者的心脏功能,但监测时间比较短,且设备昂贵,不易携带。人们针对便携性和低成本,设计发明了指夹式心率测试仪。但长期佩戴会导致使用者指尖血液循环不畅,引起使用者的不适。The most common inspection method in current hospitals is to use an electrocardiograph (ECG) to check the patient's heart function, but the monitoring time is relatively short, and the equipment is expensive and difficult to carry. For portability and low cost, people designed and invented the finger-clip heart rate tester. However, wearing it for a long time will lead to poor blood circulation in the user's fingertips, causing discomfort to the user.
近年来,随着计算机和实时视频图像处理等技术的普及,图像PPG(Photoplethysmography)技术的提出为实现无创、非接触的实时心率测量提供了切实可行的思路。图像PPG技术是指,由于人的心脏在不断地收缩和舒张,会导致人的血管中的血液的充盈程度也会随着心跳而不断地发生变化,并且对光的吸收会随着血液容积的变化呈现出与心跳一致的脉动性变化,同时皮肤表面反射的光的强度也会发生相应的周期性变化,从而表现为皮肤颜色的变化。In recent years, with the popularization of computer and real-time video image processing and other technologies, the proposal of image PPG (Photoplethysmography) technology provides a practical idea for realizing non-invasive and non-contact real-time heart rate measurement. Image PPG technology means that because the human heart is constantly contracting and relaxing, the blood filling degree in the human blood vessels will also change continuously with the heartbeat, and the absorption of light will change with the volume of blood. The change presents a pulsating change consistent with the heartbeat, and at the same time, the intensity of the light reflected by the skin surface also undergoes a corresponding periodic change, which is manifested as a change in skin color.
目前基于视频的心率检测方法中普遍存在环境光的干扰和噪声,导致图像数据的不稳定,从而增大了人脸面部参数微小信号提取的难度。At present, the interference and noise of ambient light commonly exist in the heart rate detection method based on video, which leads to the instability of image data, thus increasing the difficulty of extracting the small signal of facial parameters.
发明内容Contents of the invention
本发明的目的在于提供一种基于双摄像头的实时心率检测方法,该方法能够有效地提高基于摄像头的实时心率检测精度和稳定度。The purpose of the present invention is to provide a real-time heart rate detection method based on dual cameras, which can effectively improve the accuracy and stability of the camera-based real-time heart rate detection.
实现本发明目的的技术方案是:The technical scheme that realizes the object of the present invention is:
一种基于双摄像头的实时心率检测方法,该方法包括以下步骤:A method for real-time heart rate detection based on dual cameras, the method comprising the following steps:
(1)打开主摄像头和参考摄像头,以相同的帧率交替拍摄20s包含人脸的主视频和参考视频;(1) Turn on the main camera and the reference camera, and alternately shoot the main video and the reference video containing faces for 20 seconds at the same frame rate;
(2)创建两个人脸检测器,分别获取所述主视频和所述参考视频每一帧感兴趣区域,并提取每一帧U通道数据,获得主摄像头感兴趣区域和参考摄像头感兴趣区域的U通道序列;(2) create two face detectors, obtain the region of interest of each frame of the main video and the reference video respectively, and extract each frame of U channel data, obtain the region of interest of the main camera and the region of interest of the reference camera U channel sequence;
(3)截取主摄像头背景区域和参考摄像头背景区域,并获得主摄像头背景区域和参考摄像头背景区域的U通道序列;(3) Intercept the main camera background area and the reference camera background area, and obtain the U channel sequence of the main camera background area and the reference camera background area;
(4)根据参考摄像头感兴趣区域和背景区域提取的噪声信息,计算每一帧感兴趣区域和背景区域特征值,得到时间轴上的一维特征序列;(4) According to the noise information extracted from the region of interest and the background region of the reference camera, calculate the feature values of the region of interest and the background region of each frame to obtain a one-dimensional feature sequence on the time axis;
(5)采用低通滤波器对一维特征序列进行滤波,将滤波后的一维特征序列作为心率信号;(5) A low-pass filter is used to filter the one-dimensional feature sequence, and the filtered one-dimensional feature sequence is used as a heart rate signal;
(6)采用峰值计数法得到心跳次数,计算心率值;(6) Obtain the number of heartbeats by using the peak counting method, and calculate the heart rate value;
(7)实时采集数据更新信号序列,重复(2)-(6)步骤计算实时心率值。(7) Collect data in real time to update the signal sequence, and repeat steps (2)-(6) to calculate the real-time heart rate value.
所述步骤(1)具体为:Described step (1) is specifically:
创建主摄像头对象和参考摄像头对象,设置相同帧率fs;Create the main camera object and reference camera object, set the same frame rate fs;
设置帧计数器frame_counter=0;set frame counter frame_counter = 0;
先打开参考摄像头,经过Δt时间后打开主摄像头,Δt<1/fs,并均以fs的帧率拍摄20s包含人脸的视频,分别记参考摄像头当前帧和主摄像头当前帧为Frame_ref(i)和Frame(i),frame_counter=K=20*fs。First turn on the reference camera, then turn on the main camera after Δt time, Δt<1/fs, and shoot 20s of videos containing faces at the frame rate of fs, record the current frame of the reference camera and the current frame of the main camera as Frame_ref(i) and Frame(i), frame_counter=K=20*fs.
所述步骤(2)具体为:Described step (2) is specifically:
利用维奥拉-琼斯算法创建人脸检测器,分别获得所述主视频和所述参考视频每一帧人脸区域的起始点坐标(x,y)、(x_ref,y_ref)和人脸尺寸(w,h)、(w_ref,h_ref);Utilize the Viola-Jones algorithm to create a face detector, obtain the starting point coordinates (x, y), (x_ref, y_ref) and the face size ( w, h), (w_ref, h_ref);
根据人脸比例,分别截取Frame(i)和Frame_ref(i)高度范围为x+0.5*h~x+0.7*h,宽度范围为y+0.1*w~y+0.3*w的区域,即主摄像头感兴趣区域Interest(i)和参考摄像头感兴趣区域Interest_ref(i),获得当前帧所述主摄像头感兴趣区域和所述参考摄像头感兴趣区域的RGB通道;According to the proportion of the face, intercept Frame(i) and Frame_ref(i) with a height range of x+0.5*h~x+0.7*h and a width range of y+0.1*w~y+0.3*w, namely the main camera region of interest Interest(i) and reference camera region of interest Interest_ref(i), obtain the RGB channels of the main camera region of interest and the reference camera region of interest in the current frame;
将所述主摄像头感兴趣区域和所述参考摄像头感兴趣区域的RGB通道转换为YUV通道,分别提取二维U通道U(i)、U_ref(i);U通道计算公式如下:The RGB channels of the main camera region of interest and the reference camera region of interest are converted into YUV channels, and two-dimensional U channels U(i), U_ref(i) are extracted respectively; the U channel calculation formula is as follows:
U=-0.169*R-0.331*G+0.5*BU=-0.169*R-0.331*G+0.5*B
每一帧所述主摄像头感兴趣区域的U通道构成序列U(1),U(2),…,U(K)、每一帧所述参考摄像头感兴趣区域的U通道构成序列U_ref(1),U_ref(2),…,U_ref(K)。The U channel of the region of interest of the main camera in each frame constitutes a sequence U(1), U(2),..., U(K), and the U channel of the region of interest of the reference camera in each frame constitutes a sequence U_ref(1 ), U_ref(2), ..., U_ref(K).
所述步骤(3)具体为:Described step (3) is specifically:
创建n*n背景窗口,n的取值为8,16,32,64或128,可根据视频分辨率大小进行调整;Create n*n background windows, the value of n is 8, 16, 32, 64 or 128, which can be adjusted according to the video resolution;
使用所述背景窗口分别在每一帧所述主摄像头感兴趣区域和所述参考摄像头感兴趣区域的补集区域逐行逐列移动,每次取出背景窗口覆盖的临时像素矩阵,分别记为TempArray和TempArray_ref,分别计算其U通道矩阵的方差U_Sigma和U_Sigma_ref;Use the background window to move row-by-row and column-by-column in the complement area of the main camera ROI and the reference camera ROI in each frame, and take out the temporary pixel matrix covered by the background window each time, which is respectively recorded as TempArray and TempArray_ref, respectively calculate the variance U_Sigma and U_Sigma_ref of its U channel matrix;
若U_Sigma>预设方差阈值Tr,则继续移动背景窗口,重新计算,否则将当前TempArray作为当前帧主摄像头背景区域,提取当前帧所述主摄像头背景区域的U通道U_back(i),每一帧所述主摄像头背景区域的U通道构成序列U_back(1),U_back(2),…,U_back(K);If U_Sigma>preset variance threshold Tr, then continue to move the background window and recalculate, otherwise use the current TempArray as the background area of the main camera in the current frame, and extract the U channel U_back(i) of the background area of the main camera in the current frame, each frame The U channel in the background area of the main camera constitutes a sequence U_back(1), U_back(2),..., U_back(K);
若U_Sigma_ref>预设方差阈值Tr,则继续移动背景窗口,重新计算,否则将当前TempArray_ref作为当前帧参考摄像头背景区域,提取当前帧所述参考摄像头背景区域的U通道U_back_ref(i),每一帧所述参考摄像头背景区域的U通道构成序列U_back_ref(1),U_back_ref(2),…,U_back_ref(K);If U_Sigma_ref>preset variance threshold Tr, continue to move the background window and recalculate; otherwise, use the current TempArray_ref as the reference camera background area of the current frame, and extract the U channel U_back_ref(i) of the reference camera background area in the current frame, each frame The U channel of the background area of the reference camera constitutes a sequence U_back_ref(1), U_back_ref(2),..., U_back_ref(K);
所述步骤(4)具体为:Described step (4) is specifically:
计算当前帧所述参考摄像头感兴趣区域二维U通道矩阵U_ref(i)的均值m_ref(i)和标准差σ(i),计算U_ref(i)与m_ref(i)的差值矩阵,统计差值矩阵中绝对值超过3*σ(i)的像素数,记为Num(i);Calculate the mean value m_ref(i) and standard deviation σ(i) of the two-dimensional U channel matrix U_ref(i) of the reference camera region of interest in the current frame, calculate the difference matrix between U_ref(i) and m_ref(i), and calculate the difference The number of pixels whose absolute value exceeds 3*σ(i) in the value matrix is denoted as Num(i);
对当前帧所述主摄像头感兴趣区域的二维U通道矩阵U(i)中U通道值由小到大进行排序,去掉Num(i)/2个最大值,去掉Num(i)/2个最小值,计算出剩余U通道值的平均值m(i),作为当前帧所述感兴趣区域二维U通道矩阵特征值;Sort the U channel values in the two-dimensional U channel matrix U(i) of the main camera region of interest in the current frame from small to large, remove Num(i)/2 maximum values, and remove Num(i)/2 The minimum value calculates the average value m (i) of the remaining U channel values, as the two-dimensional U channel matrix eigenvalue of the region of interest described in the current frame;
计算当前帧所述参考摄像头背景区域二维U通道矩阵的均值m_back_ref(i)和标准差σ_ref(i),计算U_back_ref(i)与m_back_ref(i)的差值矩阵,统计差值矩阵中绝对值超过3*σ_ref(i)的像素数,记为Num_back(i);Calculate the mean m_back_ref(i) and standard deviation σ_ref(i) of the two-dimensional U channel matrix of the reference camera background area in the current frame, calculate the difference matrix between U_back_ref(i) and m_back_ref(i), and count the absolute value in the difference matrix The number of pixels exceeding 3*σ_ref(i) is recorded as Num_back(i);
对当前帧所述主摄像头背景区域的二维U通道矩阵U_back(i)中U通道值由小到大进行排序,去掉Num_back(i)/2个最大值,去掉Num_back(i)/2个最小值,计算出剩余U通道值的平均值m_back(i),作为当前帧所述背景区域二维U通道矩阵特征值;Sort the U channel values in the two-dimensional U channel matrix U_back(i) of the main camera background area in the current frame from small to large, remove Num_back(i)/2 maximum values, and remove Num_back(i)/2 minimum values value, calculate the mean value m_back(i) of the remaining U channel values, as the two-dimensional U channel matrix eigenvalue of the background area of the current frame;
根据环境光变化剧烈程度设置噪声去除因子,记为β,其中β的取值范围为0~1,环境光变化越剧烈,β值越大,计算U_character(i)=m(i)-β*m_back(i),作为第i帧的特征值;Set the noise removal factor according to the severity of the ambient light change, denoted as β, where the value range of β is 0 to 1, the more severe the ambient light change, the larger the β value, calculate U_character(i)=m(i)-β* m_back(i), as the feature value of the i-th frame;
每一帧特征值组成所述时间轴上的一维特征序列U_character(1),U_character(2),…,U_character(K)。The feature values of each frame form a one-dimensional feature sequence U_character(1), U_character(2), . . . , U_character(K) on the time axis.
所述步骤(5)具体为:Described step (5) is specifically:
采用P阶3Hz巴特沃斯滤波器对所述一维特征序列U_character(1),U_character(2),…,U_character(K)滤波,得到长度为J的滤波序列Fil(1),Fil(2),…,Fil(J);Use a P-order 3Hz Butterworth filter to filter the one-dimensional feature sequence U_character(1), U_character(2),..., U_character(K) to obtain a length J filter sequence Fil(1), Fil(2) ,...,Fil(J);
其中,J=K+P-1;Among them, J=K+P-1;
所述步骤(6)具体为:Described step (6) is specifically:
扫描所述长度为J的滤波序列Fil(1),Fil(2),…,Fil(J),寻找峰值:所述Fil(1)不做计算处理;所述Fil(2),Fil(3),…,Fil(J-1)与相邻两点进行比较,若该点值均比相邻两点大,则认为该点为一个峰值;即:若Fil(j)>Fil(j-1)且Fil(j)>Fil(j+1),则Fil(j)为一个峰值点;Scan the filter sequence Fil(1), Fil(2),...,Fil(J) whose length is J, and find the peak value: the Fil(1) does not perform calculation processing; the Fil(2), Fil(3 ),…, Fil(J-1) is compared with two adjacent points, if the value of this point is larger than the two adjacent points, then this point is considered as a peak value; that is: if Fil(j)>Fil(j- 1) and Fil(j)>Fil(j+1), then Fil(j) is a peak point;
统计所述20s内峰值点个数,记为Peak_20s;Count the number of peak points within the 20s, which is recorded as Peak_20s;
计算Heart_Rate=Peak_20s*3,即所述20s对应的实时心率值。Calculate Heart_Rate=Peak_20s*3, that is, the real-time heart rate value corresponding to the 20s.
所述步骤(7)具体为:Described step (7) is specifically:
丢弃所述20s中的前5s数据Umid(1),Umid(2),…Umid(L1),其中L1=fs*5,后15s数据组成临时序列Umid(1),Umid(2),…Umid(L2),其中L2=fs*15;Discard the first 5s data Umid(1), Umid(2),...Umid(L1) in the 20s, where L1=fs*5, and the last 15s data form a temporary sequence Umid(1), Umid(2),...Umid (L2), wherein L2=fs*15;
更新采集5s数据,补充在所述临时序列Umid(1),Umid(2),…Umid(L2)之后,从而实现序列Umid(1),Umid(2),…Umid(K)的更新,K=L1+L2,重复步骤(2)~(6),即得到实时心率值。Update and collect 5s data, supplement after the temporary sequence Umid(1), Umid(2),...Umid(L2), so as to realize the update of the sequence Umid(1), Umid(2),...Umid(K), K =L1+L2, repeat steps (2)~(6) to get the real-time heart rate value.
本发明提供的技术方案的有益效果是:本发明所提出的一种基于双摄像头的实时心率检测方法,所述方法,在信号提取上根据人脸特征确定特定感兴趣区域范围,提高信噪比,根据背景特征确定背景区域范围,增加数据稳定度;在特征值提取上,通过参考帧提供的帧内噪声参数降低帧内噪声,通过背景区域提供的帧间噪声参数降低环境变化的影响,从而提高心率值计算的准确性。本发明适用于司机等易疲劳人群的实时心率检测场景。本发明中提出的方法降低了由于环境光干扰引起的信号失真和抖动,大大提高了非接触式心率检测中的准确度和稳定性。The beneficial effects of the technical solution provided by the present invention are: a real-time heart rate detection method based on dual cameras proposed by the present invention, the method determines the range of a specific region of interest according to the facial features in signal extraction, and improves the signal-to-noise ratio , determine the range of the background area according to the background features, and increase the stability of the data; in terms of feature value extraction, the intra-frame noise parameters provided by the reference frame are used to reduce the intra-frame noise, and the inter-frame noise parameters provided by the background area are used to reduce the impact of environmental changes, thereby Improve the accuracy of heart rate value calculation. The present invention is applicable to real-time heart rate detection scenarios of fatigue-prone people such as drivers. The method proposed in the invention reduces signal distortion and jitter caused by ambient light interference, and greatly improves the accuracy and stability in non-contact heart rate detection.
附图说明Description of drawings
图1为本发明流程图;Fig. 1 is a flowchart of the present invention;
图2为本发明提取主摄像头感兴趣区域和参考摄像头感兴趣区域U通道序列流程图;Fig. 2 is the U channel sequence flowchart of the present invention extracting main camera ROI and reference camera ROI;
图3为本发明提取主摄像头背景区域和参考摄像头背景区域U通道序列流程图;Fig. 3 is the U channel sequence flowchart of the present invention extracting main camera background area and reference camera background area;
图4为本发明基于双摄像头的算法流程图;Fig. 4 is the algorithm flowchart based on dual camera of the present invention;
图5为本发明采用3Hz巴特沃斯低通滤波器对特征序列进行滤波的流程图;Fig. 5 is the flow chart that the present invention adopts 3Hz Butterworth low-pass filter to filter feature sequence;
图6为本发明采用峰值计数法计算心率值的流程图;Fig. 6 is the flow chart that the present invention adopts peak counting method to calculate heart rate value;
图7为本发明实时心率更新的流程图。Fig. 7 is a flow chart of real-time heart rate update in the present invention.
具体实施方式Detailed ways
为了更加清楚明白地说明本发明所述的技术手段、技术改进及有益效果,以下结合附图对本发明进行详细的说明。In order to more clearly illustrate the technical means, technical improvements and beneficial effects of the present invention, the present invention will be described in detail below in conjunction with the accompanying drawings.
实施例Example
参阅图1-7,本实施例包括以下步骤:Referring to Fig. 1-7, present embodiment comprises the following steps:
S101:打开主摄像头和参考摄像头,以相同的帧率交替拍摄20s包含人脸的主视频和参考视频。以30岁健康女性为例进行说明。S101: Turn on the main camera and the reference camera, and alternately shoot the main video and the reference video including the human face for 20 seconds at the same frame rate. Take a 30-year-old healthy woman as an example for illustration.
该步骤具体为:The steps are specifically:
创建所述主摄像头对象和参考摄像头对象,设置相同帧率fs=20Hz;Create the main camera object and reference camera object, set the same frame rate fs=20Hz;
设置帧计数器frame_counter=0;set frame counter frame_counter = 0;
先打开参考摄像头,经过Δt=0.025s时间后打开主摄像头,并均以20Hz的帧率拍摄包含人脸的视频20s,分别记参考摄像头当前帧和主摄像头当前帧为Frame_ref(i)和Frame(i),frame_counter=K=400。Turn on the reference camera first, then turn on the main camera after Δt=0.025s, and shoot videos containing human faces at a frame rate of 20Hz for 20s, record the current frame of the reference camera and the current frame of the main camera as Frame_ref(i) and Frame( i), frame_counter=K=400.
S102:创建两个人脸检测器,分别获取主摄像头感兴趣区域和参考摄像头感兴趣区域,基于每一帧RGB数据,提取U通道数据,并获得主摄像头感兴趣区域和参考摄像头感兴趣区域的U通道序列,参阅图2。S102: Create two face detectors to obtain the ROI of the main camera and the ROI of the reference camera respectively, extract the U channel data based on the RGB data of each frame, and obtain the U of the ROI of the main camera and the ROI of the reference camera Channel sequence, see Figure 2.
该步骤具体为:The steps are specifically:
利用维奥拉-琼斯算法创建所述人脸检测器,分别获得人脸主区域及人脸参考区域的起始点坐标(x,y)、(x_ref,y_ref)和人脸尺寸(w,h)、(w_ref,h_ref);例如,人脸主区域起始点坐标为(400,500)、人脸尺寸为(100,200),人脸参考区域的起始点坐标(450,550)、人脸尺寸为(100,200)。Use the Viola-Jones algorithm to create the face detector, and obtain the starting point coordinates (x, y), (x_ref, y_ref) and face size (w, h) of the main area of the face and the reference area of the face respectively , (w_ref, h_ref); For example, the starting point coordinates of the main face area are (400, 500), the face size is (100, 200), the starting point coordinates of the face reference area are (450, 550), the face size is (100, 200).
根据人脸比例,截取Frame(i)高度范围为500~540,宽度范围为510~530的区域,即主摄像头感兴趣区域Interest(i),截取Frame_ref(i)高度范围为550~590,宽度范围为560~580的区域,即参考摄像头感兴趣区域Interest_ref(i),获得当前帧所述主摄像头感兴趣区域和所述参考摄像头感兴趣区域的RGB通道;According to the proportion of the face, intercept the area of Frame(i) whose height ranges from 500 to 540 and whose width ranges from 510 to 530, that is, the area of interest of the main camera Interest(i), and intercept Frame_ref(i) whose height range is from 550 to 590 and whose width The area with a range of 560 to 580, that is, the area of interest of the reference camera Interest_ref(i), obtains the RGB channels of the area of interest of the main camera and the area of interest of the reference camera in the current frame;
将所述主摄像头感兴趣区域和所述参考摄像头感兴趣区域的RGB通道转换为YUV通道,分别提取二维U通道U(i)、U_ref(i);U通道计算公式如下:The RGB channels of the main camera region of interest and the reference camera region of interest are converted into YUV channels, and two-dimensional U channels U(i), U_ref(i) are extracted respectively; the U channel calculation formula is as follows:
U=-0.169*R-0.331*G+0.5*BU=-0.169*R-0.331*G+0.5*B
每一帧所述主摄像头感兴趣区域的U通道构成序列U(1),U(2),…,U(400)、每一帧所述参考摄像头感兴趣区域的U通道构成序列U_ref(1),U_ref(2),…,U_ref(400)。The U channel of the region of interest of the main camera in each frame constitutes a sequence U(1), U(2),..., U(400), and the U channel of the region of interest of the reference camera in each frame constitutes a sequence U_ref(1 ), U_ref(2), ..., U_ref(400).
S103:截取主摄像头背景区域和参考摄像头背景区域,并获得主摄像头背景区域和参考摄像头背景区域的U通道序列,参阅图3。S103: Intercept the background area of the main camera and the background area of the reference camera, and obtain the U channel sequence of the background area of the main camera and the background area of the reference camera, see FIG. 3 .
该步骤具体为:The steps are specifically:
创建8*8背景窗口;Create an 8*8 background window;
使用所述背景窗口分别在每一帧所述主摄像头感兴趣区域和所述参考摄像头感兴趣区域的补集区域逐行逐列移动,每次取出背景窗口覆盖的临时像素矩阵,分别记为TempArray和TempArray_ref,分别计算其U通道矩阵的方差U_Sigma和U_Sigma_ref;Use the background window to move row-by-row and column-by-column in the complement area of the main camera ROI and the reference camera ROI in each frame, and take out the temporary pixel matrix covered by the background window each time, which is respectively recorded as TempArray and TempArray_ref, respectively calculate the variance U_Sigma and U_Sigma_ref of its U channel matrix;
若U_Sigma>10,则继续移动背景窗口,重新计算;否则将当前TempArray作为当前帧主摄像头背景区域,提取当前帧所述主摄像头背景区域的U通道U_back(i),每一帧所述主摄像头背景区域的U通道构成序列U_back(1),U_back(2),…,U_back(400);If U_Sigma>10, continue to move the background window and recalculate; otherwise, use the current TempArray as the background area of the main camera in the current frame, extract the U channel U_back(i) of the background area of the main camera in the current frame, and the main camera in each frame The U channel in the background area constitutes the sequence U_back(1), U_back(2),..., U_back(400);
若U_Sigma_ref>10,则继续移动背景窗口,重新计算;否则将当前TempArray_ref作为当前帧参考摄像头背景区域,提取当前帧所述参考摄像头背景区域的U通道U_back_ref(i),每一帧所述参考摄像头背景区域的U通道构成序列U_back_ref(1),U_back_ref(2),…,U_back_ref(400);If U_Sigma_ref>10, continue to move the background window and recalculate; otherwise, use the current TempArray_ref as the reference camera background area of the current frame, extract the U channel U_back_ref(i) of the reference camera background area in the current frame, and refer to the reference camera in each frame The U channel in the background area constitutes the sequence U_back_ref(1), U_back_ref(2),..., U_back_ref(400);
S104:基于双摄像头计算每一帧感兴趣区域和背景区域特征值,得到时间轴上的一维特征序列,参阅图4,图中,a图为U_ref(i)均值及标准差计算,b图为U_back_ref(i)均值及标准差计算,c图为第i帧的特征值计算。S104: Calculate the eigenvalues of the region of interest and the background region of each frame based on the dual cameras, and obtain the one-dimensional feature sequence on the time axis, refer to Fig. 4, in the figure, a is the calculation of the mean and standard deviation of U_ref(i), and b is the calculation It is the calculation of the mean and standard deviation of U_back_ref(i), and the figure c is the calculation of the feature value of the i-th frame.
该步骤具体为:The steps are specifically:
计算当前帧所述参考摄像头感兴趣区域二维U通道矩阵的均值m_ref(i)和标准差σ(i),例如,m_ref(i)=133,σ(i)=6;获得U_ref(i)与m_ref(i)的差值矩阵,统计差值矩阵中绝对值超过3*σ(i)=18的像素数,记为Num(i),例如,Num(i)=10;Calculate the mean value m_ref(i) and standard deviation σ(i) of the two-dimensional U channel matrix of the reference camera region of interest in the current frame, for example, m_ref(i)=133, σ(i)=6; obtain U_ref(i) For the difference matrix with m_ref(i), the number of pixels whose absolute value exceeds 3*σ(i)=18 in the statistical difference matrix is recorded as Num(i), for example, Num(i)=10;
对当前帧所述主摄像头感兴趣区域的二维U通道矩阵U(i)中U通道值由小到大进行排序,去掉5个最大值,去掉5个最小值,计算出U(i)的平均值m(i),作为当前帧所述感兴趣区域二维U通道矩阵特征值,例如,m(i)=130;Sort the U channel values in the two-dimensional U channel matrix U(i) of the area of interest of the main camera in the current frame from small to large, remove 5 maximum values, remove 5 minimum values, and calculate U(i) Average value m(i), as the eigenvalue of the two-dimensional U channel matrix of the region of interest in the current frame, for example, m(i)=130;
计算当前帧所述参考摄像头背景区域二维U通道矩阵的均值m_back_ref(i)和标准差σ_ref(i),例如,m_back_ref(i)=212,σ_ref(i)=2;获得U_back_ref(i)与m_back_ref(i)的差值矩阵,统计差值矩阵中绝对值超过3*σ_ref(i)的像素数,记为Num_back(i),例如,Num_back(i)=4;Calculate the mean m_back_ref(i) and standard deviation σ_ref(i) of the two-dimensional U channel matrix of the reference camera background area in the current frame, for example, m_back_ref(i)=212, σ_ref(i)=2; obtain U_back_ref(i) and The difference matrix of m_back_ref(i), the number of pixels whose absolute value exceeds 3*σ_ref(i) in the statistical difference matrix, is recorded as Num_back(i), for example, Num_back(i)=4;
对当前帧所述主摄像头背景区域的二维U通道矩阵U_back(i)中U通道值由小到大进行排序,去掉2个最大值,去掉2个最小值,计算出U_back(i)的平均值m_back(i),作为所述背景区域二维U通道矩阵特征值,例如,m_back(i)=210;Sort the U channel values in the two-dimensional U channel matrix U_back(i) of the main camera background area in the current frame from small to large, remove 2 maximum values, remove 2 minimum values, and calculate the average of U_back(i) Value m_back(i), as the eigenvalue of the two-dimensional U channel matrix of the background area, for example, m_back(i)=210;
根据环境光变化剧烈程度设置噪声去除因子,记为β=0.5,计算U_character(i)=130-0.5*210=25,作为第i帧的特征值;Set the noise removal factor according to the severity of the ambient light change, denoted as β=0.5, calculate U_character(i)=130-0.5*210=25, as the feature value of the i-th frame;
每一帧特征值组成所述时间轴上的一维特征序列U_character(1),U_character(2),…,U_character(400)。The feature values of each frame form a one-dimensional feature sequence U_character(1), U_character(2), . . . , U_character(400) on the time axis.
S105:采用低通滤波器对一维特征序列进行滤波,将滤波后的一维特征序列作为心率信号,参阅图5。S105: Use a low-pass filter to filter the one-dimensional feature sequence, and use the filtered one-dimensional feature sequence as the heart rate signal, see FIG. 5 .
该步骤具体为:The steps are specifically:
采用40阶3Hz巴特沃斯滤波器对所述一维特征序列U_character(1),U_character(2),…,U_character(400)进行滤波,得到长度为439的滤波序列Fil(1),Fil(2),…,Fil(439);A 40-order 3Hz Butterworth filter is used to filter the one-dimensional feature sequence U_character(1), U_character(2),..., U_character(400) to obtain a filter sequence Fil(1), Fil(2) with a length of 439 ),...,Fil(439);
S106:采用峰值计数法得到心跳次数,计算心率值,参阅图6。S106: Use the peak counting method to obtain the number of heartbeats, and calculate the heart rate value, see FIG. 6 .
该步骤具体为:The steps are specifically:
扫描所述长度为J的滤波序列Fil(1),Fil(2),…,Fil(439),寻找峰值:所述Fil(1)不做计算处理;所述Fil(2),Fil(3),…,Fil(438)与相邻两点进行比较,若该点值均比相邻两点大,则认为该点为一个峰值;即:若Fil(j)>Fil(j-1)且Fil(j)>Fil(j+1),则Fil(j)为一个峰值点;Scan the filter sequence Fil(1), Fil(2),...,Fil(439) whose length is J, and find the peak value: the Fil(1) does not perform calculation processing; the Fil(2), Fil(3 ),…, Fil(438) are compared with two adjacent points, if the value of this point is larger than the two adjacent points, then this point is considered to be a peak; that is: if Fil(j)>Fil(j-1) And Fil(j)>Fil(j+1), then Fil(j) is a peak point;
统计所述20s内峰值点个数,记为Peak_20s,例如Peak_20s=28;Count the number of peak points within the 20s, recorded as Peak_20s, for example Peak_20s=28;
计算所述20s对应的实时心率值Heart_Rate=Peak_20s*3=84,即该人在所述20s的实时心率值为84次/分。Calculate the real-time heart rate value Heart_Rate=Peak_20s*3=84 corresponding to the 20s, that is, the real-time heart rate value of the person in the 20s is 84 beats/min.
S107:实时采集数据更新信号序列,重复(2)-(6)步骤计算实时心率值,参阅图7。S107: Collect data in real time to update the signal sequence, and repeat steps (2)-(6) to calculate the real-time heart rate value, see FIG. 7 .
丢弃所述20s中的前5s数据Umid(1),Umid(2),…Umid(100),后15s数据组成临时序列Umid(1),Umid(2),…Umid(300);Discard the first 5s data Umid(1), Umid(2),...Umid(100) in the 20s, and the latter 15s data form a temporary sequence Umid(1), Umid(2),...Umid(300);
更新采集5s数据,补充在所述临时序列Umid(1),Umid(2),…Umid(300)之后,从而实现序列Umid(1),Umid(2),…Umid(400)的更新,重复步骤(2)~(6),即得到实时心率值。Update and collect 5s data, supplement after the temporary sequence Umid(1), Umid(2),...Umid(300), so as to realize the update of the sequence Umid(1), Umid(2),...Umid(400), repeat Steps (2) to (6), namely to obtain the real-time heart rate value.
以上所述仅为本发明的较佳实施例,并不用以限制本发明,凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910802268.XA CN110353648A (en) | 2019-08-28 | 2019-08-28 | A kind of real-time heart rate detection method based on dual camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910802268.XA CN110353648A (en) | 2019-08-28 | 2019-08-28 | A kind of real-time heart rate detection method based on dual camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110353648A true CN110353648A (en) | 2019-10-22 |
Family
ID=68225432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910802268.XA Pending CN110353648A (en) | 2019-08-28 | 2019-08-28 | A kind of real-time heart rate detection method based on dual camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110353648A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114220152A (en) * | 2021-12-17 | 2022-03-22 | 嘉兴学院 | Non-contact real-time heart rate detection method based on RGB-NIR camera |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103826532A (en) * | 2011-08-22 | 2014-05-28 | Isis创新有限公司 | Remote monitoring of vital signs |
| WO2017125763A1 (en) * | 2016-01-21 | 2017-07-27 | Oxehealth Limited | Method and apparatus for estimating heart rate |
| CN107506716A (en) * | 2017-08-17 | 2017-12-22 | 华东师范大学 | A kind of contactless real-time method for measuring heart rate based on video image |
| CN109512418A (en) * | 2018-10-22 | 2019-03-26 | 哈尔滨理工大学 | A kind of volumetric blood flow pulse imaging reduction motion artifacts method |
-
2019
- 2019-08-28 CN CN201910802268.XA patent/CN110353648A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103826532A (en) * | 2011-08-22 | 2014-05-28 | Isis创新有限公司 | Remote monitoring of vital signs |
| WO2017125763A1 (en) * | 2016-01-21 | 2017-07-27 | Oxehealth Limited | Method and apparatus for estimating heart rate |
| CN107506716A (en) * | 2017-08-17 | 2017-12-22 | 华东师范大学 | A kind of contactless real-time method for measuring heart rate based on video image |
| CN109512418A (en) * | 2018-10-22 | 2019-03-26 | 哈尔滨理工大学 | A kind of volumetric blood flow pulse imaging reduction motion artifacts method |
Non-Patent Citations (2)
| Title |
|---|
| 印辉云: "基于FPGA的指纹识别系统", 《中国优秀博硕士学位论文全文数据库(硕士)》 * |
| 桑海峰等: "基于光照矫正的面部局部区域心率检测", 《计算机应用》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114220152A (en) * | 2021-12-17 | 2022-03-22 | 嘉兴学院 | Non-contact real-time heart rate detection method based on RGB-NIR camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Remote heart rate measurement from face videos under realistic situations | |
| US11672436B2 (en) | Pulse detection from head motions in video | |
| Wang et al. | A comparative survey of methods for remote heart rate detection from frontal face videos | |
| US9443289B2 (en) | Compensating for motion induced artifacts in a physiological signal extracted from multiple videos | |
| US9436984B2 (en) | Compensating for motion induced artifacts in a physiological signal extracted from a single video | |
| CN110662480B (en) | Device, system and method for measuring and processing physiological signals of a subject | |
| Chen et al. | Respiratory rate estimation from face videos | |
| US9928607B2 (en) | Device and method for obtaining a vital signal of a subject | |
| CN103908236B (en) | An automatic blood pressure measurement system | |
| US9521954B2 (en) | Video acquisition system for monitoring a subject for a desired physiological function | |
| CN108272448B (en) | A non-contact day and night monitoring method for infant physiological parameters | |
| Hassan et al. | Towards health monitoring using remote heart rate measurement using digital camera: A feasibility study | |
| CN110647815A (en) | Non-contact heart rate measurement method and system based on face video image | |
| US20160317041A1 (en) | System and methods for measuring physiological parameters | |
| CN112233813A (en) | A non-contact non-invasive heart rate and respiration measurement method and system based on PPG | |
| CN107358220A (en) | A kind of human heart rate and the contactless measurement of breathing | |
| CN106778695A (en) | A kind of many people's examing heartbeat fastly methods based on video | |
| Banerjee et al. | Noise cleaning and Gaussian modeling of smart phone photoplethysmogram to improve blood pressure estimation | |
| WO2013165550A1 (en) | Methods and systems for measuring dynamic changes in the physiological parameters of a subject | |
| WO2011127487A2 (en) | Method and system for measurement of physiological parameters | |
| WO2019030124A1 (en) | Device, system and method for determining a physiological parameter of a subject | |
| CN114387479A (en) | A non-contact heart rate measurement method and system based on face video | |
| JP6620999B2 (en) | Biological information measuring device, biological information measuring program, and biological information measuring method | |
| CN111714105A (en) | Human vital signs perception system based on IPPG | |
| CN112294282A (en) | Self-calibration method of emotion detection device based on RPPG |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20191022 |
|
| WD01 | Invention patent application deemed withdrawn after publication |