CN110674697B - Filtering method, device and related product - Google Patents

Abstract

The invention discloses a filtering method, a filtering device and related products. The filtering method comprises the following steps: acquiring one or more correlation peaks obtained by carrying out phase correlation calculation on a frame sequence corresponding to a detection video, wherein the correlation peaks are frequency domain signals; determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak; and carrying out filtering treatment on each correlation peak according to a filtering strategy corresponding to each correlation peak. By selecting a proper filtering strategy according to the position of each correlation peak and the corresponding frequency band to carry out filtering treatment, noise information in each correlation peak can be reduced, so that the signal-to-noise ratio of a frequency signal obtained from a detection video is higher, and the running condition of a device to be detected obtained according to the detection video is more accurate.

Description

Filtering method, device and related products

technical field

The invention relates to the technical field of detection, in particular to a filtering method, device and related products.

Background technique

In related technologies, vibration can reflect the operating conditions of certain mechanical structures. It is possible to take a video of the vibration device during operation, then extract the vibration signal from the video, and then obtain the operating status of the device according to the vibration signal. However, when shooting the video, A variety of noise information will be introduced due to the influence of the environment, and the vibration signal extracted from the video also contains a large number of noise signals, which makes it impossible to accurately obtain the operating status of the device.

Contents of the invention

Embodiments of the present invention provide a filtering method, device and related products.

In a first aspect, an embodiment of the present invention provides a filtering method, including:

Obtaining one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detected video, where the correlation peaks are frequency domain signals;

Determine the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band of the correlation peak;

Each correlation peak is filtered according to the filtering strategy corresponding to each correlation peak.

In some embodiments, the filtering strategy includes a filtering bandwidth, and determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak includes:

Determine the initial filtering bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;

Determine the bandwidth adjustment coefficient corresponding to the correlation peak according to the corresponding position of each correlation peak in the detection video;

The product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient is used as the filter bandwidth corresponding to each correlation peak.

In some embodiments, the determining the bandwidth adjustment coefficient corresponding to the correlation peak according to the corresponding position of each correlation peak in the detection video includes:

Obtain the preset bandwidth adjustment coefficient corresponding to the region where the corresponding position of each correlation peak in the detected video is located, and use the preset bandwidth adjustment coefficient as the bandwidth adjustment coefficient corresponding to the correlation peak.

In some embodiments, before determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak, the filtering method further includes:

Use the principal component decomposition method to reduce the dimensionality of each correlation peak;

According to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak, the filtering strategy corresponding to each correlation peak is determined to include:

The filtering strategy corresponding to each correlation peak is determined according to the corresponding position of each correlation peak after the dimensionality reduction processing in the detection video and the frequency band corresponding to the correlation peak.

In some embodiments, before performing dimensionality reduction processing on each correlation peak by using the principal component decomposition method, the filtering method further includes:

Comparing the peak value of each correlation peak with the preset peak range, respectively judging whether the peak value of each correlation peak is within the preset peak range;

The method of utilizing the principal component decomposition method to carry out dimensionality reduction processing on each correlation peak includes:

Dimensionality reduction is performed on related peaks whose peaks are within the preset peak range.

In some embodiments, the filtering processing of each correlation peak according to the filtering strategy corresponding to each correlation peak includes:

According to the filtering strategy corresponding to each correlation peak, filter processing is performed on each correlation peak by way of interpolation filtering.

In a second aspect, the present invention also provides a filtering device, including:

The first acquisition module is used to acquire one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detected video, and the correlation peaks are frequency domain signals;

A filter strategy determination module, configured to determine the filter strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak;

The filtering module is configured to perform filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak.

In the third aspect, the embodiment of the present invention also provides an electronic device, including a processor, a memory, and an information recommendation program stored on the memory and executable by the processor, wherein the information recommendation program is executed by the When executed by a processor, the instructions to implement the steps in the filtering method described in any one of the above embodiments.

In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a filtering program is stored, wherein when the filtering program is executed by a processor, the filtering method described in any one of the above is implemented.

In a fifth aspect, the present invention further provides a detection device for detecting the operation status of the device to be detected, the detection device includes the filter device of the above embodiment or the electronic device of the above embodiment.

The filtering method of the embodiment of the present invention obtains one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detection video; and determines each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak Corresponding filtering strategy: filter each correlation peak according to the filtering strategy corresponding to each correlation peak. By selecting an appropriate filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, the noise information in each correlation peak can be reduced, so that the signal-to-noise ratio of the state change signal obtained from the detection video is higher, so that according to The operation status of the device to be detected obtained by detecting the video is more accurate.

Description of drawings

Some drawings related to the embodiments of the present invention will be described below.

FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a filtering method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of the process of zooming in and detecting video involved when the filtering method according to an embodiment of the present invention is used to detect the operating status of the device to be detected;

FIG. 4 is another schematic flowchart of a filtering method according to an embodiment of the present invention;

FIG. 5 is another schematic flowchart of a filtering method according to an embodiment of the present invention;

FIG. 6 is another schematic flowchart of a filtering method according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.

Please refer to FIG. 1 . FIG. 1 is a schematic diagram of a hardware structure of an electronic device 100 provided by an embodiment of the present invention. The electronic device 100 includes a processor 101, a memory 102, an input and output interface 103, and one or more programs, one or more programs are stored in the memory 102, and are configured to be executed by the processor 101, and the programs include any of the following Instructions for the steps of the filtering method of an embodiment. The electronic device 100 may be a server device or a terminal device. The memory 102 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory, and the memory 102 may optionally be a storage device independent of the aforementioned processor 101. Optionally, the input and output interface 103 may include a USB interface, a standard wired interface, a wireless interface (such as a WI-FI interface), and the like.

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of a filtering method provided by an embodiment of the present invention. This method may include but not limited to the following steps:

01. Obtain one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detected video, and the correlation peaks are frequency domain signals;

The filtering method of the embodiment of the present invention can be used for but not limited to detecting the vibration of the device to be detected. The vibration condition of the device to be detected can reflect the operation status of the device to be detected. The inspection video can be a video obtained by shooting the device to be inspected by the imaging module. By analyzing the detection video, the vibration information of the device to be detected can be extracted from the detection video, and then the vibration information can be analyzed to obtain the operation status of the device to be detected.

In a preferred embodiment, the detection video is a vibration amplified video after the vibration in the video is amplified by using the Euler algorithm to capture the video of the device to be detected by the imaging module. The devices to be tested include but are not limited to various types of mechanical equipment, building structures, etc. After the detected video is obtained, phase correlation calculation is performed on the frame sequence corresponding to the detected video to obtain a cross-cross power spectrum, and the cross-cross power spectrum includes one or more correlation peaks. It can be understood that the correlation peak is a frequency domain signal, and each correlation peak can reflect the severity of the state change of a certain frequency band at a certain position in the detection video, and the phase correlation calculation is performed on the frame sequence corresponding to the detection video to obtain the cross cross power spectrum. It is understood that the state change information in the video picture is extracted from the detected video. The status change information includes vibration information and other noise information, and the vibration information can reflect the operating status of the device to be detected. The filtering method in the embodiment of the present invention can be used to filter each correlation peak in the cross-cross power spectrum to reduce noise information in each correlation peak, so that the operating status of the device to be detected obtained from the detection video is more accurate.

02. Determine the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak;

Each correlation peak can be analyzed to obtain the corresponding position of each correlation peak in the detection video, and the corresponding position of the correlation peak in the detection video can be understood as the position in the detection video corresponding to the state change reflected by the correlation peak. For example, the brightness of a position in the detection video changes periodically with time, and one or more correlation peaks obtained by performing phase correlation calculation on the corresponding frame sequence of the detection video include at least one correlation peak to reflect the position brightness changes.

In the detection video, vibrations at different locations have different effects on detecting the operating conditions of the device to be detected. For example, the vibration at the edge position in the picture of the detection video is less effective in detecting the operating condition of the device to be detected than the vibration at the middle position, so the corresponding positions of each correlation peak in the detection video can be used as the basis, Determine the filtering strategy for each correlation peak. The frequency range (ie, frequency band) corresponding to each correlation peak is different, and the filtering strategy of each correlation peak is determined according to the frequency band corresponding to the correlation peak, so that the filtering strategy is more suitable for each correlation peak, thus helping to obtain the corresponding frequency range from each correlation peak. Extract useful state change signals from . The filtering strategy may include, but not limited to, the bandwidth of the filter, the type of the filter, and the like, for example.

03. Filter each correlation peak according to the filtering strategy corresponding to each correlation peak.

After the filtering strategy of each correlation peak is obtained, each correlation peak is filtered according to the corresponding filtering strategy, so that the signal-to-noise ratio of the signal obtained from the detection video is higher. Preferably, each correlation peak can be filtered by means of interpolation filtering according to the filtering strategy corresponding to each correlation peak, so that the filtering effect is better.

The filtering method of the embodiment of the present invention obtains one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detection video; and determines each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak Corresponding filtering strategy: filter each correlation peak according to the filtering strategy corresponding to each correlation peak. By selecting an appropriate filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, the noise information in each correlation peak can be reduced, so that the signal-to-noise ratio of the state change signal obtained from the detection video is higher, so that according to The operation status of the device to be detected obtained by detecting the video is more accurate.

When detecting the running status of the device to be detected, vibration information needs to be extracted from the detection video including the picture of the detection device running. As shown in Figure 3, the frame sequence in the detected video can be converted from the RGB color space to the YIQ color space, and the brightness information and chrominance information of the video frame can be separated. The conversion relationship between RGB and YIQ is:

Y＝0.299*R+0.587*G+0.114*B;

I＝0.596*R-0.275*G-0.321*B;

Q=0.212*R−0.523*G+0.311*B.

Then keep the I and Q channels unchanged, perform FFT operation on the Y channel, and then use the Euler motion amplification algorithm to amplify the video data. It specifically includes: decomposing the Y-channel image after FFT transformation into a complex number controllable pyramid space domain to obtain a pyramid structure composed of multiple sub-images with different spatial resolutions; for each sub-image in the multiple sub-images in the pyramid structure The time-domain band-pass filtering process is performed to obtain the transformed signal corresponding to the target frequency band. It can be understood that in a video picture, vibration can be reflected by the brightness of the video frame sequence, then the vibration information in the detected video can be obtained by analyzing the Y channel in the detected video. Then amplify the converted signal corresponding to the target frequency band obtained after time-domain bandpass filtering and extract the motion information of interest, perform complex manipulation pyramid reconstruction on the motion information of interest, and obtain the enlarged Y channel image; finally, the reconstructed Y The channel image is added to the original I and Q channel images, and then converted into RGB color space to obtain the output video.

Then, the phase correlation algorithm is used to calculate the cross power spectrum between the frame sequences after video motion amplification processing. The phase correlation algorithm uses the following formula to calculate the cross-cross power spectrum.

为b帧图像的傅里叶变换的共轭信号，除式的下边为两个傅里叶变换的信号的相关积的模。R为本步骤的计算结果交叉互功率谱(包含频域噪音)。In the above formula, Fa is the Fourier transform of the frame a image,

is the conjugate signal of the Fourier transform of the b-frame image, and the lower side of the division formula is the modulus of the correlation product of the two Fourier transformed signals. R is the cross-cross power spectrum (including frequency domain noise) of the calculation result of this step.

Finally, according to the above-mentioned filtering method of the embodiment of the present invention, the correlation peaks in the cross-cross power spectrum are filtered to improve the signal-to-noise ratio of the useful signal, so that Information, the signal-to-noise ratio of the useful signal is higher.

Specifically, after performing pyramid decomposition on the image frame sequence of the target vibration video, a pyramid structure is obtained. The pyramid structure includes multiple sub-images, and the image resolution and spatial frequency decrease successively from top to bottom. The time-domain band-pass filtering process is performed on each layer of images to obtain the target frequency band, so that the sub-image resolution of the target frequency band can clearly express the motion characteristics of the image, and at the same time, the calculation amount will not be too large due to the high resolution. Therefore, according to the pyramid structure, the frequency bands of each sub-image at different spatial frequencies are obtained from bottom to top, and compared with the standard frequency bands in the order of acquisition. The frequency band acquisition and matching of the spatial frequency of one or more layers of sub-images above the layer of sub-images improves the efficiency of time-domain band-pass filtering. In addition, the maximum number of layers for pyramid decomposition is determined: log2(min(xres,yres)), where xres is the width pixel value of the image, and yres is the height pixel value of the image.

Further, performing time-domain band-pass filtering processing on each sub-image in the multiple sub-images in the pyramid structure to obtain a transformed signal corresponding to the target frequency band, specifically including:

Obtain the total number of layers N of the pyramid structure, wherein the number of layers of the pyramid structure increases sequentially from bottom to top, and the resolution of the corresponding sub-images increases sequentially;

Group multiple sub-images in the pyramid structure, where the first group includes sub-images corresponding to 2n+1 layers, and the second group includes sub-images corresponding to 2(n+1) layers, where n is an integer greater than or equal to 0 , N≥max(2(n+1),2n+1);

The first group of sub-images is processed by the first processor according to the number of layers from small to large, and the second group of sub-images is processed by the second processor according to the number of layers from small to large. The time-domain band-pass filtering process is to match the frequency bands of different spatial frequencies corresponding to the sub-images with the standard frequency bands, and the first processor and the second processor are independent processors;

When the first processor or the second processor determines that the frequency band matches the standard frequency band successfully, determine that the frequency band is the target frequency band;

Obtaining the sub-image corresponding to the target frequency band as the first target sub-image, and obtaining the sub-image above the first target sub-image as the second target sub-image;

Determine the first target sub-image and the second target sub-image as transformed signals corresponding to the target frequency band.

Specifically, after performing spatial pyramid decomposition on the frame sequence composed of multiple frames of images of the target vibration video, the total number of layers N of the pyramid structure can be obtained, and the resolutions of the corresponding sub-images of the pyramid structure from bottom to top 1-N layers are sequentially raised. Then group the N-layer sub-images, including the base group and the even group. The base group uses the first processor to perform time-domain band-pass filtering in the order of 1, 3, 5...2n+1, and the even group uses the second processor. Perform time-domain band-pass filter processing in the order of 2, 4, 6, ... 2(n+1), the two processors can start running at the same time, or have a certain processing time interval. In addition, time-domain band-pass filter processing In order to acquire the target frequency band, the acquisition method may be to match the frequency bands of different spatial frequencies corresponding to the sub-images with the standard frequency band. When the first processor or the second processor determines that the frequency band matches the standard frequency band successfully, determine that the frequency band is the target frequency band.

Because the target frequency band represents the spatial frequency of the lowest resolution sub-image that can reflect image vibration information, then determine the sub-image corresponding to the target frequency band as the first target sub-image, and then obtain the sub-image above the sub-image corresponding to the target frequency band as the second For the target sub-image, the first target sub-image and the second target sub-image are used as transformed signals corresponding to the target frequency band, and subsequently enlarged and transformed. In this way, the frame image corresponding to the target vibration video can be amplified more accurately, and more accurate motion information can be obtained, and at the same time, the increased computational load required for amplifying a higher-resolution image can be avoided.

In the embodiment of the present application, the sub-images corresponding to the pyramid structure are grouped, and then the sub-images of different groups are processed by time-domain band-pass filtering through two independently operating processors, which can improve the filtering processing efficiency, and at the same time, the acquired After the sub-image of the layer corresponding to the target frequency band is used as the first target sub-image, the sub-image of the upper layer is obtained as the second target sub-image, and then the first target sub-image and the second target sub-image are transformed signals corresponding to the target frequency band , so that more accurate zoom-in motion information can be obtained, and at the same time, the increased computation required for zooming in on higher-resolution images can be avoided.

Please refer to FIG. 4 , based on the above embodiments, in some embodiments, the filtering strategy includes filtering bandwidth, and the filtering strategy corresponding to each correlation peak is determined according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak, including but Not limited to the following steps:

021. Determine the initial filter bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;

It can be understood that the correlation peak obtained by performing phase correlation calculation on the frame sequence corresponding to the detected video includes similar multiple state change information of multiple pixels with similar positions in the picture corresponding to the detected video, so that multiple pixel points can be reflected The status of the location changes. Then, in the picture corresponding to the detection video, the state changes of different positions are different, and the frequency distributions of the corresponding correlation peaks are also different. If the same filter bandwidth is used for filtering, in order to avoid useful signals from being filtered, the filter bandwidth needs to be set to a larger value, but this may cause noise signals to be unable to be filtered. And in the filtering method of the present application, determine the corresponding initial filter bandwidth of each correlation peak according to the frequency band corresponding to each correlation peak, the frequency band corresponding to the correlation peak is positively correlated with the initial filter bandwidth, the larger the frequency band corresponding to the correlation peak, the corresponding initial filter bandwidth The wider it is, the better the filtering bandwidth can be adapted to each correlation peak, which can not only effectively filter the noise signal, but also help prevent the useful signal from being filtered, thus making the filtering effect better.

For example, B*(1-a) can be used as the initial filter bandwidth, B is the frequency band corresponding to each correlation peak, and a can be determined according to actual needs. When a better denoising effect is required, a can be set to a larger value value, when more useful signals need to be acquired, a can be set to a smaller value. Of course, in other embodiments, the method for setting the initial filtering bandwidth is not limited to the above example, and no limitation is made here.

022. Determine the bandwidth adjustment coefficient corresponding to the correlation peak according to the corresponding position of each correlation peak in the detection video;

Detecting vibrations at different positions in the video has different effects on detecting the operating status of the device to be detected. Then, the bandwidth adjustment coefficient corresponding to the correlation peak can be determined according to the corresponding position of each correlation peak in the detection video. The bandwidth adjustment coefficient is used to adjust the initial filtering bandwidth to make the filtering bandwidth more reasonable.

For example, according to the effect of the vibration of the position corresponding to the correlation peak in the detection video on the detection of the operating status of the device to be detected, the screen corresponding to the detection video can be divided into multiple regions, and the preset bandwidth adjustment can be set for each region coefficient, when determining the bandwidth adjustment coefficient in step 022, can directly obtain the preset bandwidth adjustment coefficient corresponding to the region where each correlation peak corresponds to the position in the detection video, and use the preset bandwidth adjustment coefficient as the bandwidth corresponding to the correlation peak Adjustment coefficient. In this way, the bandwidth adjustment coefficient can be obtained quickly and accurately.

Further, according to the structure of the device to be detected, the user can set the bandwidth weighting coefficient for each area in the screen corresponding to the detection video, and then use the product of the preset bandwidth adjustment coefficient and the bandwidth weighting coefficient as the relevant bandwidth adjustment of the corresponding area coefficient, which makes the filtering bandwidth of each correlation peak more reasonable. For example, the vibration of some key parts in the device to be detected is more able to reflect the operating conditions of the device to be detected than other parts, then the bandwidth weighting coefficient of the corresponding area of the key part in the detection video screen can be set to a larger value , so that the filtering bandwidth is larger, so that more useful information can be extracted.

023. Use the product of the initial filtering bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filtering bandwidth corresponding to each correlation peak.

The filtering bandwidth obtained in this way comprehensively considers the corresponding position of the correlation peak in the detection video and the frequency band corresponding to the correlation peak, so that the filtering bandwidth is more suitable for each correlation peak, thereby helping to extract useful information from each correlation peak. State change signal.

It should be noted that the calculation method of the filter bandwidth is not limited to the above method, and in other embodiments, an appropriate calculation method may be selected according to the frequency band corresponding to each correlation peak and the corresponding position of each correlation peak in the detection video to determine the filter bandwidth.

Please refer to FIG. 5, based on the above embodiments, in some embodiments, the filtering method further includes the steps of:

04. Use the principal component decomposition method to reduce the dimensionality of each correlation peak;

Step 04 is executed after Step 01 and before Step 02. Principal Component Analysis (PCA) is used to reduce the data dimensionality of the correlation peak data in the two main directions of vibration detection. It can be understood that when the vibration information is obtained from the correlation peaks, in the process of filtering the correlation peaks, it is necessary to calculate each correlation peak, and the correlation peaks extracted from the detection video include multiple dimensions of the picture corresponding to the detection video. State change status information, which will lead to cumbersome calculations and low efficiency in the filtering process. Using the principal component decomposition method to reduce the dimensionality of each correlation peak, and then perform other calculations related to the filtering process, which can greatly reduce the calculation amount of the filtering process.

Step 02 includes:

024. Determine the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak after dimensionality reduction processing in the detection video and the frequency band corresponding to the correlation peak.

After reducing the dimensionality of the correlation peak data, the correlation peaks after dimensionality reduction processing are calculated to obtain the filtering strategy corresponding to each correlation peak, which can reduce the calculation amount of the filtering process and improve the filtering efficiency.

Further, referring to Fig. 6, before step 04, the filtering method also includes:

05. Compare the peak value of each correlation peak with the preset peak range, and judge whether the peak value of each correlation peak is within the preset peak range;

Step 05 is executed after Step 01 and before Step 04. When the imaging module shoots the detection video of the device to be detected, small changes in some pixels can also form correlation peaks, but the degree of frequency change is not large enough, and the peak value of the correlation peak is small. These signals are essential for analyzing the operating status of the device to be detected The effect of is small or even has no effect, which can be understood as a noise signal. Then, some common noise signals can be filtered out by presetting a preset peak range, and comparing the peak values of each correlation peak with the preset peak range during filtering.

Step 04 includes:

041. Perform dimensionality reduction processing on related peaks whose peak values are within the preset peak range.

Dimensionality reduction processing is performed on the correlation peaks whose peak value is within the preset peak range, and the correlation peaks not within the preset peak range are filtered, so that on the one hand, the signal-to-noise ratio of the state change signal obtained from the detection video can be improved, and there are more useful signals , on the other hand, since part of the noise signal is removed, the calculation amount of subsequent filtering can be reduced, and the filtering efficiency can be improved.

Further, the vibration of the device to be detected is a periodic reciprocating motion, and the state change caused by the vibration is also periodic. Although a lot of noise information can lead to the detection of the state changes of each pixel in the video, the state changes caused by noise are often not periodic, for example, in outdoor scenes, the state changes caused by the light and dark changes of outdoor optical fibers. Moreover, when analyzing the operating conditions of the device to be detected based on the vibration of the device to be detected, only periodic vibrations can be used to reflect the operating status of the device to be detected, because non-periodic vibrations are often caused by the external environment, not by the device to be detected. If it is caused by the detection device itself, this part of the aperiodic signal cannot be used to analyze the operating status of the device to be detected. Then, after filtering each correlation peak according to the filtering strategy corresponding to each correlation peak, the correlation peak can also be inversely Fourier transformed to convert the frequency domain signal into a time domain signal to obtain multiple state change signals, and then The aperiodic signal in the state change signal is removed to further remove the noise, so that the operation status of the device to be detected obtained according to the detection video is more accurate.

An embodiment of the present invention also provides a filtering device, including:

The first acquisition module is used to acquire one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detection video, and the correlation peaks are frequency domain signals;

The filter strategy determination module is used to determine the filter strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the corresponding frequency band of the correlation peak;

The filtering module is configured to perform filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak.

The filter device of the embodiment of the present invention obtains one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detection video, and the correlation peaks are frequency domain signals; according to the corresponding positions and correlation peaks of each correlation peak in the detection video The corresponding frequency band determines the filtering strategy corresponding to each correlation peak; performs filtering processing on each correlation peak according to the filtering strategy corresponding to each correlation peak. By selecting an appropriate filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, the noise information in each correlation peak can be reduced, so that the signal-to-noise ratio of the state change signal obtained from the detection video is higher, so that according to The operation status of the device to be detected obtained by detecting the video is more accurate. It should be noted that the explanations and technical effects of the above filtering method embodiments are also applicable to the filtering device of this embodiment, and will not be repeated here to avoid redundancy.

In some embodiments, the filtering strategy includes a filtering bandwidth. The filtering strategy determination module includes:

An initial filter bandwidth determination unit, configured to determine the initial filter bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak;

A bandwidth adjustment coefficient determination unit is used to determine the bandwidth adjustment coefficient corresponding to the correlation peak according to the corresponding position of each correlation peak in the detection video;

The filter bandwidth determination unit is configured to use the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filter bandwidth corresponding to each correlation peak.

In some embodiments, the bandwidth adjustment coefficient determination unit is configured to obtain the preset bandwidth adjustment coefficient corresponding to the area where the corresponding position of each correlation peak in the detection video is located, and use the preset bandwidth adjustment coefficient as the bandwidth adjustment corresponding to the correlation peak coefficient.

In some embodiments, the filtering device also includes:

A dimensionality reduction module is used to perform dimensionality reduction processing on each correlation peak by using a principal component decomposition method;

The filtering strategy determination module is used to determine the corresponding filtering strategy of each correlation peak according to the corresponding position in the detection video and the corresponding frequency band of each correlation peak after dimensionality reduction processing.

In some embodiments, the filtering device also includes:

A judging module, configured to compare the peak value of each correlation peak with a preset peak range, and determine whether the peak value of each correlation peak is within the preset peak range;

The dimensionality reduction module is used to perform dimensionality reduction processing on related peaks whose peak value is within a preset peak range.

In some embodiments, the filtering module is configured to filter each correlation peak through interpolation filtering according to a filtering strategy corresponding to each correlation peak.

Wherein, the function implementation and technical effect of each module in the above filtering device correspond to each step in the above filtering method embodiment, and the functions and implementation processes thereof will not be repeated here.

The present invention also provides a computer-readable storage medium, on which a filtering program is stored, wherein when the filtering program is executed by a processor, the steps of the filtering method in any of the foregoing embodiments are implemented.

Wherein, the method and the corresponding technical effect realized when the filtering program is executed can refer to the various embodiments of the filtering method of the present invention, which will not be repeated here.

An embodiment of the present invention also provides a detection device for detecting the operation status of the device to be detected, and the detection device includes the filter device of the above embodiment or the electronic device of the above embodiment.

The detection device in the embodiment of the present invention obtains one or more correlation peaks obtained by performing phase correlation calculation on the frame sequence corresponding to the detection video; and determines each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak Corresponding filtering strategy: filter each correlation peak according to the filtering strategy corresponding to each correlation peak. By selecting an appropriate filtering strategy for filtering according to the position of each correlation peak and the corresponding frequency band, the noise information in each correlation peak can be reduced, so that the signal-to-noise ratio of the state change signal obtained from the detection video is higher, so that according to The operation status of the device to be detected obtained by detecting the video is more accurate.

When the detection equipment detects the operating status of the device to be detected, the imaging module can be used to shoot the detection video of the device to be detected when it is running, and then the state change information can be extracted from the detection video containing the picture of the detection device running. The state change information can reflect The vibration information of the device to be detected can obtain the operation status of the device to be detected according to the vibration information of the device to be detected. As shown in Figure 3, the frame sequence in the detected video can be converted from the RGB color space to the YIQ color space, and the brightness information and chrominance information of the video frame can be separated. The conversion relationship between RGB and YIQ is:

Y＝0.299*R+0.587*G+0.114*B;

I＝0.596*R-0.275*G-0.321*B;

Q=0.212*R−0.523*G+0.311*B.

Then keep the I and Q channels unchanged, perform FFT operation on the Y channel, and then use the Euler motion amplification algorithm to amplify the video data. It specifically includes: decomposing the Y-channel image after the FFT transformation into a complex number steerable pyramid space domain. The images of different scales decomposed in the Y channel space domain are subjected to time domain bandpass filtering. It can be understood that in the video picture, the vibration can be reflected by the brightness of the video frame sequence, so the Y channel in the detection video can be obtained by analyzing and detecting the Y channel in the video. vibration information. ; Then amplify the motion information of interest after time-domain bandpass filtering, perform complex manipulation pyramid reconstruction on the motion information of interest, and obtain an enlarged Y channel image; finally combine the reconstructed Y channel image with the original I, Q channel The images are added and converted to RGB color space to obtain the output video.

Then, the phase correlation algorithm is used to calculate the cross power spectrum between the frame sequences after video motion amplification processing. The phase correlation algorithm uses the following formula to calculate the cross-cross power spectrum.

为b帧图像的傅里叶变换的共轭信号，除式的下边为两个傅里叶变换的信号的相关积的模。R为本步骤的计算结果交叉互功率谱(包含频域噪音)。交叉互功率谱中包括一个或多个相关峰，相关峰为频域信号。可以理解，每个相关峰可以反应检测视频中的某个位置的状态变化，对检测视频对应的真序列进行相位相关计算得到交叉互功率谱可以理解为，从检测视频中提取出视频画面中的状态变化信息。状态变化信息包括振动信息和其他噪声信息。可通过本发明实施例的滤波方法，对交叉互功率谱中的各相关峰进行滤波处理，以减少各相关峰中的噪声信息，使得从包含有检测装置运行时的画面的检测视频中提取出的信息中，有用信号的信噪比更高。In the above formula, Fa is the Fourier transform of the frame a image,

is the conjugate signal of the Fourier transform of the b-frame image, and the lower side of the division formula is the modulus of the correlation product of the two Fourier transformed signals. R is the cross-cross power spectrum (including frequency domain noise) of the calculation result of this step. The cross-cross power spectrum includes one or more correlation peaks, and the correlation peaks are frequency-domain signals. It can be understood that each correlation peak can reflect the state change of a certain position in the detection video, and the phase correlation calculation of the true sequence corresponding to the detection video to obtain the cross-cross power spectrum can be understood as extracting the Status change information. State change information includes vibration information and other noise information. Through the filtering method of the embodiment of the present invention, each correlation peak in the cross-cross power spectrum can be filtered to reduce the noise information in each correlation peak, so that it can be extracted from the detection video containing the picture when the detection device is running. In the information, the signal-to-noise ratio of the useful signal is higher.

In the above-mentioned embodiments, all or part may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present invention will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as an optical disk), or a semiconductor medium (such as a solid-state disk), and the like. In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

In the several embodiments provided by the present invention, it should be understood that the disclosed device can also be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated to another system, or some features may be ignored or not implemented. In another point, the shown or discussed mutual indirect coupling or direct coupling or communication connection may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can also be implemented in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. Several instructions are included to make a computer device (which may be a personal computer, server or network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present invention. And the aforementioned storage medium can include, for example: U disk, mobile hard disk, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc., which can store program codes. medium.

Claims (9)

1. A method of filtering comprising:

acquiring one or more correlation peaks obtained by carrying out phase correlation calculation on a frame sequence corresponding to a detection video, wherein the correlation peaks are frequency domain signals;

determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band of the correlation peak; the filtering strategy comprises a filtering bandwidth, and the determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak comprises the following steps: determining initial filtering bandwidths corresponding to the correlation peaks according to the frequency bands corresponding to the correlation peaks; determining a bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of the correlation peak in the detection video; taking the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filter bandwidth corresponding to each correlation peak;

and carrying out filtering treatment on each correlation peak according to a filtering strategy corresponding to each correlation peak.

2. The filtering method according to claim 1, wherein determining the bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of the correlation peak in the detected video comprises:

and acquiring a preset bandwidth adjustment coefficient corresponding to the area where the corresponding position of each correlation peak in the detection video is located, and taking the preset bandwidth adjustment coefficient as the bandwidth adjustment coefficient corresponding to the correlation peak.

3. The filtering method according to claim 1, wherein before determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak, the filtering method further comprises:

performing dimension reduction treatment on each relevant peak by using a principal component decomposition method;

the determining the filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detected video and the frequency band corresponding to the correlation peak comprises the following steps:

and determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak after the dimension reduction processing in the detection video and the frequency band corresponding to the correlation peak.

4. The filtering method according to claim 3, wherein before the dimensionality reduction process is performed on each correlation peak by the principal component decomposition method, the filtering method further comprises:

comparing the peak value of each correlation peak with a preset peak value range, and respectively judging whether the peak value of each correlation peak is in the preset peak value range;

the method for performing dimension reduction processing on each correlation peak by using the principal component decomposition method comprises the following steps:

and performing dimension reduction treatment on the correlation peak with the peak value within a preset peak value range.

5. The filtering method according to claim 1, wherein the filtering each correlation peak according to the filtering strategy corresponding to each correlation peak comprises:

and carrying out filtering treatment on each correlation peak in a mode of interpolation filtering according to a filtering strategy corresponding to each correlation peak.

6. A filtering apparatus, comprising:

the first acquisition module is used for acquiring one or more correlation peaks obtained by carrying out phase correlation calculation on a frame sequence corresponding to the detection video, wherein the correlation peaks are frequency domain signals;

the filtering strategy determining module is used for determining a filtering strategy corresponding to each correlation peak according to the corresponding position of each correlation peak in the detection video and the frequency band corresponding to the correlation peak; the filtering strategy comprises a filtering bandwidth, and the filtering strategy determining module is further used for determining an initial filtering bandwidth corresponding to each correlation peak according to the frequency band corresponding to each correlation peak; determining a bandwidth adjustment coefficient corresponding to each correlation peak according to the corresponding position of the correlation peak in the detection video; taking the product of the initial filter bandwidth corresponding to each correlation peak and the bandwidth adjustment coefficient as the filter bandwidth corresponding to each correlation peak;

and the filtering module is used for carrying out filtering processing on each correlation peak according to a filtering strategy corresponding to each correlation peak.

7. An electronic device comprising a processor, a memory, and a filtering program stored on the memory and executable by the processor, wherein the filtering program, when executed by the processor, implements the instructions of the steps of the filtering method of any one of claims 1 to 5.

8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a filter program, wherein the filter program, when executed by a processor, implements the steps of the filtering method of any of claims 1 to 5.

9. A detection apparatus for detecting an operating condition of a device to be detected, characterized in that the detection apparatus comprises a filter device according to claim 6 or an electronic device according to claim 7.

Images