CN109960960A - Video fingerprint generation and matching method and device, computer equipment and storage medium - Google Patents

Abstract

The invention discloses a video fingerprint generation and matching method and device, computer equipment and storage medium. The generating method includes: preprocessing the video to be processed, and extracting key frame images of the preprocessed video; performing dimensionality reduction processing on the key frame images through a compressed sensing algorithm to obtain a sparse matrix; performing block processing on the sparse matrix, Extract the preset block block matrix, calculate and obtain a feature matrix composed of the average value of the preset block block matrix; perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the video fingerprint. According to the embodiment of the present invention, a video fingerprint can be generated based on a compressed sensing algorithm, so that the generated video fingerprint can effectively resist the changes brought by the external environment, and the sparseness of the compressed sensing can reduce the fingerprint on the basis of retaining the basic characteristics of the original video. dimension to simplify the calculation process.

Description

Video fingerprint generation and matching method and device, computer equipment and storage medium

technical field

The invention belongs to the field of computer technology, and in particular relates to a method for generating video fingerprints, a method for matching video fingerprints, a device for generating video fingerprints, a device for matching video fingerprints, computer equipment and a computer storage medium.

Background technique

At present, the multimedia technology has made great progress in the storage and dissemination of data information, which makes the available video information expand day by day. However, video information has a complex structure, large information capacity, and low degree of abstraction, which also brings a very serious problem of information expansion. Therefore, how to effectively organize, express, manage, query and retrieve these unstructured massive data has become the focus of research. . To this end, video fingerprinting came into being, which is a technology for software to identify, extract, and compress videos. The features extracted from video files that can uniquely identify video files are used as video fingerprints. Compared with the original video content, its data volume It is significantly reduced, and at the same time, it needs to meet the requirements of robustness, distinguishability, and real-time performance. Therefore, in order to better protect the commercial interests of video program manufacturers and distributors and effectively retrieve infringing videos, video fingerprinting technology has received more and more attention.

However, when generating a video fingerprint in the related art, the video fingerprint is generated by selecting a region of interest, and the region cannot express the characteristics of the original video, resulting in the loss of some characteristics of the video, or the combination of time domain and space domain is used to generate video fingerprints. The large amount of data results in long retrieval time and high bit error rate.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a video fingerprint generation method, a matching method, a generation device, a matching device, computer equipment and a computer storage medium, which can generate a video fingerprint based on a compressed sensing algorithm, so that the generated video fingerprint can effectively resist the external environment. The sparseness of the compressed sensing can reduce the dimension of the fingerprint, reduce the amount of fingerprint data, and simplify the calculation process on the basis of retaining the basic characteristics of the original video.

On the one hand, an embodiment of the present invention provides a method for generating a video fingerprint, including: preprocessing a video to be processed, and extracting a key frame image of the preprocessed video; performing dimensionality reduction processing on the key frame image through a compressed sensing algorithm , to obtain a sparse matrix; perform block processing on the sparse matrix, extract a preset block block matrix, and calculate a feature matrix composed of the average value of the preset block block matrix; perform singular value decomposition on the feature matrix, And the singular value obtained by decomposing the singular value of the feature matrix is defined as the fingerprint of the video.

In the above technical solution, preferably, the steps of preprocessing the video to be processed and extracting key frame images of the preprocessed video specifically include: converting the video to be processed into frame images at a preset fixed frame rate, and processing the frame images Extract the key frame image according to the preset ratio.

In any of the above technical solutions, preferably, the steps of preprocessing the video to be processed and extracting the key frame image of the preprocessed video are the same as performing dimensionality reduction processing on the key frame image through compressed sensing theory to obtain a sparse matrix The steps further include: performing grayscale conversion on the key frame image, adjusting the height of the converted key frame image to a preset height, and adjusting the width of the converted key frame image to a preset width.

In any of the above technical solutions, preferably, performing singular value decomposition on the feature matrix, and defining the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video, the step further includes: encoding the fingerprint using Gray code. .

In any of the above technical solutions, preferably, the specific step of extracting the preset block-blocking matrix is: arranging the values of all the block-blocking matrices obtained after dividing the sparse matrix into blocks in descending order , and select the block matrix located in the previous preset as the preset block block matrix.

On the other hand, an embodiment of the present invention provides a device for generating a video fingerprint, and the device for generating a video fingerprint is applied to the method for generating a video fingerprint provided by any one of the above technical solutions of the present invention. The device for generating video fingerprints includes: a preprocessing unit for preprocessing the video to be processed and extracting key frame images of the preprocessed video to be processed; a dimensionality reduction processing unit for passing the key frame images through a compressed sensing algorithm Dimension reduction processing is performed to obtain a sparse matrix; a block processing unit is used to perform block processing on the sparse matrix, and extract the preset block block matrix, and calculate the average value of the preset block block matrix. The feature matrix; the fingerprint generating unit is used for singular value decomposition of the feature matrix, and the singular value obtained after the singular value decomposition of the feature matrix is defined as the fingerprint of the video to be processed.

In another aspect, an embodiment of the present invention provides a method for matching video fingerprints. The method for generating video fingerprints provided by any of the above technical solutions of the present invention is used to generate fingerprints of a target video and a video to be matched, and the matching of video fingerprints is performed. The method includes: extracting the first fingerprint of the first key frame of the target video; comparing the first fingerprint with the fingerprint of each key frame of the video to be matched, and obtaining the second key frame in the video to be matched, and the second key frame The frame is the key frame with the smallest Euclidean distance from the fingerprint of each key frame in the video to be matched to the first fingerprint; with the second key frame as the center, select the key frame contained in the video to be matched forward and backward respectively according to the preset duration. The number of frames, so that the number of key frames contained in the selected video to be matched is equal to the number of key frames of the target video; calculate the fingerprints of all key frames in the target video one by one and the Euclidean distance between the key frames selected in the video to be detected, in When each Euclidean distance is less than the threshold, it is determined that the video to be matched selected at this time is a similar video to the target video.

In the above technical solution, preferably, the step of extracting the first fingerprint of the first key frame of the target video specifically includes: randomly selecting any key frame in the target video as the first key frame.

In any of the above technical solutions, preferably, the preset duration is 3 seconds.

In another aspect, an embodiment of the present invention provides a video fingerprint matching device, the video fingerprint matching device is applied to the video fingerprint matching method provided by any of the above technical solutions of the present invention, and the video fingerprint matching device includes: an extraction unit , is used to extract the first fingerprint of the first key frame of the target video; the comparison unit is used to compare the first fingerprint with the fingerprint of each key frame of the video to be matched, and obtain the second key frame of the video to be matched. , and the second key frame is the key frame with the smallest Euclidean distance from the fingerprint of each key frame in the video to be matched to the first fingerprint; the selection unit is used for taking the second key frame as the center, respectively forward and backward according to the preset Set the time length to select the number of key frames contained in the video to be matched, so that the number of key frames contained in the selected video to be matched is equal to the number of key frames of the target video; the computing unit is used to calculate the fingerprints of all the key frames in the target video one by one. The Euclidean distance between the selected key frames in the video to be detected, when each Euclidean distance is less than a threshold, it is determined that the selected video to be matched at this time is a similar video to the target video.

In another aspect, an embodiment of the present invention provides a computer device, the device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the generation of the video fingerprint provided by any of the above technical solutions is realized The method or the video fingerprint matching method provided by any one of the above technical solutions.

On the other hand, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method for generating a video fingerprint provided by any of the above technical solutions or The video fingerprint matching method provided by any one of the above technical solutions.

The video fingerprint generation method, matching method, generation device, matching device, computer equipment, and a computer storage medium according to the embodiments of the present invention can generate video fingerprints by using compressed sensing theory. Video fingerprints can well resist the changes brought by the external environment such as viewing angle, light, brightness transformation, frame ratio transformation, cropping and label coverage, and make full use of the temporal and spatial characteristics of the original video to improve the robustness and distinguishability of fingerprints. At the same time, the sparsity of compressed sensing can reduce the dimension of the fingerprint on the basis of retaining the basic characteristics of the original video, and finally use Gray code to generate the video fingerprint, which effectively reduces the bit error rate of the generated video and ensures that the video fingerprint can be Realistically and effectively characterize the features of the video to be processed. In addition, by comparing the Euclidean distance between the fingerprint of the target video generated by the above method and the key frame of the video to be matched in the database, the similarity between the target video and the video to be matched is calculated, which can improve the retrieval efficiency during video fingerprint retrieval. speed.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments of the present invention. For those of ordinary skill in the art, without creative work, the Additional drawings can be obtained from these drawings.

FIG. 1 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention.

FIG. 2 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention.

FIG. 3 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention.

FIG. 4 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention.

FIG. 5 shows a schematic diagram of a method for generating a video fingerprint provided by an embodiment of the present invention.

FIG. 6 shows a schematic diagram of a framework of an apparatus for generating video fingerprints provided by an embodiment of the present invention.

FIG. 7 shows a schematic flowchart of a video fingerprint matching method provided by an embodiment of the present invention.

FIG. 8 shows a schematic diagram of a video fingerprint matching method provided by an embodiment of the present invention.

FIG. 9 shows a schematic diagram of a framework of a video fingerprint matching apparatus provided by an embodiment of the present invention.

FIG. 10 shows a real-time analysis and comparison diagram of a video fingerprint matching method provided by an embodiment of the present invention and related technologies.

FIG. 11 shows a schematic diagram of a hardware structure of a computer device provided by an embodiment of the present invention.

Detailed ways

The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention, and are not configured to limit the present invention. It will be apparent to those skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present invention by illustrating examples of the invention.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element defined by the phrase "comprising" does not preclude the presence of additional identical elements in the process, method, article or device that includes the element.

In order to solve the problems of the prior art, the embodiments of the present invention provide a video fingerprint generation method, a matching method, a generation apparatus, a matching apparatus, computer equipment and a computer storage medium. The following first introduces the method for generating a video fingerprint provided by the embodiment of the present invention.

FIG. 1 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention. As shown in FIG. 1 , a method for generating a video fingerprint provided by this embodiment includes:

S102, preprocessing the video to be processed, and extracting key frame images of the preprocessed video to be processed;

S104, performing dimension reduction processing on the key frame image through a compressed sensing algorithm to obtain a sparse matrix;

S106, performing block processing on the sparse matrix, extracting a preset block block matrix, and calculating a feature matrix composed of an average value of the preset block block matrix;

S108, perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video.

The method for generating a video fingerprint provided by the present invention firstly preprocesses the video to be processed, and extracts key frame images of the preprocessed video. The key frame images of the video to be processed need to be extracted for subsequent steps.

Secondly, the key frame image is dimensionally reduced by the compressed sensing algorithm to obtain the sparse matrix S _i . At this time, the key frame image needs to be projected on the discrete wavelet transform basis, and the sparse matrix is obtained by calculation. Since most of the obtained changes The absolute value of the coefficient is very small, and the resulting transformation vector is sparse or approximately sparse, so it is named as a sparse matrix S _i to reduce the dimension of the generated video fingerprint and effectively remove the redundant information in the key frame image. At the same time, the main idea of compressed sensing is to combine compression and sampling. First, the non-adaptive linear projection (measurement value) of the signal is collected, and then the measurement value reconstructs the original signal according to the corresponding reconstruction algorithm. The advantage is that the sampling process breaks through the constraints of the Nyquist sampling theorem, and there is no or only little information loss during signal reconstruction, making high-resolution acquisition possible.

Subsequently, the sparse matrix S _i is subjected to block processing, and the preset block block matrix is extracted, and the feature matrix G=[g ₁ , g ₂ , . . . ....g _i ] ^T .

Preferably, K blocks are extracted from the blocks in the _sparse matrix Si, and the step of extracting the K blocks is to _sort the blocks in the sparse matrix Si according to the size of the energy.

Thirdly, perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video. The algorithm of singular value decomposition is as follows:

G=UDV ^H (Formula 1)

where:

Call d _r , i=1,2,.....,k the singular value of matrix D, namely

D _v =[diag(d ₁ ,d ₂ ,...,d _r )] is the feature description value that finally represents the video to be processed, that is, the fingerprint of the video to be processed.

In addition, the number of video fingerprints depends on the number of extracted key frames. Each key frame is relative to a video fingerprint. The more fingerprints are calculated, the more detailed the video can be characterized, but it will also reduce the amount of fingerprint calculation and The content is too long, which is not conducive to information carrying; therefore, the number of extracted video fingerprints can be controlled according to actual needs.

FIG. 2 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention. As shown in Figure 2, the method for generating a video fingerprint provided by this embodiment includes:

S202, converting the video to be processed into a frame image at a preset fixed frame rate, and extracting a key frame image from the frame image according to a preset ratio;

S204, performing dimension reduction processing on the key frame image through a compressed sensing algorithm to obtain a sparse matrix;

S206, performing block processing on the sparse matrix, extracting a preset block block matrix, and calculating a feature matrix composed of an average value of the preset block block matrix;

S208 , perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video.

In this embodiment, when extracting video fingerprints, the video to be processed is first converted into a frame image at a preset fixed frame rate, and a key frame image is extracted from the frame image according to a preset ratio. Frame image, which can resist the attack of frame ratio change, so as to ensure the stability of the video. Preferably, the preset fixed frame rate is 20 frames per second.

FIG. 3 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention. As shown in FIG. 3 , the method for generating a video fingerprint provided by this embodiment includes:

S302, preprocessing the video to be processed, and extracting key frame images of the preprocessed video to be processed;

S304, performing grayscale conversion on the key frame image, adjusting the height of the converted key frame image to a preset height, and adjusting the width of the converted key frame image to a preset width;

S306, performing dimension reduction processing on the key frame image through a compressed sensing algorithm to obtain a sparse matrix;

S308, performing block processing on the sparse matrix, extracting a preset block block matrix, and calculating a feature matrix composed of an average value of the preset block block matrix;

S310, perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video.

In this embodiment, the key frame image is gray-scaled to make it robust to color changes, which can ensure that the video fingerprint generation method provided by the embodiment of the present invention can not only be applied to color videos, but also to traditional black and white videos. The same goes for video. In addition, the height of the converted key frame image is adjusted to a preset height, and the width of the converted key frame image is adjusted to a preset width, and the video fingerprint provided by the embodiment of the present invention is made by unifying the values of the height and width. The generation method is robust to images of any size, has a unified specification scale, and also has a unified standard for subsequent processing, which reduces the error caused by different image sizes.

FIG. 4 shows a schematic flowchart of a method for generating a video fingerprint provided by an embodiment of the present invention. As shown in FIG. 4 , the method for generating a video fingerprint provided by this embodiment includes:

S402, preprocessing the video to be processed, and extracting key frame images of the preprocessed video to be processed;

S404, performing dimension reduction processing on the key frame image through a compressed sensing algorithm to obtain a sparse matrix;

S406, performing block processing on the sparse matrix, extracting a preset block block matrix, and calculating a feature matrix composed of an average value of the preset block block matrix;

S408, perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video;

S410, the fingerprint is encoded using Gray code.

In this embodiment, using the Gray code for coding can effectively reduce the bit error rate. Specifically, since the Gray code belongs to reliability encoding and is an error-minimizing encoding method, the use of the Gray code for encoding to generate a video fingerprint can effectively reduce the bit error rate.

In an embodiment of the present invention, preferably, the specific step of extracting the preset block-blocking matrix is: the values of all the block-blocking matrices obtained after dividing the sparse matrix into blocks are in descending order Arrangement, selecting the block matrix located in the previous preset as the preset block block matrix.

In this technical solution, when extracting the preset block sub-block matrix, the preset block may be one block or multiple blocks, and the preset block is a preset matrix with large energy in all sub-block matrices, and the extraction steps are as follows: After dividing the sparse matrix into blocks, sort all the obtained block matrices in descending order of value, and extract the block matrix located in the previous preset, so that the energy of the extracted block matrix (that is, the value of the matrix) It is larger than the unextracted block matrix, because more feature information of the image can be preserved in the block with high energy.

FIG. 5 shows a schematic diagram of a method for generating a video fingerprint provided by an embodiment of the present invention.

As shown in Figure 5, firstly, S502 is each frame image in the original video; in S504, key frame extraction is performed, and some frame images in S502 are selected as key frame images; the key frame frame images in S504 are converted into grayscale images , and unify the height and width to get S506; and introduce the compressed sensing theory in S506, make the image sparse, and then divide the sparse frame image into blocks, (the baseball image is divided into blocks in the figure); finally extract For blocks with large energy values, the average value is used to form a feature matrix, and singular value SVD decomposition is performed on the feature matrix to obtain an S-value production fingerprint, and the result is shown in S508.

The method for generating a video fingerprint provided by the embodiment of the present invention can well resist the changes brought by the external environment such as viewing angle, light, brightness transformation, frame ratio transformation, clipping and label coverage, and retains the basic characteristics and characteristics of the original video through the minimum information. information. Table 1 shows the video fingerprint stability analysis and comparison of the related art and the technical solution of the present invention:

Table 1

It can be seen from Table 1 that the method for generating a video fingerprint provided by the present invention can have a good recall rate and precision rate when subjected to various types of attacks.

On the other hand, an embodiment of the present invention provides a device for generating video fingerprints, and the device for generating video fingerprints is applied to the method for generating video fingerprints provided by the technical solutions of any of the foregoing embodiments of the present invention. Therefore, the implementation of the present invention The apparatus for generating a video fingerprint provided in this example has all the beneficial effects of the method for generating a video fingerprint provided in any of the previous embodiments, which will not be listed one by one here.

FIG. 6 shows a schematic diagram of a framework of an apparatus for generating video fingerprints provided by an embodiment of the present invention. As shown in FIG. 6 , the apparatus 600 for generating a video fingerprint provided in this embodiment includes:

A preprocessing unit 602, configured to preprocess the video to be processed, and extract the key frame image of the preprocessed video to be processed;

A dimensionality reduction processing unit 604, configured to perform dimensionality reduction processing on the key frame image through a compressed sensing algorithm to obtain a sparse matrix;

A block processing unit 606, configured to perform block processing on the sparse matrix, extract a preset block block matrix, and calculate a feature matrix composed of an average value of the preset block block matrix;

The fingerprint generating unit 608 is configured to perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video to be processed.

The device for generating video fingerprints provided by the present invention first preprocesses the video to be processed by the preprocessing unit 602, and extracts the key frame images of the preprocessed video, because the video can be understood as a combination of a large number of frame images in a continuous time Therefore, it is necessary to extract the key frame images of the video to be processed for subsequent operations.

Secondly, the dimensionality reduction processing unit 604 performs the dimensionality reduction process on the key frame image through the compressed sensing algorithm to obtain the sparse matrix S _i . At this time, the key frame image needs to be projected on the discrete wavelet transform basis to obtain the sparse matrix by calculation. The absolute value of most of the obtained change coefficients is very small, and the obtained transformation vector is sparse or approximately sparse, so it is named as the sparse matrix S _i to reduce the dimension of the generated video fingerprint and effectively remove the key frame image. redundant information. At the same time, the main idea of compressed sensing is to combine compression and sampling. First, the non-adaptive linear projection (measurement value) of the signal is collected, and then the measurement value reconstructs the original signal according to the corresponding reconstruction algorithm. The advantage is that the sampling process breaks through the constraints of the Nyquist sampling theorem, and there is no or only little information loss during signal reconstruction, making high-resolution acquisition possible.

Subsequently, the sparse matrix S _i is subjected to block processing by the block processing unit 606, and a preset block block matrix is extracted, and a feature matrix G=[g ₁ ,g ₂ ,...g _i ] ^T .

Preferably, K blocks are extracted from the blocks in the _sparse matrix Si, and the step of extracting the K blocks is to _sort the blocks in the sparse matrix Si according to the size of the energy.

Once again, singular value decomposition is performed on the feature matrix by the fingerprint generating unit 608, and the singular value obtained after singular value decomposition of the feature matrix is defined as the fingerprint of the video. The algorithm of singular value decomposition is as above (Equation 1), and the calculated The D _v =[diag(d ₁ ,d ₂ ,...,d _r )] finally represents the feature description value of the video to be processed, that is, the fingerprint of the video to be processed.

In addition, the number of video fingerprints depends on the number of extracted key frames. Each key frame is relative to a video fingerprint. The more fingerprints are calculated, the more detailed the video can be characterized, but it will also reduce the amount of fingerprint calculation and The content is too long, which is not conducive to information carrying; therefore, the number of extracted video fingerprints can be controlled according to actual needs.

FIG. 7 shows a schematic flowchart of a video fingerprint matching method provided by an embodiment of the present invention. As shown in FIG. 7 , the method for matching video fingerprints provided in this embodiment includes:

S702, extract the first fingerprint of the first key frame of the target video;

S704, compare the first fingerprint with the fingerprint of each key frame of the video to be matched, and obtain the second key frame in the video to be matched, and the second key frame is the fingerprint of each key frame in the video to be matched to the first key frame The key frame with the smallest Euclidean distance of a fingerprint;

S706, with the second key frame as the center, select the number of key frames contained in the video to be matched forward and backward respectively according to the preset duration, so that the number of key frames contained in the selected video to be matched and the number of key frames of the target video are equal;

S708, calculate the Euclidean distances between the fingerprints of all key frames in the target video and the selected key frames in the video to be detected one by one, and when each Euclidean distance is less than the threshold, determine that the selected video to be matched at this time is a similar video of the target video .

The video fingerprint matching method provided by this embodiment first extracts the first fingerprint of the first key frame of the target video, and this step is to roughly select the key frame; then the first fingerprint is matched with the fingerprint of each key frame of the video to be matched. By comparison, the second key frame in the video to be matched is obtained, and the second key frame is the key frame with the smallest Euclidean distance from the fingerprint of each key frame in the video to be matched to the first fingerprint; secondly, the second key frame is centered , select the number of key frames contained in the video to be matched forward and backward respectively according to the preset duration, so that the number of key frames contained in the selected video to be matched is equal to the number of key frames of the target video, and this step is to roughly select Based on the precise selection of keyframes. Finally, calculate the Euclidean distances between the fingerprints of all key frames in the target video and the selected key frames in the video to be detected one by one. When each Euclidean distance is less than the threshold, it is determined that the selected video to be matched is a similar video to the target video. .

As shown in FIG. 8 , it is a schematic diagram of a matching process of a video fingerprint matching method provided by an embodiment of the present invention. Among them, randomly select any key frame in the target video (10s length in Figure 8), extract fingerprints by using compressed sensing, and then use the key frame S _i fingerprint with the video to be matched (20s length in Figure 8), each time A keyframe fingerprint is compared one by one to find the keyframe t _{i with the smallest Euclidean distance from S i .}

Among them, assuming that the threshold value is T, it is determined that the video to be matched selected at this time is a similar video of the target video and needs to satisfy:

in: target video fingerprint sequence of length r, is the fingerprint sequence of the video to be matched in the video library.

In an embodiment provided in the present invention, the fingerprints formed by compressed sensing are used to select key frames according to rough filtering and precise selection respectively, and calculate the fingerprint search to select the video key frames, and calculate the Euclidean distance between the fingerprints of the video key frames. Determine the similarity and reduce the amount of calculation during video retrieval, so as to improve the speed of retrieval and meet the requirements of real-time.

In any embodiment provided by the present invention, preferably, the step of extracting the first fingerprint of the first key frame of the target video is specifically: randomly selecting any key frame in the target video as the first key frame.

In the technical solution of this embodiment, the step of extracting the first fingerprint of the first key frame of the target video is specifically to randomly select any key frame in the target video as the first key frame, and randomly extract any key frame from the target video. The number of first fingerprints are matched separately to provide similarity for matching retrieval.

In the technical solutions of any of the above embodiments, preferably, the preset duration is 3 seconds.

In this technical solution, the preset duration is 3 seconds, so that the number of key frames included in the video to be detected is selected (precisely select key frames) forward and backward respectively every 3 seconds with the second key frame as the center.

In addition, due to the huge amount of data and information in the video database, a video will generate multiple frame images, and the fingerprint of only one key frame is not enough to represent all the information of the video fingerprint, and in order to achieve the reliability of matching, a large amount of video frame information, and there will be overlapping selected parts, which causes a huge computational burden and cannot meet the real-time requirements. A coarse-fine matching video retrieval scheme proposed in the embodiments of the present invention can reduce the amount of calculation and meet the real-time performance by comparing the Euclidean distance between fingerprints.

In another aspect, an embodiment of the present invention provides a video fingerprint matching apparatus, and the video fingerprint matching apparatus is applied to the video fingerprint matching method provided by any of the foregoing embodiments of the present invention. Therefore, the video fingerprint matching apparatus provided by the embodiment of the present invention has all the beneficial effects of the video fingerprint matching method provided by any of the foregoing embodiments, which will not be listed one by one here.

FIG. 9 shows a schematic diagram of a video fingerprint matching apparatus provided by an embodiment of the present invention. As shown in FIG. 9 , the apparatus 900 for matching video fingerprints provided in this embodiment includes:

Extraction unit 902, for extracting the first fingerprint of the first key frame of the target video;

The comparison unit 904 is used to compare the first fingerprint with the fingerprint of each key frame of the video to be matched, and obtain the second key frame in the video to be matched, and the second key frame is each key frame in the video to be matched The key frame with the smallest Euclidean distance from the fingerprint to the first fingerprint;

The selection unit 906 is used to select the number of key frames contained in the video to be matched forward and backward according to the preset duration respectively, with the second key frame as the center, so that the number of key frames contained in the selected video to be matched is the same as the target video. The number of keyframes is equal;

The calculation unit 908 is used to calculate the Euclidean distance between the fingerprints of all key frames in the target video and the selected key frame in the video to be detected one by one, and when each Euclidean distance is less than the threshold, it is determined that the selected video to be matched is the target at this time. Similar videos of videos.

The video fingerprint matching device provided by this embodiment firstly extracts the first fingerprint of the first key frame of the target video through the extraction unit 902, which is a unit for roughly selecting the key frame; The fingerprints of each key frame of the matching video are compared, and the second key frame in the video to be matched is obtained, and the second key frame is the key of the minimum Euclidean distance from the fingerprint of each key frame in the video to be matched to the first fingerprint. frame; secondly take the second key frame as the center, select the number of key frames included in the video to be matched by selecting unit 906 forward and backward respectively according to the preset duration, so that the number of key frames included in the selected video to be matched and the target The number of key frames in the video is equal, and this unit is the unit for accurately selecting key frames on the basis of rough selection. Finally, the Euclidean distance between the fingerprints of all key frames in the target video and the selected key frame in the video to be detected is calculated by the calculation unit 908 one by one, and when each Euclidean distance is less than the threshold, it is determined that the selected video to be matched at this time is the target Similar videos of videos.

Fig. 10 shows the real-time analysis and comparison diagram of the present invention and the related art. As shown in Fig. 10, under the same situation, the real-time analysis and comparison results of the present invention and the related art show that the matching method of the video fingerprint provided by the present invention The real-time performance is significantly reduced.

In another aspect, an embodiment of the present invention provides a computer device, including: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for generating a video fingerprint provided by any of the above embodiments or the above A method for matching video fingerprints provided by any embodiment. Therefore, the computer device provided by the embodiment of the present invention has all the beneficial effects of the video fingerprint production method provided by any of the foregoing embodiments and the video fingerprint matching method provided by any of the foregoing embodiments, which will not be listed one by one here.

In another aspect, an embodiment of the present invention provides a computer storage medium, where computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the method for generating a video fingerprint provided by any of the foregoing embodiments or any of the foregoing embodiments is implemented. An embodiment provides a video fingerprint matching method. Therefore, the computer device provided by the embodiment of the present invention has all the beneficial effects of the video fingerprint production method provided by any of the foregoing embodiments and the video fingerprint matching method provided by any of the foregoing embodiments, which will not be listed one by one here.

FIG. 11 shows a schematic diagram of a hardware structure of a computer device provided by an embodiment of the present invention.

The computer device may include a processor 301 and a memory 302 storing computer program instructions.

Specifically, the above-mentioned processor 301 may include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured as one or more integrated circuits implementing the embodiments of the present invention.

Memory 302 may include mass storage for data or instructions. By way of example and not limitation, memory 302 may include a Hard Disk Drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive or two or more A combination of more than one of the above. Memory 302 may include removable or non-removable (or fixed) media, where appropriate. Storage 302 may be internal or external to the integrated gateway disaster recovery device, where appropriate. In certain embodiments, memory 302 is non-volatile solid state memory. In particular embodiments, memory 302 includes read only memory (ROM). Where appropriate, the ROM may be a mask programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically rewritable ROM (EAROM) or flash memory or A combination of two or more of the above.

The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any of the methods for generating video fingerprints and the method for matching video fingerprints in the foregoing embodiments.

In one example, the computer device may also include a communication interface 303 and a bus 310 . Among them, as shown in FIG. 11 , the processor 301 , the memory 302 , and the communication interface 303 are connected through the bus 310 and complete the mutual communication.

The communication interface 303 is mainly used to implement communication between modules, apparatuses, units and/or devices in the embodiments of the present invention.

The bus 310 includes hardware, software, or both, coupling the components of the online data flow metering device to each other. By way of example and not limitation, the bus may include Accelerated Graphics Port (AGP) or other graphics bus, Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), HyperTransport (HT) Interconnect, Industry Standard Architecture (ISA) Bus, Infiniband Interconnect, Low Pin Count (LPC) Bus, Memory Bus, Microchannel Architecture (MCA) Bus, Peripheral Component Interconnect (PCI) Bus, PCI-Express (PCI-X) Bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus or other suitable bus or a combination of two or more of the above. Bus 310 may include one or more buses, where appropriate. Although embodiments of the present invention describe and illustrate a particular bus, the present invention contemplates any suitable bus or interconnect.

It is to be understood that the present invention is not limited to the specific arrangements and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above-described embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the sequence of steps after comprehending the spirit of the present invention.

The functional blocks shown in the above structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, elements of the invention are programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted over a transmission medium or communication link by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments may be downloaded via a computer network such as the Internet, an intranet, or the like.

It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps may be performed in the order mentioned in the embodiments, or may be different from the order in the embodiments, or several steps may be performed simultaneously.

The above are only specific implementations of the present invention, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, modules and units can be referred to in the foregoing method embodiments. The corresponding process is not repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should all cover within the protection scope of the present invention.

Claims (12)

1. a generation method of video fingerprint, is characterized in that, the generation method of described video fingerprint comprises: Preprocessing the video to be processed, and extracting the preprocessed key frame image of the video to be processed; The key frame image is subjected to dimensionality reduction processing through a compressed sensing algorithm to obtain a sparse matrix; The sparse matrix is subjected to block processing, and a preset block block matrix is extracted, and a feature matrix composed of an average value of the preset block block matrix is calculated; Perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video to be processed. 2. The method for generating a video fingerprint according to claim 1, wherein the step of preprocessing the video to be processed, and extracting the key frame image of the preprocessed video to be processed specifically comprises: The processed video is converted into a frame image at a preset fixed frame rate, and the key frame image is extracted from the frame image according to a preset ratio. 3. The method for generating video fingerprints according to claim 1, wherein the video to be processed is preprocessed, and the step of extracting the key frame image of the preprocessed video to be processed is the same as the step of extracting the key frame image of the video to be processed. The steps of performing dimensionality reduction processing through compressed sensing theory to obtain a sparse matrix also include: Grayscale conversion is performed on the key frame image, the height of the converted key frame image is adjusted to a preset height, and the width of the converted key frame image is adjusted to a preset width. 4. The method for generating a video fingerprint according to any one of claims 1 to 3, wherein the feature matrix is subjected to singular value decomposition, and the singular value obtained after the singular value decomposition of the feature matrix is performed After the step of defining the fingerprint of the video, the method further includes: encoding the fingerprint by using a Gray code. 5. The method for generating video fingerprints according to any one of claims 1 to 3, wherein, The specific steps of extracting the preset block-blocking matrix are: arranging the values of all the block-blocking matrices obtained by dividing the sparse matrix into blocks in descending order, and selecting presets in descending order. The block matrix of the number of blocks is used as the preset block block matrix. 6. A device for generating video fingerprints, wherein the device for generating video fingerprints comprises: a preprocessing unit for preprocessing the video to be processed, and extracting the preprocessed key frame image of the video to be processed; a dimensionality reduction processing unit, configured to perform dimensionality reduction processing on the key frame image through a compressed sensing algorithm to obtain a sparse matrix; a block processing unit, configured to perform block processing on the sparse matrix, extract a preset block block matrix, and calculate a feature matrix composed of an average value of the preset block block matrix; The fingerprint generating unit is configured to perform singular value decomposition on the feature matrix, and define the singular value obtained after the singular value decomposition of the feature matrix as the fingerprint of the video to be processed. 7. a matching method of video fingerprint, is characterized in that, applying the generation method of video fingerprint as described in any one of claim 1 to 4 to generate the fingerprint of target video and video to be matched, and the fingerprint of described video fingerprint is generated. Matching methods include: extracting the first fingerprint of the first key frame of the target video; The first fingerprint is compared with the fingerprint of each key frame of the video to be matched, and the second key frame in the video to be matched is obtained, and the second key frame is each key frame in the video to be matched. The key frame with the smallest Euclidean distance from the fingerprint of the key frame to the first fingerprint; Taking the second key frame as the center, select the number of key frames contained in the video to be matched forward and backward respectively according to the preset duration, so that the number of key frames contained in the selected video to be matched is the same as the number of the key frames contained in the selected video to be matched. The number of key frames of the target video is equal; Calculate the Euclidean distances between the fingerprints of all key frames in the target video and the selected key frames in the video to be detected one by one, and when each Euclidean distance is less than the threshold, it is determined that the selected video to be matched at this time is the target. Similar videos of videos. 8. The method for matching video fingerprints according to claim 7, wherein the step of extracting the first fingerprint of the first key frame of the target video specifically comprises: Any key frame in the target video is randomly selected as the first key frame. 9 . The method for matching video fingerprints according to claim 7 , wherein the preset duration is 3 seconds. 10 . 10. A matching device of a video fingerprint, wherein the matching device of the video fingerprint applies the generation method of the video fingerprint according to any one of claims 1 to 5 to carry out the fingerprint of the target video and the video to be matched. Generating, the matching device of the video fingerprint includes: an extraction unit for extracting the first fingerprint of the first key frame of the target video; A comparison unit, configured to compare the first fingerprint with the fingerprint of each key frame of the video to be matched, and obtain a second key frame in the video to be matched, and the second key frame is the key frame of the to-be-matched video Match the fingerprint of each key frame in the video to the key frame with the smallest Euclidean distance of the first fingerprint; The selection unit is used to select the number of key frames contained in the video to be matched forward and backward according to the preset duration respectively, with the second key frame as the center, so that the key frame contained in the selected video to be matched is selected. The number of frames is equal to the number of key frames of the target video; The calculation unit is used to calculate the Euclidean distance between the fingerprints of all key frames in the target video and the selected key frames in the video to be detected one by one, and when each Euclidean distance is less than a threshold, it is determined that the selected at this time is to be matched. The video is a similar video to the target video. 11. A computer device, characterized in that the device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for generating video fingerprints according to any one of claims 1 to 5 or the method for matching video fingerprints according to any one of claims 6 to 9 is implemented. 12. A computer storage medium, characterized in that, computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the video fingerprint according to any one of claims 1 to 5 is realized The generation method or the video fingerprint matching method according to any one of claims 6 to 9.