JP5702269B2 - Parallel video analysis device, parallel video analysis method, and parallel video analysis program - Google Patents

Description

  The present invention relates to a technique for analyzing video data in parallel.

  There are video analysis techniques for detecting various events such as cut scenes, camera work, telops, and music in a video by using physical characteristics of image data and sound data of an image frame in the video.

  This video analysis technique is a technique for finding differences in physical characteristics between image frames using two or more image frames that are temporally related, and detecting the differences as events. For example, the telop display is detected on the basis of the difference or the degree of the gray value of the pixel between time-sequential image frames.

  Such a video analysis technique sequentially compares image frames from the beginning to the end of the video, and therefore requires a video analysis time proportional to the number of image frames constituting the video. This shortening of time is a major issue in the field of video analysis technology.

  In order to deal with such a problem, there is a method of performing video analysis every several image frames by thinning out image frames to be analyzed instead of using all image frames of the video. Thereby, since the number of image frames to be analyzed is reduced, the video analysis time can be shortened.

  Patent Document 1 describes a method of dividing an analysis target video into a plurality of partial videos, analyzing the partial videos in parallel, and integrating the results into one. As a result, the number of image frames to be analyzed by one processing worker is reduced to the number of frames in the divided partial images, so that the image analysis time can be shortened.

  By adopting these two methods, it is possible to further shorten the video analysis time required for video analysis.

JP 2003-69946 A

  However, if these two methods are executed at the same time, the event detection results are likely to be different from the case where the video is sequentially analyzed from the beginning to the end by one processing worker.

  When video analysis is started from the divided partial video, for example, as shown in FIG. 10, seek is performed to the head of the partial video, and analysis is performed while skipping image frames at the same thinning interval from the image frame at the seek position. Will proceed.

  Therefore, depending on the seek position, the image may be displayed using an image frame (◯ in FIG. 10) that is different from the image frame (● in FIG. 10) when “sequentially analyzing the image from the beginning to the end with one processing worker”. Since the analysis is performed, the same event detection result cannot be obtained.

  The present invention has been made in view of the above-mentioned problems, and the object of the present invention is to obtain the same event detection result as that obtained when video analysis is sequentially performed from the top of the video.

The parallel video analysis apparatus according to claim 1 reads out video data from the storage means and obtains a plurality of image frames located at an arbitrary integral multiple of a predetermined thinned-out image frame interval from the first image frame of the video data . and 1 means, and having a second means for initiating each detection processing of event information in the video data temporally parallel the decimated image frame interval from each image frame to the position.
The parallel video analysis device according to claim 2, wherein the first means divides the video data and obtains a number of a divided frame existing at the divided position. The image frame located at the integer multiple is obtained by correcting the number of the divided frame with a floor function.

4. The parallel video analysis method according to claim 3, wherein in the parallel video analysis method performed by the parallel video analysis device, the video data is read from the storage means, and an arbitrary integer of predetermined thinned image frame intervals from the first image frame of the video data. a first step of obtaining a plurality of image frames located doubled th first starts each detection processing of event information in the video data from each image frame to the temporally located parallel by the thinned image frame interval 2 and having the steps of, a.
The parallel video analysis method according to claim 4 is the parallel video analysis method according to claim 3, wherein in the first step, the video data is divided and a number of divided frames existing at the divided position is obtained. The image frame located at the integer multiple is obtained by correcting the number of the divided frame with a floor function.

A parallel video analysis program according to claim 5 causes a computer to execute the parallel video analysis method according to claim 3 or 4 .

  As described above, according to the present invention, a plurality of image frames located at an arbitrary integer multiple of a predetermined thinned-out image frame interval are obtained from the first image frame of video data, and the thinned-out image frame interval is determined from the respective image frames. Therefore, the event information detection process in the video data is started in parallel with each other in time, so that the same event detection result as that obtained when the video analysis is sequentially performed from the top of the video data can be obtained.

  According to the present invention, it is possible to obtain the same event detection result as that obtained when video is sequentially analyzed from the top of the video.

It is a figure which shows the functional block structure of a parallel video analyzer. It is a flowchart which shows the production | generation process of an analysis task. It is a figure explaining correction | amendment of a video analysis start point. It is a figure explaining the specific example of correction | amendment of a video analysis start point. It is a figure explaining the example of analysis task data. It is a flowchart which shows the video analysis process performed by each processing worker. It is a figure which shows the example of analysis task allocation to a processing worker. It is a figure which shows the construction example of a parallel video analysis apparatus. It is a figure which shows the other example of a production | generation of an analysis task. It is a figure explaining the conventional video-analysis result method.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. However, the present invention can be implemented in many different modes and should not be construed as being limited to the description of the present embodiment.

  FIG. 1 is a diagram showing a functional block configuration of the parallel video analysis apparatus according to the present embodiment. The parallel video analysis device 1 is communicably connected to a data input device 2 that can input video data to the parallel video analysis device 1, and includes a video storage unit 11, an analysis task generation unit 12, and an analysis task management unit 13. The video analysis unit 14, the analysis result integration unit 15, the analysis result storage unit 16, and the data output unit 17.

  The parallel video analysis apparatus 1 can be realized by an information processing apparatus such as a personal computer provided with storage means such as a memory and control means such as a CPU, and is executed by a program. Hereinafter, each functional unit will be described in detail.

  The video storage unit 11 holds the video data input by the data input device 2 in a memory or a hard disk and stores it. Video data is composed of a collection of a plurality of time-series image frames, and unique frame numbers are continuously assigned in ascending order.

  This video data is transmitted by a communication protocol such as HTTP (Hypertext Transfer Protocol) or SMB (Server Message Block), and is copied and stored in a file format. A video capture board or the like may be used to input as a video signal and hold it in a file format.

The analysis task generation unit 12 acquires the video data from the video storage unit 11, and based on the video length L of the video data and the thinned image frame interval S at the time of video analysis, each processing worker W _i (i = 0, 1, 2,..., W-1, W = number of processing workers), the number of the image frame to start video analysis (hereinafter referred to as video analysis start image frame number) s _i and the number of image frames to end video analysis. A number (hereinafter, image analysis end image frame number) e _i is calculated to generate an analysis task T _i .

Here, the image frame of the number s _i is video data so that the same event detection result as that obtained when the video analysis unit 14 sequentially analyzes the video from the first image frame of the video data can be obtained at the time of parallel video analysis by the video analysis unit 14. A plurality of image frames located at an arbitrary integer multiple of a predetermined thinned-out image frame interval S from the first image frame are detected. This will be described in detail below.

  FIG. 2 is a flowchart showing analysis task generation processing.

  First, a video data file to be analyzed is read from the video storage unit 11 (step S101), and the video length L of the video data is acquired from the file (step S102). The video length L of the video data is, for example, the total number of image frames constituting the video data.

Next, in order to generate the analysis task T ₀ assigned to the first processing worker W ₀ , i is set to 0 (step S 103), and “subscript _i of processing worker W i ≧ number of processing workers W” is established. It is determined whether or not to perform (step S104).

Next, when the determination is not satisfied in step S104, using expression (1), the video data is divided into partial video data for the number of W processing workers, that is, at the division position. The number of divided image frames to be performed (hereinafter referred to as divided image frame number) d _i is calculated (step S105).

d _i = i · (L / W) (1)
Next, the divided image calculated in step S105 using the floor function floor of Expression (2) so that the divided image frame number d _i is an integral multiple of the thinned image frame interval S from the first image frame of the video data. The frame number d _i is corrected (step S106).

d _i ′ = floor (d _i / S) · S (2)
At this time, using the equation (3), the corrected divided image frame number d _i ′ is set as the video analysis start image frame number s _i, and using the equation (4), the video analysis end image frame number e _i. Calculate

s _i = d _i ′ (3)
e _i = s _{i + 1} (4)
Then, it stored in the analysis task management unit 13 in association with the calculated video analysis start image frame number _{s i} and video analysis end image frame number _{e i} in step S106 the analysis task _{T i} (step S107).

  Thereafter, after i is incremented (step S108), the process returns to step S104, and steps S105 to S108 are repeated until the determination in step S104 is established. If the determination is satisfied in step S104, the analysis task generation process ends.

From the above, the divided image frame number d _i is corrected, for example, as shown in FIG. 3, and the corrected number d _i is set as the video analysis start image frame number s _i . Specifically, for example, when L = 7000, W = 6, and S = 100, the correction is performed as shown in FIG. 4, and an analysis task as shown in FIG. 5 is generated.

The analysis task management unit 13 stores the analysis task T _i generated by the analysis task generation unit 12 as analysis task data in association with the video data that was the generation target of the analysis task.

The video analysis unit 14 includes a plurality of processing workers W _i (video analysis processing execution unit), and each processing worker W _i reads video data to be analyzed from the video storage unit 11 and relates to the video data. The analysis task data is read from the analysis task management unit 13, and the analysis task T _i having the same i in the analysis task data is executed in parallel in time as the analysis task assigned to itself.

FIG. 6 is a flowchart showing video analysis processing performed by each processing worker. Each processing worker W _i seeks to the image frame of the assigned video analysis start image frame number s _i , and starts event information detection processing in parallel in time for each thinned image frame interval S. Details will be described below.

First, the processing worker W _i obtains the video analysis start image frame number s _i and the video analysis end image frame number e _i having the same _i from the analysis task data (step S201), and sets the thinned image frame interval S. The coefficient n is set to 0 (step S202).

Next, the image frame of the video analysis start image frame number s _i is set as the current image frame, the data of the current image frame is obtained from the video data (step S203), and the number of the current image frame (hereinafter referred to as “the current image frame number”). , Current image frame number) c is acquired (step S204).

Next, it is determined whether or not “current image frame number c ≧ video analysis start image frame number s _i + n × thinned image frame interval S” is satisfied (step S205).

  Next, when the determination is satisfied in step S205, since the current image frame is an image frame located at the preset thinned-out image frame interval S, event detection is performed using the image frame data acquired in step S203. The process is executed (step S206), and n is incremented (step S207).

  The event detection process is executed using two or more image frames that are temporally related, but how many image frames are used at which position as viewed from the current image frame. Is preset.

  On the other hand, if the determination is not satisfied in step S205, the current image frame is not an image frame located at the preset thinned-out image frame interval S, so the process proceeds to step S208 without executing steps S206 and S207.

Finally, it is determined whether or not the event detection process within the range acquired in step S201 is completed (step S208). For example, the current image frame number c is available for determining when it reaches the video analysis end image frame number e _i a (c ≧ e _i) as a termination condition.

  If it is determined in step S208 that the process has not been completed, the process returns to step S203, and steps S203 to S208 are repeated with the next image frame as the current image frame. On the other hand, when the processing is finished, the video analysis processing is finished and transmitted to the analysis result integration unit 15.

  Note that each processing worker does not communicate its execution state with other processing workers and can independently execute the video analysis processing, so that the video data shown in FIG. Video analysis processing can be executed in parallel in time.

  As a specific method of event detection processing performed in step S206, any detection method can be adopted as long as event information can be detected, and the method is not limited in this embodiment. The same applies to the detection of the type of event information.

Figure 7 shows an example of assigning the analysis task T _i to the processing worker W _i on the basis of the analysis task data shown in FIG. In this example, six processing workers W ₀ to ₅ share one video file, and each processing worker W _{0 to 5} has a video analysis start image frame in the analysis task T ₀ to ₅ respectively assigned _thereto. The analysis process is started from the image frame with the number s _i .

Analysis result integration unit 15 integrates the analysis result of each analysis task T _i analyzed by the processing worker W _i of the image analysis unit 14, and records the final analysis result in the analysis result storage unit 16.

  The information to be recorded includes “detection event” indicating the type of event detection such as cut, camera work, and telop, “detection start point (for example, detection start time or detection start frame number)” and “detection end” of the detection event. Point (for example, detection end time or detection end frame number) "or the like.

  The data output unit 17 reads out the analysis result of the video data from the analysis result storage unit 16 and outputs it to the outside.

  The user can read out this final analysis result by the data output unit 17 and use it as the video analysis result. For example, it may be displayed on a computer display or read out in a format that can be used by another device such as a text file, and the other device may perform further processing.

  Finally, a specific construction example of the parallel video analysis apparatus described above will be described. FIG. 8 is a diagram illustrating a construction example of a parallel video analysis apparatus. As shown in (a) of the figure, one core of the multi-core CPU may be used as one processing worker, and as shown in (b) of the same figure, one server machine may be used as one processing worker.

  In this embodiment, the analysis task management unit assigns an analysis task to each processing worker, and analyzes the analysis task to which the processing worker is assigned independently. For this reason, the setting method of a processing worker is not ask | required if it is an environment which can operate a processing worker independently.

  However, as described above, since each processing worker shares video data and proceeds with video analysis processing, it is necessary to consider transfer of video data. Therefore, in the case of FIG. 8B, it is necessary to manage the analysis task including the communication network state between the analysis servers.

  In addition to the construction example shown in FIG. 8, parallel processing of video analysis according to the present embodiment may be executed by a plurality of server machines having multi-core CPUs.

  Further, when a plurality of pieces of video data are input to the parallel video analysis apparatus, it is possible to cope with the input video data by managing them in a queue.

  That is, as shown in FIG. 9A, the analysis task management unit manages the generated analysis task by adding a video ID for identifying video data and a task state indicating whether or not the task is executed. At the time of video analysis processing execution, as shown in FIG. 5B, among analysis tasks whose task status is not executed, analysis tasks having the same video ID are arranged in order from the top (for example, task ID and analysis start frame are in ascending order). Will be assigned to.

  As described above, according to the present embodiment, the analysis task generation unit 12 obtains a plurality of image frames located at an arbitrary integral multiple of a predetermined thinned image frame interval from the first image frame of the video data, and the video analysis unit Since each of the 14 processing workers starts the event information detection process in the video data in parallel in time from the obtained image frames at the thinned image frame intervals, the video data is divided into a plurality of partial video data. In this case, it is possible to prevent erroneous detection of an event that occurs at the time, and to always analyze only the same image frame. As a result, it is possible to obtain the same event detection result as when video analysis is sequentially performed from the beginning of the video data.

  In addition, according to the present embodiment, since video analysis is executed in parallel and thinned out in time, when video analysis is performed sequentially from the beginning of the video data with the video analysis time required for video analysis shortened The same event detection result can be obtained.

DESCRIPTION OF SYMBOLS 1 ... Parallel video analysis apparatus 11 ... Video | video memory | storage part 12 ... Analysis task production | generation part 13 ... Analysis task management part 14 ... Video | video analysis part 15 ... Analysis result integration part 16 ... Analysis result memory | storage part 17 ... Data output part S101-S108, S201 ~ S208 ... Step

Claims (5)

From the storage means reads the video data, first means for obtaining a plurality of image frames located at any integer multiple eyes of the image given decimated image frame interval from the leading image frame of data,
A second means for initiating each detection processing of event information in the video data from each image frame to the location in parallel temporally in the thinned image frame interval,
A parallel video analysis apparatus comprising: The first means includes
Dividing the video data, obtaining a number of a divided frame existing at the divided position, and correcting the number of the divided frame with a floor function to obtain an image frame located at the integer multiple The parallel video analysis apparatus according to claim 1. In the parallel video analysis method performed by the parallel video analysis device,
A first step of reading video data from the storage means and obtaining a plurality of image frames located at an arbitrary integral multiple of a predetermined thinned-out image frame interval from the first image frame of the video data;
A second step of starting detection processing of event information in video data in parallel in time from each image frame located at the thinned image frame interval;
A parallel video analysis method comprising: In the first step,
Dividing the video data, obtaining a number of a divided frame existing at the divided position, and correcting the number of the divided frame with a floor function to obtain an image frame located at the integer multiple The parallel video analysis method according to claim 3. A parallel video analysis program for causing a computer to execute the parallel video analysis method according to claim 3.

Images