KR101037710B1 - Automatic production system of photographed video - Google Patents

Abstract

The present invention relates to a system for automatically directing a photographed video to effectively organize a video and a composition of a video by automatically performing a video for a subject simultaneously taken through a plurality of cameras, the board (B) and the lecturer (subject) First and second cameras 11 and 12 respectively photographing M); Photographing position adjusting means (20) for adjusting photographing directions and magnifications of the first and second cameras (11, 12); An image analysis module 30 for identifying a facial region among the images of the lecturer M photographed by the second camera 12 and at least one selected region among the regions of the eye area within the facial region; An action checking module 40 for checking a position bias of at least one selected area of each area of the eye area based on the face area, and confirming the behavior of the lecturer M; Check the setting data according to the behavior of the lecturer (M), edit the video photographed by the first and second cameras 11 and 12 according to the setting data, and operate the first and second cameras 11 and 12. An image editing module 50 for transmitting control data according to the setting data; A control module 60 for manipulating the photographing position adjusting means 20 according to the control data of the image editing module 50; And an image DB 70 that stores the produced image data edited by the image editing module 50.

Description

Automatic presentation system of filmed images {Presentation system of the picture}

The present invention relates to an automatic production system of a photographed video object to automatically organize a moving picture for a subject photographed simultaneously through a plurality of cameras to effectively organize the structure and configuration of the moving picture.

With the advancement and popularization of devices that can record video such as moving pictures or still images in an analog or digital manner, the general public, as well as experts, can shoot and record still images and videos for various purposes and purposes. .

On the other hand, as the use of various communication means such as the Internet has rapidly increased and the utilization thereof is easy, people frequently upload the contents they want to express to the public in writing, or produce and upload still images or videos. In addition, he recorded his own video on videotape or recorded on various recording media such as CDs or discs, so that his contents could be notified or propagated offline.

Among these videos are special purpose videos such as lectures. These videos are mixed with the filming of the lecturer (hereinafter referred to as the "lecturer") and the recording of the blackboard or whiteboard (hereinafter referred to as the "board") recorded by the lecturer, improving the efficiency of delivering the lecture contents. It is directed and edited by an expert as much as possible. More specifically, the video for the purpose of teaching the lecture contents through the directing technique such as showing only the lecturer, close-up the contents recorded on the board according to the lecturer's action, or show the lecturer and the board together. It is designed to be effectively delivered to viewers.

However, the general public who is not an expert can not effectively apply the above-described directing technique when producing a video for the purpose of teaching, especially when the video production using two cameras, there was a limit to apply the directing technique alone. In the end, producing a special-purpose video that required professional directing techniques such as lectures was not easy for the general public.

Accordingly, the present invention has been made to solve the above problems, and has a special purpose, such as lectures of the automatic production system of the photographing video that allows the non-experts to proceed effectively the video production work that should be applied to a variety of directing techniques alone Provision is a technical task.

According to an aspect of the present invention,

First and second cameras for photographing a board and a lecturer as a subject;

Photographing position adjusting means for adjusting photographing directions and magnifications of the first and second cameras;

An image analysis module configured to identify a facial area of the lecturer's image photographed by the second camera and at least one selected area among respective areas of the eye area within the facial area;

An action checking module for checking a position bias of at least one selected area of each area of the eye area based on the face area, and confirming the lecturer's behavior;

An image editing module that checks the setting data according to the lecturer's actions, edits the image taken by the first and second cameras according to the setting data, and transmits the control data according to the setting data for the operation of the first and second cameras. ;

A control module for operating the photographing position adjusting means according to the control data of the image editing module; And

Image material DB for storing the produced image data edited in the image editing module;

It is an automatic rendering system of the photographed video including.

According to the present invention, a non-specialist alone can easily produce a video such as a lecture through various directing techniques, and the produced video can be optimized according to the lecturer's behavior and the contents of the lecture. It is effective to learn the contents effectively.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a plan view schematically showing a state of a shooting site to which the directing system according to the present invention is applied, and FIG. 2 is a block diagram showing a state of the directing system according to the present invention.

The production system according to the present invention comprises at least two or more first and second cameras 11 and 12, photographing position adjusting means 20 for adjusting a photographing direction of the first and second cameras 11 and 12, and a first one. , An image analysis module 30 for analyzing images captured by the two cameras 11 and 12, an action checking module 40 for checking the lecturer's behavior, and first and second cameras 11 set according to the lecturer's behavior. 12, the image editing module 50 for checking the shooting direction and the shooting method, and the shooting position adjusting means 20, the image analysis module 30, the behavior checking module 40, and the image editing module 50. A control module 60 for controlling the driving, an image DB 70 for storing the edited image, an output module 80 for outputting the image as necessary, and a setting module for resetting the captured image ( 90) may be further included.

The first and second cameras 11 and 12 are conventional photographing means for photographing the lecturer M and the board B, which are subjects, respectively. Possible known devices.

The photographing position adjusting means 20 adjusts the photographing direction of the first and second cameras 11 and 12, and ascends and descends the first and second cameras 11 and 12, and the first and second cameras 11 and 12. Adjust mechanical movements such as up, down, left, and right rotation. In general, the first and second cameras 11 and 12 and the photographing position adjusting means 20 are combined to be called tilt cameras. In addition, the photographing position adjusting means 20 may be adjusted so that the second camera 12 continuously senses the movement of the lecturer M by detecting the movement of the lecturer M. FIG. To this end, the photographing position adjusting means 20 detects the lecturer M through a sensor (not shown) that detects the body temperature or the pronunciation sound of the lecturer M, and the second camera 12 in the detected direction. Aim the shooting direction.

Meanwhile, the photographing position adjusting means 20 may be applied to any device capable of adjusting the mechanical movement of each of the first and second cameras 11 and 12. For example, the raising or lowering or horizontal movement of the first and second cameras 11 and 12 may enable a linear movement along the ball screw and the guide by applying a ball screw structure, and the first and second cameras 11 and 12. , 12) may be enabled by a rotating motor and a rotating shaft gear.

In addition, the photographing position adjusting means 20 may be variously modified within the scope of the following claims as long as the device can adjust the position and the photographing direction of the first and second cameras 11 and 12.

The image analysis module 30 analyzes an image to distinguish subjects photographed by the first and second cameras 11 and 12, and the lecturer M photographed by the second camera 12 as illustrated. Check your face, throat, and body. Driving of the image analysis module 30 for this purpose will be described in detail below.

The behavior checking module 40 checks the behavior of the lecturer M, which is the subject identified in the image analysis module 30, and detects what behavior the lecturer M is currently taking.

The image editing module 50 checks the photographed video object according to the behavior of the lecturer M identified in the action confirming module 40, and controls the photographing position adjusting unit 20 to the photographed video object. Send control data for

The control module 60 controls the operation of the photographing position adjusting means 20 according to the control data transmitted from the image editing module 50, and controls the image analysis module 30, the behavior checking module 40, and the image editing module. The on / off of 50 is controlled.

The image object DB 70 stores the edited image object as image data.

The output module 80 is a conventional displayer for outputting an image.

The setting module 90 may be configured to reset the setting values, such as the color set in the image analysis module 30 or the photographed image of the image editing module 50 in accordance with the subject's operation. Here, the setting module 90 is a conventional computer, and is programmed to process input and editing of setting values, which are the execution criteria of the image analysis module 30 and the image editing module 50.

3 is a flowchart sequentially showing a first embodiment of the rendering method according to the present invention, Figure 4 is a view showing a face labeling state performed by the rendering system according to the present invention, Figure 5 according to the present invention The drawing system shows the principle of checking the lecturer's gaze.

S11; Object shooting stage

The first and second cameras 11 and 12 are disposed toward the subject.

For reference, in order to clearly describe the implementation of the production system according to the present invention, the subject is a lecture of the lecturer (M).

The first camera 11 photographs the board B, and the second camera 12 photographs the lecturer M. FIG. In this case, the first camera 11 may selectively photograph the entire board B or only one point of the board B, and the second camera 12 selectively selects only the whole body, the upper body or the hand of the lecturer M, and the like. You can shoot. In other words, the first and second cameras 11 and 12 are photographed by the photographing position adjusting means 20, and the lecturer M and the board B are generally used as necessary while the position adjustment and the photographing direction and the magnification are adjusted. Or you can shoot partially.

The subject lecturer M and the board B may be continuously photographed, and the photographed image may be temporarily stored. In addition, the first and second cameras 11 and 12 may be selectively driven only when necessary.

S12; Checking the posture of the subject

The image analysis module 30 analyzes the appearance of the lecturer M photographed by the second camera 12.

To this end, the image analysis module 30 separates the image photographed by the second camera 12 in units of pixels and checks the detected color of each pixel. At this time, the image analysis module 30 checks the human lecturer (M), the color will be the skin color. In this case, the skin color may be a normal color, and the color to be checked may be variously set according to race. For reference, the setting value for setting the skin color may be directly performed by the operator in the setting module 90.

In an embodiment of the present invention, the image analysis module 30 analyzes an image by detecting a pixel in which the color (skin color) is identified and labeling it.

The image analysis module 30 checks the pixel for detecting the skin color in the image captured by the second camera 12, and then labels the pixel as '1' and other pixels as '0'. That is, as shown in FIG. 4 (a), the face area of the lecturer M is primarily sensed. For reference, FIG. 4 (a) shows the entire face of the lecturer M as labeled '1', but is actually labeled as '1' along the edge identified as the face of the lecturer M, while It may be possible to estimate the contour.

When the detection of the facial area is completed, the color detected by the pixels in the facial area is checked again, and the non-skin part is labeled as '1', and the skin part is labeled as '0'. That is, as shown in FIG. 4 (b), the second eye area of the lecturer M is detected in the facial area.

As shown in FIG. 4 (b), the eye gaze of the lecturer M may be estimated by identifying the face area of the lecturer M and the position of the eyes. To explain this in more detail, the behavior checking module 40 checks the position of the eye and the nose or the mouth based on the face region of the lecturer M through the photographed image analyzed by the image analysis module 30, The lecturer (M) can see the eye.

For example, FIG. 5 (a) illustrates the appearance of the lecturer M photographed by the second camera 12, and the lecturer M of the figure stares at the camera. The behavior check module 40 confirms that the eye, nose or mouth of the lecturer M is located at the center of the face area, and the gaze of the lecturer M faces the front, that is, the second camera 12. Estimate.

On the other hand, as shown in Figure 5 (b), the behavior check module 40 confirms that the eye and nose or mouth of the lecturer (M) to the right side of the face area (based on the lecturer), lecturer (M) It is assumed that the line of sight is facing to the right.

For reference, whether the gaze is biased may be determined by the number of left and right pixels of the pixel corresponding to the eye area in the facial area.

S13; Eye output check step

The image editing module 50 receives the gaze information of the lecturer M estimated by the behavior checking module 40 and checks the photographed image in which the behavior of the lecturer M is set according to the corresponding gaze information.

Here, the setting data is set by the director information about the composition, editing, and the like of the video images photographed by the first and second cameras 11 and 12 according to the direction of the lecturer's eye. That is, when it is confirmed that the gaze of the lecturer M faces the front, the direction information outputs only an image of the lecturer M photographed on the second camera 12, and the gaze of the lecturer M is left or right. When it is confirmed that the camera is facing the first or second cameras (11, 12), the image taken by each of the composite, or the lecturer (M) and the board (B) of either the first camera 11 or the second camera 12 This is to control the video output of the camera being photographed. For reference, when it is confirmed that the eye of the lecturer M faces upward or downward, the lecturer M reads a memo or takes a specific gesture, so the lecturer M may be regarded as facing. will be.

In addition, the degree of gaze bias of the lecturer M may be checked, and it may be estimated where the gaze of the lecturer M faces. In more detail, when the front line of sight of the lecturer M is 90 degrees, if the line of sight of the lecturer M photographed by the second camera 12 is within 20 degrees (reference value), the lecturer M is the current board ( B) can be considered to be viewed, and if it exceeds 20 degrees, the lecturer (M) can be considered to be facing the front. This criterion may be directly adjusted through the setting module 90 according to the shape and pattern of the actual lecturer (M).

S17; Image processing step

The image editing module 50 checks the eye angle of the lecturer M and synthesizes or combines the lecturer M and the board B photographed by the first and second cameras 11 and 12 when the eye angle is within the reference value. The output may be edited or edited so that only the board B is output. When the reference value is exceeded, only the lecturer M photographed by the second camera 12 may be edited.

S18; Output stage

The control module 60 checks the image edited in the image editing module 50 and transmits it to the output module 80, and the output module 80 outputs the image.

Of course, since the output module 80 is not an essential configuration, the control module 60 may not proceed to output the image.

In addition, the video material may be broadcast in real time through an online (network) transmission. In other words, there is no need to proceed a separate editing process, it is possible to proceed live.

S19; Save step

The control module 60 checks the image edited in the image editing module 50 and transmits the image to the image DB 70, and the image DB 70 stores the image.

For reference, the image object DB 70 may be conventional hardware, but may also be a conventional recording medium (software) such as a video tape or a CD.

FIG. 6 is a flowchart sequentially illustrating a second embodiment of the directing method according to the present invention, and FIG. 7 is a view illustrating a face labeling performed by the directing system according to the present invention.

The production system according to the present invention can check not only the eyes of the lecturer M but also the behavior of the lecturer M, so as to enhance the delivery power and understanding of the lecturer.

Hereinafter, a description of the parts described in the first embodiment according to the present invention will be omitted.

S15; Check output point

The image analysis module 30 checks the color of the lower part of the face area after checking the face of the lecturer (M). At this time, the identified color is regarded as the body (upper body) color of the lecturer M, and the area for the same or similar color is identified.

For reference, since the clothing worn by the lecturer M may be different from the skin color, the image analysis module 30 automatically converts the color detected by the pixel checked after the lower boundary of the facial area into the clothing color of the lecturer M. Will set. However, through the setting module 90, the operator may set the clothes color of the lecturer M separately so that the image analysis module 30 may check this. On the other hand, the image analysis module 30 is labeled in the same way as the facial area, the body portion of the lecturer (M) as the label '1' and other pixel portion is labeled as '0', body area It can be specifically confirmed.

Subsequently, the behavior check module 40 may estimate the state of the body of the lecturer M by checking the horizontal width of the body region identified by the image analysis module 30. In more detail, as shown in Figure 7 (a), (b) there is a difference in the horizontal width of the portion corresponding to the body region, this difference means that the lecturer (M) is different, FIG. 7 (a) shows that the lecturer M is standing toward the second camera 12, and FIG. 7 (b) shows that the lecturer M is standing toward the board B for lecture. 40) This can be confirmed.

On the other hand, as shown in Figure 7 (b), the lecturer (M) may define the target point (P) through the movement of the arm. Of course, whether or not the movement of the arm can be estimated through the withdrawal of the body region, the behavior check module 40 may be regarded as the target (P) to indicate the end of the drawn body region.

The image editing module 50 determines whether to output the pointing object P when the pointing object P is confirmed. That is, when the lecturer M points to a point P of the board B, the board B photographed by the first camera 11 is output so that the viewer can concentrate on the point P. It is. Of course, the image editing module 50 may output only the board B, or may output the board B and the lecturer M together.

Meanwhile, when the lecturer M temporarily points to the board B, the image editing module 50 continuously outputs only the lecturer M without outputting the pointed object P, in order to prevent the screen transition from occurring. You may. Criteria for determining whether or not the output of the target object (P) will be the pointing time of the lecturer (M), which can also be reset through the setting module 90, of course.

S16; Target status adjustment stage

When the lecturer (M) writes on the board (B) or continuously points to a point of the video such as a picture posted on the board (B), the viewer can effectively recognize and recognize the pointed object (P). In order to ensure that, only the point P may be greatly expanded. That is, the pointed object (P) can be zoomed in.

Therefore, when it is set to adjust the state of the object to be pointed (P), the image editing body 50 controls the photographing position adjusting means 20 via the control module 60, the first and second cameras (11, 12) ) Adjust the shooting direction or magnification. As a result, the first and second cameras 11 and 12 operate to zoom in and output only the pointed object P. FIG.

On the other hand, the image editing module 50 may visually highlight the point indicated by the lecturer (M) by image processing. In more detail, if the point indicated by the lecturer (M) is a letter, the character image is checked and color processing or automatic underlining is performed to emphasize the letter image, or when the point is a picture, the boundary of the picture image is defined. You can check and draw a border line on the picture image. In addition, various chromatographic techniques may be applied.

The processed image is processed through an output step S18 or a storage step S19, so that the viewer can watch it.

1 is a plan view schematically showing the appearance of the shooting scene to which the directing system according to the present invention is applied,

2 is a block diagram showing a state of the production system according to the present invention,

3 is a flowchart sequentially showing a first embodiment of the directing method according to the present invention;

4 is a view showing a face labeling state performed by the rendering system according to the present invention,

5 is a view showing a principle of the system to check the lecturer's gaze according to the present invention,

6 is a flowchart sequentially showing a second embodiment of the rendering method according to the present invention;

7 is a view showing a body labeling state performed by the rendering system according to the present invention.

Claims (5)

First and second cameras 11 and 12 photographing the board B and the lecturer M, which are the subjects, respectively; Photographing position adjusting means (20) for adjusting photographing directions and magnifications of the first and second cameras (11, 12); The lecturer M photographed by the second camera 12 while checking the facial area and at least one selected area among the areas of the eye area in the facial area of the image of the lecturer M photographed by the second camera 12. An image analysis module 30 for identifying a face region of the lecturer M and at least one region selected from among regions of the eye area based on the color detected by the pixel in the image of the image; An action checking module 40 for checking a position bias of at least one selected area of each area of the eye area based on the face area, and confirming the behavior of the lecturer M; Check the setting data according to the behavior of the lecturer (M), edit the video photographed by the first and second cameras 11 and 12 according to the setting data, and operate the first and second cameras 11 and 12. An image editing module 50 for transmitting control data according to the setting data; A control module 60 for manipulating the photographing position adjusting means 20 according to the control data of the image editing module 50; And An image material DB 70 for storing the produced image data edited by the image editing module 50; Automatic production system of the photographed video, characterized in that it comprises a. The method of claim 1, The setting module 90 processes inputting and editing of setting values which are driving standards of the image editing module 50 or the image analysis module 30 so that the setting values which are the setting data or the color information may be reset according to the subject. Automatic production system of the photographed video, characterized in that it further comprises a). First and second cameras 11 and 12 photographing the board B and the lecturer M, which are the subjects, respectively; Photographing position adjusting means (20) for adjusting photographing directions and magnifications of the first and second cameras (11, 12); The facial area and the one or more selected areas of the eye area in the facial area are checked among the images of the lecturer M photographed by the second camera 12, but the lecturer (based on the color detected by the pixel below the facial area) Checking the body region of M), the behavior check module 40 checks the horizontal width of the identified body region to determine the posture of the lecturer (M) (30); An action checking module 40 for checking a position bias of at least one selected area of each area of the eye area based on the face area, and confirming the behavior of the lecturer M; Check the setting data according to the behavior of the lecturer (M), edit the video photographed by the first and second cameras 11 and 12 according to the setting data, and operate the first and second cameras 11 and 12. An image editing module 50 for transmitting control data according to the setting data; A control module 60 for manipulating the photographing position adjusting means 20 according to the control data of the image editing module 50; And An image material DB 70 for storing the produced image data edited by the image editing module 50; Automatic production system of the photographed video, characterized in that it comprises a. The method of claim 3, wherein The image analysis module 30 checks the body region drawn out from the identified body region, sets the target point P to a point of the board B where the end of the drawn body region is located, and sets the image editing module ( 50) is an automatic rendering system of the photographed video, characterized in that for adjusting the state of the pointing object (P). delete

Images