CN106603912B - Video live broadcast control method and device - Google Patents
Video live broadcast control method and device Download PDFInfo
- Publication number
- CN106603912B CN106603912B CN201611104572.XA CN201611104572A CN106603912B CN 106603912 B CN106603912 B CN 106603912B CN 201611104572 A CN201611104572 A CN 201611104572A CN 106603912 B CN106603912 B CN 106603912B
- Authority
- CN
- China
- Prior art keywords
- camera
- interest point
- view
- shot
- suitability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000009432 framing Methods 0.000 claims abstract description 29
- 238000011217 control strategy Methods 0.000 claims abstract description 18
- 239000013589 supplement Substances 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 8
- 210000000887 face Anatomy 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000011410 subtraction method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Closed-Circuit Television Systems (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a video live broadcast control method and a video live broadcast control device, wherein the method comprises the following steps: acquiring a video image input signal of a positioning camera, and acquiring position information of an interest point in an image according to the video image input signal; according to the acquired position information of the interest points, calculating shooting parameters of each framing camera and generating control signals, and controlling each framing camera to shoot according to the shooting parameters; acquiring shot pictures of a plurality of view finding cameras, and calculating the suitability of the shot pictures of each view finding camera; according to a switching control strategy, one or more shooting pictures in a plurality of view finding cameras are output.
Description
Technical Field
The invention relates to the technical field of video playing, in particular to a live video control method and device for multi-camera automatic switching.
Background
In an existing multi-camera live video system, a plurality of cameras are generally equipped to shoot stage pictures from multiple angles, and then the most suitable camera is selected for live broadcasting with the picture effect. In order to achieve the effect of shooting at multiple angles, multiple cameras are required to be arranged at different positions to respectively shoot pictures, each camera is provided with a camera shooting person to track shooting of actors on a stage, and work division cooperation is carried out, for example, someone is responsible for close-up, someone is responsible for the full-view of the stage, the pictures of the multiple cameras are finally input to a broadcasting guide table, the broadcasting guide table is provided with a broadcasting guide, the broadcasting guide is used for watching the picture effect of each camera, and then the most appropriate camera picture is output for broadcasting according to the position and the face direction of the actors on the stage.
Therefore, in the existing multi-camera live broadcast technology, each camera needs to be manually tracked and shot, and the broadcast guide station needs to be manually screened out appropriate pictures for broadcasting, so that a professional and complete shooting broadcast guide team is needed, the cost is high, automation cannot be achieved, and the efficiency is low.
Disclosure of Invention
In order to overcome the defects in the prior art, the present invention provides a method and a device for controlling live video broadcast, so as to realize automation of multi-machine live video broadcast.
In order to achieve the above object, the present invention provides a video live broadcast control method, which includes the following steps:
acquiring a video image input signal of a positioning camera, and acquiring position information of an interest point in an image according to the video image input signal;
step two, according to the acquired position information of the interest point, calculating the shooting parameters of each framing camera and generating control signals, and controlling each framing camera to shoot according to the shooting parameters;
thirdly, acquiring the shot pictures of the plurality of view finding cameras, and calculating the suitability of the shot pictures of each view finding camera;
and step four, outputting one or more shooting pictures of the plurality of view finding cameras according to a switching control strategy.
Further, in the first step, a background difference method for detecting moving objects in the video image is adopted to obtain a moving target in the video image as an interest point, and the geometric center coordinate of the minimum enclosing rectangle of the moving target is the position information of the interest point.
Further, in the second step, the shooting parameters include a shooting direction parameter and/or a zoom parameter.
Further, the shooting direction parameter is a shooting direction angle of the view camera, and is obtained through the position information of the interest point and a preset position of the view camera.
Further, the zooming parameters are obtained through the shooting angle of the framing camera and the visual angle of the moving target at the position of the camera, and the visual angle of the moving target at the position of the camera is the included angle between the two diagonal vertices of the minimum enclosing rectangle of the moving target and the framing camera.
Further, the third step further comprises:
calculating the interest point area of the acquired shooting picture;
calculating to obtain the actual interest point ratio of the shot picture according to the obtained interest point area;
and calculating the suitability of the pictures shot by each view camera according to the actual interest point ratio and the preset expected interest point ratio.
Further, the switching control strategy selects corresponding shot pictures to output according to the service duration of each viewfinder camera and the suitability of the shot pictures of each viewfinder camera.
Further, the handover control policy includes one or more of the following 1) to 4):
1) if the current framing camera is continuously used for a time exceeding the longest tolerance time, switching to other framing cameras with the highest suitability;
2) if the fitness of other view finding cameras exceeds the fitness of the current view finding camera and the current use time of the current view finding camera is longer than the minimum use threshold, switching to other view finding cameras with the highest fitness;
3) if the proportion of the current use time length of the view-finding camera to the total use time length is lower than the lowest tolerance value and the current use time length of the current view-finding camera is greater than the minimum use threshold value, switching to the view-finding camera with the shortest current use time length;
4) and if the current use time of the view camera is lower than the minimum use threshold, switching of the shooting picture of the view camera is not performed no matter the suitability is.
Furthermore, the positioning camera adopts an infrared camera and an infrared light supplement lamp to shoot to obtain the video image.
In order to achieve the above object, the present invention further provides a live video control apparatus, including:
the positioning unit is used for acquiring a video image input signal of the positioning camera and acquiring the position information of an interest point in an image according to the video image input signal;
the control unit is used for calculating shooting parameters of each framing camera according to the acquired position information of the interest point, generating control signals and controlling each framing camera to shoot according to the shooting parameters;
the suitability processing unit is used for acquiring the shot pictures of the plurality of view finding cameras and calculating the suitability of the shot pictures of each view finding camera;
and the output selection unit is used for outputting one or more shooting pictures of the plurality of framing cameras according to a switching control strategy.
Further, the fitness processing unit further includes:
an interest point region calculation unit for calculating an interest point region of the acquired shot picture;
the actual interest point ratio calculating unit is used for calculating the actual interest point ratio of the shot picture according to the obtained interest point area;
and the fitness calculating unit is used for calculating the fitness of the pictures shot by each view camera according to the actual interest point ratio and the preset expected interest point ratio.
Further, the switching control strategy selects corresponding shot pictures to output according to the service duration of each viewfinder camera and the suitability of the shot pictures of each viewfinder camera.
Further, the handover control policy includes one or more of the following 1) to 4):
1) if the current framing camera is continuously used for a time exceeding the longest tolerance time, switching to other framing cameras with the highest suitability;
2) if the fitness of other view finding cameras exceeds the fitness of the current view finding camera and the current use time of the current view finding camera is longer than the minimum use threshold, switching to other view finding cameras with the highest fitness;
3) if the proportion of the current use time length of the view-finding camera to the total use time length is lower than the lowest tolerance value and the current use time length of the current view-finding camera is greater than the minimum use threshold value, switching to the view-finding camera with the shortest current use time length;
4) and if the current use time of the view camera is lower than the minimum use threshold, switching of the shooting picture of the view camera is not performed no matter the suitability is.
Compared with the prior art, the live video broadcast control method and the live video broadcast control device have the advantages that the position information of the interest point is obtained, the shooting parameters of all the view cameras are calculated according to the position information of the interest point, the corresponding control signals are generated to control all the view cameras, the suitability of all the view cameras is calculated for the shot pictures of all the view cameras, one or more paths of shot pictures are selected according to a switching control strategy to be output, the automatic control of a plurality of cameras is realized, the live video broadcast efficiency is improved, and the live video broadcast control method and the live video broadcast control device are particularly suitable for self-service scenes.
Drawings
FIG. 1 is a diagram illustrating a portion of the present invention;
fig. 2 is a flowchart illustrating steps of a video live broadcast control method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating location information of interest points in an embodiment of the present invention;
FIG. 4 is a schematic view of a shooting direction in an embodiment of the present invention;
FIG. 5 is a schematic view of a point of interest region of a frame captured by a viewfinder camera in accordance with an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a video live broadcast control apparatus according to a second embodiment of the present invention;
FIG. 7 is a detailed structure diagram of the fitness processing unit 603 according to an embodiment of the present invention;
FIGS. 8 and 9 are top and front views of a live video system in which embodiments of the present invention are applied;
fig. 10 is a schematic diagram illustrating an application scenario of a video live broadcast control apparatus according to an embodiment of the present invention.
Detailed Description
Other advantages and capabilities of the present invention will be readily apparent to those skilled in the art from the present disclosure by describing the embodiments of the present invention with specific embodiments thereof in conjunction with the accompanying drawings. The invention is capable of other and different embodiments and its several details are capable of modification in various other respects, all without departing from the spirit and scope of the present invention.
Before describing the present invention, some terms used in the present invention will be described below with reference to fig. 1:
the shooting direction indicates which direction the camera shoots towards the three-dimensional space;
zooming: representing zoom of a camera lens;
the interest point proportion represents the ratio of the interest points (such as human faces) to the whole picture area, and if the proportion is less, the camera is more inclined to shoot a panoramic picture;
the expected interest point ratio represents the percentage of the interest point expected to be shot by the camera to the area of the picture;
the actual interest point accounts for the percentage of the interest points shot by the camera in the picture area;
the total use duration indicates how long the current camera has been used for live broadcasting pictures in live broadcasting;
the use duration of the time indicates how long the time is passed after the switch is switched to the user, and the value is-1 when the user is not used at present.
Fig. 2 is a flowchart illustrating a video live broadcast control method according to a first embodiment of the present invention. As shown in fig. 2, the method for controlling live video includes the following steps:
In the specific embodiment of the invention, the position area of the interest point in the video image is obtained by adopting a background difference method for detecting a moving object in the video image. The basic idea of the background subtraction method is to input a frame of picture as a background image, compare the current image with the background image, separate a moving object from the background, where the moving object is an interest point, and the position information of the interest point is represented by the geometric center coordinate of the minimum bounding rectangle of the moving object.
Taking the position information of the interest point of the stage as an example, as shown in fig. 3, assuming that the stage size 1920 × 1080, the upper left-hand location coordinate origin (0,0), the upper left-hand coordinates (800, -400) of the moving object, and the length and width are both 100, the central area is (800+100/2, -400+ -100/2), i.e., (850, -450).
Because the background difference method is greatly interfered by the environment lamp, the positioning camera in the embodiment of the invention adopts the infrared camera and the infrared light supplement lamp for shooting, and does not adopt visible light for shooting, thereby eliminating the interference of visible environment light.
And step 202, calculating shooting parameters of each viewfinder camera according to the acquired position information of the interest point, generating a control signal, and controlling each viewfinder camera to shoot according to the shooting parameters.
In this step, the shooting parameters include a shooting direction and/or a zoom parameter, where the shooting direction indicates which direction the finder camera shoots in the three-dimensional space, and the zoom parameter refers to a lens zoom parameter of the finder camera. The calculation of the two parameters will be described below:
in this step, for each shooting direction parameter of the view camera, the shooting direction angle of the view camera is obtained by calculation according to the position of the interest point and the preset position of the view camera. For example, the direction of the viewfinder cameras is represented by an angle θ, the position of the upper left corner of the stage is taken as a starting angle, the angle is 0 degrees, the right deflection (i.e., clockwise) is taken as a positive direction, each viewfinder camera position and the stage position are firstly calibrated, as shown in fig. 4, the upper left corner of the stage is assumed to be a (0,0) point, the stage width is 1080, the length is 1920, the cameras are located at (300, -1200), the minimum bounding rectangle of the moving object is a rectangle formed by (800, -400) and (900, -500), the interest point is (850, -450), the angle of each camera towards the (0,0) point is defined to be 0 °, and the shooting direction parameter θ can be obtained by a slope formula and an included angle formula:
the slope formula is that k is (y2-y1)/(x2-x1)
The included angle is given by tan theta | (k 1-k 2)/(1+ k1 × k2) |
In the embodiment of the invention, the straight line from the coordinate origin (0,0) point to the viewfinder camera (300, -1200) point is L1, and the slope k1 of the straight line is-4 according to the slope calculation formula; when the straight line from the point of interest (850, -450) to the point of the viewfinder camera (300, -1200) is L2, the slope k2 of the straight line is 15/11 according to the slope calculation formula, and the included angle theta between the straight line L1 and the straight line L2 is as follows:
the shooting direction parameter is a direction 50.29 ° clockwise with the line connecting the origin of coordinates to the viewfinder camera as a start line.
In this step, for the zoom parameter of each view camera, the zoom parameter may be calculated from the shooting angle of the camera/the viewing angle of the moving object at the camera position, where the shooting angle of the camera is a fixed value of the camera, and is 90 ° in the present invention, and the viewing angle of the moving object at the camera position is an angle between the two diagonal vertices (point (800, -400) and point (900, -500)) of the minimum bounding rectangle of the moving object and the view camera, and in the specific embodiment of the present invention, the viewing angle of the moving object at the camera position may be found to be 8.6 ° similarly according to the above slope and angle formulas, and the zoom parameter is 90/8.6, and is about 10.
And step 203, acquiring the shot pictures of the plurality of view cameras, and calculating the suitability of the shot pictures of each view camera.
In an embodiment of the present invention, step 203 further comprises:
in step S1, a point of interest region of the acquired captured picture is calculated. In this step, the number and the position of the faces in the shot image are calculated by a face recognition method, and the shot image is surrounded by a minimum rectangle, which is the interest point region, as shown in fig. 5, the interest point region in the shot image is 400 × 1600.
And step S2, calculating the actual interest point ratio of the shooting picture according to the obtained interest point area. In the invention, the interest point ratio refers to the ratio of interest points (such as human faces) in the whole picture area, if the ratio is less, the camera is more prone to shoot a panoramic picture, and the actual interest point ratio represents the percentage of the interest points shot by the camera in the whole picture area. As shown in fig. 5, assuming that the stage size 1920 × 1080 is 1920 × 1080, that is, the entire screen area is 1920 × 1080, if the interest point area in the shot is 400 × 1600, the actual interest point percentage is 30.8% (400 × 1600)/(1080 × 1920).
Step S3, calculating the suitability of the picture taken by each viewfinder camera according to the actual interest point ratio and the expected interest point ratio. In the present invention, the expected interest point ratio represents the percentage of the interest point expected to be captured by the camera to the area of the picture, and is generally a preset value, the actual interest point ratio is obtained through step S2, and the suitability of the captured picture is obtained through the following formula:
fitness is 1- | (expected interest point ratio-actual interest point ratio) |/expected interest point ratio
In a specific embodiment of the present invention, assuming that the desired interest point percentage is 30%, the camera has a fitness of 1- | (30% -30.8%) |/30% > -97.3%.
And step 204, outputting one or more shooting pictures of the plurality of viewfinder cameras according to a switching control strategy.
Among the plurality of view cameras, it is necessary to select a view camera for live broadcasting according to a certain switching control strategy, so that a directly broadcasted picture has a plurality of machine positions (angles) to be switched, and interest points (people) in the picture should be relatively prominent.
In an embodiment of the present invention, the switching control strategy is to select an optimal shot picture to output according to the usage duration of each viewfinder camera and the suitability of the shot picture of each viewfinder camera. Specifically, the use duration here includes a total use duration and the use duration this time, and the total use duration indicates how long the current camera has been used for live broadcasting pictures in live broadcasting; the using time length indicates how long the user has been after switching to the user, and the value is-1 if the user is not currently using the device. Specifically, the handover control policy includes one or more of the following 1) to 4):
1) if the current viewfinder camera is continuously used for more than the maximum tolerance time (preset, for example, 20 seconds), switching to other viewfinder cameras with the highest suitability;
2) if the fitness of other view finding cameras exceeds the fitness of the current view finding camera and the current use time length of the current view finding camera is larger than the minimum use threshold (for example, 5s), switching to other view finding cameras with the highest fitness;
3) if the proportion of the current use time length of the viewfinder camera to the total use time length is lower than the minimum tolerance value (for example, 10%) and the current use time length of the current viewfinder camera is greater than the minimum use threshold value (for example, 5s), switching to the viewfinder camera with the shortest current use time length;
4) if the current time length of use of the current view camera is lower than the minimum use threshold (for example, 5s), the view camera is not switched regardless of the suitability.
By the switching control strategy, the camera with the highest suitability can be used as far as possible, and meanwhile, the situation that the lens is not switched for a long time or the picture flickers due to the fact that the lens is switched too fast is avoided.
Fig. 6 is a schematic structural diagram of a video live broadcast control apparatus according to a second embodiment of the present invention. As shown in fig. 6, a video live broadcast control apparatus according to the present invention includes: a positioning unit 601, a control unit 602, a suitability processing unit 603, and an output selection unit 604.
The positioning unit 601 is configured to acquire a video image input signal for positioning the camera, and acquire position information of a point of interest in an image according to the video image input signal.
In the embodiment of the present invention, the positioning unit 601 obtains the location area of the interest point in the video image by using a background subtraction method for detecting a moving object in the video image. The basic idea of the background subtraction method is to input a frame of picture as a background image, compare the current image with the background image, and separate a moving object from the background, where the moving object is an interest point, and the position information of the interest point is represented by the geometric center coordinate of the minimum bounding rectangle of the moving object.
Because the background difference method is greatly interfered by the environment lamp, the positioning camera in the embodiment of the invention adopts the infrared camera and the infrared light supplement lamp for shooting, and does not adopt visible light for shooting, thereby eliminating the interference of visible environment light.
And a control unit 602, which calculates the shooting parameters of each viewfinder camera according to the acquired position information of the interest point and generates a control signal to control each viewfinder camera to shoot according to the shooting parameters.
In the invention, the shooting parameters comprise a shooting direction and/or a zooming parameter, wherein the shooting direction represents the direction of the viewfinder camera to shoot in the three-dimensional space, and the zooming parameter refers to the lens zooming parameter of the viewfinder camera. The calculation of the two parameters will be described below:
and calculating the shooting direction parameters of the framing cameras according to the positions of the interest points and the preset positions of the framing cameras. For example, the shooting direction of the viewfinder camera is represented by an angle θ, the position towards the upper left corner of the stage is used as a starting angle, the starting angle is 0 degrees, the right deflection (i.e. clockwise) is used as a positive direction, and the shooting direction parameter θ can be obtained by an angle formula:
the included angle is given by tan theta | (k 1-k 2)/(1+ k1 × k2) |
Where (x1, y1) is the slope of a straight line from the coordinate origin (0,0) point to the coordinate point of the viewfinder camera with K1, K2 is the slope of a straight line from the point of interest to the coordinate point of the viewfinder camera, and the slope K1/K2 can be obtained from the slope formula, where K is (y2-y1)/(x2-x 1).
For the zoom parameters of each view camera, the zoom parameters can be calculated from the shooting angle of the camera/the viewing angle of the moving object at the camera position, the shooting angle of the camera is a fixed value of the camera, in the present invention, the shooting angle is 90 °, and the viewing angle of the moving object at the camera position is the angle between the two diagonal vertices (points (800, -400) and points (900, -500)) of the minimum bounding rectangle of the moving object and the view camera.
The suitability processing unit 603 is configured to obtain the captured images of the plurality of view cameras, and calculate the suitability of the captured images of each view camera.
Fig. 7 is a detailed structure diagram of the fitness processing unit 603 according to an embodiment of the invention. In an embodiment of the present invention, the fitness processing unit 603 further includes:
a point-of-interest region calculating unit 6031 for calculating a point-of-interest region of the acquired photographing screen. In the embodiment of the present invention, the interest point region calculating unit 6031 calculates the number and the position of faces in a shot image by using a face recognition method, and surrounds the faces with a minimum rectangle, where the rectangle is an interest point region;
an actual interest point ratio calculating unit 6032, configured to calculate an actual interest point ratio of the captured picture according to the obtained interest point region. In the invention, the interest point ratio refers to the ratio of interest points (such as human faces) in the whole picture area, if the ratio is less, the camera is more prone to shoot a panoramic picture, and the actual interest point ratio represents the percentage of the interest points shot by the camera in the whole picture area.
The suitability calculation unit 6033 calculates the suitability of the picture taken by each view camera according to the actual interest point ratio and the expected interest point ratio. In the present invention, the expected interest point ratio represents the percentage of the interest point expected to be captured by the camera to the area of the picture, and is generally a preset value, the actual interest point ratio is obtained through step S2, and the suitability of the captured picture is obtained through the following formula:
fitness is 1- | (expected-actual-interest ratio) |/expected-interest ratio.
And an output selection unit 604, configured to output one or more captured images of the plurality of viewfinder cameras according to a switching control policy.
In a system with a plurality of view cameras, the view cameras for live broadcasting need to be selected according to a certain switching control strategy, so that the directly broadcasted picture has a plurality of machine positions (angles) to be switched, and interest points (people) in the picture are relatively prominent.
In an embodiment of the present invention, the switching control strategy is to select an appropriate shot picture to output according to the duration of each viewfinder camera and the suitability of the shot picture of each viewfinder camera. Specifically, the use duration here includes a total use duration and the use duration this time, and the total use duration indicates how long the current camera has been used for live broadcasting pictures in live broadcasting; the using time length indicates how long the user has been after switching to the user, and the value is-1 if the user is not currently using the device. Specifically, the handover control policy includes one or more of the following 1) to 4):
1) if the current viewfinder camera is continuously used for more than the maximum tolerance time (preset, for example, 20 seconds), switching to other viewfinder cameras with the highest suitability;
2) if the fitness of other view finding cameras exceeds the fitness of the current view finding camera and the current use time length of the current view finding camera is larger than a minimum use threshold (for example, 5s), switching to other view finding cameras with the highest fitness;
3) if the proportion of the current use time length of the viewfinder camera to the total use time length is lower than the minimum tolerance value (for example, 10%) and the current use time length of the current viewfinder camera is greater than the minimum use threshold value (for example, 5s), switching to the viewfinder camera with the shortest current use time length;
4) if the current time length of use of the current view camera is lower than the minimum use threshold (for example, 5s), the view camera is not switched regardless of the suitability.
Through the switching control strategy, the viewfinder camera with the highest suitability can be used as far as possible, and meanwhile, the condition that the lens is not switched for a long time or the image flickers due to the fact that the lens is switched too fast is avoided.
Fig. 8 and 9 are top and front views of a video live broadcast system according to an embodiment of the present invention. In the embodiment of the present invention, the system is applied to a stage environment, and includes an infrared fill light 80, a positioning camera (infrared camera) 81, a plurality of view cameras 82, and a live video control device 83.
The positioning camera 81 is arranged at the top of the stage and shoots the stage panorama from the top, the infrared light supplement lamp 80 is obliquely arranged above the stage and emits infrared light obliquely, the infrared light is invisible to naked eyes and does not influence the stage lighting effect, when a person (or a moving object) walks onto the stage, the body or the head can reflect the infrared light to the positioning camera 81, the positioning camera 81 can shoot a bright spot, the positioning camera 81 is adjusted to just shoot all stages, and the width and height value of the resolution of the shot picture is defined as the width and height value of the stage; the view cameras 82 are distributed around the stage as required, are directed toward the center of the stage, and can take a panoramic view of the stage.
Fig. 10 is a schematic diagram illustrating an application scenario of a video live broadcast control apparatus according to an embodiment of the present invention. The live video control device comprises a positioning unit, a control unit, a suitability processing unit and an output selection unit, wherein the positioning unit receives picture signals of a positioning camera, the control unit outputs control signals to each viewing camera, and shot pictures of each viewing camera are output to the suitability processing unit and the output selection unit and are output live pictures according to a switching selection strategy.
In summary, the live video broadcast control method and apparatus of the present invention obtain the location information of the interest point, calculate the shooting parameters of each view camera according to the location information of the interest point and generate corresponding control signals to control each view camera, calculate the suitability of each view camera for the shot pictures of each view computer, and select one or more channels of shot pictures to output according to a switching control strategy, thereby implementing automatic control of multiple cameras, improving the live video broadcast efficiency, and being particularly suitable for self-service scenes.
Modifications and variations can be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the present invention. Therefore, the scope of the invention should be determined from the following claims.
Claims (11)
1. A video live broadcast control method comprises the following steps:
acquiring a video image input signal of a positioning camera, and acquiring position information of an interest point in an image according to the video image input signal;
step two, according to the acquired position information of the interest point, calculating the shooting parameters of each framing camera and generating control signals, and controlling each framing camera to shoot according to the shooting parameters;
acquiring the shot pictures of a plurality of view cameras, and calculating the suitability of the pictures shot by each view camera, wherein the suitability is related to the expected interest point ratio and the actual interest point ratio, the expected interest point ratio represents the percentage of the interest points shot by the camera to the picture area, and the actual interest point ratio represents the percentage of the interest points shot by the camera to the picture area; the fitness is used for representing the closeness degree of the actual interest point proportion relative to the expected interest point proportion;
outputting one or more paths of shot pictures in the plurality of view finding cameras according to a switching control strategy; and selecting corresponding shot pictures to output according to the service duration of each framing camera and the suitability of the shot pictures of each framing camera by the switching control strategy.
2. A video live broadcast control method as claimed in claim 1, characterized in that: in the first step, a background difference method for detecting moving objects in a video image is adopted to obtain a moving target in the video image as an interest point, and the geometric center coordinate of the minimum enclosing rectangle of the moving target is the position information of the interest point.
3. A video live broadcast control method as claimed in claim 1, characterized in that: in the second step, the shooting parameters include a shooting direction parameter and/or a zoom parameter.
4. A video live broadcast control method as claimed in claim 3, characterized in that: and the shooting direction parameter is the shooting direction angle of the view finding camera and is obtained through the position information of the interest point and the preset position of the view finding camera.
5. A video live broadcast control method as claimed in claim 3, characterized in that: the zooming parameters are obtained through the shooting angle of the framing camera and the visual angle of the moving target at the position of the camera, and the visual angle of the moving target at the position of the camera is the included angle between the vertex of the two diagonals of the minimum bounding rectangle of the moving target and the framing camera.
6. A video live broadcast control method as claimed in claim 1, wherein the step three further comprises:
calculating the interest point area of the acquired shooting picture;
calculating to obtain the actual interest point ratio of the shot picture according to the obtained interest point area;
and calculating the suitability of the pictures shot by each view camera according to the actual interest point ratio and the preset expected interest point ratio.
7. A live video control method as claimed in claim 1, wherein the switching control policy includes one or more of the following 1) to 4):
1) if the current framing camera is continuously used for a time exceeding the longest tolerance time, switching to other framing cameras with the highest suitability;
2) if the fitness of other view finding cameras exceeds the fitness of the current view finding camera and the current use time of the current view finding camera is longer than the minimum use threshold, switching to other view finding cameras with the highest fitness;
3) if the proportion of the current use time length of the view-finding camera to the total use time length is lower than the lowest tolerance value and the current use time length of the current view-finding camera is greater than the minimum use threshold value, switching to the view-finding camera with the shortest current use time length;
4) and if the current use time of the view camera is lower than the minimum use threshold, switching of the shooting picture of the view camera is not performed no matter the suitability is.
8. A video live broadcast control method as claimed in claim 1, characterized in that: the positioning camera adopts an infrared camera and an infrared light supplement lamp to shoot to obtain the video image.
9. A video live control apparatus comprising:
the positioning unit is used for acquiring a video image input signal of the positioning camera and acquiring the position information of an interest point in an image according to the video image input signal;
the control unit is used for calculating shooting parameters of each framing camera according to the acquired position information of the interest point, generating control signals and controlling each framing camera to shoot according to the shooting parameters;
the suitability processing unit is used for acquiring the shot pictures of the plurality of view cameras and calculating the suitability of the shot pictures of each view camera, wherein the suitability is related to the expected interest point ratio and the actual interest point ratio, the expected interest point ratio represents the percentage of the interest points expected to be shot by the camera to the picture area, and the actual interest point ratio represents the percentage of the interest points shot by the camera to the picture area; the fitness is used for representing the closeness degree of the actual interest point proportion relative to the expected interest point proportion;
the output selection unit is used for outputting one or more paths of shot pictures in the plurality of view finding cameras according to a switching control strategy; and selecting corresponding shot pictures to output according to the service duration of each framing camera and the suitability of the shot pictures of each framing camera by the switching control strategy.
10. A video live broadcast control apparatus as claimed in claim 9, wherein the suitability processing unit further comprises:
an interest point region calculation unit for calculating an interest point region of the acquired shot picture;
the actual interest point ratio calculating unit is used for calculating the actual interest point ratio of the shot picture according to the obtained interest point area;
and the fitness calculating unit is used for calculating the fitness of the pictures shot by each view camera according to the actual interest point ratio and the preset expected interest point ratio.
11. A live video control apparatus as claimed in claim 9 wherein the handover control policy comprises one or more of the following 1) to 4):
1) if the current framing camera is continuously used for a time exceeding the longest tolerance time, switching to other framing cameras with the highest suitability;
2) if the fitness of other view finding cameras exceeds the fitness of the current view finding camera and the current use time of the current view finding camera is longer than a minimum use threshold, switching to other view finding cameras with the highest fitness;
3) if the proportion of the current use time length of the view-finding camera to the total use time length is lower than the lowest tolerance value and the current use time length of the current view-finding camera is greater than the minimum use threshold value, switching to the view-finding camera with the shortest current use time length;
4) and if the current use time of the view camera is lower than the minimum use threshold, switching of the shooting picture of the view camera is not performed no matter the suitability is.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611104572.XA CN106603912B (en) | 2016-12-05 | 2016-12-05 | Video live broadcast control method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611104572.XA CN106603912B (en) | 2016-12-05 | 2016-12-05 | Video live broadcast control method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106603912A CN106603912A (en) | 2017-04-26 |
| CN106603912B true CN106603912B (en) | 2020-09-11 |
Family
ID=58597023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611104572.XA Active CN106603912B (en) | 2016-12-05 | 2016-12-05 | Video live broadcast control method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106603912B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107124662B (en) * | 2017-05-10 | 2022-03-18 | 腾讯科技（上海）有限公司 | Video live broadcast method and device, electronic equipment and computer readable storage medium |
| CN107197320B (en) * | 2017-06-21 | 2021-01-15 | 北京小米移动软件有限公司 | Video live broadcast method, device and system |
| US11082599B2 (en) * | 2017-07-18 | 2021-08-03 | Hangzhou Taruo Information Technology Llc | Controlling camera field-of-view based on remote viewer voting |
| WO2021081756A1 (en) * | 2019-10-29 | 2021-05-06 | 苏州随闻智能科技有限公司 | Video image display method and apparatus |
| CN110956106B (en) * | 2019-11-20 | 2023-10-10 | 广州方硅信息技术有限公司 | Live broadcast on-demand processing method, device, storage medium and equipment |
| CN111931564A (en) * | 2020-06-29 | 2020-11-13 | 北京大学 | Target tracking method and device based on face recognition |
| CN112423139A (en) * | 2020-11-20 | 2021-02-26 | 千脉文化传媒(上海)有限公司 | Multi-machine live broadcast method, system, equipment and storage medium based on mobile terminal |
| CN115223384B (en) * | 2021-03-29 | 2024-01-16 | 东风汽车集团股份有限公司 | Vehicle data display method and device, electronic equipment and storage medium |
| CN113141518B (en) | 2021-04-20 | 2022-09-06 | 北京安博盛赢教育科技有限责任公司 | Control method and control device for video frame images in live classroom |
| CN114979689B (en) * | 2022-05-05 | 2023-12-08 | 咪咕视讯科技有限公司 | Multi-machine-position live broadcast guide method, equipment and medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05328343A (en) * | 1992-05-19 | 1993-12-10 | Fujitsu Ltd | Multi-screen video conference system |
| NO327899B1 (en) * | 2007-07-13 | 2009-10-19 | Tandberg Telecom As | Procedure and system for automatic camera control |
| CN102300043B (en) * | 2010-06-23 | 2014-06-11 | 中兴通讯股份有限公司 | Method for adjusting meeting place camera of remote presentation meeting system |
| CN102148965B (en) * | 2011-05-09 | 2014-01-15 | 厦门博聪信息技术有限公司 | Multi-target tracking close-up shooting video monitoring system |
| WO2016074123A1 (en) * | 2014-11-10 | 2016-05-19 | 深圳锐取信息技术股份有限公司 | Video generating method and device of video generating system |
-
2016
- 2016-12-05 CN CN201611104572.XA patent/CN106603912B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106603912A (en) | 2017-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106603912B (en) | Video live broadcast control method and device | |
| JP4153146B2 (en) | Image control method for camera array and camera array | |
| US11860511B2 (en) | Image pickup device and method of tracking subject thereof | |
| US10262692B2 (en) | Method and system for automatic television production | |
| US8687041B2 (en) | Stereoscopic panorama image creating apparatus, stereoscopic panorama image creating method, stereoscopic panorama image reproducing apparatus, stereoscopic panorama image reproducing method, and recording medium | |
| US10911694B2 (en) | System and method for creating metadata model to improve multi-camera production | |
| US9786064B2 (en) | Multi-camera control apparatus and method to maintain location and size of object in continuous viewpoint switching service | |
| US8970704B2 (en) | Network synchronized camera settings | |
| JP2009284452A (en) | Hybrid video camera imaging apparatus and system | |
| CN112330736A (en) | Scene picture shooting method and device, electronic equipment and storage medium | |
| CN102739955A (en) | Image capturing apparatus for enabling generation of data of panoramic image with wide dynamic range | |
| US9667853B2 (en) | Image-capturing apparatus | |
| JP2022128489A (en) | Image capture device | |
| CN116055696A (en) | Projection equipment and projection method | |
| JP6426308B2 (en) | Image pickup system, lens apparatus, and method of operating lens apparatus | |
| TW202002606A (en) | Image-capturing device and method for operating the same | |
| US20180095347A1 (en) | Information processing device, method of information processing, program, and image display system | |
| JP2012175215A (en) | Method of operating television monitor screen of numerical control device with television camera mounted therein | |
| JP6426307B2 (en) | Image pickup system, image pickup apparatus and operation method of image pickup apparatus | |
| JP3891402B2 (en) | Image clipping control device | |
| CN114615435A (en) | Processing method and device | |
| CN214384918U (en) | Audio and video acquisition equipment | |
| JP2010152135A (en) | Safe area warning device | |
| Thomas | Virtual Graphics for Broadcast Production | |
| JP2020016883A (en) | Image capture device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |