CN1148062C - Realization method of remote picture-in-picture - Google Patents

Abstract

A method for realizing remote picture-in-picture, specifically relating to the field of video conferencing. The image captured by the main camera is first converted into CIF format data; the images captured by multiple auxiliary cameras are respectively converted into small images to be superimposed; Frame image data in CIF format; then send it to the encoder for encoding; the encoded data is sent to the channel through the multiplexing module. The method of the invention realizes that the video conference terminal equipment simultaneously transmits moving video images picked up by two or more cameras, and simultaneously saves the trouble of switching video sources.

Description

Realization method of remote picture-in-picture

The invention belongs to the electric communication technology, and in particular relates to the field of video conferencing.

In the video conferencing system, the multi-point control unit (MCU) controls the terminals of multiple venues below through the network, and the terminals of each venue have the function of sending the information of the local end and receiving the information of the other party at the same time. Generally, each terminal also displays a local image, that is, a local picture-in-picture, while receiving the other party's image, and this image is generally the image of the local main camera.

Usually, the main venue is an auditorium that can accommodate hundreds of people. Because the venue is too large, one camera cannot capture the entire scene of the venue, so two or more cameras can be arranged, and one camera can shoot The picture on the rostrum, the rest of the cameras shoot the picture in the auditorium, etc.

As shown in Figure 1, the terminal equipment of each meeting place can be connected to multiple cameras. The terminal equipment not only needs to transmit the image of the main camera, but also needs to transmit the image of one or more auxiliary cameras (such as: teletext camera). The more, the more information the other party gets. At present, domestic and foreign conference TV manufacturers use the method of switching video sources to make the other party see the images captured by other cameras, but this method can only transmit the images captured by one camera at the same time. When the conference is in progress, switch the two video sources of the same conference site as needed, so that participants in other conference sites can see the scene of the entire conference site.

The purpose of the present invention is to propose a remote picture-in-picture implementation method, which can overcome the above-mentioned shortcomings of the existing technology, so that the two parties in the meeting can avoid the trouble of frequently switching video sources, and can see both parties at the same time. images of the scene.

The concept of remote picture-in-picture is proposed relative to local picture-in-picture. That is to say, each terminal of the remote picture-in-picture video conferencing system can see three pictures at the same time, one picture is the main picture sent by the other party, and the other is the local picture-in-picture superimposed on the main picture , and a remote picture-in-picture sent from the other party superimposed on the main image.

The method of the present invention is realized by the following technical solutions: the image captured by the main camera is first converted into the standard format data required by the encoder through the format conversion module, such as: CIF (common intermediate format), QCIF (fourth of CIF) One); the image taken by any one of the auxiliary cameras in the plurality of auxiliary cameras is converted into a small image to be superimposed; then the main image and the small image to be superimposed are sent to the video superimposition module for superimposition, and become the requirements of a frame encoder The standard format data; then handed over to the encoder for encoding; the encoded data is sent to the channel through the multiplexing module.

The method of the present invention realizes that the conference TV terminal equipment simultaneously transmits moving video images taken by two cameras, and when a video conference is held, the receiving party in the meeting can see the images of the other party's two scenes at the same time, that is, the remote picture-in-picture is realized ; At the same time, it also saves the trouble of switching video sources.

The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Fig. 1 is the connection schematic diagram of video conference terminal camera in the prior art;

Fig. 2 is a schematic flow chart of video conference terminal image encoding in the prior art;

Fig. 3 is the application example of the present invention with two video sources as example;

Fig. 4 (a) is the superimposition schematic diagram of luminance signal Y in the video superposition process;

Fig. 4 (b) is the superposition schematic diagram of chrominance signal U in the video superposition process;

Fig. 4(c) is a schematic diagram of the superimposition of the chrominance signal V during the video superimposition process.

FIG. 1 is a description of the connection situation of the videoconferencing terminal camera in the prior art, which has been introduced in detail in the previous background technology.

Fig. 2 is a schematic diagram of an image encoding process of a videoconferencing terminal in the prior art. In the prior art, the image encoding process of a videoconferencing terminal is shown in Fig. 2: first, the analog video signal is converted into a standard format through a format conversion module, such as: CIF (common intermediate format), QCIF (one quarter of CIF) format, and then handed over to the encoder for encoding, and the encoded data is sent to the channel after multiplexing. This method is obviously only suitable for transmitting one image at the same time.

Fig. 3 is to connect two video sources as example, has illustrated the realization process of the inventive method:

Video source 1 is the image captured by the main camera, which is first converted into the format data required by the encoder through the format conversion module, such as: CIF (common intermediate format), QCIF (quarter of CIF) format data, video source 2 is The image captured by the auxiliary camera is converted into a small image to be superimposed through the format conversion module; then the main image and the small image to be superimposed are sent to the video superposition module for superimposition, and the format data required by a frame encoder is synthesized, such as: CIF , the image data in QCIF format; and then hand it over to the encoder for encoding. Here, the size and position of the small image to be superimposed, that is, the remote picture-in-picture, can be set according to the needs of customers. For the encoder, there is no difference between the superimposed image and the original image data with only one image, so the speed and effect of encoding will not be affected; the encoded data is sent to the channel through the multiplexing module. Compared with the image encoding process of the videoconferencing terminal in the prior art in FIG. 2 , the present invention adds a video superimposition step before the usual image encoding.

Figure 4 takes two video sources as an example to illustrate the specific implementation process of video superposition: an image is composed of several rows of pixels, and each pixel generally contains a luminance signal Y and two chrominance signals U and V. Human eyes are not very sensitive to chrominance signals, but are more sensitive to luminance signals. Therefore, generally, less chrominance signals are extracted for encoding, which not only reduces the computational load of image encoding, but also does not affect image quality. For example, the pixels of a CIF image in YUV420 format are Y: 352×288, U: 176×144, V: 176×144.

In Figure 4, the format of video source 1 for video superimposition is: YUV420, CIF, (ie Y: 352×288, U: 176×144, V: 176×144), video source 2 for video superposition, that is, far The format of the end picture-in-picture is: YUV420, 100×80 (ie Y: 100×80, U: 50×40, V: 50×40). Assuming that the user wants to place the remote picture-in-picture at the position (200, 100) on the screen, the overlay process is as follows:

(1) The superposition of the luminance signal Y as shown in Figure 4(a): Find out where the video source 1 is located

The brightness signal Y1 of 200<x<300, 100<y<180, the brightness signal of this area uses the video source 2

Brightness signal Y2 instead. Wherein, Y1 and Y2 are brightness signals collected by video source 1 and video source 2 respectively;

(2) The superposition of chrominance signal U as shown in Fig. 4 (b): Find the chrominance signal U1 where video source 1 is located at 100<x<150, 50<y<90, and the chrominance signal in this area uses the video source 2 of the chrominance signal U2 instead. Where U1 and U2 are the chrominance signals collected by video source 1 and video source 2 respectively;

(3) The superposition of chrominance signal V as shown in Fig. 4 (c): Find the chrominance signal V1 where video source 1 is located at 100<x<150, 50<y<90, and the chrominance signal in this area uses the video source 2's chrominance signal V2 instead. Wherein, V1 and V2 are the chrominance signals collected by video source 1 and video source 2 respectively;

At this point, the video overlay process is complete.

Because the present invention uses the method of video superimposition, both parties in a meeting can see two or more scenes of each other at the same time, and can realize remote picture-in-picture transmission, which is simple and convenient to use.

Claims (2)

1, a kind of implementation method of far-end picture-in-picture may further comprise the steps:

1) image that main camera is absorbed is converted into and meets the formatted data that encoder requires;

2) image of any one auxiliary camera picked-up is converted into the little image that needs stack in a plurality of auxiliary cameras;

3) the little image that described master image and need are superposeed is sent into together and is carried out video superimpose, becomes the formatted data that a frame meets the encoder requirement;

4) view data after the stack is encoded;

5) data behind the coding are through serving channel after multiplexing.

2, the implementation method of a kind of far-end picture-in-picture as claimed in claim 1 is characterized in that: described video superimpose process comprises the stack of brightness signal Y, the stack of carrier chrominance signal U and the stack of carrier chrominance signal V:

Need at first to determine the original position of little image in big image of stack; According to the original position and the size of little image, find out the zone of little image in big image then; To be positioned at the brightness signal Y of the big image in this zone at last, carrier chrominance signal U, V use the brightness signal Y of little image respectively, carrier chrominance signal U, and V replaces.

Images