WO2022130514A1 - Video processing method, video processing device, and program - Google Patents

Abstract

A video processing method according to the present disclosure comprises: a step (S11) for obtaining a video (CV) generated by a camera (1); a step (S12) for generating a processed video (PV) by, with respect to the video (CV), not performing data quantity reduction processing in a non-processing region (R2) based on a display region (R1) displayed on a display (34) of a video display device (3), and performing data quantity reduction processing in a processing region (R3) which is different from the non-processing region (R2); a step (S13) for encoding the processed video (PV) so that the data quantity of the processed video (PV) is reduced; and a step (S14) for transmitting, to the video display device 3, the encoded processed video (PV) and setting information which includes the position of the display region (R1) of the video (CV) and a magnification ratio of the display region (R1) when the display region (R1) is displayed on the display (34).

Description

Video processing methods, video processing equipment, and programs

The present disclosure relates to a video processing method, a video processing device, and a program for displaying a video on a display.

Display technology for displaying images has been developed, and it is highly portable, and various displays are known that are composed of various shapes such as circles and polygons as well as rectangles. It is also known that one virtual display configured by combining the plurality of displays displays the entire image by displaying a part of the image on each of the plurality of displays. Further, as shown in Non-Patent Document 1, a plurality of mobile displays in which each of the plurality of moving objects is equipped with a display displays a part of an image corresponding to each position, whereby the plurality of mobile displays are displayed. It is known that one virtual display configured by the above displays the entire image.

In the technique described in Patent Document 1, it is conceivable that a plurality of mobile displays (hereinafter, simply referred to as "displays") receive an image taken by a camera in real time and display the image by a streaming technique. .. However, in order to properly display the video by the streaming technique, it is necessary to secure a sufficient wireless transmission band so that the video is transmitted from the camera to the display at high speed. It is conceivable to transmit images from the camera to the display via a communication network using a communication standard such as a wireless LAN (Local Area Network) or LTE (registered trademark) (LongTermEvolution). However, since it is assumed that a large number of users send and receive information via a communication network using these communication standards, sufficient bandwidth cannot be secured when transmitting video from the camera to the display, which is good. Communication may not be possible. As a result, there may be a delay in displaying the image on the display.

An object of the present disclosure made in view of such circumstances is to provide a video processing method, a video processing device, and a program capable of displaying an appropriate video while reducing the amount of video data.

In order to solve the above problems, the video processing method according to the present disclosure includes steps for acquiring video and
In the video, the non-processed area based on the display area displayed on the display of the image display device is not processed to reduce the amount of data, and the processed image is processed by applying the data amount reduction processing to the processed area different from the non-processed area. Steps to generate and
A step of encoding the processed video so that the amount of data of the processed video is reduced,
A step of transmitting to the video display device setting information including the encoded video, the position of the display area in the video, and the enlargement ratio of the display area when the display is displayed on the display. And, including.

Further, in order to solve the above-mentioned problems, the video processing apparatus according to the present disclosure reduces the amount of data in the acquisition unit for acquiring a video and the non-processed region based on the display area displayed on the display of the video display device in the video. An image processing unit that generates a processed image by performing data amount reduction processing on a processing area different from the non-processed area without processing, and the processed image so that the transmission amount of the processed image is reduced. The setting information including the encoder to be encoded, the processed image encoded by the encoder, the position of the display area in the image, and the enlargement ratio of the display area when the display area is displayed on the display is described. It is provided with a transmission unit for transmitting to a video display device.

Further, in order to solve the above-mentioned problems, the program according to the present disclosure causes the computer to function as the above-mentioned video processing device.

According to the video processing method, the video processing device, and the program according to the present disclosure, it is possible to display an appropriate video while reducing the amount of video data.

It is a functional block diagram of the video processing system which concerns on 1st Embodiment of this disclosure. It is a schematic diagram which shows an example of the image generated by the camera shown in FIG. It is a schematic diagram for demonstrating the non-processed area and the processed area in the image shown in FIG. It is a flowchart for demonstrating the operation of the image processing apparatus shown in FIG. It is a flowchart for demonstrating the operation of the image display device shown in FIG. It is a functional block diagram of the video processing system which concerns on the 2nd Embodiment of this disclosure. It is a schematic diagram for demonstrating the arrangement relation of the display shown in FIG. It is a schematic diagram for demonstrating the display area in an image based on the arrangement relation of the display shown in FIG. It is a flowchart for demonstrating the operation of the image processing apparatus shown in FIG. It is a schematic diagram which shows the other example of a processed image. It is a functional block diagram of the video processing system which concerns on the 3rd Embodiment of this disclosure. It is a figure for demonstrating another example which determines a non-machining area. It is a schematic diagram which shows the non-processed area and the processed area determined by another example shown in FIG. It is a flowchart for demonstrating operation of the image processing apparatus shown in FIG. It is a hardware block diagram of the image processing apparatus and the image display apparatus which concerns on 1st to 3rd Embodiment.

First, the embodiments of the present disclosure will be described with reference to the drawings.

<< First Embodiment >>
The overall configuration of the first embodiment will be described with reference to FIG. FIG. 1 is a functional block diagram of the video processing system 100 according to the first embodiment of the present invention.

As shown in FIG. 1, the video processing system 100 according to the first embodiment includes a camera 1, a video processing device 2, and a video display device 3. The camera 1 and the video processing device 2 communicate with each other via a communication cable or a communication network. The video processing device 2 and the video display device 3 communicate with each other via a communication network.

In the first embodiment, the video processing system 100 does not have to include the camera 1, and in such a configuration, the video processing device 2 may generate video by any method such as computer graphics. good. Further, the video processing device 2 may acquire the video generated by another information processing device. Hereinafter, "acquiring an image" may include generating an image by the image processing device 2 and acquiring an image generated by the camera 1 or the information processing device. Hereinafter, as an example, an example of acquiring an image generated by the camera 1 will be described.

<Camera configuration>
As shown in FIG. 2, the camera 1 captures a subject in the background of the main subject OB and the main subject OB included in the shooting area, and thereby includes an image OBI of the main subject OB and an image including the background image. Generate CV. The shooting area can be, for example, a rectangular area having a predetermined aspect ratio. The camera 1 transmits a video CV to the video processing device 2. The camera 1 may transmit a video CV to the video processing device 2 using the communication standard of SDI (Serial Digital Interface), or may use the communication standard of HDMI (registered trademark) (High-Definition Multimedia Interface). , The video CV may be transmitted to the video processing device 2. Further, the camera 1 may transmit the video CV to the video processing device 2 by using any other communication standard.

<Configuration of video processing equipment>
As shown in FIG. 1, the video processing device 2 includes an acquisition unit 21, a display setting holding unit 22, a video processing unit 23, an encoder 24, and a transmission unit 25. The acquisition unit 21 and the transmission unit 25 are configured by a communication interface. The display setting holding unit 22 is composed of, for example, a memory such as a semiconductor memory, a magnetic memory, or an optical memory. The video processing unit 23 and the encoder 24 form a control unit (controller). The control unit may be configured by dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array), may be configured by a processor, or may be configured by including both. May be good.

The acquisition unit 21 acquires the video CV generated by the camera 1 as shown in FIG. 2 transmitted from the camera 1. The acquisition unit 21 can acquire the video CV by using the communication standard corresponding to the communication standard used when the camera 1 transmits the video CV as described above.

The display setting holding unit 22 holds the setting information for the display 34, which will be described later, to display the video, which is provided in the video display device 3. The setting information includes the position of the display area R1 in the video CV and the enlargement ratio of the display area R1 when the video display device 3 displays the display area R1 as shown in FIG. The enlargement ratio of the display area R1 is determined by the size of the display 34 in the real space included in the video display device 3. The display area R1 in the video CV is an area displayed on the display 34 in the video CV. The display area R1 is determined by the position of the display area R1 in the video CV, the enlargement ratio of the display area R1 when displaying the display area R1 on the display 34, and the shape and size of the display 34, which are included in the setting information.

The video processing unit 23 processes the video CV to generate the processed video PV based on the setting information held in the display setting holding unit 22. Specifically, the video processing unit 23 does not perform data amount reduction processing on the non-processing region R2 based on the display region R1 in the video CV, but performs data amount reduction processing on the processing region R3 different from the non-processing region R2. In the present embodiment, the non-processed region R2 based on the display region R1 can be the same region as the display region R1.

The data amount reduction processing is processing for generating a processed video PV in which the amount of data when transmitted via a communication network is reduced by being encoded by the encoder 24 which will be described in detail next. ..

In one example, the data amount reduction processing can be a processing for making the feature amount of the pixel uniform. The feature amount of the pixel may be, for example, the brightness of the pixel. In such a configuration, the image processing unit 23 may set the luminance of the pixels constituting the processing region R3 to, for example, zero, that is, the pixels constituting the processing region R2 may be black.

In another example, the data amount reduction processing may be processing for converting the processing region R3 in each of the images of a plurality of frames constituting the video into the same still image. The still image may be any image. The still image may be, for example, an image having a grid pattern. The still image may be, for example, an image of the processing region R3 in the image of any one frame (reference frame) among the images of a plurality of frames constituting the image. In yet another example, the data amount reduction process may be a process of deleting the processing region R3.

As shown in FIG. 1, the encoder 24 encodes the processed video PV so that the amount of data of the processed video PV generated by the video processing unit 23 is reduced. Specifically, the encoder 24 has a data amount smaller than the data amount when transmitting the processed video PV according to the processing method applied in generating the processed video PV by the image processing unit 23. The processed video PV is encoded. The encoder 24 may encode the processed video PV by an MPEG (Moving Picture Experts Group) format video data compression method, or may encode the processed video PV by any other method. For example, when the encoder 24 has been processed by the image processing unit 23 to reduce the amount of data for converting the processing area R3 in each of the images of a plurality of frames constituting the image into the same still image, the processed image is predicted by inter-frame prediction. PV may be encoded.

The transmission unit 25 transmits the processed video PV encoded by the encoder 24 and the setting information held in the display setting holding unit 22 to the video display device 3. The transmission unit 25 may transmit the encoded processed video PV to the video display device 3 using a communication protocol such as UDP (User Datagram Protocol). Further, the transmission unit 25 may transmit the encoded processed video PV and the setting information to the video display device 3 using a wireless communication network such as LTE, or the encoded processed video PV and the setting information. May be transmitted to the video display device 3 using any other wireless communication standard. Further, the transmission unit 25 may transmit the encoded processed video PV and the setting information to the video display device 3 using the wired communication standard.

The video display device 3 includes a receiving unit 31, a decoder 32, a display control unit 33, and a display 34.

The receiving unit 31 receives the encoded processed video PV and the setting information transmitted from the transmitting unit 25 of the video processing device 2. At this time, the receiving unit 31 can use the communication protocol used for transmission by the transmitting unit 25. Further, the receiving unit 31 may receive the encoded processed video PV and the setting information using the same communication method as the communication method used by the transmitting unit 25, or may receive the encoded video PV using a different communication method. You may.

The decoder 32 outputs the processed video PV by decoding the encoded processed video PV received by the receiving unit 31.

The display control unit 33 displays the processed video PV output by the decoder 32 on the display 34 based on the setting information received by the reception unit 31. Specifically, the display control unit 33 causes the display 34 to display the display area R1 at the position shown in the setting information in the processed video PV at the enlargement ratio shown in the setting information. As described above, since the display region R1 in the processed video PV is the non-processed region R2, the data amount reduction processing is not performed by the image processing unit 23. Therefore, the display control unit 33 can display the image corresponding to the display area R1 in the image CV generated by the camera 1 on the display 34.

The display 34 displays the display area R1 in the processed video PV, that is, the display area R1 in the video CV, based on the control of the display control unit 33.

<Operation of video processing device>
Here, the operation of video processing in the video processing device 2 according to the first embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing an example of the operation in the video processing of the video processing apparatus 2 according to the first embodiment. The operation in the video processing of the video processing apparatus 2 described with reference to FIG. 4 corresponds to the video processing method according to the first embodiment.

In step S11, the acquisition unit 21 receives the video CV from the camera 1.

In step S12, the video processing unit 23 generates a processed video PV obtained by processing the video CV based on the setting information held in the display setting holding unit 22. At this time, the image processing unit 23 does not perform data amount reduction processing on the non-processing area R2 based on the display area R1 in the image CV, but performs data amount reduction processing on the processing area R3 different from the non-processing area R2. The data amount reduction processing may be processing for making the feature amounts of the pixels constituting the processing region R3 uniform, or the processing region R3 in each of the images of a plurality of frames constituting the video CV is converted into the same still image. It may be processed.

In step S13, the encoder 24 encodes the processed video PV so that the amount of data of the processed video PV processed in step S13 is reduced. In step S12, when the image processing unit 23 has performed data amount reduction processing for converting the processing area R3 in each of the images of a plurality of frames constituting the image into the same still image, in step S13, the encoder 24 sets the encoder 24. The processed video PV may be encoded by inter-frame prediction.

In step S14, the transmission unit 25 transmits the processed video PV encoded by the encoder 24 and the setting information held by the display setting holding unit 22 to the video display device 3 via the communication network. As described above, the setting information includes the position of the display area R1 in the video CV and the enlargement ratio of the display area R1 when the display area R1 is displayed on the display 34.

<Operation of video display device>
Subsequently, the operation of the video display device 3 according to the first embodiment in the video display will be described with reference to FIG. FIG. 5 is a flowchart showing an example of an operation in the video display of the video display device 3 according to the first embodiment. The operation in the video processing of the video display device 3 described with reference to FIG. 5 corresponds to the video display method according to the first embodiment.

In step S21, the receiving unit 31 receives the encoded processed video PV and the setting information transmitted from the transmitting unit 25 of the video processing device 2.

In step S22, the decoder 32 outputs the processed video PV by decoding the encoded processed video PV received by the receiving unit 31.

In step S23, the display control unit 33 causes the display 34 to display the processed video PV output by the decoder 32 based on the setting information received by the reception unit 31.

As described above, the image processing apparatus 2 of the first embodiment generates the processed image PV by performing the data amount reduction processing on the processing area R3 without performing the data amount reduction processing on the non-processing area R2, and processing it. The processed video PV is encoded so that the amount of data in the video PV is reduced. This makes it possible to reduce the amount of video data transmitted to the video display device 3. Therefore, it is possible to prevent the reception of the video by the video display device 3 from being delayed and causing problems such as delay in the display of the video by the video display device 3.

Further, in the encoder 24 of the video processing device 2 of the first embodiment, the video processing unit 23 performs data amount reduction processing for converting the processing region R3 in each of the images of a plurality of frames constituting the video into the same still image. If so, the processed video PV may be encoded by inter-frame prediction. In the processed video PV in which the processed area R3 in each of the images of the plurality of frames is the same still image, there is no difference in the processed area R3 in the images of the preceding and following frames. Therefore, in this way, the encoder 24 encodes the processed video PV by inter-frame prediction, so that the amount of data in the processed video PV is greatly reduced. Therefore, it is possible to further suppress the delay in receiving the video by the video display device 3 and the occurrence of problems such as delay in the display of the video by the video display device 3.

<< Second Embodiment >>
The overall configuration of the second embodiment will be described with reference to FIG. FIG. 6 is a functional block diagram of the video processing system 101 according to the second embodiment of the present invention.

As shown in FIG. 6, the video processing system 101 of the second embodiment includes a camera 4, a video processing device 5, and a plurality of video display devices 61, 62, ..., 6n (n is an integer). Be prepared. Hereinafter, each of the plurality of video display devices 61, 62, ..., 6n may be referred to as a video display device 6k (k is an integer of 1 to n). The camera 4 is the same as the camera 1 of the first embodiment.

In the second embodiment and the third embodiment described in detail later, the video processing system 101 does not have to include the camera 4, and in such a configuration, the video processing device 5 is a computer graphic or the like. The video may be generated by any method of. Further, the video processing device 2 may acquire the video generated by another information processing device. Hereinafter, "acquiring an image" may include generating an image by the image processing device 5 and acquiring an image generated by the camera 4 or the information processing device. Hereinafter, as an example, an example of acquiring an image generated by the camera 4 will be described.

<Functional configuration of video processing equipment>
The video processing device 5 includes an acquisition unit 51, a display setting holding unit 52, a video processing unit 53, an encoder 54, a transmission unit 55, and an arrangement information acquisition unit 56.

The acquisition unit 51 is the same as the acquisition unit 21 of the first embodiment.

The arrangement information acquisition unit 56 acquires arrangement information regarding the arrangement such as the position, orientation, size, and shape of the display 6k4 which is described in detail later in the video display device 6k. As shown in FIG. 7, the displays 6k4 included in each of the plurality of video display devices 6k constitute one virtual display VD. In the example shown in FIG. 7, n = 8, and the display 684 is shown from the display 614, but the number n of the display 6k4 is not limited to 8.

For example, the placement information acquisition unit 56 may acquire placement information including a position measured based on a satellite signal received by a receiver of a GPS (Global Positioning System) system attached to the display 6k4. Further, the arrangement information acquisition unit 56 may acquire arrangement information including the direction indicated by the compass attached to the display 6k4. Further, the arrangement information acquisition unit 56 includes the position of the display 6k4 measured by the infrared camera based on the infrared rays reflected from the infrared reflecting element by irradiating the infrared reflecting element attached to the display 6k4 with infrared light. You may acquire the arrangement information. The arrangement information acquisition unit 56 may acquire arrangement information including the position of the display 6k4 measured based on the image taken by the camera that receives visible light. The placement information acquisition unit 56 is not limited to these, and can acquire placement information by any method.

The display setting holding unit 52 holds the setting information for the display 6k4 to display the image, as in the first embodiment. In the second embodiment, the setting information includes the position of the display area R1-k in the video CV and the enlargement ratio when displaying the display area R1-k on the display 6k4, as in the first embodiment. .. The position of the display area R1 in the present embodiment is the position of the display area R1-k to be displayed on each of the displays 6k4. Further, the position and the enlargement ratio of the display area R1-k are determined by the arrangement information acquired by the arrangement information acquisition unit 56.

The video processing unit 53 processes the video CV to generate the processed video PV based on the setting information held in the display setting holding unit 52. Specifically, the video processing unit 53 has, as shown in FIG. 8, each display 6k4 in the video CV in the video space, based on the setting information determined based on the placement information acquired by the placement information acquisition unit 56. Each display area R1-k is determined.

In the example shown in FIG. 8, the video processing unit 53 has a display area R1-1 displayed on the display 614, a display area R1-2 displayed on the display 624, and a display area R1-displayed on the display 634 in the video CV. 3 is decided. Further, the video processing unit 53 determines the display area R1-4 to be displayed on the display 644, the display area R1-5 to be displayed on the display 654, and the display area R1-6 to be displayed on the display 664 in the video CV. .. Further, the video processing unit 53 determines the display area R1-7 to be displayed on the display 674, the display area R1-8 to be displayed on the display 684, and the display area R1-8 to be displayed on the display 684 in the video CV. ..

Further, the image processing unit 53 determines the non-processed area R2-k based on each of the display areas R1-k of the display 6k4. In one example, as shown in FIG. 8, the video processing unit 53 may determine the display area R1-k as the non-processed area R2-k.

Further, the video processing unit 53 determines a region different from any non-processed region R2-k in the video CV as the processed region R3.

The video processing unit 53 generates the processed video PV by performing the data amount reduction processing on the processed region R3 in the video CV without performing the data amount reduction processing on the non-processed region R2-k in the video CV.

The encoder 54 is the same as the encoder 24 of the first embodiment.

The transmission unit 55 is the same as the transmission unit 25 of the first embodiment.

<Functional configuration of video display device>
As shown in FIG. 6, the video display device 6k includes a receiving unit 6k1, a decoder 6k2, a display control unit 6k3, and a display 6k4. The receiving unit 6k1, the decoder 6k2, the display control unit 6k3, and the display 6k4 are the same as the receiving unit 31, the decoder 32, the display control unit 33, and the display 34 of the video display device 3 of the first embodiment, respectively.

<Operation of video processing device>
Here, the operation of video processing in the video processing apparatus 5 according to the second embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing an example of the operation in the video processing of the video processing apparatus 5 according to the second embodiment. The operation in the video processing of the video processing apparatus 5 described with reference to FIG. 9 corresponds to the video processing method according to the second embodiment.

In step S31, the video processing unit 53 acquires the video CV from the camera 4.

In step S32, the arrangement information acquisition unit 56 acquires the arrangement information of the display 6k4 included in each of the video display devices 6k.

In step S33, the display setting holding unit 52 holds the setting information determined based on the placement information acquired by the placement information acquisition unit 56.

In step S34, the video processing unit 53 determines the non-processing area R2 and the processing area R3 based on the display area R1 corresponding to the setting information held in the display setting holding unit 52.

In step S34, the video processing unit 53 generates a processed video PV obtained by processing the video CV. At this time, the image processing unit 53 generates the processed image PV by performing the data amount reduction processing on the processing area R3 different from the non-processing area R2 without performing the data amount reduction processing on the non-processing area R2.

In step S35, the encoder 54 encodes the processed video PV processed in step S13.

In step S36, the transmission unit 55 transmits the processed video PV encoded by the encoder 54 and the setting information held by the display setting holding unit 52 to each of the video display devices 6k via the communication network.

<Operation of video display device>
The operation in the video display of each of the video display devices 6k of the second embodiment is the same as the operation in the video display of the video display device 3 according to the first embodiment.

As described above, according to the second embodiment, the video processing device 5 acquires the layout information of the display 6k4 included in each of the plurality of video display devices 6k, and based on the layout information of the display 6k4, in the video CV. , The display area R1-k to be displayed on each of the displays 6k4 is determined. Therefore, even when each of the plurality of video display devices 6k constituting one virtual display VD displays an area based on each display area R1-k in the video CV, the video transmitted to the video display device 3 The amount of data can be reduced.

In the second embodiment, the video processing unit 53 may not perform data amount reduction processing only on the non-processed region R2-k of any of the displays 6k4 from the display 614 to the display 6n4. In this case, the image processing unit 53 performs data amount reduction processing by setting a region different from the non-processed region R2-k as the processed region R3.

For example, as shown in FIG. 10, the image processing unit 53 does not perform data amount reduction processing on the non-processed area R2-1 of the display 614, and forms a region different from the non-processed area R2-1 of the display 614 as a non-processed area. The processed image PV for the display 614 may be generated by performing the data amount reduction processing as R3. Then, the encoder 54 encodes the processed video PV for the display 614, and the transmission unit 55 transmits the encoded processed video PV to the video display device 61. Similarly, the image processing unit 53 does not perform data amount reduction processing on the portion included in the non-processed area R2-2 of the display 624, and the data amount is in the processing area R3 different from the non-processed area R2-2 of the display 624. The reduced processing may be performed to generate a processed video PV for the display 624. Then, the encoder 54 encodes the processed video PV for the display 624, and the transmission unit 55 transmits the encoded processed video PV to the video display device 62. Similarly, the video processing unit 53 may generate processed video PV for each of the displays 634 to 6n4. Then, the encoder 54 encodes the processed video PV for each of the displays 34 to 6n4, and the transmission unit 55 transmits the encoded processed video PV to the video display devices 63 to 6n, respectively.

<< Third Embodiment >>
The overall configuration of the third embodiment will be described with reference to FIG. FIG. 11 is a functional block diagram of the video processing system 102 according to the third embodiment of the present invention.

As shown in FIG. 11, the video processing system 102 of the third embodiment has a camera 4, a video processing device 5, and a plurality of video display devices 61, similarly to the video processing system 101 of the second embodiment. 62, ..., 6n and the like. The camera 4 is the same as the camera 4 of the second embodiment.

<Functional configuration of video processing equipment>
The video processing device 5 includes an acquisition unit 51, a display setting holding unit 52, a video processing unit 53, an encoder 54, a transmission unit 55, and an arrangement information determination unit 57. The acquisition unit 51, the encoder 54, and the transmission unit 55 are the same as the acquisition unit 51, the encoder 54, and the transmission unit 55 of the second embodiment, respectively.

The arrangement information determination unit 57 determines the target position TP of each of the plurality of displays 6k4 based on the image CV captured by the camera 4. Specifically, first, the arrangement information determination unit 57 extracts the subject region R4 in which the image OBI of the main subject OB is captured in the video CV acquired by the acquisition unit 51. The arrangement information determination unit 57 extracts, for example, the difference between the background image and the video CV generated by imaging a background that is a subject different from the main subject OB in a state where the main subject OB does not exist in the imaging range. Can extract the subject area R4. The arrangement information determination unit 57 can extract the subject area R4 by using any method, not limited to this. When the image is not an image generated by the camera, for example, an image generated by computer graphics, the subject area R4 may be an area in which the main part of the image CV is displayed. The main part can be, for example, a part of the video CV that changes more than the other parts.

Next, the arrangement information determination unit 57 calculates the center of gravity of the subject area R4. The arrangement information determination unit 57 can calculate the center of gravity of the subject area R4 by using, for example, the arithmetic mean of each coordinate of the subject area R4. Not limited to this, the center of gravity of the subject area R4 can be calculated by using any method.

The arrangement information determination unit 57 repeats the process of calculating the center of gravity of the subject area R4 every time the video CV is updated. Then, the arrangement information determination unit 57 calculates the amount of change in the center of gravity of the subject area R4, which is calculated every time the video CV is updated.

The arrangement information determination unit 57 determines the target position TP of each of the displays 6k4 based on the subject area R4. Specifically, the arrangement information determination unit 57 sets a target position TP for each of the displays 6k4 so that one virtual display VD composed of the displays 614, 624, ..., And 6n4 displays the subject area R4. To decide.

After that, when the video CV is updated and the amount of change in the center of gravity is calculated, the arrangement information determination unit 57 updates the target position TP of each of the displays 6k4 based on the amount of change in the center of gravity. Specifically, the placement information determination unit 57 updates the target position TP by adding the amount of change in the center of gravity to the target position TP to the current target position TP. After that, the arrangement information determination unit 57 updates the target position TP of each of the displays 6k4 based on the change amount of the center of gravity each time the change amount of the center of gravity of the subject area R4 where the video CV is updated is calculated. repeat.

The arrangement information determination unit 57 determines the display area R1 based on the target position TP of each of the displays 6k4. Specifically, the arrangement information determination unit 57 determines a display area R1-k including at least a part of the subject area R4 to be displayed on each of the displays 6k4 when each of the displays 6k4 is arranged at the target position TP. do. The arrangement information determination unit 57 determines the display areas R1-1 to R1-8 so as to include, for example, a part of each of the subject areas R4 representing the image OBI of the subject, as shown in FIG. Further, the arrangement information determination unit 57 determines the enlargement ratio when each of the displays 6k4 displays the display area R1-k.

The display setting holding unit 52 holds the setting information for the display 6k4 to display the image, as in the second embodiment. In the third embodiment, the setting information includes the position of the display area R1 in the video CV corresponding to the target position TP of the display 6k4 determined by the arrangement information determination unit 57, and the display area R1 in the video display device 6k. Is included with the enlargement ratio of the display area R1 when displaying.

The video processing unit 53 generates a processed video PV obtained by processing the video CV, as in the second embodiment.

Specifically, the image processing unit 53 determines the non-processed area R2-k based on each of the display areas R1-k of the display 6k4. In one example, as in the second embodiment, the video processing unit 53 may determine the display area R1-k as the non-processed area R2-k. In another example, the video processing unit 53 may determine the processing region R2-k as a region composed of a display region R1-k and a region adjacent to the display region R1-k.

Here, another example in which the image processing unit 53 determines the non-processed region R2 will be described in detail with reference to FIG. FIG. 12 is an enlarged view showing one of the display areas R1-k shown in FIG. As shown in FIG. 12, the video processing unit 53 determines the display area R1-k to be displayed on the display 6k4 based on the arrangement information acquired by the arrangement information acquisition unit 56. Here, an error may occur between the target position TP of the display 6k4 and the actual position of the display 6k4 driven based on the target position TP by the drive unit 6k5 of the image display device 6k, which will be described in detail later. .. In this case, the video display device 6k causes the display 6k4 to display the display area R1-k based on the target position TP of the display 6k4, whereas the actual position of the display 6k4 is indicated by a dashed line. It may be appropriate to display the area a. In such a case, in order for the video display device 6k to correct the display area R1-k to be displayed on the display 6k4 to the area a based on the error, the area a is not processed to reduce the amount of data. It is necessary. Therefore, as shown in FIG. 12, the video processing unit 53 is composed of a region including the region a, specifically, a display region R1-k and a region adjacent to the display region R1-k. The unprocessed region R2-k is determined.

For example, when it is expected that the position error of the display 6k4 is the distance d, the non-processed region R2-k covers the region from the outer edge of the display region R1-k to the distance d1 as shown in FIG. It can be an area adjacent to the display area R1-k. The distance d1 is a distance in the video space corresponding to the distance d in the real space. For example, when the display is a square of 50 cm × 50 cm and an error of 10 cm is expected in the position of the display, the non-processed area R2-k is the area corresponding to the square of 70 cm × 70 cm in the video CV. Further, when the accuracy of the angle of the display indicated by the arrangement information is θ °, the region included in the range of the display region R1-k when the display is rotated by ± θ ° is defined as the non-processed region R2-k. be able to. For example, when an error of 5 ° is expected in the angle of the display, the non-processed region R2-k corresponds to the display 6k4 when the display 6k4 is rotated to 5 ° from the angle indicated by the arrangement information in the video CV. It becomes the area to do.

Then, as shown in FIG. 13, the video processing unit 53 does not perform data amount reduction processing on the non-processed region R2 based on the display region R1 in the video CV, and reduces the data amount to the processed region R3 different from the non-processed region R2. Apply processing.

The transmission unit 55 transmits the processed video PV encoded by the encoder 54 and the setting information held in the display setting holding unit 52 to the video display device 6k. Further, the transmission unit 55 transmits drive information indicating the target position TP of each of the plurality of displays 6k4 to the video display device 6k provided with the displays 6k4.

<Functional configuration of video display device>
The video display device 6k includes a reception unit 6k1, a decoder 6k2, a display control unit 6k3, a display 6k4, and a drive unit 6k5. The decoder 6k2 is the same as the decoder 32 of the video display device 3 of the first embodiment, respectively.

The receiving unit 6k1 receives the encoded processed video PV and the setting information in the same manner as the receiving unit 6k1 of the video display device 6k of the second embodiment. Further, the receiving unit 6k1 receives the drive information transmitted from the transmitting unit 55 of the video processing device 5.

The display control unit 6k3 causes the display 6k4 to display the display area R1-k included in the non-processed area R3 of the processed image PV received by the receiving unit 6k1 and coded by the decoder 6k2. At this time, the display control unit 6k3 may correct the display area R1-k to be displayed on the display 6k4 based on the target position TP included in the drive information and the actual position of the display 6k4.

The display 6k4 displays an area included in the non-processed area R3 of the processed video PV based on the control of the display control unit 6k3.

The drive unit 6k5 drives the display 6k4. Specifically, the drive unit 6k5 drives the display 6k4 so as to arrange the display 6k4 at the target position TP indicated by the drive information received by the reception unit 6k1.

<Operation of video processing device>
Here, the operation of video processing in the video processing apparatus 5 according to the third embodiment will be described with reference to FIG. FIG. 14 is a flowchart showing an example of the operation in the video processing of the video processing apparatus 5 according to the third embodiment. The operation in the video processing of the video processing apparatus 5 described with reference to FIG. 14 corresponds to the video processing method according to the third embodiment.

In step S41, the acquisition unit 51 acquires the video CV from the camera 4.

In step S42, the video processing unit 53 determines the subject area R4 in the video CV.

In step S43, the video processing unit 53 determines the target position TP of each of the displays 6k4 based on the subject area R4.

In step S44, the position of the display area R1-k to be displayed on the display 6k4 and the enlargement ratio of the display area R1-k when the image processing unit 53 displays the display area R1-k on the display 6k4 in the video CV. The display setting holding unit 52 holds the setting information including the position and the enlargement ratio of the display area R1-k.

In step S45, the image processing unit 53 determines the non-processed area R2 and the processed area R3 based on the display area R1-k in the target position TP of the display 6k4. At this time, the non-processed region R2 may be composed of a display region R1 and a region adjacent to the display region R1.

In step S46, the video processing unit 53 does not perform data amount reduction processing on the non-processed region R2, but performs data amount reduction processing on the processed region R3 to generate a processed video PV processed by the video CV. At this time, the image processing unit 53 generates the processed image PV by performing the data amount reduction processing on the processing area R3 different from the non-processing area R2 without performing the data amount reduction processing on the non-processing area R2.

In step S47, the encoder 54 encodes the processed video PV processed in step S45.

In step S48, the transmission unit 55 transmits drive information indicating the target position TP of each of the displays 6k4 to the video display device 6k.

In step S49, the transmission unit 55 transmits the processed video PV encoded by the encoder 54 and the setting information held by the display setting holding unit 52 to the video display device 3.

<Operation of video display device>
The operation in the video display of each of the video display devices 6k of the third embodiment is the same as the operation in the video display of the video display device 6k according to the second embodiment. At this time, the drive unit 6k5 drives the display 6k4 based on the drive information received by the reception unit 6k1. Then, the display control unit 6k3 causes the display 6k4 to display an image in the state where the display 6k4 is in the position driven by the drive unit 6k5, as in the second embodiment.

As described above, according to the third embodiment, the image processing apparatus 5 sets the display area R1-k to be displayed on the display 6k4 and the target position TP of the display 6k4 based on the main subject area R4 in the image CV. decide. Then, the video processing device 5 transmits the drive information including the target position TP to the video display device 6k, so that the display 6k4 is driven to the target position TP. Therefore, even when each of the plurality of video display devices 6k constituting one virtual display VD displays an area based on each display area R1-k in the video CV, the video transmitted to the video display device 3 The amount of data can be reduced.

Further, according to the third embodiment, the processing region R2 may be composed of a display region R1 and a region adjacent to the display region R1. As a result, even if the actual position of the display 6k4 has an error with respect to the target position TP, the image processing unit 53 has low data in the area to be displayed by the display 6k4 at the position where the display 6k4 is actually arranged. No quantity processing is applied. Therefore, the video display device 6k appropriately displays the video that has not been processed to reduce the amount of data when the display area R1 to be displayed on the display 6k4 is corrected in the processed video PV received from the video processing device 5. Can be done.

<Program>
In order to function as the video processing device 2, the video display device 3, the video processing device 5, and the video display device 6 described above, it is also possible to use a computer capable of executing a program instruction, respectively. FIG. 15 is a block diagram showing a schematic configuration of a computer 103 that functions as a video processing device 2, a video display device 3, a video processing device 5, and a video display device 6. Here, the computer 103 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, or the like. The program instruction may be a program code, a code segment, or the like for executing a necessary task.

As shown in FIG. 15, the computer 103 includes a processor 110, a ROM (ReadOnlyMemory) 120, a RAM (RandomAccessMemory) 130, a storage 140, an input unit 150, an output unit 160, and a communication interface ( I / F) 170 and. Each configuration is communicably connected to each other via bus 180. Specifically, the processor 110 is a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and is of the same type or different types. It may be composed of a plurality of processors.

The processor 110 controls each configuration and executes various arithmetic processes. That is, the processor 110 reads the program from the ROM 120 or the storage 140, and executes the program using the RAM 130 as a work area. The processor 110 controls each of the above configurations and performs various arithmetic processes according to the program stored in the ROM 120 or the storage 140. In the present embodiment, the program according to the present disclosure is stored in the ROM 120 or the storage 140.

The program may be recorded on a recording medium that can be read by the computer 103. By using such a recording medium, it is possible to install the program on the computer 103. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transient recording medium is not particularly limited, but may be, for example, a CD-ROM, a DVD-ROM, a USB (Universal Serial Bus) memory, or the like. Further, this program may be downloaded from an external device via a network.

ROM 120 stores various programs and various data. The RAM 130 temporarily stores a program or data as a work area. The storage 140 is composed of an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

The input unit 150 includes one or more input interfaces that accept user input operations and acquire information based on the user's operations. For example, the input unit 150 is, but is not limited to, a pointing device, a keyboard, a mouse, and the like.

The output unit 160 includes one or more output interfaces that output information. For example, the output unit 160 is a display that outputs information as a video, or a speaker that outputs information as audio, but is not limited thereto. The output unit 160 also functions as an input unit 150 in the case of a touch panel type display.

The communication interface 170 is an interface for communicating with an external device.

Although the above-described embodiment has been described as a representative example, it is clear to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the invention should not be construed as limiting by the embodiments described above, and various modifications or modifications can be made without departing from the claims. For example, it is possible to combine a plurality of the constituent blocks described in the configuration diagram of the embodiment into one, or to divide one constituent block into one.

1, 4 Cameras 2, 5 Video processing device 3, 6k Video display device 21, 51 Acquisition unit 22, 52 Display display setting holding unit 23, 53 Video processing unit 24, 54 Encoder 25, 55 Transmission unit 31, 6k1 Receiver unit 32 , 6k2 decoder 33, 6k3 control unit 34, 6k4 display 56 placement information acquisition unit 57 placement information determination unit 6k5 drive unit 100, 101, 102 video processing system 103 computer 120 ROM
130 RAM
140 Storage 150 Input unit 160 Output unit 170 Communication interface 180 Bus

Claims (8)

1. Steps to get the video and
   In the video, the non-processed area based on the display area displayed on the display of the image display device is not processed to reduce the amount of data, and the processed image is processed by applying the data amount reduction processing to the processed area different from the non-processed area. Steps to generate and
   A step of encoding the processed video so that the amount of data of the processed video is reduced,
   A step of transmitting to the video display device setting information including the encoded video, the position of the display area in the video, and the enlargement ratio of the display area when the display is displayed on the display. When,
   Video processing methods including.
2. The video processing method according to claim 1, wherein the data amount reduction processing is a processing for making the feature amount of the pixels constituting the processing area uniform.
3. The video processing method according to claim 1, wherein the data amount reduction processing is a processing for converting the processed area in each of a plurality of frames of images constituting the video into the same still image.
4. Further including a step of acquiring the arrangement information of the plurality of video display devices.
   The step of generating the processed image is
   A step of holding the setting information based on the arrangement information of the plurality of video processing devices, and
   The image processing method according to any one of claims 1 to 3, further comprising a step of determining the processed image and the non-processed image based on the display area according to the setting information.
5. A step of determining a target position for each of the plurality of displays based on the image, and
   The video processing method according to any one of claims 1 to 3, further comprising a step of transmitting drive information indicating the target position of each of the plurality of displays to the video display device including the display.
6. The video processing method according to claim 5, wherein the processed area is composed of the display area and an area adjacent to the display area.
7. The acquisition department that acquires video, and
   In the video, the non-processed area based on the display area displayed on the display of the image display device is not processed to reduce the amount of data, and the processed image is processed by applying the data amount reduction processing to the processed area different from the non-processed area. The video processing unit to generate and
   An encoder that encodes the processed video so that the transmission amount of the processed video is reduced,
   Setting information including the processed image encoded by the encoder, the position of the display area in the image, and the enlargement ratio of the display area when displaying the display area on the display is transmitted to the image display device. Transmitter and
   A video processing device equipped with.
8. A program for causing a computer to function as the video processing device according to claim 7.
   

Images