CN103517052B - Visual point synthesizing method, device and encoder during a kind of coding depth information - Google Patents

Abstract

Embodiments of the present invention provide a viewpoint synthesis method, device, and encoder when encoding depth information. The method includes: during the viewpoint synthesis process when encoding depth information, for each line of depth value of a depth image coding unit, according to the depth The mapping relationship between the value and the horizontal disparity, the original depth value and the distorted depth value are mapped to the original horizontal disparity and the distorted horizontal disparity respectively; according to the difference between the original horizontal disparity and the distorted horizontal disparity, and use the horizontal disparity distortion threshold to judge the current row distortion depth value Whether it will result in a change in the pixel value of the synthesized viewpoint; if it is determined that the distorted depth value of the current row does not cause a change in the pixel value of the synthesized viewpoint, then this row will be skipped during the viewpoint synthesis process, and the distorted depth value of the row will not be used for viewpoint synthesis. The device includes: a mapping unit, a judging unit, and a view synthesis processing unit. The encoder comprises the above-mentioned means. The above technical solution of the present invention can reduce the time complexity of the encoding end while ensuring the encoding performance.

Description

A View Synthesis Method, Device, and Encoder When Encoding Depth Information

technical field

The present invention relates to the field of multimedia technology, in particular to a viewpoint synthesis method, device and encoder when encoding depth information.

Background technique

In the 98th MPEG (Moving Pictures Experts Group) meeting, HHI (Heinrich Hertz Institute, Rhein-Hertz Institute) proposed a rate-distortion optimization algorithm based on synthetic viewpoint distortion information in deep image compression . In this algorithm, the distortion of the depth image is measured by the distortion change of the synthesized viewpoint, which can be expressed as:

∑ _{(x, y)} ∈ l[s' _T (x, y)-s' _{T, R} (x, y)] ² (1)

where s' _T,R (x,y) represents the virtual viewpoint synthesized by applying the original texture image and the original depth information. At the same time, when encoding depth information, a depth image is divided into three categories: encoded, currently encoded and unencoded. for and s' _T (x,y), both choose to apply the distorted texture image for synthesis. The difference is that s' _T (x,y) applies the reconstructed encoded depth information, the original current encoded depth information and the original other pixel depth information for synthesis, while Apply the reconstructed encoded depth information, the distorted currently encoded depth information and the original other pixel depth information for synthesis. It can be seen that ΔD=0 if the current distorted depth information does not cause a change in the synthetic viewpoint.

The existing technical solution is to estimate the synthetic view distortion caused by the distortion of the current depth information based on the change of the synthetic view distortion. Therefore, the following operations need to be performed in actual coding:

1. Before encoding the current depth image, perform viewpoint synthesis on the original depth image and the original texture image, that is, synthesize s' _T,R (x,y). Perform view synthesis on the original depth image and the distorted texture image, that is, synthesize s' _T (x, y) and before the encoding process begins

2. When encoding the current block, rate-distortion calculation is required. Let the current depth block be B, the distorted block to calculate the rate distortion is B', the original depth information of the encoded area is P, the distortion depth information is P', and the original depth information of the unencoded area is H, then the current s ' _T (x, y) is drawn from the depth information of P', B, H, and does not need to be calculated at this time (updated in the following steps). However, in order to calculate ΔD in formula (1), it is necessary to apply B' to update the synthetic viewpoint information after the current depth block information is distorted to update In this process, only the synthetic viewpoint pixels affected by B' are drawn, instead of updating the entire image, so that ΔD can be obtained for rate-distortion calculation.

3. When the current block is encoded, use the currently encoded distortion information to perform view synthesis to update the synthesized view information. Similarly, assuming that the current depth block is B, the reconstructed block is B", the original depth information of the encoded area is P, the distortion depth information is P', and the original depth information of the unencoded area is H. In the process of encoding the current block In the same way, s' _T (x, y) is obtained from the depth information of P', B, and H. However, after encoding the current block to obtain B", it is necessary to apply B" to s' _T (x, y) Update. Same as 2, the update process still only draws the synthetic viewpoint pixels affected by B", instead of updating the entire image, so as to obtain the s' _T (x, y) drawn by P', B", and H ).

It can be seen from the above process that the existing technical solutions need to continuously draw and update viewpoints during the encoding process, which requires a relatively large time complexity (time overhead). Therefore, it is necessary to design a fast technical solution to reduce the time complexity of the encoding end while ensuring the encoding performance.

Contents of the invention

Embodiments of the present invention provide a view synthesis method, device, and encoder when encoding depth information, so as to reduce time complexity at an encoding end while ensuring encoding performance.

On the one hand, an embodiment of the present invention provides a method for synthesizing views when encoding depth information, and the method for synthesizing views when encoding depth information includes:

In the view synthesis process when encoding depth information, for each depth value of the depth image coding unit, according to the mapping relationship between the depth value and the horizontal disparity, the original depth value and the distorted depth value are respectively mapped to the original horizontal disparity and the distorted horizontal disparity ;

According to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to determine whether the current row's distorted depth value will cause a change in the pixel value of the synthetic viewpoint;

If it is determined that the distorted depth value of the current row does not result in a change in the pixel value of the synthesized view, the row is skipped during the view synthesis process, and the distorted depth value of the row is not used for viewpoint synthesis.

Preferably, in an embodiment of the present invention, the horizontal parallax distortion threshold is a horizontal parallax distortion threshold that causes a change in the composite viewpoint obtained according to the smoothness of the texture image and occlusion information in the composite image.

Preferably, in an embodiment of the present invention, according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to judge whether the current line distortion depth value will cause a change in the synthetic viewpoint pixel value, including: the current pixel When other pixels are blocked, the horizontal parallax distortion threshold is zero, that is, the depth value corresponding to the current pixel cannot produce any parallax distortion; if the current depth value does not produce any parallax distortion, it is further judged whether the current row distortion depth value corresponding to the current pixel is Satisfies that the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal parallax distortion threshold; if the current line of distorted depth values all satisfy the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal parallax distortion threshold, then It is determined that the current row distortion depth value does not result in a change in the synthetic view pixel value.

Preferably, in an embodiment of the present invention, according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to judge whether the current line distortion depth value will cause a change in the synthetic viewpoint pixel value, including: the current pixel When being blocked by other pixels or when the current pixel neither blocks other pixels nor is blocked by other pixels, judge whether the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, if the original horizontal parallax and the distortion level If the disparity difference is within the range defined by the horizontal parallax distortion threshold, then it is further judged whether the current row distortion depth value corresponding to the current pixel satisfies the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal parallax distortion threshold; If the distorted depth values of the current row satisfy that the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal disparity distortion threshold, it is determined that the distorted depth value of the current row does not cause a change in the pixel value of the synthetic viewpoint.

On the other hand, an embodiment of the present invention provides a viewpoint synthesis device when encoding depth information, and the viewpoint synthesis device when encoding depth information includes:

The mapping unit is used to map the original depth value and the distorted depth value to the original level according to the mapping relationship between the depth value and the horizontal disparity for each line of depth value of the depth image coding unit during the view synthesis process when encoding the depth information Parallax and Distortion Horizontal Parallax;

A judging unit, configured to judge whether the pixel distortion depth value of the current row will cause a change in the synthetic viewpoint pixel value according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold;

The view synthesis processing unit is configured to skip the line during the view synthesis process if it is determined that the distorted depth value of the current line does not result in a change in the pixel value of the synthesized view, and not use the distorted depth value of the line for view synthesis.

Preferably, in an embodiment of the present invention, the horizontal parallax distortion threshold is a horizontal parallax distortion threshold that causes a change in the composite viewpoint obtained according to the smoothness of the texture image and occlusion information in the composite image.

Preferably, in an embodiment of the present invention, the judging unit includes: a first judging module, used for when the current pixel blocks other pixels, the horizontal parallax distortion threshold is zero, that is, the depth value corresponding to the current pixel cannot produce any parallax distortion ; If the current depth value does not produce any parallax distortion, it is further judged whether the distorted depth value of the current row corresponding to the current pixel satisfies whether the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal parallax distortion threshold; if the current row If the distorted depth values satisfy that the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal disparity distortion threshold, it is determined that the distorted depth value of the current row does not cause a change in the pixel value of the synthetic viewpoint.

Preferably, in an embodiment of the present invention, the judging unit includes: a second judging module for judging the original level when the current pixel is blocked by other pixels or when the current pixel neither blocks nor is blocked by other pixels Whether the difference between the parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, if the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, then further judge the current row corresponding to the current pixel Whether the distorted depth values satisfy the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal disparity distortion threshold; If it is within the range, it is determined that the current row distortion depth value does not cause a change in the pixel value of the synthetic viewpoint.

In yet another aspect, an embodiment of the present invention provides an encoder, where the encoder includes the above-mentioned view synthesis apparatus when encoding depth information.

The above technical solution has the following beneficial effects: because the depth value of each row of the depth image coding unit is used in the process of view synthesis when encoding depth information, the original depth value and the distorted depth value are combined according to the mapping relationship between the depth value and the horizontal disparity. They are respectively mapped to the original horizontal disparity and the distorted horizontal disparity; according to the difference between the original horizontal disparity and the distorted horizontal disparity, and using the horizontal disparity distortion threshold to judge whether the distortion depth value of the current line will cause the change of the pixel value of the synthesized viewpoint; if the current line distortion depth is determined value does not lead to changes in the pixel value of the synthesized viewpoint, skip this row during the viewpoint synthesis process, and do not use the technical means of distorted depth values in this row for viewpoint synthesis, so the time complexity of the encoding end can be reduced while ensuring the encoding performance .

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a flow chart of a viewpoint synthesis method when encoding depth information according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a view synthesis device for encoding depth information according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a judging unit according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of the original parallax and the distorted parallax corresponding to the same 1/4 pixel precision parallax in the application example of the present invention;

Fig. 5(a) is a schematic diagram of the synthetic viewpoint corresponding to the original depth of the application example of the present invention;

Fig. 5(b) is a schematic diagram of synthetic viewpoint corresponding to the distortion depth of the application example of the present invention;

Fig. 5(c) is a schematic diagram of the difference between the pixel values of the synthetic viewpoint in Fig. 5(a) and Fig. 5(b) of the application example of the present invention;

Fig. 6 is a schematic diagram of occluded pixels of an application example of the present invention;

FIG. 7 is a flow chart of a view synthesis method when encoding depth information in an application example of the present invention.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

As shown in FIG. 1 , it is a flowchart of a viewpoint synthesis method when encoding depth information according to an embodiment of the present invention. The viewpoint synthesis method when encoding depth information includes:

101. During the view synthesis process when encoding depth information, for each depth value of a depth image coding unit, according to the mapping relationship between the depth value and the horizontal disparity, the original depth value and the distorted depth value are respectively mapped to the original horizontal disparity and distortion horizontal parallax;

102. According to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold, it is judged whether the distorted depth value of the current line will cause a change in the pixel value of the synthesized viewpoint;

103. If it is determined that the distorted depth value of the current row does not result in a change in the pixel value of the synthesized view, skip this row during the view synthesis process, and perform viewpoint synthesis without using the distorted depth value of the row.

Preferably, the horizontal parallax distortion threshold is a horizontal parallax distortion threshold that causes a change in the composite viewpoint obtained according to the smoothness of the texture image and occlusion information in the composite image.

Preferably, according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to determine whether the current row's distorted depth value will cause a change in the pixel value of the synthetic viewpoint, it includes: when the current pixel blocks other pixels, the horizontal parallax distortion The threshold is zero, that is, the depth value corresponding to the current pixel cannot produce any parallax distortion; if the current depth value does not produce any parallax distortion, it is further judged whether the current row distortion depth value corresponding to the current pixel satisfies the original horizontal parallax and the distorted horizontal parallax The difference is within the range defined by the horizontal parallax distortion threshold; if the current row distortion depth value satisfies that the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, it is determined that the current row distortion depth value does not cause Composites changes in viewpoint pixel values.

Preferably, according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to determine whether the current row's distorted depth value will cause a change in the pixel value of the synthetic viewpoint, including: when the current pixel is blocked by other pixels or the current pixel When neither blocking other pixels nor being blocked by other pixels, judge whether the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, if the difference between the original horizontal parallax and the distorted horizontal parallax is within the horizontal parallax distortion threshold Within the defined range, it is further judged whether the current row distortion depth value corresponding to the current pixel satisfies whether the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal parallax distortion threshold; if the current row distortion depth value satisfies If the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal disparity distortion threshold, it is determined that the distorted depth value of the current row does not cause a change in the pixel value of the synthetic viewpoint.

Corresponding to the above method embodiment, as shown in FIG. 2 , it is a schematic structural diagram of a viewpoint synthesis device when encoding depth information according to an embodiment of the present invention. The viewpoint synthesis device when encoding depth information includes:

The mapping unit 21 is used to map the original depth value and the distorted depth value to the original Horizontal Parallax and Distortion Horizontal Parallax;

The judging unit 22 is configured to judge whether the distorted depth value of the current line will cause a change in the pixel value of the synthetic viewpoint according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold;

The view synthesis processing unit 23 is configured to skip the line during the view synthesis process if it is determined that the distorted depth value of the current line does not cause a change in the synthesized view pixel value, and not use the distorted depth value of the line for view synthesis.

Preferably, the horizontal parallax distortion threshold is a horizontal parallax distortion threshold that causes a change in the composite viewpoint obtained according to the smoothness of the texture image and occlusion information in the composite image.

Preferably, as shown in FIG. 3 , which is a schematic structural diagram of the judging unit of the embodiment of the present invention, the judging unit 22 includes: a first judging module 221, used for when the current pixel blocks other pixels, the horizontal parallax distortion threshold is zero, that is, the current The depth value corresponding to the pixel cannot produce any parallax distortion; if the current depth value does not produce any parallax distortion, it is further judged whether the current row distortion depth value corresponding to the current pixel satisfies the difference between the original horizontal parallax and the distorted horizontal parallax in the horizontal parallax distortion Within the range defined by the threshold; if the current row distortion depth value satisfies the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal parallax distortion threshold, it is determined that the current row distortion depth value does not cause a change in the synthetic viewpoint pixel value . The judging unit includes: a second judging module 222, which is used to judge whether the difference between the original horizontal parallax and the distorted horizontal parallax is at the horizontal level when the current pixel is blocked by other pixels or when the current pixel neither blocks other pixels nor is blocked by other pixels. Within the range defined by the parallax distortion threshold, if the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, it is further judged whether the current row distortion depth value corresponding to the current pixel satisfies the original horizontal parallax and the The difference between the distorted horizontal disparity is within the range defined by the horizontal parallax distortion threshold; if the current row distortion depth value satisfies the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal parallax distortion threshold, then the current row distortion depth is determined The value does not result in a change in the synthetic viewpoint pixel value.

In yet another aspect, an embodiment of the present invention provides an encoder, where the encoder includes the above-mentioned view synthesis apparatus when encoding depth information.

The above-mentioned method, device or encoder technical solution in the embodiment of the present invention has the following beneficial effects: because the depth value of each line of the depth image coding unit is used in the process of view synthesis when encoding depth information, according to the mapping between the depth value and the horizontal disparity The relationship between the original depth value and the distorted depth value is mapped to the original horizontal disparity and the distorted horizontal disparity respectively; according to the difference between the original horizontal disparity and the distorted horizontal disparity, and using the horizontal disparity distortion threshold to judge whether the distorted depth value of the current line will result in synthetic viewpoint pixels value change; if it is determined that the distorted depth value of the current line does not result in a change in the pixel value of the synthesized view, then this line will be skipped during the view synthesis process, and the technical means of view synthesis will not be performed using the distorted depth value of this line, so it can be guaranteed while encoding performance while reducing the time complexity of the encoding side.

The technical solution proposed by the above method, device or encoder in the embodiment of the present invention is to use the original depth information and current depth information after obtaining the original depth information and the current depth Information, as well as texture image characteristics and occlusion characteristics of synthetic viewpoints are analyzed to determine whether a viewpoint synthesis operation is required. (It should be noted that this technical solution has no impact on the decoder.) The specific technical solution is as follows:

In the case where the cameras are horizontally arranged in parallel, the relationship between the real depth information z and the disparity information d can be expressed as:

Among them, f represents the focal length of the camera. l represents the reference distance between two viewpoints.

Assuming that the quantized depth corresponding to the current depth z is v=Q(z), the relationship between the disparity information and the quantized depth can be expressed as:

It can be seen from formula (3) that different depth information will lead to different horizontal disparity. However, in practical applications, a rounding operation (rounding) is generally performed on the horizontal disparity. For example, in the existing 3DV-HEVC, the horizontal disparity is rounded with 1/4 pixel precision, as shown in FIG. 4 , which is a schematic diagram of the original disparity and distorted disparity corresponding to the same 1/4 pixel precision disparity in the application example of the present invention. After rounding, it can be seen that the original depth and the distorted depth correspond to the same horizontal disparity with 1/4 pixel precision. Therefore, distortion of depth information does not necessarily result in distortion of horizontal disparity.

Let the position of the current pixel be p, and the rounding operation of the parallax d corresponding to the pixel depth value at 1/N precision is R _N (d). Suppose d _op is the original disparity and d _sp is the distorted disparity. Then the parallax distortion can be expressed as

D _N (d _op ,d _sp )＝R _N (d _op )-R _N (d _sp ) (4)

In the existing 3DV-HEVC, the horizontal disparity is rounded with 1/4 pixel accuracy, and N=4.

When the parallax distortion is not 0, the distortion of the synthetic virtual viewpoint has a great relationship with the characteristics of the texture image, as shown in Figure 5(a), which is a schematic diagram of the synthetic viewpoint corresponding to the original depth of the application example of the present invention; Figure 5(b) ) is a schematic diagram of the synthetic viewpoint corresponding to the distortion depth of the application example of the present invention; Fig. 5(c) is a schematic diagram of the difference between the pixel values of the synthetic viewpoint in Fig. 5(a) and Fig. 5(b) of the application example of the present invention. It can be seen from the figure that the pixel values of positions 2 to 5 in the texture image are the same, and the gradient of the texture image at these positions is very small or 0. In this case, the pixels in the synthetic viewpoint obtained by using the original depth information and the distorted depth information The difference is very small, as shown in Figure 5(a) and Figure 5(b), the pixel values of synthetic viewpoint positions 1~3; on the other hand, the pixel values of positions 5~9 in the texture image vary greatly, and the texture image gradient In this case, the pixels in the synthetic viewpoint obtained by using the original depth information and the distorted depth information are very different, as shown in Figure 5(a) and Figure 5(b) for the pixel values of synthetic viewpoint positions 4-7.

The embodiment of the present invention defines the thresholds G _l and G _r to represent the smoothness of the left and right sides of the current pixel after 1/4 pixel interpolation, specifically, G _l represents the range of pixels on the left side of the current pixel p, For any pixel between the current pixel p and G _l , the value of this pixel is the same as that of the current pixel p. Similarly, G _r means that it is within the range of pixels on the right side of the current pixel p. For the current pixel p and G _r Any pixel in between, the value of this pixel is the same as the current pixel p.

G _l = min{p _l } where p _l satisfies: I(m)=I(p)

G _r = max{p _r } where p _r satisfies: I(m)=I(p) (5)

Among them, p ₁ <p indicates the pixel on the left side of pixel p, p _r >p indicates the pixel on the right side of pixel p, and I indicates the high-resolution reconstructed image after interpolation.

When the current pixel is occluded, the depth information of the current pixel has no effect in view synthesis. As shown in FIG. 6 , it is a schematic diagram of occluded pixels in an application example of the present invention. c, d Pixel depth information does not affect the result of view synthesis. After the depth information of the current pixel is distorted, if the current pixel is still occluded, it can be considered that the distortion of the depth information will not affect the view synthesis. When the front image pixel is blocked, the application example definition of the present invention:

Where p _o is the position of the depth pixel that blocks the current pixel, and p is the position of the current pixel. When T _o <0, it means that the current pixel is occluded by the pixel to its left on the synthetic viewpoint. If the current pixel's depth value is distorted causing the current pixel's position on the composite view to be shifted to the left but not to On the left, the depth value distortion does not affect the distortion of the synthetic view. Similarly, T _o >0 indicates that the current pixel is occluded by the pixel to its right in the synthetic view.

When the current pixel blocks other pixels, the application example of the present invention requires that the current pixel cannot produce any parallax distortion.

In summary, the application examples of the present invention define the thresholds T _l and T _r as follows (T _l indicates how much the horizontal parallax after distortion can be shifted to the left at most compared with the original horizontal parallax, and T _r indicates that the horizontal parallax after distortion is different from the original horizontal parallax How much can be shifted to the right compared to the parallax):

and define the condition:

C={R _N (v)-R _N (v')≤T _l and R _N (v')-R _N (v)≤T _r }, where C represents the range of the distorted horizontal parallax Will cause distortion of the synthesized viewpoint, v represents the original horizontal disparity, and v' represents the distorted horizontal disparity.

As shown in FIG. 7 , it is a flow chart of the viewpoint synthesis method when encoding depth information in an application example of the present invention, including the following steps:

701. Initialize j=0;

702. For the j-th row pixel S _j in the current block

703. Judging whether d∈C, if yes, it can be considered that the distortion of the depth information pixel of the current line has no influence on the distortion of the synthetic viewpoint, then go to step 704, if not, then go to step 706;

704. Skip the jth row, and do not use the jth row of pixels to perform view synthesis;

705. Assign j+1 to the newest j, i.e. j=j+1, and then turn to step 702;

706. Use the j-th row of pixels S _j to synthesize viewpoints;

707. Assign j+1 to the newest j, that is, j=j+1, and go to step 702.

It can be seen that when it is necessary to use a distorted depth value for viewpoint synthesis in encoding, it is necessary to judge the depth value of each row to determine whether the depth value of the row satisfies the conditions of set C, and if they all meet, the depth of the row is not used. value, perform view synthesis, otherwise, perform view synthesis according to the normal process.

The technical scope (field) of application of this technical solution is the view synthesis process in the process of encoding depth information. In the process of computing distortion for encoding depth information, existing methods use each row of pixels of each depth image coding unit for view synthesis. In the solution proposed by this technical solution, it is first judged whether each row of the depth image coding unit has an influence on the viewpoint synthesis, and if not, the row is skipped during the viewpoint synthesis process to reduce the encoding time complexity.

Experiments show that for a sequence of 1024x768, the algorithm proposed in this technical solution can reduce the total time complexity of the encoding end by about 4% compared with the previous algorithm without losing encoding performance.

Those skilled in the art can also understand that various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units and steps above have generically described their functions. Whether such functions are implemented by hardware or software depends on the specific application and overall system design requirements. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the protection scope of the embodiments of the present invention.

The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented by general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to implement or operate the described functions. The general-purpose processor may be a microprocessor, and optionally, the general-purpose processor may also be any conventional processor, controller, microcontroller or state machine. A processor may also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration to accomplish.

The steps of the method or algorithm described in the embodiments of the present invention may be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software modules may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other storage medium in the art. Exemplarily, the storage medium can be connected to the processor, so that the processor can read information from the storage medium, and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be set in the ASIC, and the ASIC can be set in the user terminal. Optionally, the processor and the storage medium may also be set in different components in the user terminal.

In one or more exemplary designs, the above functions described in the embodiments of the present invention may be implemented in hardware, software, firmware or any combination of the three. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special computer. For example, such computer-readable media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other device that can be used to carry or store instructions or data structures and Other medium of program code in a form readable by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. In addition, any connection is properly defined as a computer-readable medium, for example, if the software is transmitted from a web site, server, or other remote source via a coaxial cable, fiber optic computer, twisted pair, digital subscriber line (DSL) Or transmitted by wireless means such as infrared, wireless and microwave are also included in the definition of computer readable media. Disks and discs include compact discs, laser discs, optical discs, DVDs, floppy discs, and Blu-ray discs. Disks usually reproduce data magnetically, while discs usually reproduce data optically with lasers. Combinations of the above can also be contained on a computer readable medium.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, within the spirit and principles of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.

Claims (5)

1. A method for synthesizing views when encoding depth information, characterized in that, the method for synthesizing views when encoding depth information comprises: In the view synthesis process when encoding depth information, for each depth value of the depth image coding unit, according to the mapping relationship between the depth value and the horizontal disparity, the original depth value and the distorted depth value are respectively mapped to the original horizontal disparity and the distorted horizontal disparity ; According to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to determine whether the current row's distorted depth value will cause a change in the pixel value of the synthetic viewpoint; If it is determined that the distorted depth value of the current row does not result in a change in the pixel value of the synthesized viewpoint, the row is skipped during the viewpoint synthesis process, and the distorted depth value of the row is not used for viewpoint synthesis; According to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to judge whether the current row distortion depth value will cause a change in the synthetic viewpoint pixel value, including: When the current pixel blocks other pixels, the horizontal parallax distortion threshold is zero, that is, the depth value corresponding to the current pixel cannot produce any parallax distortion; if the current depth value does not produce any parallax distortion, then further judge the current row distortion depth value corresponding to the current pixel Whether the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold; if the current line of distorted depth values all satisfy the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold , it is determined that the current line distortion depth value does not cause a change in the pixel value of the synthetic viewpoint; According to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold to judge whether the current row distortion depth value will cause a change in the synthetic viewpoint pixel value, including: When the current pixel is blocked by other pixels or when the current pixel neither blocks other pixels nor is blocked by other pixels, it is judged whether the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, if the original horizontal parallax and If the difference between the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, it is further judged whether the current line of distortion depth values corresponding to the current pixel satisfies the difference between the original horizontal parallax and the distorted horizontal parallax within the range defined by the horizontal parallax distortion threshold. If the distorted depth value of the current row satisfies that the difference between the original horizontal disparity and the distorted horizontal disparity is within the range defined by the horizontal disparity distortion threshold, it is determined that the distorted depth value of the current row does not cause a change in the pixel value of the synthetic viewpoint. 2. The viewpoint synthesis method when encoding depth information as claimed in claim 1, wherein the horizontal parallax distortion threshold is the level that causes the synthetic viewpoint to change according to the smoothness characteristics of the texture image and the occlusion information in the synthesized image. Parallax distortion threshold. 3. A view synthesis device when encoding depth information, characterized in that, the view synthesis device when encoding depth information comprises: The mapping unit is used to map the original depth value and the distorted depth value to the original level according to the mapping relationship between the depth value and the horizontal disparity for each line of depth value of the depth image coding unit during the view synthesis process when encoding the depth information Parallax and Distortion Horizontal Parallax; A judging unit, configured to judge whether the current row's distorted depth value will cause a change in the synthetic viewpoint pixel value according to the difference between the original horizontal parallax and the distorted horizontal parallax, and using the horizontal parallax distortion threshold; The viewpoint synthesis processing unit is used to skip the row during the viewpoint synthesis process if it is determined that the current row's distorted depth value does not result in a change in the composite viewpoint pixel value, and does not use the row's distorted depth value for viewpoint synthesis; The judging unit includes: The first judging module is used when the current pixel blocks other pixels, and the horizontal parallax distortion threshold is zero, that is, the depth value corresponding to the current pixel cannot produce any parallax distortion; if the current depth value does not produce any parallax distortion, then further judge the current pixel. Whether the corresponding distorted depth values of the current row satisfy the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal disparity distortion threshold; If it is within the range defined by the distortion threshold, it is determined that the distortion depth value of the current line does not cause a change in the pixel value of the synthetic viewpoint; The judging unit includes: The second judging module is used to determine whether the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold when the current pixel is blocked by other pixels or when the current pixel neither blocks other pixels nor is blocked by other pixels If the difference between the original horizontal parallax and the distorted horizontal parallax is within the range defined by the horizontal parallax distortion threshold, it is further judged whether the current line of the current row of distorted depth values corresponding to the current pixel satisfies the difference between the original horizontal parallax and the distorted horizontal parallax in the horizontal Within the range defined by the parallax distortion threshold; if the distorted depth value of the current row satisfies the difference between the original horizontal disparity and the distorted horizontal disparity within the range defined by the horizontal parallax distortion threshold, then it is determined that the current row distorted depth value does not result in a synthetic viewpoint pixel value The change. 4. The view synthesis device when encoding depth information as claimed in claim 3, wherein the horizontal parallax distortion threshold is a level that results in a change in the synthesized viewpoint according to the smoothness of the texture image and the occlusion information in the synthesized image. Parallax distortion threshold. 5. An encoder, characterized in that the encoder comprises the view synthesis device when encoding depth information according to any one of claims 3-4.