JP2020184059A - Circuit devices, electronic devices and mobiles - Google Patents

Abstract

An object of the present invention is to provide a circuit device and the like capable of high-speed processing of distortion correction corresponding to a curved display. A circuit device (100) includes a coordinate transformation circuit (20) and a mapping processing circuit (30). The coordinate conversion circuit 20 performs coordinate conversion from input coordinates IXY1 to output coordinates QXY1. Mapping processing circuit 30 performs mapping processing based on output coordinates QXY1 on input first image IMG1 to generate second image IMG2 to be displayed on a display panel for displaying an image on a curved display. do. The coordinate transformation circuit 20 performs coordinate transformation from the input coordinates IXY1 to the output coordinates QXY1 by performing arithmetic processing using a polynomial of degree 2 or higher representing the coordinate transformation. [Selection diagram] Fig. 1

Description

The present invention relates to circuit devices, electronic devices, mobile objects, and the like.

A head-up display (HUD) is known in which information is superimposed on the user's field of view by displaying an image on a curved display such as a transparent screen. When the head-up display displays the input image as it is, the displayed image looks distorted due to the distortion of the curved display. Therefore, the head-up display performs distortion correction on the input image and displays the image after the distortion correction to display an undistorted image. Patent Document 1 discloses a prior art of a head-up display that performs distortion correction. The head-up display of Patent Document 1 includes a memory for storing a mathematical formula for coordinate transformation, performs coordinate transformation using the mathematical formula for coordinate transformation, and maps an input image based on the result to distort the display. Make corrections.

JP-A-2010-164869

The above-mentioned Patent Document 1 discloses a mathematical formula for coordinate transformation in distortion correction, but the mathematical formula is a principle one and includes a complicated operation such as integration or square root. Therefore, the technique of Patent Document 1 has a problem that high-speed processing such as real-time processing of distortion correction for moving images is difficult. Patent Document 1 does not disclose a specific processing process for performing high-speed processing.

One aspect of the present disclosure is to display an image on a curved display by performing a coordinate conversion circuit that converts coordinates from input coordinates to output coordinates and a mapping process based on the output coordinates on the input first image. The coordinate conversion circuit includes a mapping processing circuit that generates a second image to be displayed on a display panel for display, and the coordinate conversion circuit performs arithmetic processing using a second-order or higher polymorphism representing the coordinate conversion. It relates to a circuit device that performs the coordinate conversion.

First configuration example of a circuit device. The figure explaining the operation of the circuit apparatus in 1st configuration example. Detailed configuration example of the coordinate conversion circuit. Detailed configuration example of the mapping processing circuit. A second configuration example of a circuit device. The figure explaining the operation of the circuit apparatus in the 2nd configuration example. The figure explaining the relationship between the mounting tolerance of a head-up display and rotation correction. A third configuration example of a circuit device. The figure explaining the operation of the circuit apparatus in 3rd configuration example. A fourth configuration example of a circuit device. The figure explaining the operation of the circuit apparatus in 4th configuration example. A fifth configuration example of a circuit device. Detailed configuration example of the first processing circuit and the second processing circuit. An example of an image displayed on a head-up display. An image of ROI extracted from the image before mapping processing. An image in which the ROI image extracted from the image displayed on the head-up display is reverse-mapped. An example of an edge image. Configuration example of electronic equipment. An example of a mobile body.

Hereinafter, preferred embodiments of the present disclosure will be described in detail. It should be noted that the present embodiment described below does not unreasonably limit the contents described in the claims, and not all of the configurations described in the present embodiment are essential constituent requirements.

1. 1. First Configuration Example FIG. 1 is a first configuration example of the circuit device 100 of the present embodiment. FIG. 2 is a diagram illustrating the operation of the circuit device 100 in the first configuration example.

The circuit device 100 is a HUD controller that controls the image display of the head-up display. However, the circuit device 100 is not limited to the HUD controller, and may be, for example, a display driver that drives the display panel of the head-up display. In this case, the circuit device 100 may include a drive circuit that drives the display panel based on the image IMG2. The circuit device 100 is an integrated circuit device called an IC (Integrated Circuit). The circuit device 100 is an IC manufactured by a semiconductor process, and is a semiconductor chip in which a circuit element is formed on a semiconductor substrate.

As shown in FIG. 2, the circuit device 100 maps the image IMG1 to the image IMG2. The image IMG1 is an input image and is also called a first image. The image IMG2 is an output image and is also called a second image. The mapping process gives the image IMG 2 a distortion that cancels the distortion caused by the curved display of the head-up display. That is, the image IMG2 displayed on the display panel of the head-up display is a distorted image. By displaying this image IMG2 on the curved display, a display image without distortion is presented to the user.

An image processing engine that performs image conversion in such distortion correction is called a warp engine. In the first configuration example, the case where the circuit device 100 is an inverse warp engine will be described. The inverse warp engine is a warp engine having an inverse warp function. The inverse warp is a conversion in which each pixel of the output image of the warp engine is obtained from a pixel at an arbitrary position in the input image. As shown in FIG. 2, the circuit device 100 converts the coordinates (x, y) in the image IMG2 which is the output image into the coordinates (x', y') in the image IMG1 which is the input image, and is based on the coordinates. Inverse warp is realized by performing mapping processing.

As described in the second configuration example, the circuit device 100 may be a forward warp engine. Further, as described in the third and fourth configuration examples, the circuit device 100 may further perform image rotation.

Hereinafter, the details of the first configuration example in which the circuit device 100 is an inverse warp engine will be described. As shown in FIG. 1, the circuit device 100 includes a coordinate counter 10, which is a first coordinate counter, a coordinate conversion circuit 20, and a mapping processing circuit 30.

The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1. In the first configuration example, the input coordinates IXY1 are the coordinates (x, y) corresponding to the coordinates on the image IMG2, and the output coordinates QXY1 are the coordinates (x', y') corresponding to the coordinates on the image IMG1. is there. The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1 by performing arithmetic processing using a polynomial of degree 2 or higher representing the coordinate conversion. The following equation (1) shows the coordinate transformation equation when the polynomial is quadratic.

In the first line of the above equation (1), a1 × x ^{2 on the} right side is the first term of the polynomial for obtaining x'. Similarly, a2 × y ² , a3 × xy, a4 × x, a5 × y, and a6 are the second, third, fourth, fifth, and sixth terms of the polynomial for finding x'. .. a1 is the coefficient of the first term, that is, the first coefficient. Similarly, a2, a3, a4, a5, and a6 are the second coefficient, the third coefficient, the fourth coefficient, the fifth coefficient, and the sixth coefficient. The same applies to the formula for obtaining y'.

The coordinate conversion formula is not limited to the quadratic polynomial, and may be a polynomial of degree 3 or higher. For example, the following equation (2) shows a coordinate transformation equation when the polynomial is cubic.

The mapping processing circuit 30 generates the image IMG2 by performing the mapping processing based on the output coordinate QXY1 on the input image IMG1. Specifically, the pixel value of the coordinates (x, y) in the image IMG2 is obtained from the pixel value of the coordinates (x', y') in the image IMG1. (X, y) is the input coordinate IXY1, and (x', y') is the output coordinate QXY1. However, the mapping processing circuit 30 does not use the input coordinate IXY1, but performs the mapping process using the internally generated coordinates and the output coordinate QXY1. This point will be described later in FIG. The pixel value is, for example, color data of a pixel for display at the coordinates.

The image IMG2 is displayed on the curved display of the head-up display. For example, a head-up display includes a light source, a liquid crystal display panel, an optical system, and a transparent screen. The image IMG2 is displayed on the liquid crystal display panel. The light source emits light, the emitted light passes through the liquid crystal display panel, and the optical system projects the transmitted light onto the transparent screen, so that the image IMG2 is projected on the transparent screen. In this case, the transparent screen corresponds to a curved display. Alternatively, the head-up display includes a transparent display such as an organic EL display panel. In this case, the image IMG2 is displayed directly on the transparent display. That is, the transparent display, which is a display panel, also serves as a curved display.

According to the present embodiment, the coordinate conversion circuit 20 performs coordinate conversion by arithmetic processing using a polynomial, so that high-speed processing of distortion correction is possible. Specifically, as shown in the above equation (1) and the like, the polynomial consists of a combination of multiplication and addition, and does not include complicated calculations such as integration. Therefore, by using a polynomial, the calculation time is shortened as compared with the case of performing a calculation including integration and the like.

Further, in the polynomial, the input coordinate IXY1 (x, y) and the output coordinate QXY1 (x', y') have a one-to-one correspondence. This correspondence can be used as it is as a correspondence between the pixels of the output image IMG2 and the pixels of the input image IMG1 in the mapping process. That is, when (x, y) is sequentially input to the coordinate conversion circuit 20 pixel by pixel, (x', y') is sequentially output one pixel at a time correspondingly. By mapping the mapping processing circuit 30 pixel by pixel using this correspondence, it is possible to configure the image IMG2 for one screen. Such sequential processing is more suitable for real-time processing than the case where a table or the like is once created and mapping processing is performed. Further, in the display processing, since pixel data is input and output by the pixel clock, the sequential processing of each pixel is suitable for real-time processing.

Hereinafter, a case where the coordinate conversion formula is a quadratic polynomial as in the above formula (1) will be described as an example.

The coefficient information CF1 of the polynomial and the input coordinates IXY1 are input to the coordinate conversion circuit 20. The coordinate conversion circuit 20 obtains the output coordinate QXY1 from the input coordinate IXY1 by arithmetic processing based on the coefficient information CF1. The coefficient information CF1 is information on the coefficients a1 to a6 and b1 to b6 in the above equation (1). For example, the circuit device 100 includes a storage unit that stores the coefficient information CF1, and the coefficient information CF1 is input from the storage unit to the coordinate conversion circuit 20. The storage unit is a register or a memory. The memory is a semiconductor memory such as RAM or non-volatile memory. The coefficient information CF1 is written in the storage unit, for example, when the circuit device 100 is started up or when the head-up display is manufactured.

In coordinate transformation using a polynomial, the correspondence between the input coordinate IXY1 and the output coordinate QXY1 is determined by the coefficient of the polynomial. That is, by inputting the coefficient information CF1 that matches the shape of the curved display into the coordinate conversion circuit 20, distortion correction suitable for various curved displays is realized. Further, since the coordinate conversion circuit 20 obtains the output coordinate QXY1 from the input input coordinate IXY1, it is possible to perform coordinate conversion for each pixel suitable for real-time processing as described above.

The coordinate counter 10 outputs the first count coordinates, which are the coordinates of each pixel based on the pixel clock. The coordinate conversion circuit 20 performs coordinate conversion using the first count coordinate as the input coordinate IXY1. In the first configuration example, since the input coordinates IXY1 correspond to the coordinates on the image IMG2, the pixel clock used by the mapping processing circuit 30 to process the image IMG2 is input to the coordinate counter 10.

The size of the image IMG2 is N × M pixels. N and M are integers of 2 or more. For example, the coordinate counter 10 increments x by 1 for each pixel clock at y = 0, and (0,0), (1,0), (2,0), ..., (N-1,0). ) Is output. Next, the coordinate counter 10 sets y = 1 and increments x by 1 for each pixel clock to increase (0,1), (1,1), (2,1), ..., (N-1). , 1) is output. The coordinate counter 10 repeats this until (N-1, M-1). The order of counting the coordinates is not limited to the above, and the counting order may be set according to the order in which the pixels of the image IMG2 are processed.

In the present embodiment, the coordinate counter 10 outputs the coordinates (x, y) pixel by pixel to the coordinate conversion circuit 20 based on the pixel clock, and the coordinate conversion circuit 20 performs coordinate conversion to coordinate the coordinates (x', one pixel at a time. Output y'). As a result, the mapping processing circuit 30 can perform the mapping processing pixel by pixel based on the pixel clock. As a result, distortion correction suitable for real-time processing is realized.

The processing time of the calculation for converting the input coordinate IXY1 to the output coordinate QXY1 may be a plurality of cycles of the pixel clock. The output coordinate QXY1 corresponding to the input coordinate IXY1 is output with a delay of the processing time, but it is sufficient that one output coordinate QXY1 can be obtained in one cycle of the pixel clock as the throughput.

FIG. 3 is a detailed configuration example of the coordinate conversion circuit 20. The coordinate conversion circuit 20 includes a first calculation circuit 21 and a second calculation circuit 22. The first arithmetic circuit 21 and the second arithmetic circuit 22 are logic circuits, each of which is composed of individual hardware circuits. Although only the arithmetic circuit for obtaining x'is shown in FIG. 3, the arithmetic circuit for obtaining y'has the same configuration.

The first arithmetic circuit 21, when the input coordinates IXY1 and ^{(x, y), x 2} , y 2, and obtains the xy. More specifically, the first arithmetic circuit 21 includes a ^{x 2} arithmetic circuit MC1 seeking ^{x 2} by squaring the x, and ^{y 2} arithmetic circuit MC2 seeking ^{y 2} by squaring y, by multiplying the x and y The xy arithmetic circuit MC3 for obtaining xy is included. The arithmetic circuits MC1 to MC3 are individual hardware multipliers.

The second arithmetic circuit 22 obtains the output coordinates QXY1 based on the arithmetic result of the first arithmetic circuit 21 and the coefficient information CF1. Here, the second arithmetic circuit 22 obtains x'of (x', y'). The coefficients a1 to a6 of the above equation (1) are input to the second arithmetic circuit 22 as the coefficient information CF1. The second arithmetic circuit 22 includes the first to fifth arithmetic circuits AC1 to AC5 and the addition circuit ADDC. The first to fifth term arithmetic circuits AC1 to AC5 are individual hardware multipliers.

The first term calculation circuit AC1 calculates the first term a1 × ^{x 2} by multiplying the coefficients a1 and ^{x 2.} The second term arithmetic circuit AC2 multiplies the coefficients a2 and y ² to obtain the second term a2 × y ² . The third term arithmetic circuit AC3 obtains the third term a3 × xy by multiplying the coefficient a3 and xy. The fourth term arithmetic circuit AC4 obtains the fourth term a4 × x by multiplying the coefficients a4 and x. The fifth term arithmetic circuit AC5 multiplies the coefficients a5 and y to obtain the fifth term a5 × y. The adder circuit ADDC adds the first to fifth terms and the coefficient a6 which is the sixth term, and outputs the result as x'. As a result, the operation of x'in the above equation (1) is realized.

The configuration of the first arithmetic circuit 21 and the second arithmetic circuit 22 is not limited to FIG. For example, the first arithmetic circuit 21 includes three arithmetic circuits, x ² its three arithmetic circuit as pipeline ^processing, y ^2, and may be calculated xy. Then, the second arithmetic circuit 22 includes five arithmetic circuits, and the five arithmetic circuits may calculate the first to fifth terms as in the pipeline processing.

According to this embodiment, the first arithmetic circuit ^21, x 2, ^{y 2,} and includes an operational circuit MC1~MC3 seeking xy, second arithmetic circuit 22, first to determine the first to fifth term Item 5 The arithmetic circuits AC1 to AC5 are included. As a result, each term can be calculated in parallel, and high-speed processing suitable for real-time processing becomes possible.

The second arithmetic circuit 22 may include the first to nth arithmetic circuits. The i-term arithmetic circuit computes the i-term of the polynomial. n is an integer of 2 or more, and i is an integer of 1 or more and n or less. When the coordinate transformation formula is a quadratic polynomial as shown in FIG. 3, n = 5. For example, when the coordinate conversion formula is a cubic polynomial as in the above formula (2), n = 9 may be set.

FIG. 4 is a detailed configuration example of the mapping processing circuit 30. The mapping processing circuit 30 includes a memory control circuit 31, a coordinate counter 32 which is a second coordinate counter, and an image memory 33.

The coordinate counter 32 outputs the second count coordinate CXYB1, which is the coordinate of each pixel based on the pixel clock. Let the second count coordinate CXYB1 be (xb, yb). In the first configuration example, since (xb, yb) specifies the coordinates on the image IMG1, the pixel clock input to the mapping processing circuit 30 together with the image IMG1 is input to the coordinate counter 32. The coordinate counter 32 counts (xb, yb) by the same counting operation as the coordinate counter 10.

The memory control circuit 31 controls access to the image memory 33. The image memory 33 is a buffer memory that temporarily stores the image IMG 1, and is, for example, a semiconductor memory such as a RAM. The memory control circuit 31 writes the image IMG1 to the image memory 33 based on the second count coordinate CXYB1, reads the pixel value from the image memory 33 based on the output coordinate QXY1 from the coordinate conversion circuit 20, and reads the pixel value from the image IMG2. It is output as the pixel value of. Specifically, the memory control circuit 31 includes a write address controller 34 and a read address controller 35.

The write address controller 34 decodes (xb, yb), which is the second count coordinate CXYB1, into the write address ADWR, and writes the pixel value at (xb, yb) of the image IMG1 to the address ADWR of the image memory 33. By repeating this, the image IMG1 is written to the image memory 33.

The read address controller 35 decodes the output coordinates QXY1 (x', y') into the read address ADRD, and reads the pixel value from the address ADRD of the image memory 33. This corresponds to reading out the pixel value at (x', y') of the image IMG1. The read address controller 35 outputs the read pixel value as the pixel value at (x, y) of the image IMG2. By repeating this, the image IMG2 is output from the mapping processing circuit 30.

In the above mapping process, the pixel value at (x, y) of the image IMG2 is acquired from the pixel value at (x', y') of the image IMG1. That is, the mapping processing circuit 30 operates as the inverse warp engine described with reference to FIG.

2. Second Configuration Example FIG. 5 is a second configuration example of the circuit device 100. FIG. 6 is a diagram illustrating the operation of the circuit device 100 in the second configuration example.

In the second configuration example, the case where the circuit device 100 is a forward warp engine will be described. The forward warp engine is a warp engine having a forward warp function. The forward warp is a conversion in which each pixel of the input image of the warp engine is obtained from a pixel at an arbitrary position in the output image. As shown in FIG. 6, the circuit device 100 converts the coordinates (x', y') in the image IMG1 which is an input image into the coordinates (x, y) in the image IMG2 which is an output image, and is based on the coordinates. The forward warp is realized by performing the mapping process.

As shown in FIG. 5, the circuit device 100 includes a coordinate counter 10, a coordinate conversion circuit 20, and a mapping processing circuit 30.

In the second configuration example, the coordinate counter 10 outputs (x', y'), which is the first count coordinate, to the coordinate conversion circuit 20 as the input coordinate IXY1. Since (x', y') correspond to the coordinates on the image IMG1, the pixel clock input to the mapping processing circuit 30 together with the image IMG1 is input to the coordinate counter 10.

The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1. In the second configuration example, the output coordinate QXY1 is the coordinate (x, y) corresponding to the coordinate on the image IMG2. The following equation (3) shows the coordinate transformation equation when the polynomial is quadratic. The lower equation (3) corresponds to the inverse coordinate transformation of the upper equation (1).

The coordinate conversion circuit 20 performs the arithmetic processing of the above equation (3) based on the coefficient information CF1. In the second configuration example, the coefficient information CF1 is the information of the coefficients c1 to c6 and d1 to d6 of the above equation (3). The configuration of the coordinate conversion circuit 20 is the same as that of the first configuration example described with reference to FIG. However, in the second configuration example, (x', y') is input to the first calculation circuit 21 and the second calculation circuit 22, and the coefficients c1 to c6 are input to the second calculation circuit 22, and the output coordinates are output. X is output from the second arithmetic circuit 22.

The mapping processing circuit 30 moves the pixel value of the coordinates (x', y') in the image IMG1 to the coordinates (x, y) in the image IMG2. (X', y') is the output coordinate QXY1, and (x, y) is the input coordinate IXY1. The mapping processing circuit 30 includes a memory control circuit 31, a coordinate counter 32, and an image memory 33.

The coordinate counter 32 outputs (xb, yb) as the second count coordinate CXYB1. In the second configuration example, since (xb, yb) specifies the coordinates on the image IMG2, the pixel clock used when the mapping processing circuit 30 processes the image IMG2 is input to the coordinate counter 32.

The memory control circuit 31 writes the image IMG1 to the image memory 33 based on the output coordinates QXY1 from the coordinate conversion circuit 20, reads the pixel value from the image memory 33 based on the second count coordinate CXYB1, and reads the pixel value from the image IMG2. It is output as the pixel value of.

Specifically, the write address controller 34 decodes (x, y), which is the output coordinate QXY1, into the write address ADWR, and sets the pixel value at (x', y') of the image IMG1 to the address ADWR of the image memory 33. Write to. This corresponds to moving the pixel value at (x', y') of the image IMG1 to the coordinates (x, y). That is, the image IMG1 written in the image memory 33 is an image deformed by the distortion correction.

The read address controller 35 decodes the second count coordinate CXYB1 (xb, yb) into the read address ADRD, and reads the pixel value from the address ADRD of the image memory 33. The read address controller 35 outputs the read pixel value as the pixel value at (x, y) of the image IMG2. By repeating this, the image IMG2 is output from the mapping processing circuit 30.

In the above mapping process, the pixel value at (x', y') of the image IMG1 is moved to the pixel value at (x, y) of the image IMG2. That is, the mapping processing circuit 30 operates as the forward warp engine described with reference to FIG.

3. 3. Third Configuration Example In the third configuration example, the circuit device 100 performs rotation correction in addition to distortion correction of the displayed image. First, the relationship between the mounting tolerance of the head-up display and the rotation correction will be described with reference to FIG. 7. In FIG. 7, the case where the head-up display is installed on the dashboard of an automobile will be described as an example, but the installation location of the head-up display is not limited to this. Further, although the display unit DSP of the head-up display is shown in a plane in FIG. 7, it is actually a curved display.

The direction DZ shown in FIG. 7 is a direction orthogonal to the display unit DSP of the head-up display. Specifically, the direction DZ and the display unit DSP are orthogonal to each other at any position of the display unit DSP which is a curved surface. For example, the direction DZ and the display unit DSP are orthogonal to each other at the center of the display unit DSP. The direction DX is a direction orthogonal to the direction DZ, and the direction DY is a direction orthogonal to the direction DX and the direction DZ. The direction DX corresponds to the horizontal direction. That is, the direction DX is parallel to the horizontal plane when the vehicle is in a horizontal position. When the heads-up display is installed vertically on the dashboard of an automobile, the direction DY corresponds to the vertical direction and the direction DY is perpendicular to the horizontal plane. However, the direction DY is not limited to the vertical direction, and the direction DY may be inclined with respect to the horizontal plane. When the heads-up display is installed on the dashboard at an angle θX in the depth direction, the direction DY is tilted by an angle θX with respect to the horizontal plane.

The rotation RZ indicates the rotation of the display unit DSP with the axis parallel to the direction DZ as the rotation axis. The rotation angle of the rotation RZ varies depending on the tolerance when the head-up display is attached to the dashboard. The rotation angle when the tolerance is zero is 0 degrees, the clockwise rotation is positive, and the counterclockwise rotation is negative. When the rotation RZ of the display unit DSP is a positive rotation angle, the display image is rotated clockwise when viewed from the user. The circuit device 100 rotates the image in the negative direction, that is, counterclockwise. As a result, even if the display unit DSP is tilted due to the tolerance, it is possible to display an image that is not tilted when viewed from the user.

FIG. 8 is a third configuration example of the circuit device 100. FIG. 9 is a diagram illustrating the operation of the circuit device 100 in the third configuration example.

In the third configuration example, the case where the circuit device 100 is an inverse warp engine will be described. As shown in FIG. 9, the circuit device 100 rotationally transforms the coordinates (xa, ya) in the image IMG2 which is the output image, and the coordinates (x, y) after the rotational transformation are the coordinates in the image IMG1 which is the input image. Image rotation and inverse warp are realized by converting to (x', y') and performing mapping processing based on the coordinates.

As shown in FIG. 8, the circuit device 100 includes a coordinate counter 10, a coordinate conversion circuit 20, a mapping processing circuit 30, and a rotation conversion circuit 40. The same configuration and operation as in the first configuration example will be omitted as appropriate.

The coordinate counter 10 outputs (xa, ya) as the first count coordinate CXYA1. Since (xa, ya) correspond to the coordinates on the image IMG2, the pixel clock used by the mapping processing circuit 30 to process the image IMG2 is input to the coordinate counter 10.

The rotation conversion circuit 40 performs rotation conversion on the first count coordinate CXYA1 and outputs the coordinate after the rotation conversion to the coordinate conversion circuit 20 as the input coordinate IXY1. In the third configuration example, the input coordinate IXY1 is (x, y). As shown in the following equation (4), the rotation conversion circuit 40 performs coordinate rotation using the affine transformation. θ is the rotation angle.

The rotation conversion circuit 40 converts the first count coordinate CXYA1 into the input coordinate IXY1 according to the above equation (4) based on the angle information RT1 indicating the rotation angle θ. For example, the circuit device 100 includes a storage unit that stores the angle information RT1 and the coefficient information CF1, the angle information RT1 is input from the storage unit to the rotation conversion circuit 40, and the coefficient information CF1 is input from the storage unit to the coordinate conversion circuit 20. Will be done. The storage unit is a register or a memory. The memory is a semiconductor memory such as RAM or non-volatile memory. The angle information RT1 and the coefficient information CF1 are written in the storage unit, for example, at the time of starting the circuit device 100 or manufacturing the head-up display.

The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1 based on the coefficient information CF1. In the third configuration example, the output coordinates QXY1 are the coordinates (x', y') corresponding to the coordinates on the image IMG1. The coordinate conversion formula is the same as that of the first configuration example.

The mapping processing circuit 30 obtains the pixel value of the coordinates (xa, ya) in the image IMG2 from the pixel value of the coordinates (x', y') in the image IMG1. (Xa, ya) is the first count coordinate CXYA1, and (x', y') is the output coordinate QXY1. The configuration and operation of the mapping processing circuit 30 are the same as those in the first configuration example.

In the mapping process of the third configuration example, the pixel value at (xa, ya) of the image IMG2 is acquired from the pixel value at (x', y') of the image IMG1. That is, the mapping processing circuit 30 operates as the inverse warp engine described with reference to FIG. Further, when the rotation conversion circuit 40 performs coordinate rotation, inverse warp including image rotation is performed in the mapping process.

4. Fourth Configuration Example FIG. 10 is a fourth configuration example of the circuit device 100. FIG. 11 is a diagram illustrating the operation of the circuit device 100 in the fourth configuration example.

In the fourth configuration example, the case where the circuit device 100 is a forward warp engine will be described. As shown in FIG. 11, the circuit device 100 rotationally transforms the coordinates (xa, ya) in the image IMG1 which is an input image, and converts the coordinates (x', y') after the rotational transformation into the image IMG2 which is an output image. Image rotation and forward warp are realized by converting to the coordinates (x, y) in the above and performing mapping processing based on the coordinates.

As shown in FIG. 10, the circuit device 100 includes a coordinate counter 10, a coordinate conversion circuit 20, a mapping processing circuit 30, and a rotation conversion circuit 40. The same configurations and operations as those of the second configuration example and the third configuration example will be omitted as appropriate.

The coordinate counter 10 outputs (xa, ya) as the first count coordinate CXYA1. In the fourth configuration example, since (xa, ya) corresponds to the coordinates on the image IMG1, the pixel clock input to the mapping processing circuit 30 together with the image IMG1 is input to the coordinate counter 10.

The rotation conversion circuit 40 performs rotation conversion on the first count coordinate CXYA1 and outputs the coordinate after the rotation conversion to the coordinate conversion circuit 20 as the input coordinate IXY1. In the fourth configuration example, the input coordinate IXY1 is (x', y'). As shown in the following equation (5), the rotation conversion circuit 40 performs coordinate rotation using the affine transformation. θ is the rotation angle.

The rotation conversion circuit 40 converts the first count coordinate CXYA1 into the input coordinate IXY1 according to the above equation (5) based on the angle information RT1 indicating the rotation angle θ.

The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1 based on the coefficient information CF1. In the fourth configuration example, the output coordinate QXY1 is the coordinate (x, y) corresponding to the coordinate on the image IMG2. The coordinate conversion formula is the same as that of the second configuration example.

The mapping processing circuit 30 moves the pixel value of the coordinates (xa, ya) in the image IMG1 to the coordinates (x, y) in the image IMG2. (Xa, ya) is the first count coordinate CXYA1, and (x, y) is the output coordinate QXY1. The configuration and operation of the mapping processing circuit 30 are the same as those of the second configuration example.

In the mapping process of the fourth configuration example, the pixel value in (x, ya) of the image IMG1 is moved to the pixel value of (x, y) in the image IMG2. That is, the mapping processing circuit 30 operates as the forward warp engine described with reference to FIG. Further, when the rotation conversion circuit 40 performs coordinate rotation, forward warp including image rotation is performed in the mapping process.

5. Fifth Configuration Example FIG. 12 is a fifth configuration example of the circuit device 100. In the fifth configuration example, the circuit device 100 generates the image IMG3 by performing the reverse mapping process of the image IMG2, and detects the error of the image IMG2 by comparing the image IMG1 and the image IMG3.

The circuit device 100 includes an interface 110, a storage unit 133, an image processing circuit 135, an interface 140, a comparison circuit 145, an error detection circuit 150, a storage unit 160, a register circuit 170, and an interface 190. Including.

The interface 110 receives image data transmitted from, for example, a processing device 200 or the like to the circuit device 100. The interface 110 converts the received image data into a format used inside the circuit device 100, and outputs the converted image data as the image IMA1. For example, the interface 110 is Open LDI (Open LVDS Display Interface), and converts a serial signal received by LVDS (Low Voltage Differential Signaling) into an RGB parallel signal. The processing device 200 is, for example, a SoC (System on a Chip), an MCU (Micro Control Unit), or a CPU (Central Processing Unit).

The image processing circuit 135 includes a first processing circuit 131 that maps an image according to the surface shape of the curved display of the head-up display, and a second processing circuit 132 that performs reverse mapping thereof. The first processing circuit 131 is also referred to as a first warp engine, and the second processing circuit 132 is also referred to as a second warp engine. Hereinafter, an example in which the first processing circuit 131 and the second processing circuit 132 perform image rotation together with the distortion correction will be described, but the first processing circuit 131 and the second processing circuit 132 may perform only the distortion correction.

The curved display is a screen or a display panel in a head-up display. The screen is also called a projected object. When the curved display is a screen, the heads-up display includes a projection device that projects the image IMA2 onto the screen. The projection device includes, for example, a liquid crystal display panel, a display driver for driving the liquid crystal display panel, a light source, and a lens. The display driver displays an image on the liquid crystal display panel based on the received image data, the light source outputs light to the liquid crystal display panel, and the light passing through the liquid crystal display panel is projected onto the screen by the lens. The screen is a transparent object and has a reflective surface that reflects the projected light. For example, in an in-vehicle head-up display, the screen is a transparent screen mounted on the dashboard or the windshield of an automobile. When the curved display is a display panel, the head-up display is configured so that the display image of the display panel is directly visible to the user, and the head-up display displays the image IMA2 on the display panel. The display panel is, for example, a transparent display using an organic EL panel, and the transparent display has a curved surface.

The first processing circuit 131 performs the first mapping processing using the coefficient information CF1 and the first rotation processing using the angle information RT1 on the image IMA1, and outputs the processed image IMA2. Image IMA1 is the first image and image IMA2 is the second image. In addition, the first processing circuit 131 extracts the image IMA1'of the region of interest from the image IMA1. The area of interest is also called ROI (Region Of Interest). The image IMA1'may be the entire image IMA1. In the following, a case where the image IMA1'is an image of the region of interest extracted from the image IMA1 will be described as an example, and the image IMA'of the region of interest will also be referred to as a first image.

The second processing circuit 132 performs the second mapping processing using the coefficient information CF2 and the second rotation processing using the angle information RT2 on the image IMA2, and outputs the processed image IMA3. Image IMA3 is the third image. Specifically, the second processing circuit 132 extracts an image of the region of interest from the image IMA2, and performs a second mapping process and a second rotation process on the image. The second rotation process is a reverse rotation process of the first rotation process. The image IMA3 is an image in which the image of the region of interest extracted from the image IMA2 is reverse-mapped and reverse-rotated.

The interface 140 outputs the image IMA2 to the outside of the circuit device 100. The outside of the circuit device 100 is, for example, a display driver that drives a display panel of a head-up display. For example, the interface 140 is an LVDS interface and converts RGB parallel signals from the image processing circuit 135 into LVDS serial signals.

The storage unit 133 is the first storage unit. The first processing circuit 131 stores the image IMA1'in the region of interest in the storage unit 133. The storage unit 160 is a memory. For example, the memory is a semiconductor memory such as RAM or non-volatile memory. The storage unit 133 and the storage unit 160 may be configured by individual memories or may be configured by one memory.

The comparison circuit 145 performs comparison processing between the image IMA1'stored in the storage unit 133 and the image IMA3, and outputs the comparison result. This comparison result is used to detect an error in image IMA2. That is, it is used to verify whether or not the first mapping process and the first rotation process performed by the first processing circuit 131 are normal. The comparison circuit 145 obtains an index indicating the degree of similarity between the image IMA1 and the image IMA3. The index is a shape index or a visibility index described later. Alternatively, the comparison circuit 145 may be obtained using SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), NCC (Normalized Cross Correlation), or the like as an index.

The error detection circuit 150 performs error detection of the second image IMA2 by comparing the index and the threshold value. The threshold value is a threshold value indicating how similar the image IMA1'and the image IMA3 should be tolerable.

When an error is detected by the error detection circuit 150, the image processing circuit 135 stops the output of the image IMA2 to the interface 140. Alternatively, the interface 140 stops the output of the image IMA2 when an error is detected by the error detection circuit 150. The interface 140 may output the image IMA2 together with the error information, and the display driver receiving the error information may perform an operation based on the error information. Alternatively, the interface 190 may output error information to the processing device 200, and the processing device 200 that has received the error information may perform an operation based on the error information. The error information is, for example, an error determination flag, an index, or the like. The operation based on the error information is, for example, stopping the display of the head-up display.

The interface 190 performs inter-circuit communication between the circuit device 100 and the processing device 200. For example, the interface 190 is a serial communication interface such as an SPI (Serial Peripheral Interface) system or an I2C system. The setting information and control information from the processing device 200 are written to, for example, the register circuit 170, and the circuit device 100 operates according to the setting information and control information.

The register circuit 170 is configured to be accessible from the processing apparatus 200 via the interface 190. The register circuit 170 includes an error detection result register 176 and a threshold register 178.

The error detection result register 176 stores the error detection result output by the error detection circuit 150. The error detection result is, for example, an error determination flag indicating whether or not the display image is determined to be an error. The processing device 200 can determine whether or not an error has occurred by reading the error detection result from the error detection result register 176 via the interface 190.

A threshold value is set in the threshold value register 178 from the processing device 200 via the interface 190. The error detection circuit 150 compares the index with the threshold value set in the threshold value register 178 to perform error detection.

The storage unit 160 is a second storage unit. The storage unit 160 stores the coefficient information CF1 and CF2 and the angle information RT1 and RT2. Specifically, the processing device 200 transmits CF1, CF2, RT1 and RT2 to the interface 190, and the storage unit 160 stores the CF1, CF2, RT1 and RT2 received by the interface 190. The image processing circuit 135 performs mapping processing and rotation processing based on CF1, CF2, RT1 and RT2 read from the storage unit 160. The storage unit 160 is, for example, a memory or a register. For example, the memory is a semiconductor memory such as RAM or non-volatile memory.

The image processing circuit 135, the comparison circuit 145, and the error detection circuit 150 are logic circuits. The image processing circuit 135, the comparison circuit 145, and the error detection circuit 150 may be configured as individual circuits, or may be configured as an integrated circuit by automatic placement and routing or the like. Further, a part or all of these logic circuits may be realized by a processor such as a DSP (Digital Signal Processor). In this case, a program or instruction set in which the function of each circuit is described is stored in the memory, and the processor executes the program or instruction set to realize the function of each circuit.

In FIG. 12, the circuit device 100 includes the error detection circuit 150 and the error detection result register 176, but the circuit device 100 may not include the error detection circuit 150 and the error detection result register 176. In this case, the interface 190 may output the index obtained by the comparison circuit 145 to the processing device 200, and the processing device 200 that has received the index may detect an error by comparing the index with the threshold value. When the processing device 200 detects an error, the processing device 200 may perform an error handling operation such as stopping the display of the head-up display.

FIG. 13 is a detailed configuration example of the first processing circuit 131 and the second processing circuit 132.

The first processing circuit 131 is a coordinate counter 10 which is a first coordinate counter, a rotation conversion circuit 40 which is a first rotation conversion circuit, a coordinate conversion circuit 20 which is a first coordinate conversion circuit, and a first mapping processing circuit. Includes a mapping processing circuit 30. These operations are as described in the first to fourth configuration examples. When the first processing circuit 131 only performs distortion correction, the rotation conversion circuit 40 may be omitted.

The second processing circuit 132 is a coordinate counter 50 which is a second coordinate counter, a rotation conversion circuit 80 which is a second rotation conversion circuit, a coordinate conversion circuit 60 which is a second coordinate conversion circuit, and a second mapping processing circuit. Includes a mapping processing circuit 70. The coordinate counter 50 outputs the count coordinate CXYA2. The rotation conversion circuit 80 rotationally transforms the count coordinate CXYA2 into the input coordinate IXY2 based on the angle information RT2. The coordinate conversion circuit 60 converts the input coordinates IXY2 into the output coordinates QXY2 by arithmetic processing using a polynomial of degree 2 or higher based on the coefficient information CF2. The mapping processing circuit 70 maps the image IMA2 to the image IMA3 based on the output coordinates QXY2. The detailed configuration and operation of these circuits are the same as those of the coordinate counter 10, the rotation conversion circuit 40, the coordinate conversion circuit 20, and the mapping processing circuit 30. When the second processing circuit 132 only performs distortion correction, the rotation conversion circuit 80 may be omitted.

The configuration of the second processing circuit 132 is not limited to FIG. For example, a map table describing the correspondence of coordinates in the second mapping process may be input to the second processing circuit 132. The second processing circuit 132 may perform the second mapping processing and the second rotation processing based on the map table and the angle information RT2.

The image comparison performed by the comparison circuit 145 will be described.

FIG. 14 is an example of the image IMG2 displayed on the head-up display. In FIG. 14, the icon ICA is superimposed on the meter image DIM. The icon ICA is blended into the meter image DIM with a certain transmittance. In the present embodiment, the circuit device 100 verifies whether or not the icon ICA is properly displayed. In this case, as shown by the dotted rectangle in FIG. 14, the area including the icon ICA is set to ROI.

FIG. 15 is an image IMG1'of ROI extracted from the image IMG1 before the mapping process. The ROI includes an icon ICA and a meter image DIM which is a background image of the icon ICA. FIG. 16 is an image IMG3 in which the ROI image IMG2'extracted from the image IMG2 is reverse-mapped. FIG. 16 shows an example in which the icon ICA was not displayed correctly. In this case, the image IMG1'and the image IMG3 match only the background image, and the icon ICA portion is different. Therefore, the similarity indicated by the index that is the comparison result is low. If the icon ICA is correctly displayed also in the image IMG3, the similarity is high. The index indicating the degree of similarity can take a continuous or gradual value. The error detection circuit 150 compares the index with the threshold value, and by adjusting the threshold value, it is possible to adjust how much similarity is allowed.

The comparison circuit 145 obtains a shape index, a visibility index, or both as an index indicating the degree of similarity between the image IMG1'and the image IMG3. As described above, the comparison circuit 145 may be obtained using SSD, SAD, NCC, or the like as an index. Whether the index becomes larger as the degree of similarity is higher or the index becomes smaller as the degree of similarity is higher depends on the calculation method of the index. The threshold value is set for each of the shape index and the visibility index.

First, the first calculation method of the shape index will be described. The comparison circuit 145 obtains the distance between the images of the image IMG1'and the image IMG3 in the color space. The color space is, for example, RGB or YCrCb. Specifically, the comparison circuit 145 obtains the square value of the distance between the pixels of the image IMG1'and the pixels of the image IMG3 corresponding to the pixels in the color space. The comparison circuit 145 integrates the squared values in the image, and uses the integrated value as the inter-image distance. In the first calculation method, the distance between images corresponds to the shape index.

Next, the second calculation method of the shape index will be described. As shown in FIG. 17, the comparison circuit 145 obtains the edge image EIMG1'by extracting the edge of the image IMG1'. Further, the comparison circuit 145 obtains an edge image of the image IMG3 by extracting the edge of the image IMG3. Hereinafter, the edge image of the image IMG3 will be referred to as EIMG3. The comparison circuit 145 compares the edge image EIMG1'and the edge image EIMG3. Specifically, the comparison circuit 145 extracts an edge from the image IMA1'and the image IMG3 using a sobel filter or the like, and obtains a correlation value between the edge image EIMG1'and the edge image EIMG3. In the second calculation method, the correlation value of the edge image corresponds to the shape index.

Next, the calculation method of the visibility index will be described. Here, the color space is YCrCb, but the color space may be RGB or the like. The comparison circuit 145 obtains a histogram from the Y channel of the image IMG1'. Similarly, the comparison circuit 145 obtains a histogram from the Cr channel and Cb channel of the image IMG1'and obtains a histogram from the Y channel, Cr channel and Cb channel of the image IMG3.

The comparison circuit 145 performs cross-correlation calculation on the histograms of the image IMG1'and the image IMG3 in the Y channel. The cross-correlation operation is an operation in which two histograms are shifted by a delay (lag) to obtain a correlation value, and the correlation value is obtained while changing the delay. If there is a place where the correlation value of the two histograms becomes high by changing the delay, the delay will have a peak. There can be multiple peaks. Similarly, the comparison circuit 145 performs cross-correlation calculation on the histograms of the image IMG1'and the image IMG3 in the Cr channel and the Cb channel.

The comparison circuit 145 examines the delay value at which the peak is set in the cross-correlation signals of all channels, and obtains the maximum delay value among the delay values. This maximum delay value corresponds to the visibility index. When the color contrast between the icon and the background image is high, the maximum delay value becomes large, so that the visibility index indicates the color contrast between the icon and the background image. It is considered that the higher the color contrast is, the higher the visibility is. Therefore, it is judged that the larger the visibility index is, the higher the similarity is.

By using the shape index, the visibility index, or both described above, the similarity between the ROI image IMG1'and the IMG3 can be determined. When the shape index is used, the similarity is highest when the image IMG1'and the image IMG3 match. That is, the degree of similarity can be rephrased as the degree of agreement. For example, when the icon ICA is rotated and displayed, the similarity is lower than when the icon ICA is not rotated. On the other hand, since the visibility index indicates the color contrast, the similarity does not change much even when the icon ICA is rotated. Therefore, when the case where the icon ICA is rotated is allowed, the visibility index may be used. Further, since the calculation method is different between the shape index and the visibility index, the accuracy of error detection can be improved by using both of them.

6. Electronic device, mobile FIG. 18 is a configuration example of an electronic device including the circuit device of the present embodiment. The electronic device 300 includes a processing device 310, a circuit device 320, a storage device 350, an operation device 360, a communication device 370, and a head-up display 400. The circuit device 320 corresponds to the circuit device 100 of the first to fifth configuration examples. The head-up display 400 includes a display driver 330 and a display panel 340. The processing device 310 is, for example, an MCU or the like. In the configuration example of FIG. 18, the circuit device 320 corresponds to the display controller. However, the circuit device 100 of this embodiment is applicable not only to the display controller but also to any circuit device that generates a display image for a head-up display.

The processing device 310 transfers the image data stored in the storage device 350 or the image data received by the communication device 370 to the circuit device 320. The circuit device 320 performs image processing on the image data, display timing control, generation of image data to be transferred to the display driver, and the like. Further, the circuit device 320 may perform error detection of image data as described in the fifth configuration example. The display driver 330 drives the display panel 340 to display an image based on the image data transferred from the circuit device 320 and the display timing control by the circuit device 320. The display panel 340 is, for example, a liquid crystal display panel, an EL display panel, or the like. The storage device 350 is, for example, a memory, a hard disk drive, an optical disk drive, or the like. The operation device 360 is a device for a user to operate the electronic device 300, and is, for example, a button, a touch panel, a keyboard, or the like. The communication device 370 is, for example, a device that performs wired communication or a device that performs wireless communication. Wired communication is, for example, LAN, USB, or the like. Wireless communication is, for example, wireless LAN, wireless proximity communication, or the like.

As the electronic device including the circuit device of the present embodiment, various devices such as an in-vehicle electronic device, a display terminal such as a factory facility, a display device mounted on a robot, and an information processing device can be assumed. The in-vehicle electronic device is, for example, a meter panel or the like. The information processing device is, for example, a PC or the like.

FIG. 19 is an example of a mobile body including the circuit device 320 of the present embodiment. The mobile body includes the circuit device 320 of the present embodiment and the processing device 310 that transmits image data to the circuit device 320. The processing device 310 may perform error handling processing based on the error detection result of the display image for the head-up display from the circuit device 320. The moving body includes a head-up display 400 and a control device 208. The control device 208 is an ECU (Electronic Control Unit), and a circuit device 320 and a processing device 310 are incorporated in the ECU. The circuit device 320 may be incorporated in the head-up display 400. The circuit device 320 of the present embodiment can be incorporated into various moving bodies such as cars, airplanes, motorcycles, bicycles, and ships. The moving body is, for example, a device or device provided with a drive mechanism such as an engine or a motor, a steering mechanism such as a steering wheel or a rudder, and various electronic devices, and moves on the ground, in the sky, or on the sea. FIG. 19 schematically shows an automobile 206 as a specific example of a moving body. The head-up display 400 has a transparent screen, which is installed between the driver's seat and the windshield. Alternatively, the head-up display may use the windshield as a transparent screen and project an image on the windshield. The head-up display 400 functions as, for example, a meter panel of an automobile 206.

The circuit device of the present embodiment described above includes a coordinate conversion circuit and a mapping processing circuit. The coordinate conversion circuit performs coordinate conversion from input coordinates to output coordinates. The mapping processing circuit performs mapping processing based on the output coordinates on the input first image to generate a second image to be displayed on the display panel for displaying the image on the curved display. The coordinate conversion circuit performs coordinate conversion by performing arithmetic processing using a polynomial of degree 2 or higher representing the coordinate conversion.

A polynomial consists of a combination of multiplication and addition, and does not include complicated calculations such as integration. Therefore, by performing coordinate transformation using a polynomial, the calculation time can be shortened as compared with the case of performing calculation including integration and the like. Further, in the polynomial, the input coordinates and the output coordinates have a one-to-one correspondence. By using this correspondence, the mapping processing circuit maps one pixel at a time, so that an image for one screen can be configured. Such sequential processing is more suitable for real-time processing than the case where a table or the like is once created and mapping processing is performed.

Further, in the present embodiment, the coordinate conversion circuit may input the coefficient information of the polynomial and the input coordinates, and obtain the output coordinates from the input coordinates by arithmetic processing based on the coefficient information.

In coordinate transformation using a polynomial, the correspondence between input coordinates and output coordinates is determined by the coefficients of the polynomial. That is, by inputting the coefficient information according to the shape of the curved display into the coordinate conversion circuit, distortion correction suitable for various curved displays is realized. Further, since the coordinate conversion circuit obtains the output coordinates from the input input coordinates, it is possible to perform coordinate conversion for each pixel suitable for real-time processing as described above.

Further, in the present embodiment, the coordinate conversion circuit may include the first to nth term arithmetic circuits and an addition circuit. The i-term arithmetic circuit may calculate the i-term of the first to nth terms of the polynomial based on the coefficient information. n is an integer of 2 or more, and i is an integer of 1 or more and n or less. The adder circuit may add the first to nth terms output by the first to nth term arithmetic circuits and output the output coordinates.

In this way, the term operation circuits corresponding to each term of the polynomial are individually provided, so that the operations of each term are processed in parallel. This makes it possible to process the coordinate transformation at high speed.

Further, in the present embodiment, the coordinate conversion circuit may include a first calculation circuit and a second calculation circuit. The first operation circuit, when the input coordinates ^{(x, y), x 2} , y 2, and may be obtained xy. The second arithmetic circuit may obtain the output coordinates based on the arithmetic result of the first arithmetic circuit and the coefficient information.

In this way, the coefficient first arithmetic circuit, terms ^x 2, ^{y 2} before multiplied by a coefficient, and calculates the xy, second operation circuit, for the ^x 2, ^{y 2,} and xy By multiplying by, each term of the polynomial is calculated, and the output coordinates can be output by adding the obtained terms. In this way, by using a polynomial, the operation of coordinate transformation can be constructed by multiplication and addition.

Further, in the present embodiment, the polynomial may include a1 × x ² , a2 × y ² , a3 × xy, a4 × x, a5 × y, and a6 as the first to sixth terms. The coefficient information may include information on a1, a2, a3, a4, a5, and a6 as the first to sixth coefficients. The second arithmetic circuit includes a first term calculation circuit for multiplying the a1 and ^{x 2,} and the second term calculation circuit for multiplying the a2 and ^{y 2,} and the third term calculation circuit for multiplying the a3 and xy, a4 and x A fourth binary operation circuit for multiplying, a fifth term operation circuit for multiplying a5 and y, and an addition circuit may be included. The adder circuit may add the first to fifth terms and the sixth term a6 output by the first to fifth arithmetic circuits and output the output coordinates.

In this way, term arithmetic circuits corresponding to each term are individually provided for the quadratic term and the linear term of the polynomial. As a result, the operations of each term are processed in parallel for the quadratic term and the linear term of the polynomial, so that the coordinate transformation can be processed at high speed.

Further, in the present embodiment, the circuit device may include a first coordinate counter. The first coordinate counter may output the first count coordinates, which are the coordinates of each pixel based on the pixel clock. The coordinate conversion circuit may perform coordinate conversion using the first count coordinates as input coordinates.

According to the present embodiment, the first coordinate counter outputs the first count coordinates one pixel at a time based on the pixel clock, and the coordinate conversion circuit converts the first count coordinates into the output coordinates. As a result, the output coordinates are output one pixel at a time according to the pixel clock, so that the mapping processing circuit can perform the mapping process one pixel at a time based on the pixel clock. As a result, distortion correction suitable for real-time processing is realized.

Further, in the present embodiment, the input coordinates may be the coordinates corresponding to the coordinates on one of the first image and the second image. The output coordinates may be coordinates corresponding to the coordinates on the other image of the first image and the second image.

When the mapping processing circuit performs inverse warp, the input coordinates correspond to the coordinates on the second image, and the output coordinates correspond to the coordinates on the first image. When the mapping processing circuit performs forward warping, the input coordinates correspond to the coordinates on the first image, and the output coordinates correspond to the coordinates on the second image. In the arithmetic processing using the polynomial, both the inverse warp and the forward warp can be realized by using the coefficient corresponding to the inverse warp or the coefficient corresponding to the forward warp.

Further, in the present embodiment, the mapping processing circuit controls access to the second coordinate counter that outputs the second count coordinates, which are the coordinates of each pixel based on the pixel clock, the image memory that stores the first image, and the image memory. It may include a memory control circuit to perform. The memory control circuit writes the first image to the image memory based on the second count coordinates, and outputs the pixel value read from the image memory based on the output coordinates as the pixel value at the first count coordinates of the second image. Then, the mapping process may be performed. Alternatively, the memory control circuit writes the pixel value at the first count coordinate of the first image to the image memory as the pixel value at the output coordinate, and reads the pixel value read from the image memory based on the second count coordinate of the second image. Mapping processing may be performed by outputting as a pixel value at the second count coordinate.

In the former mapping process, when the second image is read from the image memory, the pixel value at the input coordinate of the second image is acquired from the pixel value at the output coordinate of the first image. As a result, inverse warp is realized. In the latter mapping process, when the first image is written to the image memory, the pixel value in the input coordinates of the first image is moved to the pixel value in the output coordinates. As a result, forward warp is realized.

Further, in the present embodiment, the circuit device may include a rotation conversion circuit. The rotation conversion circuit may perform rotation conversion of the coordinates and output the coordinates after rotation. The coordinate conversion circuit may perform coordinate conversion using the coordinates after rotation as input coordinates.

In this way, the input coordinates, which are the coordinates after rotation, are converted into the output coordinates, and the mapping process is performed based on the output coordinates. As a result, it is possible to realize image rotation corresponding to the mounting error of the curved display and image distortion correction corresponding to the distortion of the image by the curved display.

Further, in the present embodiment, the circuit device may include a first coordinate counter. The first coordinate counter may output the first count coordinates, which are the coordinates of each pixel based on the pixel clock. The rotation conversion circuit may obtain the coordinates after rotation by performing rotation conversion on the first count coordinates.

According to the present embodiment, the first coordinate counter outputs the first count coordinates one pixel at a time based on the pixel clock, the rotation conversion circuit converts the first count coordinates into the post-rotation coordinates, and the post-rotation coordinates are converted. The coordinate conversion circuit converts to the output coordinates. As a result, the output coordinates are output one pixel at a time according to the pixel clock, so that the mapping processing circuit can perform the mapping process one pixel at a time based on the pixel clock. As a result, rotation correction and distortion correction suitable for real-time processing are realized.

Further, in the present embodiment, the first count coordinates may be coordinates that specify the coordinates on one of the first image and the second image. The output coordinates may be coordinates that specify the coordinates on the other image of the first image and the second image.

When the mapping processing circuit performs inverse warp, the first count coordinates correspond to the coordinates on the second image, and the output coordinates correspond to the coordinates on the first image. When the mapping processing circuit performs forward warping, the first count coordinates correspond to the coordinates on the first image, and the output coordinates correspond to the coordinates on the second image. In the arithmetic processing using the polynomial, both the inverse warp and the forward warp can be realized by using the coefficient corresponding to the inverse warp or the coefficient corresponding to the forward warp. Further, since the rotation conversion circuit rotationally transforms the first count coordinates, distortion correction and image rotation are performed at the same time when the first image is mapped to the second image.

Further, in the present embodiment, the mapping processing circuit controls access to the second coordinate counter that outputs the second count coordinates, which are the coordinates of each pixel based on the pixel clock, the image memory that stores the first image, and the image memory. It may include a memory control circuit to perform. The memory control circuit writes the first image to the image memory based on the second count coordinates, and outputs the pixel value read from the image memory based on the output coordinates as the pixel value at the first count coordinates of the second image. Then, the mapping process may be performed. Alternatively, the memory control circuit writes the pixel value at the first count coordinate of the first image to the image memory as the pixel value at the output coordinate, and reads the pixel value read from the image memory based on the second count coordinate of the second image. Mapping processing may be performed by outputting as a pixel value at the second count coordinate.

In the former mapping process, when the second image is read from the image memory, the pixel value at the first count coordinate of the second image is acquired from the pixel value at the output coordinate of the first image. As a result, inverse warp including image rotation and distortion correction is realized. In the latter mapping process, when the first image is written to the image memory, the pixel value at the first count coordinate of the first image is moved to the pixel value at the output coordinate. As a result, a forward warp including image rotation and distortion correction is realized.

Further, in the present embodiment, the polynomial may be a1 × x ² + a2 × y ² + a3 × xy + a4 × x + a5 × y + a6.

In this way, the coordinate transformation is performed by the arithmetic processing using the quadratic polynomial. Each term of a quadratic polynomial contains up to two multiplications. Therefore, the coordinate transformation using the quadratic polynomial is suitable for real-time processing because the calculation load is smaller than that when the higher-order polynomial is used.

Further, in the present embodiment, the circuit device may include a second mapping processing circuit and a comparison circuit. The second mapping processing circuit may generate a third image by performing the second mapping processing which is the inverse mapping processing of the mapping processing on the second image. The comparison circuit may compare the first image and the third image, and output the comparison result as information for performing error detection of the second image.

In this way, the circuit device can verify that the distortion-corrected image is suitable for display on the heads-up display. Alternatively, when the error detection information is output to the circuit device to the outside, the external device that receives the error detection information can carry out the above verification. Since the mapping processing circuit maps the first image to the second image and the second mapping processing circuit reverse-maps the second image to the third image, the third image is the first when the second image is normal. It should return to the same image as the image. The comparison circuit can output information for performing error detection of the second image by comparing the third image and the first image.

Further, the electronic device of the present embodiment includes the circuit device described in any of the above.

Further, the moving body of the present embodiment includes the circuit device described in any of the above.

Although the present embodiment has been described in detail as described above, those skilled in the art will easily understand that many modifications that do not substantially deviate from the novel matters and effects of the present disclosure are possible. Therefore, all such variations are included in the scope of the present disclosure. For example, a term described at least once in a specification or drawing with a different term in a broader or synonymous manner may be replaced by that different term anywhere in the specification or drawing. All combinations of the present embodiment and modifications are also included in the scope of the present disclosure. Further, the configuration and operation of the circuit device, the head-up display, the electronic device, and the mobile body are not limited to those described in the present embodiment, and various modifications can be performed.

10 ... coordinate counter, 20 ... coordinate conversion circuit, 21 ... first arithmetic circuit, 22 ... second arithmetic circuit, 30 ... mapping processing circuit, 31 ... memory control circuit, 32 ... coordinate counter, 33 ... image memory, 34 ... write Address controller, 35 ... read address controller, 40 ... rotation conversion circuit, 50 ... coordinate counter, 60 ... coordinate conversion circuit, 70 ... mapping processing circuit, 80 ... rotation conversion circuit, 100 ... circuit device, 110 ... interface, 131 ... 1 processing circuit, 132 ... 2nd processing circuit, 133 ... storage unit, 135 ... image processing circuit, 140 ... interface, 145 ... comparison circuit, 150 ... error detection circuit, 160 ... storage unit, 170 ... register circuit, 176 ... error Detection result register, 178 ... threshold register, 190 ... interface, 200 ... processing device, 206 ... automobile, 208 ... control device, 300 ... electronic device, 310 ... processing device, 320 ... circuit device, 330 ... display driver, 340 ... display Panel, 350 ... Storage device, 360 ... Operation device, 370 ... Communication device, 400 ... Head-up display, AC1 to AC5 ... First to fifth arithmetic circuits, ADDC ... Addition circuit, ADRD ... Read address, ADWR ... Write address , CF1 ... Coefficient information, CXYA1 ... 1st count coordinates, CXYB1 ... 2nd count coordinates, IMA1, IMG1 ... 1st image, IMA2, IMG2 ... 2nd image, IMA3 ... 3rd image, IXY1 ... Input coordinates, MC1 ... x ² arithmetic circuit, MC2 ... y ² arithmetic circuit, MC3 ... xy arithmetic circuit, QXY1 ... output coordinates, RT1 ... angle information, a1 to a6 ... coefficients

Claims (16)

A coordinate conversion circuit that converts coordinates from input coordinates to output coordinates,
A mapping processing circuit that generates a second image displayed on a display panel for displaying an image on a curved display by performing a mapping process based on the output coordinates on the input first image.
Including
The coordinate conversion circuit
A circuit device characterized in that the coordinate transformation is performed by performing arithmetic processing using a polynomial of degree 2 or higher representing the coordinate transformation. In the circuit device according to claim 1,
The coordinate conversion circuit
A circuit device characterized in that the coefficient information of the polynomial and the input coordinates are input, and the output coordinates are obtained from the input coordinates by the arithmetic processing based on the coefficient information. In the circuit device according to claim 2.
The coordinate conversion circuit
The i-term arithmetic circuit calculates the i-term of the 1st to nth terms of the polynomial based on the coefficient information (n is an integer of 2 or more, i is 1 or more n). The following integers) and
An addition circuit that adds the first to nth terms output by the first to nth arithmetic circuits and outputs the output coordinates.
A circuit device characterized by having. In the circuit device according to claim 2.
The coordinate conversion circuit
When said input coordinates ^{(x, y), x 2} , y 2, and a first arithmetic circuit for obtaining the xy,
A second arithmetic circuit that obtains the output coordinates based on the arithmetic result of the first arithmetic circuit and the coefficient information, and
A circuit device characterized by including. In the circuit device according to claim 4,
The polynomial includes a1 × x ² , a2 × y ² , a3 × xy, a4 × x, a5 × y, and a6 as the first to sixth terms.
The coefficient information includes information of a1, a2, a3, a4, a5, and a6 as the first to sixth coefficients.
The second arithmetic circuit is
A first term calculation circuit for multiplying the x ² and the a1,
A second term calculation circuit for multiplying the y ² and the a2,
A ternary operation circuit that multiplies the a3 and the xy,
A fourth binary operation circuit that multiplies the a4 and the x.
A fifth term arithmetic circuit that multiplies the a5 and the y.
An addition circuit that adds the first to fifth terms output by the first to fifth arithmetic circuits and a6, which is the sixth term, and outputs the output coordinates.
A circuit device characterized by including. In the circuit device according to any one of claims 1 to 5.
Includes a first coordinate counter that outputs first count coordinates, which are pixel-based coordinates based on the pixel clock.
The coordinate conversion circuit
A circuit device characterized in that the coordinate conversion is performed using the first count coordinates as the input coordinates. In the circuit device according to claim 6,
The input coordinates are coordinates corresponding to the coordinates on one of the first image and the second image.
A circuit device characterized in that the output coordinates are coordinates corresponding to the coordinates on the other image of the first image and the second image. In the circuit device according to claim 6,
The mapping processing circuit
A second coordinate counter that outputs second count coordinates, which are coordinates in pixel units based on the pixel clock, and
An image memory for storing the first image and
A memory control circuit that controls access to the image memory and
Including
The memory control circuit
The first image is written to the image memory based on the second count coordinates, and the pixel value read from the image memory based on the output coordinates is output as the pixel value at the first count coordinates of the second image. By doing so, the mapping process is performed, or
The pixel value at the first count coordinate of the first image is written to the image memory as the pixel value at the output coordinate, and the pixel value read from the image memory based on the second count coordinate is the pixel value of the second image. A circuit device characterized in that the mapping process is performed by outputting as a pixel value at the second count coordinates. In the circuit device according to any one of claims 1 to 5.
Includes a rotation conversion circuit that performs rotation conversion of coordinates and outputs coordinates after rotation.
The coordinate conversion circuit
A circuit device characterized in that the coordinate conversion is performed using the rotated coordinates as the input coordinates. In the circuit device according to claim 9,
Includes a first coordinate counter that outputs first count coordinates, which are pixel-based coordinates based on the pixel clock.
The rotation conversion circuit
A circuit device characterized in that the coordinates after rotation are obtained by performing the rotation conversion with respect to the first count coordinates. In the circuit device according to claim 10,
The first count coordinates are coordinates that specify the coordinates on one of the first image and the second image.
A circuit device characterized in that the output coordinates are coordinates that specify coordinates on the other image of the first image and the second image. In the circuit device according to claim 10,
The mapping processing circuit
A second coordinate counter that outputs second count coordinates, which are coordinates in pixel units based on the pixel clock, and
An image memory for storing the first image and
A memory control circuit that controls access to the image memory and
Including
The memory control circuit
The first image is written to the image memory based on the second count coordinates, and the pixel value read from the image memory based on the output coordinates is output as the pixel value at the first count coordinates of the second image. By doing so, the mapping process is performed, or
The pixel value at the first count coordinate of the first image is written to the image memory as the pixel value at the output coordinate, and the pixel value read from the image memory based on the second count coordinate is the pixel value of the second image. A circuit device characterized in that the mapping process is performed by outputting as a pixel value at the second count coordinates. In the circuit device according to any one of claims 1 to 12.
The circuit device characterized in that the polynomial is a1 × × ² + a2 × y ² + a3 × xy + a4 × x + a5 × y + a6. In the circuit device according to any one of claims 1 to 13.
A second mapping processing circuit that generates a third image by performing a second mapping process, which is a reverse mapping process of the mapping process, on the second image.
A comparison circuit that compares the first image with the third image and outputs the result of the comparison as information for performing error detection of the second image.
A circuit device characterized by including. An electronic device comprising the circuit device according to any one of claims 1 to 14. A mobile body including the circuit device according to any one of claims 1 to 14.