JP7467883B2 - Circuit device, electronic device and mobile device - Google Patents

Description

The present invention relates to circuit devices, electronic devices, mobile objects, etc.

Head-up displays (HUDs) are known that display information superimposed on the user's field of vision by displaying an image on a curved display such as a transparent screen. If the head-up display displays an input image as is, the displayed image will appear distorted due to distortion of the curved display. For this reason, the head-up display performs distortion correction on the input image and displays the image after distortion correction, thereby displaying an undistorted image. Patent Document 1 discloses a conventional technology for a head-up display that performs distortion correction. The head-up display in Patent Document 1 includes a memory that stores a formula for coordinate conversion, performs coordinate conversion using the formula for coordinate conversion, and performs a mapping process on the input image based on the result, thereby performing distortion correction.

JP 2010-164869 A

The above-mentioned Patent Document 1 discloses a formula for coordinate conversion in distortion correction, but the formula is theoretical and includes complex calculations such as integration or square root. For this reason, the technology in Patent Document 1 has an issue in that it is difficult to perform high-speed processing, such as performing real-time distortion correction on moving images. Patent Document 1 does not disclose a specific processing process for performing high-speed processing.

One aspect of the present disclosure includes a coordinate conversion circuit that performs coordinate conversion from input coordinates to output coordinates, and a mapping processing circuit that performs mapping processing on an input first image based on the output coordinates to generate a second image to be displayed on a display panel for displaying an image on a curved display, and the coordinate conversion circuit relates to a circuit device that performs the coordinate conversion by performing arithmetic processing using a polynomial of degree 2 or higher that represents the coordinate conversion.

1 shows a first configuration example of a circuit device. 5A to 5C are diagrams illustrating the operation of the circuit device in the first configuration example. 13 shows a detailed configuration example of a coordinate conversion circuit. 3 shows a detailed configuration example of a mapping processing circuit. 13 shows a second configuration example of a circuit device. 11A to 11C are diagrams illustrating the operation of a circuit device in a second configuration example. 5A and 5B are diagrams for explaining the relationship between the mounting tolerance of the head-up display and rotation correction. 13 shows a third configuration example of a circuit device. 13A to 13C are diagrams illustrating the operation of a circuit device in a third configuration example. 4 shows a fourth configuration example of a circuit device. 13A to 13C are diagrams illustrating the operation of a circuit device in a fourth configuration example. 5 shows a fifth configuration example of a circuit device. 3 shows an example of detailed configurations of a first processing circuit and a second processing circuit. An example of an image displayed on a head-up display. An image of the ROI extracted from the image before mapping processing. An image obtained by performing reverse mapping processing on an image of an ROI extracted from an image displayed on a head-up display. An example of an edge image. Example of electronic device configuration. An example of a moving object.

A preferred embodiment of the present disclosure will be described in detail below. Note that the present embodiment described below does not unduly limit the content described in the claims, and not all of the configurations described in the present embodiment are necessarily essential components.

1. First Configuration Example Fig. 1 shows a first configuration example of a circuit device 100 according to the present embodiment. Fig. 2 is a diagram for explaining the operation of the circuit device 100 in the first configuration example.

The circuit device 100 is a HUD controller that controls the image display of a head-up display. However, the circuit device 100 is not limited to a HUD controller, and may be, for example, a display driver that drives a display panel of a head-up display. In this case, the circuit device 100 may include a drive circuit that drives the display panel based on the image IMG2. The circuit device 100 is an integrated circuit device called an IC (Integrated Circuit). The circuit device 100 is an IC manufactured by a semiconductor process, and is a semiconductor chip in which circuit elements are formed on a semiconductor substrate.

As shown in FIG. 2, the circuit device 100 performs a mapping process of image IMG1 onto image IMG2. Image IMG1 is an input image, and is also called the first image. Image IMG2 is an output image, and is also called the second image. The mapping process imparts a distortion to image IMG2 that cancels the distortion caused by the curved display of the head-up display. In other words, image IMG2 displayed on the display panel of the head-up display is a distorted image. By displaying this image IMG2 on the curved display, an undistorted display image is presented to the user.

An image processing engine that performs such image conversion for distortion correction is called a warp engine. In the first configuration example, a case will be described where the circuit device 100 is an inverse warp engine. An inverse warp engine is a warp engine that has an inverse warp function. Inverse warp is a conversion that determines each pixel of the output image of the warp engine from a pixel at an arbitrary position in the input image. As shown in FIG. 2, the circuit device 100 realizes inverse warp by converting the coordinates (x, y) in the image IMG2, which is the output image, to the coordinates (x', y') in the image IMG1, which is the input image, and performing mapping processing based on the coordinates.

As described in the second configuration example, the circuit device 100 may be a forward warp engine. As described in the third and fourth configuration examples, the circuit device 100 may further perform image rotation.

The following describes in detail a first configuration example in which the circuit device 100 is an inverse warp engine. As shown in FIG. 1, the circuit device 100 includes a coordinate counter 10 that is a first coordinate counter, a coordinate conversion circuit 20, and a mapping processing circuit 30.

The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1. In the first configuration example, the input coordinate IXY1 is a coordinate (x, y) corresponding to the coordinate on the image IMG2, and the output coordinate QXY1 is a coordinate (x', y') corresponding to the coordinate on the image IMG1. The coordinate conversion circuit 20 performs coordinate conversion from the input coordinate IXY1 to the output coordinate QXY1 by performing an arithmetic process using a polynomial of degree 2 or higher that represents the coordinate conversion. The following equation (1) shows the coordinate conversion equation when the polynomial is of degree 2.

In the first line of the above equation (1), a1× ^x2 on the right hand side is the first term of the polynomial that finds x'. Similarly, a2× ^y2 , a3×xy, a4×x, a5×y, and a6 are the second, third, fourth, fifth, and sixth terms of the polynomial that finds x'. a1 is the coefficient of the first term, i.e., the first coefficient. Similarly, a2, a3, a4, a5, and a6 are the second, third, fourth, fifth, and sixth coefficients. The same is true for the equation that finds y'.

The coordinate transformation equation is not limited to a quadratic polynomial, but may be a cubic or higher polynomial. For example, the following equation (2) shows a coordinate transformation equation when the polynomial is a cubic polynomial.

The mapping processing circuit 30 generates image IMG2 by performing mapping processing on the input image IMG1 based on the output coordinates QXY1. Specifically, the pixel value of coordinates (x, y) in image IMG2 is calculated from the pixel value of coordinates (x', y') in image IMG1. (x, y) is the input coordinate IXY1, and (x', y') is the output coordinate QXY1. However, the mapping processing circuit 30 does not use the input coordinate IXY1, but performs mapping processing using coordinates generated internally and the output coordinate QXY1. This point will be described later with reference to FIG. 4. The pixel value is, for example, color data of the pixel to be displayed at the coordinate.

Image IMG2 is displayed on a curved display of a head-up display. For example, a head-up display includes a light source, a liquid crystal display panel, an optical system, and a transparent screen. Image IMG2 is displayed on the liquid crystal display panel. The light source emits light, the emitted light passes through the liquid crystal display panel, and the optical system projects the transmitted light onto the transparent screen, thereby projecting image IMG2 onto the transparent screen. In this case, the transparent screen corresponds to the curved display. Alternatively, the head-up display includes a transparent display such as an organic EL display panel. In this case, image IMG2 is displayed directly on the transparent display. In other words, the transparent display, which is the display panel, also serves as the curved display.

According to this embodiment, the coordinate transformation circuit 20 performs coordinate transformation by calculation processing using polynomials, which enables high-speed processing of distortion correction. Specifically, as shown in the above formula (1), polynomials consist of a combination of multiplication and addition, and do not include complex calculations such as integration. Therefore, by using polynomials, the calculation time is reduced compared to the case of performing calculations including integration.

In addition, in the polynomial, there is a one-to-one correspondence between the input coordinate IXY1 (x, y) and the output coordinate QXY1 (x', y'). This correspondence can be used as it is as the correspondence between the pixels of the output image IMG2 and the pixels of the input image IMG1 in the mapping process. That is, when (x, y) is input to the coordinate conversion circuit 20 one pixel at a time, (x', y') is output one pixel at a time correspondingly. By using this correspondence and the mapping processing circuit 30 mapping one pixel at a time, it is possible to construct one screen's worth of image IMG2. This type of sequential processing is more suitable for real-time processing than the case where a table or the like is created and mapping processing is performed. In addition, in the display processing, pixel data is input and output by a pixel clock, so sequential processing one pixel at a time is suitable for real-time processing.

Below, we will explain an example where the coordinate transformation equation is a second-order polynomial, as in equation (1) above.

The coordinate transformation circuit 20 receives coefficient information CF1 of the polynomial and input coordinate IXY1. The coordinate transformation circuit 20 calculates output coordinate QXY1 from input coordinate IXY1 by calculation based on the coefficient information CF1. The coefficient information CF1 is information on coefficients a1 to a6 and b1 to b6 in the above equation (1). For example, the circuit device 100 includes a storage unit that stores the coefficient information CF1, and the coefficient information CF1 is input from the storage unit to the coordinate transformation circuit 20. The storage unit is a register or memory. The memory is a semiconductor memory such as a RAM or a non-volatile memory. The coefficient information CF1 is written to the storage unit, for example, when the circuit device 100 is started up or when the head-up display is manufactured.

In coordinate transformation using a polynomial, the correspondence between input coordinates IXY1 and output coordinates QXY1 is determined by the coefficients of the polynomial. That is, by inputting coefficient information CF1 that matches the shape of the curved display to the coordinate transformation circuit 20, distortion correction suitable for various curved displays is realized. In addition, since the coordinate transformation circuit 20 determines the output coordinates QXY1 from the input coordinates IXY1 that have been input, it is possible to perform coordinate transformation on a pixel-by-pixel basis that is suitable for real-time processing, as described above.

The coordinate counter 10 outputs a first count coordinate, which is a pixel-based coordinate based on a pixel clock. The coordinate conversion circuit 20 performs coordinate conversion using the first count coordinate as an input coordinate IXY1. In the first configuration example, since the input coordinate IXY1 corresponds to a coordinate on the image IMG2, the pixel clock used by the mapping processing circuit 30 when processing the image IMG2 is input to the coordinate counter 10.

The size of image IMG2 is N x M pixels. N and M are integers of 2 or more. For example, when y = 0, the coordinate counter 10 increments x by 1 for each pixel clock, and outputs (0,0), (1,0), (2,0), ..., (N-1,0). Next, the coordinate counter 10 sets y = 1, increments x by 1 for each pixel clock, and outputs (0,1), (1,1), (2,1), ..., (N-1,1). The coordinate counter 10 repeats this process up to (N-1,M-1). Note that the coordinate counting order is not limited to the above, and the counting order may be set depending on the order in which the pixels of image IMG2 are processed.

In this embodiment, the coordinate counter 10 outputs coordinates (x, y) for each pixel to the coordinate conversion circuit 20 based on the pixel clock, and the coordinate conversion circuit 20 performs coordinate conversion and outputs coordinates (x', y') for each pixel. This enables the mapping processing circuit 30 to perform mapping processing for each pixel based on the pixel clock. This realizes distortion correction suitable for real-time processing.

The processing time for the calculation to convert the input coordinate IXY1 to the output coordinate QXY1 may be multiple cycles of the pixel clock. The output coordinate QXY1 corresponding to the input coordinate IXY1 is output with a delay of the processing time, but the throughput is sufficient if one output coordinate QXY1 can be obtained in one cycle of the pixel clock.

Figure 3 shows an example of the detailed configuration of the coordinate transformation circuit 20. The coordinate transformation circuit 20 includes a first arithmetic circuit 21 and a second arithmetic circuit 22. The first arithmetic circuit 21 and the second arithmetic circuit 22 are logic circuits, and are each configured as separate hardware circuits. Note that while Figure 3 shows only the arithmetic circuit that calculates x', the arithmetic circuit that calculates y' has a similar configuration.

The first arithmetic circuit 21 calculates ^x2 , ^y2 , and xy when the input coordinate IXY1 is (x, y). Specifically, the first arithmetic circuit 21 includes an x2 arithmetic circuit MC1 that squares x to obtain ^x2 , a ^y2 arithmetic circuit MC2 that squares y to obtain ^y2 , and an ^xy arithmetic circuit MC3 that multiplies x and y to obtain xy. Each of the arithmetic circuits MC1 to MC3 is an individual hardware multiplier.

The second arithmetic circuit 22 finds the output coordinate QXY1 based on the calculation result of the first arithmetic circuit 21 and the coefficient information CF1. Here, the second arithmetic circuit 22 finds x' from (x', y'). The coefficients a1 to a6 in the above equation (1) are input to the second arithmetic circuit 22 as the coefficient information CF1. The second arithmetic circuit 22 includes first to fifth term arithmetic circuits AC1 to AC5 and an addition circuit ADDC. The first to fifth term arithmetic circuits AC1 to AC5 are each an individual hardware multiplier.

The first term calculation circuit AC1 multiplies the coefficient a1 by ^x2 to obtain the first term a1× ^x2 . The second term calculation circuit AC2 multiplies the coefficient a2 by ^y2 to obtain the second term a2× ^y2 . The third term calculation circuit AC3 multiplies the coefficient a3 by xy to obtain the third term a3×xy. The fourth term calculation circuit AC4 multiplies the coefficient a4 by x to obtain the fourth term a4×x. The fifth term calculation circuit AC5 multiplies the coefficient a5 by y to obtain the fifth term a5×y. The adder circuit ADDC adds the first to fifth terms and the sixth term, coefficient a6, and outputs the result as x'. This realizes the calculation of x' in the above equation (1).

The configurations of the first arithmetic circuit 21 and the second arithmetic circuit 22 are not limited to those shown in Fig. 3. For example, the first arithmetic circuit 21 may include three arithmetic circuits, which may calculate ^x2 , ^y2 , and xy in a pipeline processing manner. The second arithmetic circuit 22 may include five arithmetic circuits, which may calculate the first to fifth terms in a pipeline processing manner.

According to this embodiment, the first arithmetic circuit 21 includes arithmetic circuits MC1 to MC3 that calculate x ² , y ² , and xy, and the second arithmetic circuit 22 includes 1st to 5th term arithmetic circuits AC1 to AC5 that calculate the 1st to 5th terms. This allows each term to be calculated in parallel, enabling high-speed processing suitable for real-time processing.

The second calculation circuit 22 may include 1st to nth term calculation circuits. The i-th term calculation circuit calculates the i-th term of the polynomial. n is an integer of 2 or more, and i is an integer of 1 to n. When the coordinate transformation equation is a quadratic polynomial as in FIG. 3, n=5. For example, when the coordinate transformation equation is a cubic polynomial as in the above equation (2), n=9 may be used.

Figure 4 shows an example of a detailed configuration of the mapping processing circuit 30. The mapping processing circuit 30 includes a memory control circuit 31, a coordinate counter 32 which is a second coordinate counter, and an image memory 33.

The coordinate counter 32 outputs the second count coordinate CXYB1, which is a pixel-based coordinate based on the pixel clock. The second count coordinate CXYB1 is (xb, yb). In the first configuration example, (xb, yb) specifies coordinates on the image IMG1, so the pixel clock input to the mapping processing circuit 30 together with the image IMG1 is input to the coordinate counter 32. The coordinate counter 32 counts (xb, yb) by the same counting operation as the coordinate counter 10.

The memory control circuit 31 controls access to the image memory 33. The image memory 33 is a buffer memory that temporarily stores the image IMG1, and is, for example, a semiconductor memory such as a RAM. The memory control circuit 31 writes the image IMG1 to the image memory 33 based on the second count coordinate CXYB1, reads pixel values from the image memory 33 based on the output coordinate QXY1 from the coordinate conversion circuit 20, and outputs the pixel values as pixel values of the image IMG2. Specifically, the memory control circuit 31 includes a write address controller 34 and a read address controller 35.

The write address controller 34 decodes the second count coordinate CXYB1 (xb, yb) into the write address ADWR, and writes the pixel value at (xb, yb) of the image IMG1 to the address ADWR of the image memory 33. By repeating this process, the image IMG1 is written to the image memory 33.

The read address controller 35 decodes the output coordinate QXY1 (x', y') into a read address ADRD and reads out the pixel value from address ADRD of the image memory 33. This corresponds to reading out the pixel value at (x', y') in image IMG1. The read address controller 35 outputs the read out pixel value as the pixel value at (x, y) in image IMG2. By repeating this process, image IMG2 is output from the mapping processing circuit 30.

In the above mapping process, the pixel value at (x, y) in image IMG2 is obtained from the pixel value at (x', y') in image IMG1. In other words, the mapping process circuit 30 operates as the inverse warp engine described in FIG. 2.

2. Second Configuration Example Fig. 5 shows a second configuration example of the circuit device 100. Fig. 6 is a diagram for explaining the operation of the circuit device 100 in the second configuration example.

In the second configuration example, a case will be described in which the circuit device 100 is a forward warp engine. A forward warp engine is a warp engine that has a forward warp function. Forward warp is a transformation in which each pixel of an input image of the warp engine is found from a pixel at an arbitrary position in an output image. As shown in FIG. 6, the circuit device 100 realizes forward warp by transforming coordinates (x', y') in image IMG1, which is the input image, into coordinates (x, y) in image IMG2, which is the output image, and performing a mapping process based on the coordinates.

As shown in FIG. 5, the circuit device 100 includes a coordinate counter 10, a coordinate conversion circuit 20, and a mapping processing circuit 30.

In the second configuration example, the coordinate counter 10 outputs the first count coordinate (x', y') to the coordinate conversion circuit 20 as the input coordinate IXY1. Since (x', y') corresponds to the coordinate on the image IMG1, the pixel clock input to the mapping processing circuit 30 together with the image IMG1 is input to the coordinate counter 10.

The coordinate conversion circuit 20 converts the input coordinate IXY1 to the output coordinate QXY1. In the second configuration example, the output coordinate QXY1 is a coordinate (x, y) corresponding to the coordinate on the image IMG2. The following formula (3) shows a coordinate conversion formula when the polynomial is quadratic. The following formula (3) corresponds to the inverse coordinate conversion of the above formula (1).

The coordinate transformation circuit 20 performs the calculation process of the above formula (3) based on the coefficient information CF1. In the second configuration example, the coefficient information CF1 is information on the coefficients c1 to c6 and d1 to d6 in the above formula (3). The configuration of the coordinate transformation circuit 20 is similar to the first configuration example described in FIG. 3. However, in the second configuration example, (x', y') is input as the input coordinates to the first calculation circuit 21 and the second calculation circuit 22, the coefficients c1 to c6 are input to the second calculation circuit 22, and x is output from the second calculation circuit 22 as the output coordinate.

The mapping processing circuit 30 moves the pixel value of coordinates (x', y') in image IMG1 to coordinates (x, y) in image IMG2. (x', y') is the output coordinate QXY1, and (x, y) is the input coordinate IXY1. The mapping processing circuit 30 includes a memory control circuit 31, a coordinate counter 32, and an image memory 33.

The coordinate counter 32 outputs (xb, yb) as the second count coordinate CXYB1. In the second configuration example, (xb, yb) specifies coordinates on the image IMG2, so the pixel clock used by the mapping processing circuit 30 when processing the image IMG2 is input to the coordinate counter 32.

The memory control circuit 31 writes the image IMG1 to the image memory 33 based on the output coordinate QXY1 from the coordinate conversion circuit 20, reads pixel values from the image memory 33 based on the second count coordinate CXYB1, and outputs the pixel values as pixel values of the image IMG2.

Specifically, the write address controller 34 decodes the output coordinate QXY1 (x, y) into the write address ADWR, and writes the pixel value at (x', y') of the image IMG1 to the address ADWR of the image memory 33. This is equivalent to moving the pixel value at (x', y') of the image IMG1 to the coordinates (x, y). In other words, the image IMG1 written to the image memory 33 is an image that has been transformed by distortion correction.

The read address controller 35 decodes the second count coordinate CXYB1 (xb, yb) into a read address ADRD, and reads out the pixel value from the address ADRD of the image memory 33. The read address controller 35 outputs the read out pixel value as the pixel value at (x, y) of the image IMG2. This is repeated, and the image IMG2 is output from the mapping processing circuit 30.

In the above mapping process, the pixel value at (x', y') in image IMG1 is moved to the pixel value at (x, y) in image IMG2. In other words, the mapping process circuit 30 operates as the forward warp engine described in FIG. 6.

3. Third Configuration Example In the third configuration example, the circuit device 100 performs rotation correction in addition to distortion correction of the display image. First, the relationship between the mounting tolerance of the head-up display and rotation correction will be described with reference to FIG. 7. Note that FIG. 7 describes an example in which the head-up display is installed on the dashboard of a car, but the installation location of the head-up display is not limited to this. Also, FIG. 7 shows the display unit DSP of the head-up display as a flat surface, but in reality it is a curved display.

The direction DZ shown in FIG. 7 is a direction perpendicular to the display unit DSP of the head-up display. Specifically, the direction DZ and the display unit DSP are perpendicular to each other at any position on the display unit DSP, which is a curved surface. For example, the direction DZ and the display unit DSP are perpendicular to each other at the center of the display unit DSP. The direction DX is a direction perpendicular to the direction DZ, and the direction DY is a direction perpendicular to the direction DX and the direction DZ. The direction DX corresponds to the horizontal direction. That is, when the automobile is in a horizontal position, the direction DX is parallel to the horizontal plane. When the head-up display is installed vertically on the dashboard of the automobile, the direction DY corresponds to the vertical direction, and the direction DY is perpendicular to the horizontal plane. However, the direction DY is not limited to the vertical direction, and the direction DY may be inclined with respect to the horizontal plane. When the head-up display is installed on the dashboard with an angle θX inclined in the depth direction, the direction DY is inclined with respect to the horizontal plane by the angle θX.

Rotation RZ indicates the rotation of the display unit DSP around an axis parallel to the direction DZ. The rotation angle of rotation RZ varies due to tolerances when mounting the head-up display to the dashboard. The rotation angle when the tolerance is zero is set to 0 degrees, clockwise rotation is positive, and counterclockwise rotation is negative. When the rotation RZ of the display unit DSP is a positive rotation angle, the displayed image rotates clockwise as seen by the user. The circuit device 100 rotates the image in the negative direction, i.e. counterclockwise. This makes it possible to display an image that is not tilted as seen by the user, even if the display unit DSP is tilted due to a tolerance.

Figure 8 shows a third configuration example of the circuit device 100. Figure 9 is a diagram explaining the operation of the circuit device 100 in the third configuration example.

In the third configuration example, a case will be described in which the circuit device 100 is an inverse warp engine. As shown in FIG. 9, the circuit device 100 rotates and transforms the coordinates (xa, ya) in the image IMG2, which is the output image, converts the coordinates (x, y) after the rotation and transformation into coordinates (x', y') in the image IMG1, which is the input image, and performs mapping processing based on the coordinates to achieve image rotation and inverse warp.

As shown in FIG. 8, the circuit device 100 includes a coordinate counter 10, a coordinate conversion circuit 20, a mapping processing circuit 30, and a rotation conversion circuit 40. Note that descriptions of configurations and operations similar to those in the first configuration example will be omitted where appropriate.

The coordinate counter 10 outputs (xa, ya) as the first count coordinate CXYA1. Since (xa, ya) corresponds to the coordinates on the image IMG2, the pixel clock used by the mapping processing circuit 30 when processing the image IMG2 is input to the coordinate counter 10.

The rotation transformation circuit 40 performs a rotation transformation on the first count coordinate CXYA1, and outputs the coordinate after the rotation transformation as an input coordinate IXY1 to the coordinate transformation circuit 20. In the third configuration example, the input coordinate IXY1 is (x, y). As shown in the following formula (4), the rotation transformation circuit 40 performs coordinate rotation using an affine transformation. θ is the rotation angle.

The rotational transformation circuit 40 transforms the first count coordinate CXYA1 into the input coordinate IXY1 using the above formula (4) based on the angle information RT1 indicating the rotational angle θ. For example, the circuit device 100 includes a storage unit that stores the angle information RT1 and the coefficient information CF1, and the angle information RT1 is input from the storage unit to the rotational transformation circuit 40, and the coefficient information CF1 is input from the storage unit to the coordinate transformation circuit 20. The storage unit is a register or memory. The memory is a semiconductor memory such as a RAM or a non-volatile memory. The angle information RT1 and the coefficient information CF1 are written to the storage unit, for example, when the circuit device 100 is started up or when the head-up display is manufactured.

The coordinate conversion circuit 20 converts the input coordinate IXY1 to the output coordinate QXY1 based on the coefficient information CF1. In the third configuration example, the output coordinate QXY1 is the coordinate (x', y') corresponding to the coordinate on the image IMG1. The coordinate conversion formula is the same as in the first configuration example.

The mapping processing circuit 30 obtains the pixel value of the coordinates (xa, ya) in the image IMG2 from the pixel value of the coordinates (x', y') in the image IMG1. (xa, ya) is the first count coordinate CXYA1, and (x', y') is the output coordinate QXY1. The configuration and operation of the mapping processing circuit 30 are the same as in the first configuration example.

In the mapping process of the third configuration example, the pixel value at (xa, ya) of image IMG2 is obtained from the pixel value at (x', y') of image IMG1. That is, the mapping process circuit 30 operates as the inverse warp engine described in FIG. 9. In addition, the rotation transformation circuit 40 performs coordinate rotation, and thus inverse warp including image rotation is performed in the mapping process.

4. Fourth Configuration Example Fig. 10 shows a fourth configuration example of the circuit device 100. Fig. 11 is a diagram for explaining the operation of the circuit device 100 in the fourth configuration example.

In the fourth configuration example, a case will be described in which the circuit device 100 is a forward warp engine. As shown in FIG. 11, the circuit device 100 rotates and transforms the coordinates (xa, ya) in the input image, image IMG1, and converts the coordinates (x', y') after the rotation transformation into coordinates (x, y) in the output image, image IMG2, and performs mapping processing based on the coordinates, thereby realizing image rotation and forward warp.

As shown in FIG. 10, the circuit device 100 includes a coordinate counter 10, a coordinate conversion circuit 20, a mapping processing circuit 30, and a rotation conversion circuit 40. Note that descriptions of configurations and operations similar to those of the second and third configuration examples will be omitted as appropriate.

The coordinate counter 10 outputs (xa, ya) as the first count coordinate CXYA1. In the fourth configuration example, (xa, ya) corresponds to the coordinates on the image IMG1, so the pixel clock input to the mapping processing circuit 30 together with the image IMG1 is input to the coordinate counter 10.

The rotation transformation circuit 40 performs a rotation transformation on the first count coordinate CXYA1, and outputs the coordinate after the rotation transformation as an input coordinate IXY1 to the coordinate transformation circuit 20. In the fourth configuration example, the input coordinate IXY1 is (x', y'). As shown in the following equation (5), the rotation transformation circuit 40 performs coordinate rotation using an affine transformation. θ is the rotation angle.

The rotation conversion circuit 40 converts the first count coordinate CXYA1 into the input coordinate IXY1 using the above formula (5) based on the angle information RT1 indicating the rotation angle θ.

The coordinate conversion circuit 20 converts the input coordinate IXY1 to the output coordinate QXY1 based on the coefficient information CF1. In the fourth configuration example, the output coordinate QXY1 is the coordinate (x, y) corresponding to the coordinate on the image IMG2. The coordinate conversion formula is the same as in the second configuration example.

The mapping processing circuit 30 moves the pixel value of coordinates (xa, ya) in image IMG1 to coordinates (x, y) in image IMG2. (xa, ya) is the first count coordinate CXYA1, and (x, y) is the output coordinate QXY1. The configuration and operation of the mapping processing circuit 30 are the same as in the second configuration example.

In the mapping process of the fourth configuration example, the pixel value at (xa, ya) of image IMG1 is moved to the pixel value at (x, y) of image IMG2. In other words, the mapping process circuit 30 operates as the forward warp engine described in FIG. 11. In addition, the rotation transformation circuit 40 performs coordinate rotation, so that forward warping including image rotation is performed in the mapping process.

12 shows a fifth configuration example of the circuit device 100. In the fifth configuration example, the circuit device 100 generates an image IMG3 by performing an inverse mapping process on the image IMG2, and detects an error in the image IMG2 by comparing the image IMG1 with the image IMG3.

The circuit device 100 includes an interface 110, a memory unit 133, an image processing circuit 135, an interface 140, a comparison circuit 145, an error detection circuit 150, a memory unit 160, a register circuit 170, and an interface 190.

The interface 110 receives image data transmitted from, for example, the processing device 200 to the circuit device 100. The interface 110 converts the received image data into a format used inside the circuit device 100, and outputs the converted image data as image IMA1. For example, the interface 110 is an OpenLDI (Open LVDS Display Interface) that converts a serial signal received via LVDS (Low Voltage Differential Signaling) into an RGB parallel signal. The processing device 200 is, for example, a SoC (System on a Chip), an MCU (Micro Control Unit), or a CPU (Central Processing Unit).

The image processing circuit 135 includes a first processing circuit 131 that maps an image to match the surface shape of the curved display of the head-up display, and a second processing circuit 132 that performs the inverse mapping. The first processing circuit 131 is also called the first warp engine, and the second processing circuit 132 is also called the second warp engine. Below, an example will be described in which the first processing circuit 131 and the second processing circuit 132 perform image rotation together with distortion correction, but the first processing circuit 131 and the second processing circuit 132 may perform only distortion correction.

The curved display is a screen or a display panel in a head-up display. The screen is also called a projection target. When the curved display is a screen, the head-up display includes a projection device that projects the image IMA2 onto the screen. The projection device includes, for example, a liquid crystal display panel, a display driver that drives the liquid crystal display panel, a light source, and a lens. The display driver displays an image on the liquid crystal display panel based on the received image data, the light source outputs light to the liquid crystal display panel, and the light that passes through the liquid crystal display panel is projected onto the screen by the lens. The screen is a transparent object and has a reflective surface that reflects the projected light. For example, in an in-vehicle head-up display, the screen is a transparent screen attached to the dashboard or the windshield of the automobile. When the curved display is a display panel, the head-up display is configured so that the display image on the display panel is directly visible to the user, and the head-up display displays the image IMA2 on the display panel. The display panel is, for example, a transparent display using an organic EL panel, and the transparent display has a curved surface.

The first processing circuit 131 performs a first mapping process using coefficient information CF1 and a first rotation process using angle information RT1 on image IMA1, and outputs a processed image IMA2. Image IMA1 is the first image, and image IMA2 is the second image. The first processing circuit 131 also extracts an image IMA1' of a region of interest from image IMA1. The region of interest is also called ROI (Region Of Interest). Note that image IMA1' may be the entire image IMA1. In the following, an example will be described in which image IMA1' is an image of a region of interest extracted from image IMA1, and image IMA' of the region of interest will also be called the first image.

The second processing circuit 132 performs a second mapping process using the coefficient information CF2 and a second rotation process using the angle information RT2 on the image IMA2, and outputs the processed image IMA3. The image IMA3 is the third image. Specifically, the second processing circuit 132 extracts an image of the region of interest from the image IMA2, and performs a second mapping process and a second rotation process on the image. The second rotation process is the reverse rotation process of the first rotation process. The image IMA3 is an image that has been reverse mapped and reverse rotated from the image of the region of interest extracted from the image IMA2.

The interface 140 outputs the image IMA2 to the outside of the circuit device 100. The outside of the circuit device 100 is, for example, a display driver that drives a display panel of a head-up display. For example, the interface 140 is an LVDS interface, and converts the RGB parallel signals from the image processing circuit 135 into LVDS serial signals.

The storage unit 133 is a first storage unit. The first processing circuit 131 stores the image IMA1' of the area of interest in the storage unit 133. The storage unit 160 is a memory. For example, the memory is a semiconductor memory such as a RAM or a non-volatile memory. Note that the storage unit 133 and the storage unit 160 may each be configured as separate memories, or may be configured as a single memory.

The comparison circuit 145 performs a comparison process between the image IMA1' stored in the memory unit 133 and the image IMA3, and outputs the comparison result. This comparison result is used to detect an error in the image IMA2. In other words, it is used to verify whether the first mapping process and the first rotation process performed by the first processing circuit 131 were normal. The comparison circuit 145 obtains an index indicating the similarity between the image IMA1 and the image IMA3. The index is a shape index or a visibility index, which will be described later. Alternatively, the comparison circuit 145 may obtain an index such as SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), or NCC (Normalized Cross Correlation).

The error detection circuit 150 detects errors in the second image IMA2 by comparing the index with a threshold value. The threshold value indicates how similar the images IMA1' and IMA3 must be to be acceptable.

When an error is detected by the error detection circuit 150, the image processing circuit 135 stops outputting the image IMA2 to the interface 140. Alternatively, when an error is detected by the error detection circuit 150, the interface 140 stops outputting the image IMA2. The interface 140 may output the image IMA2 together with error information, and the display driver that receives the error information may perform an operation based on the error information. Alternatively, the interface 190 may output the error information to the processing device 200, and the processing device 200 that receives the error information may perform an operation based on the error information. The error information may be, for example, an error determination flag or an indicator. The operation based on the error information may be, for example, stopping the display of the head-up display.

The interface 190 performs inter-circuit communication between the circuit device 100 and the processing device 200. For example, the interface 190 is a serial communication interface such as an SPI (Serial Peripheral Interface) type or an I2C type. Setting information and control information from the processing device 200 are written, for example, to the register circuit 170, and the circuit device 100 operates according to the setting information and control information.

The register circuit 170 is configured to be accessible from the processing device 200 via the interface 190. The register circuit 170 includes an error detection result register 176 and a threshold register 178.

The error detection result register 176 stores the error detection result output by the error detection circuit 150. The error detection result is, for example, an error determination flag indicating whether or not the display image has been determined to be an error. The processing device 200 can determine whether or not an error has occurred by reading the error detection result from the error detection result register 176 via the interface 190.

A threshold value is set in the threshold register 178 from the processing device 200 via the interface 190. The error detection circuit 150 performs error detection by comparing the index with the threshold value set in the threshold register 178.

The memory unit 160 is a second memory unit. The memory unit 160 stores coefficient information CF1, CF2 and angle information RT1, RT2. Specifically, the processing device 200 transmits CF1, CF2, RT1, and RT2 to the interface 190, and the memory unit 160 stores CF1, CF2, RT1, and RT2 received by the interface 190. The image processing circuit 135 performs mapping processing and rotation processing based on CF1, CF2, RT1, and RT2 read from the memory unit 160. The memory unit 160 is, for example, a memory or a register. For example, the memory is a semiconductor memory such as a RAM or a non-volatile memory.

The image processing circuit 135, the comparison circuit 145, and the error detection circuit 150 are logic circuits. The image processing circuit 135, the comparison circuit 145, and the error detection circuit 150 may be configured as individual circuits, or may be configured as an integrated circuit by automatic placement and wiring, etc. Furthermore, some or all of these logic circuits may be realized by a processor such as a DSP (Digital Signal Processor). In this case, a program or instruction set describing the function of each circuit is stored in memory, and the function of each circuit is realized by the processor executing the program or instruction set.

In FIG. 12, the circuit device 100 includes the error detection circuit 150 and the error detection result register 176, but the circuit device 100 does not have to include the error detection circuit 150 and the error detection result register 176. In this case, the interface 190 may output the index determined by the comparison circuit 145 to the processing device 200, and the processing device 200 that receives the index may detect an error by comparing the index with a threshold value. When the processing device 200 detects an error, it may perform an error response operation such as stopping the display of the head-up display.

Figure 13 shows an example of the detailed configuration of the first processing circuit 131 and the second processing circuit 132.

The first processing circuit 131 includes a coordinate counter 10 which is a first coordinate counter, a rotation transformation circuit 40 which is a first rotation transformation circuit, a coordinate transformation circuit 20 which is a first coordinate transformation circuit, and a mapping processing circuit 30 which is a first mapping processing circuit. The operation of these circuits is as described in the first to fourth configuration examples. Note that if the first processing circuit 131 only performs distortion correction, the rotation transformation circuit 40 may be omitted.

The second processing circuit 132 includes a coordinate counter 50 which is a second coordinate counter, a rotation transformation circuit 80 which is a second rotation transformation circuit, a coordinate transformation circuit 60 which is a second coordinate transformation circuit, and a mapping processing circuit 70 which is a second mapping processing circuit. The coordinate counter 50 outputs the count coordinate CXYA2. The rotation transformation circuit 80 rotates and transforms the count coordinate CXYA2 into the input coordinate IXY2 based on the angle information RT2. The coordinate transformation circuit 60 transforms the input coordinate IXY2 into the output coordinate QXY2 by a calculation process using a polynomial of degree 2 or higher based on the coefficient information CF2. The mapping processing circuit 70 maps the image IMA2 to the image IMA3 based on the output coordinate QXY2. The detailed configuration and operation of these circuits are the same as those of the coordinate counter 10, the rotation transformation circuit 40, the coordinate transformation circuit 20, and the mapping processing circuit 30. Note that when the second processing circuit 132 only performs distortion correction, the rotation transformation circuit 80 may be omitted.

The configuration of the second processing circuit 132 is not limited to that shown in FIG. 13. For example, a map table describing the correspondence of coordinates in the second mapping process may be input to the second processing circuit 132. The second processing circuit 132 may perform the second mapping process and the second rotation process based on the map table and the angle information RT2.

This section explains the image comparison performed by the comparison circuit 145.

Figure 14 is an example of an image IMG2 displayed on a head-up display. In Figure 14, an icon ICA is overlaid on a meter image DIM. The icon ICA is blended into the meter image DIM at a certain transparency. In this embodiment, the circuit device 100 verifies whether the icon ICA is displayed appropriately. In this case, the area including the icon ICA is set as the ROI, as shown by the dotted rectangle in Figure 14.

Figure 15 shows an image IMG1' of the ROI extracted from the image IMG1 before the mapping process. The ROI includes the icon ICA and the meter image DIM, which is the background image of the icon ICA. Figure 16 shows an image IMG3 obtained by performing the reverse mapping process on the image IMG2' of the ROI extracted from the image IMG2. Figure 16 shows an example in which the icon ICA is not displayed correctly. In this case, the images IMG1' and IMG3 only match in the background image, and the icon ICA portion is different. Therefore, the similarity indicated by the index, which is the comparison result, is low. If the icon ICA is also displayed correctly in the image IMG3, the similarity is high. The index indicating the similarity can take a continuous or stepped value. The error detection circuit 150 compares the index with a threshold value, and the degree of similarity to be tolerated can be adjusted by adjusting the threshold value.

The comparison circuit 145 obtains a shape index, a visibility index, or both, as an index indicating the similarity between the images IMG1' and IMG3. As described above, the comparison circuit 145 may obtain SSD, SAD, NCC, or the like as an index. Whether the higher the similarity, the larger the index or the smaller the index, depends on the index calculation method. A threshold is set for each of the shape index and the visibility index.

First, the first calculation method of the shape index will be described. The comparison circuit 145 calculates the inter-image distance between images IMG1' and IMG3 in color space. The color space is, for example, RGB or YCrCb. Specifically, the comparison circuit 145 calculates the squared value of the distance between a pixel of image IMG1' and its corresponding pixel of image IMG3 in color space. The comparison circuit 145 accumulates this squared value within the image, and the accumulated value is the inter-image distance. In the first calculation method, the inter-image distance corresponds to the shape index.

Next, the second calculation method of the shape index will be described. As shown in FIG. 17, the comparison circuit 145 obtains an edge image EIMG1' by extracting edges from image IMG1'. The comparison circuit 145 also obtains an edge image of image IMG3 by extracting edges from image IMG3. Hereinafter, the edge image of image IMG3 will be referred to as EIMG3. The comparison circuit 145 compares edge image EIMG1' with edge image EIMG3. Specifically, the comparison circuit 145 extracts edges from images IMA1' and IMG3 using a Sobel filter or the like, and obtains a correlation value between edge image EIMG1' and edge image EIMG3. In the second calculation method, the correlation value of the edge image corresponds to the shape index.

Next, a method for calculating the visibility index will be described. Here, the color space is YCrCb, but the color space may be RGB, etc. The comparison circuit 145 obtains a histogram from the Y channel of image IMG1'. Similarly, the comparison circuit 145 obtains a histogram from the Cr channel and Cb channel of image IMG1', and obtains a histogram from the Y channel, Cr channel, and Cb channel of image IMG3.

The comparison circuit 145 performs a cross-correlation calculation on the histograms of images IMG1' and IMG3 in the Y channel. The cross-correlation calculation is a calculation in which the two histograms are shifted by a delay (lag) to find the correlation value, and the correlation value is calculated while changing the delay. If the delay is changed and there is a point where the correlation value of the two histograms is high, a peak will appear at that delay. There may be multiple peaks. Similarly, the comparison circuit 145 performs a cross-correlation calculation on the histograms of images IMG1' and IMG3 in the Cr and Cb channels.

The comparison circuit 145 examines the delay values at which peaks appear in the cross-correlation signals of all channels, and finds the maximum delay value among those delay values. This maximum delay value corresponds to the visibility index. When the color contrast between the icon and the background image is high, the maximum delay value is large, so the visibility index indicates the color contrast between the icon and the background image. Since the higher the color contrast, the higher the visibility, it is determined that the similarity is higher.

By using the shape index or the visibility index, or both, described above, the similarity between the images IMG1' and IMG3 of the ROI can be determined. When the shape index is used, the similarity is highest when the images IMG1' and IMG3 match. In other words, the similarity can also be said to be the degree of match. For example, when the icon ICA is displayed rotated, the similarity decreases compared to when the icon ICA is not rotated. On the other hand, since the visibility index indicates the color contrast, the similarity does not change much even when the icon ICA is rotated. Therefore, when the icon ICA is allowed to be rotated, the visibility index can be used. In addition, since the shape index and the visibility index are calculated using different methods, the accuracy of error detection can be improved by using both of them.

6. Electronic device, mobile object FIG. 18 is a configuration example of an electronic device including the circuit device of this embodiment. The electronic device 300 includes a processing device 310, a circuit device 320, a storage device 350, an operation device 360, a communication device 370, and a head-up display 400. The circuit device 320 corresponds to the circuit device 100 of the first to fifth configuration examples. The head-up display 400 includes a display driver 330 and a display panel 340. The processing device 310 is, for example, an MCU. In the configuration example of FIG. 18, the circuit device 320 corresponds to a display controller. However, the circuit device 100 of this embodiment can be applied not only to a display controller but also to any circuit device that generates a display image for a head-up display.

The processing device 310 transfers image data stored in the storage device 350 or image data received by the communication device 370 to the circuit device 320. The circuit device 320 performs image processing on the image data, display timing control, and generation of image data to be transferred to the display driver. The circuit device 320 may also perform error detection of the image data as described in the fifth configuration example. The display driver 330 drives the display panel 340 to display an image based on the image data transferred from the circuit device 320 and the display timing control by the circuit device 320. The display panel 340 is, for example, a liquid crystal display panel or an EL display panel. The storage device 350 is, for example, a memory, a hard disk drive, an optical disk drive, etc. The operation device 360 is a device for a user to operate the electronic device 300, and is, for example, a button, a touch panel, or a keyboard. The communication device 370 is, for example, a device that performs wired communication or a device that performs wireless communication. The wired communication is, for example, a LAN or USB. Examples of wireless communication include wireless LAN and wireless proximity communication.

Electronic devices including the circuit device of this embodiment can be various devices such as electronic devices for vehicles, display terminals for factory equipment, display devices mounted on robots, and information processing devices. An example of an electronic device for vehicles is a meter panel. An example of an information processing device is a PC.

Figure 19 is an example of a moving body including the circuit device 320 of this embodiment. The moving body includes the circuit device 320 of this embodiment and a processing device 310 that transmits image data to the circuit device 320. The processing device 310 may perform error response processing based on an error detection result of the display image for the head-up display from the circuit device 320. The moving body includes a head-up display 400 and a control device 208. The control device 208 is an ECU (Electronic Control Unit), and the circuit device 320 and the processing device 310 are incorporated in the ECU. The circuit device 320 may be incorporated in the head-up display 400. The circuit device 320 of this embodiment can be incorporated in various moving bodies such as cars, airplanes, motorcycles, bicycles, or ships. The moving body is, for example, a device or equipment that has a driving mechanism such as an engine or motor, a steering mechanism such as a handlebar or rudder, and various electronic devices and moves on the ground, in the air, or on the sea. Figure 19 shows a schematic diagram of an automobile 206 as a specific example of a moving body. The head-up display 400 has a transparent screen, and the transparent screen is installed between the driver's seat and the windshield. Alternatively, the head-up display may use the windshield as a transparent screen and project an image onto the windshield. The head-up display 400 functions as, for example, a meter panel of the automobile 206.

The circuit device of the present embodiment described above includes a coordinate conversion circuit and a mapping processing circuit. The coordinate conversion circuit performs coordinate conversion from input coordinates to output coordinates. The mapping processing circuit performs mapping processing based on the output coordinates on the input first image to generate a second image to be displayed on a display panel for displaying images on a curved display. The coordinate conversion circuit performs coordinate conversion by performing arithmetic processing using a polynomial of degree 2 or higher that represents the coordinate conversion.

Polynomials consist of a combination of multiplication and addition, and do not include complex calculations such as integration. For this reason, using polynomials to perform coordinate transformation shortens the calculation time compared to calculations that include integration. Furthermore, with polynomials, there is a one-to-one correspondence between input coordinates and output coordinates. Using this correspondence, the mapping processing circuit maps each pixel, making it possible to create an image for one screen. This type of sequential processing is more suitable for real-time processing than mapping processing that requires the creation of a table, etc.

In addition, in this embodiment, the coordinate conversion circuit may receive polynomial coefficient information and input coordinates, and determine output coordinates from the input coordinates by performing calculation processing based on the coefficient information.

In coordinate transformation using a polynomial, the correspondence between input coordinates and output coordinates is determined by the coefficients of the polynomial. In other words, by inputting coefficient information that matches the shape of the curved display to the coordinate transformation circuit, distortion correction suitable for various curved displays can be achieved. In addition, since the coordinate transformation circuit determines the output coordinates from the input coordinates that it receives, it is possible to perform coordinate transformation on a pixel-by-pixel basis that is suitable for real-time processing, as described above.

In this embodiment, the coordinate conversion circuit may include a 1st to nth term calculation circuit and an adder circuit. The i-th term calculation circuit may calculate the i-th term of the 1st to nth terms of the polynomial based on the coefficient information. n is an integer of 2 or more, and i is an integer of 1 to n or less. The adder circuit may add the 1st to nth terms output by the 1st to nth term calculation circuit, and output the output coordinate.

In this way, a separate term calculation circuit is provided for each term of the polynomial, and the calculations for each term are processed in parallel. This makes it possible to perform coordinate transformation at high speed.

In this embodiment, the coordinate conversion circuit may include a first arithmetic circuit and a second arithmetic circuit. The first arithmetic circuit may calculate ^x2 , ^y2 , and xy when the input coordinates are (x, y). The second arithmetic circuit may calculate output coordinates based on the calculation result of the first arithmetic circuit and coefficient information.

In this way, the first arithmetic circuit calculates the terms ^x2 , ^y2 , and xy before multiplication by the coefficients, and the second arithmetic circuit multiplies ^x2 , ^y2 , and xy by the coefficients to calculate each term of the polynomial, and the output coordinates can be output by adding up the calculated terms. In this way, by using polynomials, the coordinate transformation calculation can be configured by multiplication and addition.

In this embodiment, the polynomial may include a1×x ² , a2×y ² , a3×xy, a4×x, a5×y, and a6 as the first to sixth terms. The coefficient information may include information on a1, a2, a3, a4, a5, and a6 as the first to sixth coefficients. The second arithmetic circuit may include a first term arithmetic circuit that multiplies a1 by x ² , a second term arithmetic circuit that multiplies a2 by y ² , a third term arithmetic circuit that multiplies a3 by xy, a fourth term arithmetic circuit that multiplies a4 by x, a fifth term arithmetic circuit that multiplies a5 by y, and an adder circuit. The adder circuit may add the first to fifth terms output by the first to fifth arithmetic circuits and the sixth term, a6, and output the output coordinate.

In this way, a term calculation circuit corresponding to each second-order term and first-order term of the polynomial is provided separately. This allows the calculation of each term of the second-order term and first-order term of the polynomial to be processed in parallel, making it possible to perform coordinate transformation at high speed.

In this embodiment, the circuit device may also include a first coordinate counter. The first coordinate counter may output a first count coordinate, which is a pixel-unit coordinate based on a pixel clock. The coordinate conversion circuit may perform coordinate conversion using the first count coordinate as an input coordinate.

According to this embodiment, the first coordinate counter outputs the first count coordinate for each pixel based on the pixel clock, and the coordinate conversion circuit converts the first count coordinate into an output coordinate. As a result, the output coordinate is output for each pixel according to the pixel clock, and the mapping processing circuit can perform mapping processing for each pixel based on the pixel clock. This realizes distortion correction suitable for real-time processing.

In addition, in this embodiment, the input coordinates may be coordinates corresponding to coordinates on one of the first image and the second image. The output coordinates may be coordinates corresponding to coordinates on the other of the first image and the second image.

When the mapping processing circuit performs inverse warp, the input coordinates correspond to the coordinates on the second image, and the output coordinates correspond to the coordinates on the first image. When the mapping processing circuit performs forward warp, the input coordinates correspond to the coordinates on the first image, and the output coordinates correspond to the coordinates on the second image. In the arithmetic processing using polynomials, both inverse warp and forward warp can be realized by using coefficients corresponding to inverse warp or coefficients corresponding to forward warp.

In this embodiment, the mapping processing circuit may include a second coordinate counter that outputs second count coordinates, which are pixel-unit coordinates based on a pixel clock, an image memory that stores the first image, and a memory control circuit that controls access to the image memory. The memory control circuit may perform mapping processing by writing the first image to the image memory based on the second count coordinates, and outputting pixel values read from the image memory based on the output coordinates as pixel values at the first count coordinates of the second image. Alternatively, the memory control circuit may perform mapping processing by writing pixel values at the first count coordinates of the first image to the image memory as pixel values at the output coordinates, and outputting pixel values read from the image memory based on the second count coordinates as pixel values at the second count coordinates of the second image.

In the former mapping process, when the second image is read from the image memory, the pixel values at the input coordinates of the second image are obtained from the pixel values at the output coordinates of the first image. This achieves inverse warping. In the latter mapping process, when the first image is written to the image memory, the pixel values at the input coordinates of the first image are moved to the pixel values at the output coordinates. This achieves forward warping.

In this embodiment, the circuit device may also include a rotation transformation circuit. The rotation transformation circuit may perform a rotation transformation of the coordinates and output the rotated coordinates. The coordinate transformation circuit may perform a coordinate transformation using the rotated coordinates as input coordinates.

In this way, the input coordinates, which are the rotated coordinates, are converted to output coordinates, and the mapping process is performed based on the output coordinates. This makes it possible to realize image rotation that corresponds to the installation error of the curved display, and image distortion correction that corresponds to the image distortion caused by the curved display.

In this embodiment, the circuit device may also include a first coordinate counter. The first coordinate counter may output a first count coordinate, which is a pixel-based coordinate based on a pixel clock. The rotation transformation circuit may perform a rotation transformation on the first count coordinate to obtain a rotated coordinate.

According to this embodiment, the first coordinate counter outputs the first count coordinates for each pixel based on the pixel clock, the rotation conversion circuit converts the first count coordinates into rotated coordinates, and the coordinate conversion circuit converts the rotated coordinates into output coordinates. As a result, the output coordinates are output for each pixel according to the pixel clock, making it possible for the mapping processing circuit to perform mapping processing for each pixel based on the pixel clock. This allows rotation correction and distortion correction suitable for real-time processing to be realized.

In this embodiment, the first count coordinates may be coordinates that specify coordinates on one of the first and second images. The output coordinates may be coordinates that specify coordinates on the other of the first and second images.

When the mapping processing circuit performs inverse warp, the first count coordinates correspond to coordinates on the second image, and the output coordinates correspond to coordinates on the first image. When the mapping processing circuit performs forward warp, the first count coordinates correspond to coordinates on the first image, and the output coordinates correspond to coordinates on the second image. In the arithmetic processing using polynomials, both inverse warp and forward warp can be realized by using coefficients corresponding to inverse warp or coefficients corresponding to forward warp. In addition, since the rotation transformation circuit performs rotation transformation on the first count coordinates, distortion correction and image rotation are performed simultaneously when the first image is mapped to the second image.

In this embodiment, the mapping processing circuit may include a second coordinate counter that outputs second count coordinates, which are pixel-unit coordinates based on a pixel clock, an image memory that stores the first image, and a memory control circuit that controls access to the image memory. The memory control circuit may perform mapping processing by writing the first image to the image memory based on the second count coordinates, and outputting pixel values read from the image memory based on the output coordinates as pixel values at the first count coordinates of the second image. Alternatively, the memory control circuit may perform mapping processing by writing pixel values at the first count coordinates of the first image to the image memory as pixel values at the output coordinates, and outputting pixel values read from the image memory based on the second count coordinates as pixel values at the second count coordinates of the second image.

In the former mapping process, when the second image is read from the image memory, the pixel value at the first count coordinate of the second image is obtained from the pixel value at the output coordinate of the first image. This achieves an inverse warp that includes image rotation and distortion correction. In the latter mapping process, when the first image is written to the image memory, the pixel value at the first count coordinate of the first image is moved to the pixel value at the output coordinate. This achieves a forward warp that includes image rotation and distortion correction.

Also, in this embodiment, the polynomial may be ^a1xx2 + ^a2xy2 +a3xy+a4xx+a5xy+a6.

In this way, coordinate transformation is performed by calculation using a quadratic polynomial. Each term of the quadratic polynomial contains a maximum of two multiplications. For this reason, coordinate transformation using a quadratic polynomial has a smaller calculation load than when using higher-order polynomials, making it suitable for real-time processing.

In this embodiment, the circuit device may also include a second mapping processing circuit and a comparison circuit. The second mapping processing circuit may generate a third image by performing a second mapping process, which is an inverse mapping process of the mapping process, on the second image. The comparison circuit may compare the first image with the third image and output the comparison result as information for detecting errors in the second image.

In this way, the circuit device can verify whether the distortion-corrected image is appropriate for display on the head-up display. Alternatively, if error detection information is output externally to the circuit device, the external device that receives the error detection information can perform the above verification. Since the mapping processing circuit maps the first image onto the second image and the second mapping processing circuit inversely maps the second image onto the third image, if the second image is normal, the third image should return to the same image as the first image. The comparison circuit can output information for detecting errors in the second image by comparing the third image with the first image.

The electronic device of this embodiment also includes any of the circuit devices described above.

The moving body of this embodiment also includes any of the circuit devices described above.

Although the present embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible that do not substantially deviate from the novel matters and effects of the present disclosure. Therefore, all such modifications are intended to be included in the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader or similar meaning may be replaced with that different term anywhere in the specification or drawings. All combinations of the present embodiment and modifications are also included in the scope of the present disclosure. Furthermore, the configurations and operations of the circuit device, head-up display, electronic device, and mobile object are not limited to those described in the present embodiment, and various modifications are possible.

10...coordinate counter, 20...coordinate conversion circuit, 21...first arithmetic circuit, 22...second arithmetic circuit, 30...mapping processing circuit, 31...memory control circuit, 32...coordinate counter, 33...image memory, 34...write address controller, 35...read address controller, 40...rotation conversion circuit, 50...coordinate counter, 60...coordinate conversion circuit, 70...mapping processing circuit, 80...rotation conversion circuit, 100...circuit device, 110...interface, 131...first processing circuit, 132...second processing circuit, 133...memory unit, 135...image processing circuit, 140...interface, 145...comparison circuit, 150...error detection circuit, 160...memory unit, 170...register circuit, 176...error detection Result register, 178... threshold register, 190... interface, 200... processing device, 206... automobile, 208... control device, 300... electronic device, 310... processing device, 320... circuit device, 330... display driver, 340... display panel, 350... storage device, 360... operation device, 370... communication device, 400... head-up display, AC1 to AC5... first to fifth term calculation circuits, ADDC... addition circuit, ADRD... read address, ADWR... write address, CF1... coefficient information, CXYA1... first count coordinate, CXYB1... second count coordinate, IMA1, IMG1... first image, IMA2, IMG2... second image, IMA3... third image, IXY1... input coordinate, MC1... x ² calculation circuit, MC2...y ² calculation circuit, MC3...xy calculation circuit, QXY1...output coordinates, RT1...angle information, a1 to a6...coefficients

Claims (17)

a rotation conversion circuit that performs rotation conversion of coordinates to correct the tilt of the display due to an installation error when the head-up display using a curved display is installed at an installation location , and outputs the rotated coordinates;
a coordinate conversion circuit for converting the input coordinates, which are the rotated coordinates, into output coordinates;
a mapping processing circuit that performs mapping processing based on the output coordinates on an input first image to generate a second image to be displayed on a display panel for displaying an image on the curved display;
Including,
The coordinate conversion circuit includes:
the input coordinates are inputted one pixel at a time, and an arithmetic process is performed on the input coordinates one pixel at a time using a polynomial of degree two or higher that represents the coordinate transformation, thereby outputting the output coordinates one pixel at a time;
The mapping processing circuit includes:
a circuit device which performs said mapping process sequentially for each pixel based on said output coordinates; 2. The circuit device according to claim 1,
When the input coordinates are (x, y), x is an integer between 0 and N, y is an integer between 0 and M, and N and M are integers of 2 or more,
The input coordinates sequentially input to the coordinate conversion circuit are (0,0), (1,0), (2,0), ..., (N-1,0) when y=0, then (0,1), (1,1), (2,1), ..., (N-1,1) when y=1, and thereafter y is increased sequentially and this is repeated up to (N-1,M-1). 3. The circuit device according to claim 1,
The coordinate conversion circuit includes:
A circuit device, comprising: coefficient information of the polynomial and the input coordinates; and determining the output coordinates from the input coordinates by the arithmetic processing based on the coefficient information. 4. The circuit device according to claim 3,
The coordinate conversion circuit includes:
an i-th term calculation circuit (n is an integer of 2 or more, i is an integer of 1 to n inclusive) that calculates an i-th term of a 1st to n-th term of the polynomial based on the coefficient information;
an addition circuit that adds the first to n-th terms output from the first to n-th term calculation circuits and outputs the output coordinate;
A circuit device comprising: 5. The circuit device according to claim 4,
The first to n-th term arithmetic circuits include
A circuit device which calculates the first to n-th terms in parallel. 4. The circuit device according to claim 3,
The coordinate conversion circuit includes:
a first calculation circuit for calculating x ² , y ² , and xy when the input coordinates are (x, y);
a second calculation circuit that determines the output coordinates based on the calculation result of the first calculation circuit and the coefficient information;
A circuit device comprising: 7. The circuit device according to claim 6,
The polynomial includes, as first to sixth terms, a1×x ² , a2×y ² , a3×xy, a4×x, a5×y, and a6;
The coefficient information includes information on a1, a2, a3, a4, a5, and a6 as the first to sixth coefficients,
The second arithmetic circuit includes:
a first term calculation circuit that multiplies the a1 by the ^x2 ;
a second term calculation circuit that multiplies the a2 by the ^y2 ;
a third term calculation circuit that multiplies the a3 by the x and y;
a fourth term calculation circuit that multiplies the a4 by the x;
a fifth term calculation circuit that multiplies the a5 by the y;
an adder circuit that adds the first to fifth terms output by the first to fifth arithmetic circuits and the sixth term a6, and outputs the output coordinate;
A circuit device comprising: 8. The circuit device according to claim 1,
a first coordinate counter that outputs a first count coordinate, which is a pixel unit coordinate based on a pixel clock;
The coordinate conversion circuit includes:
A circuit device which performs the coordinate transformation using the first count coordinate as the input coordinate. 8. The circuit device according to claim 7,
the input coordinates are coordinates corresponding to coordinates on one of the first image and the second image,
A circuit device, characterized in that the output coordinates are coordinates corresponding to coordinates on the other image of the first image and the second image. 9. The circuit device according to claim 8,
The mapping processing circuit includes:
a second coordinate counter that outputs a second count coordinate, which is a pixel unit coordinate based on the pixel clock;
an image memory for storing the first image;
a memory control circuit for controlling access to the image memory;
Including,
The memory control circuit includes:
performing the mapping process by writing the first image into the image memory based on the second count coordinates, and outputting pixel values read from the image memory based on the output coordinates as pixel values at the first count coordinates of the second image; or
A circuit device characterized in that the mapping process is performed by writing a pixel value at the first count coordinate of the first image to the image memory as a pixel value at the output coordinate, and outputting a pixel value read from the image memory based on the second count coordinate as a pixel value at the second count coordinate of the second image. 2. The circuit device according to claim 1,
a first coordinate counter that outputs a first count coordinate, which is a pixel unit coordinate based on a pixel clock;
The rotation conversion circuit includes:
A circuit device comprising: a first count coordinate unit that performs the rotation transformation to obtain the rotated coordinate unit. 12. The circuit arrangement according to claim 11,
the first count coordinates are coordinates that specify coordinates on one of the first image and the second image,
A circuit device, characterized in that the output coordinates are coordinates that specify coordinates on the other image of the first image and the second image. 12. The circuit arrangement according to claim 11,
The mapping processing circuit includes:
a second coordinate counter that outputs a second count coordinate, which is a pixel unit coordinate based on the pixel clock;
an image memory for storing the first image;
a memory control circuit for controlling access to the image memory;
Including,
The memory control circuit includes:
performing the mapping process by writing the first image into the image memory based on the second count coordinates, and outputting pixel values read from the image memory based on the output coordinates as pixel values at the first count coordinates of the second image; or
A circuit device characterized in that the mapping process is performed by writing a pixel value at the first count coordinate of the first image to the image memory as a pixel value at the output coordinate, and outputting a pixel value read from the image memory based on the second count coordinate as a pixel value at the second count coordinate of the second image. 14. The circuit arrangement according to claim 1,
The circuit device, wherein the polynomial is ^a1x2 + ^a2xy2 +a3xy+a4xx+a5xy+a6. 14. The circuit arrangement according to claim 1,
a second mapping processing circuit that performs a second mapping process, which is an inverse mapping process of the mapping process, on the second image to generate a third image;
a comparison circuit that compares the first image with the third image and outputs a result of the comparison as information for detecting an error in the second image;
A circuit device comprising: An electronic device comprising the circuit device according to any one of claims 1 to 15. A moving object comprising the circuit device according to any one of claims 1 to 15.

Images