JP2002094977A - Image processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the capacity of a memory that constitutes an image processing unit and to simply correct misalignment of the optical axis of a video camera. SOLUTION: The image processing unit 10 is provided with the video camera 11, the memory 13 that stores its image pickup signal and a CPU 14 that applies a prescribed processing to the read image pickup signal, is also provided with a selection write means 12 that selects only a prescribed part of the image pickup signal in response to a selection command C from the CPU and writes only the selected image pickup signal S' to the memory 13.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus which is mounted on a vehicle and which is suitable for monitoring road conditions around the vehicle with a video camera.

[0002]

2. Description of the Related Art An image processing apparatus as described above detects that the inter-vehicle distance to a vehicle traveling in front is not sufficient and issues an alarm sound to a driver, or the driver loses a lane mark. It can also be used to emit an alarm sound to the driver when the vehicle starts running.

[0003] The image processing apparatus can be used not only when the vehicle is moving forward but also when the vehicle is moving backward to confirm the rearward direction. As described above, the image processing device has a function of capturing road conditions such as other vehicles around the vehicle and lane marks as images and further recognizing the road conditions. That is, image recognition.

For this image recognition, a video camera and
An essential component is a memory for temporarily storing an image signal from the video camera, and a CPU for reading the image signal from the memory and performing predetermined processing.

[0005]

When attention is paid to the memory among the above components in the image processing apparatus, the cost required for the memory is not inexpensive among the costs required for the entire apparatus. In this case, the cost required for the memory is generally proportional to the memory capacity. However, there is a first problem that the amount of data obtained by A / D conversion of the image pickup signal of the video camera is considerably large, so that the memory capacity becomes large, and it is difficult to reduce the cost required for the memory. .

On the other hand, there is the following problem. When attention is paid particularly to a video camera among the above components of the image apparatus, it is common that a deviation occurs in the optical axis of a stereo camera constituting the video camera. In order to correct the deviation of the optical axis, the amount of deviation of the optical axis is measured by using a test pattern at the time of shipment from a factory, and is manually corrected. However, it is not practical because it requires a lot of time and labor. Therefore, the optical axis shift amount is input to the CPU, and an operation of adding / subtracting the optical axis shift amount to, for example, position data on an image plane of a vehicle traveling ahead is performed. However, this causes a second problem that the load on the CPU increases and the image processing speed decreases.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image processing apparatus capable of reducing the memory capacity of a memory and correcting an optical axis shift without lowering the image processing speed. It is intended for.

[0008]

FIG. 1 is a block diagram showing the basic configuration of the present invention. In the figure, an image processing apparatus as a premise of the present invention includes a video camera 11 for capturing an image of an object OB, a memory 13 for storing an image signal S from the video camera 11, and an image signal S read from the memory 13. The image processing apparatus includes a CPU 14 that performs predetermined processing on the image processing apparatus. Note that the reference number 20 corresponds to the memory 13.
It is an arithmetic processing unit that includes an auxiliary circuit part (see FIG. 2) in addition to the CPU 14 and the like.

A feature of the present invention to be noted here lies in the selective writing means 12 inserted between the video camera 11 and the memory 13. This selective writing means 13 is
In response to a selection command C from the camera 3, only a predetermined portion of the imaging signal S from the video camera 11 is selected, and only the selected imaging signal S ′ is written in the memory 13. .

Conventionally, there is no such selective writing means 12, and all the imaging signals S from the video camera 11 are A / D
After the conversion, the image data was directly stored in the memory 13. For this reason, the above-described first problem has occurred.
Conventionally, the above-described optical axis shift occurring in the video camera 11 is stored in advance in, for example, the ROM 15 and the optical axis shift amount is added / subtracted for each pixel by the CPU 14 with respect to the image data. , The optical axis deviation correction was executed. For this reason, the above-described second problem has been caused.

However, according to the present invention, the selection command C
Only the selected image pickup signal S ′ necessary for a predetermined process is stored in the memory
, The memory capacity is small (first
Problem). Further, only by giving the selection command C based on the data (optical axis shift amount) of the ROM 15 from the CPU 14 to the selective writing means 12, the selected image pickup signal S ′ from the image pickup area equivalent to the correction of the optical axis shift is stored in the memory 13. (Solution of the second problem).

A specific mode of the selection command C will be described later.

[0013]

FIG. 2 is a diagram showing a specific example of the basic configuration shown in FIG. In the figure, the image processing apparatus 10
Is a video camera 11 and an arithmetic processing unit 20 as shown in FIG.
Consists of The video camera 11 includes a lens 21, an image sensor 22, and the like. The video camera 11 is mounted, for example, at a rearview mirror in the vehicle and / or near the center of the rear window of the vehicle.

The analog image pickup signal S from the video camera 11 is input to an arithmetic processing unit 20, where the signal is first amplified by an amplifier 23, and then the signal is further converted to an ADC (Analog).
/ Digital Converter) for converting at 24 into a digital image signal S _d. Here, the selective writing means 12 of FIG.
This is shown as a memory I / F control circuit 25. The memory I / F control circuit 25 controls the writing of data (imaging signal S _d ) to the memory 13 under the selection command C from the CPU 14. It also controls reading to the CPU 25.
The circuit 25 also performs other controls.

FIG. 3 is a diagram schematically showing an outline of the operation of the selective writing means 12. As shown in the figure, selective writing means 12, in response to the selection command C from the CPU 14, corresponding to area a necessary a area in the imaging plane 26 defined by the imaging signal S _d predetermined processing The selected imaging signal is selected as a selected imaging signal S '.

FIG. 4 is a perspective view for explaining the deviation of the optical axis of the stereo camera. This figure relates particularly to the second problem already described. In the figure, the above-mentioned video camera 11 comprises a stereo camera 31 and is composed of a left camera and a right camera. The left camera (same for the right camera)
An image sensor 22L (22R) composed of a sensor chip formed on a camera substrate 32L (32R), and a lens 21L (21R) that forms an image of the vehicle V ahead on the image sensor 22L (22R).

The stereo camera 31 also includes a mechanical mechanism, so that the optical axis 33 is shifted, for example, in at least one direction indicated by arrows in the vertical and horizontal directions in the figure. FIG. 5 is a graph showing the distance measurement error due to the optical axis shift in an easily understandable manner. The vertical axis of this graph indicates the pixel shift amount due to the optical axis shift, and the horizontal axis indicates the distance measurement error (m) due to the pixel shift.

According to this graph, it can be understood that a shift of only one pixel causes an extremely large error of 11.0 to 13.0 m. Conventionally, in order to correct this optical axis shift, the optical axis shift amount (pixel shift amount) is recorded in the ROM 15 at the time of shipment from the factory, and the CPU 14
Have added correction to the distance measurement data by addition / subtraction operations. But this is CPU14
However, there has been a problem that the load on the device significantly increases and the processing time increases (as described above).

Therefore, the present invention changes the idea. Referring to FIG. 3, the position of the area "a" is entirely shifted by the amount of the optical axis shift with respect to the imaging surface 26, and the image data ( If S _d ) is stored as the selected image pickup signal S ′ in the memory 13, the CPU 14 can output correct distance measurement data without worrying about the optical axis shift amount.

FIG. 6 is a diagram showing how the optical axis shift is corrected by shifting the position of the area a. Reference number 34
Denotes a display, for example, a display of a navigation device is used. In the display 34, the above-described imaging surface 26 shown in FIG. 3 appears, and an area a according to the present invention is set therein.

According to the present invention, a shift is made in at least one direction of each arrow shown in FIG. 6 so as to compensate for the shift corresponding to the up, down, left and right arrows indicating the optical axis shift shown in FIG. An area a is defined, and a selected imaging signal S ′ corresponding to the area a is stored in the memory 13. The CPU 14 reads the selected image pickup signal S 'corresponding to the shifted area a from the memory 13 and performs a predetermined process, that is, a distance measurement operation. This is done according to the well-known principle of triangulation.

In summary, the selective writing means 12 according to the present invention stores the optical axis shift amount obtained by measuring the shift of the optical axis 33 of the stereo camera 31 constituting the video camera 11 in advance in, for example, the ROM 15. Then, an area a necessary for a predetermined process such as a distance measurement is set to the imaging surface 26 according to the optical axis shift amount.
It is characterized in that it is shifted within. The selection command C is issued by the CPU 14 as not only the above-described command for the optical axis deviation correction (first selection command) but also each command corresponding to various purposes. Some examples of the latter will be described.

FIG. 7 is a diagram schematically showing the function of the second selection command. The selection writing means 12 outputs the second selection command C
Based on, in the imaging surface 26, selects the imaging signal S _d of the portion corresponding to the area a of the lower half, and selects the imaging signal S 'it. That is, of the 640 × 480 pixels, the lower half of 640 × 480 pixels shot by the video camera 11
Only 240 pixels are stored in the memory 13.

This is because the information of only that part is sufficient for recognizing the vehicle V. FIG. 8 is a diagram schematically showing the function of the third selection command. Based on the third selection command C, the selection writing unit 12 selects the imaging signal S ′ based on an imaging signal obtained by thinning out pixels in the horizontal axis direction of the imaging surface 26 among a plurality of pixels constituting the imaging surface 26. And

That is, of the 640 × 480 pixels, the pixels on the horizontal axis are thinned out, and only 320 × 480 pixels are stored in the memory 13. In the image processing device of the vehicle, it is important to consider the surrounding situation where the vehicle V is placed, particularly the situation of night / day, when performing image recognition, and filter processing according to night / day is performed. .

[0026] The information in this out of the imaging signal S _d in the night / day of judgment "empty" (sky) is particularly important. For this reason, the pixels in the vertical axis direction are 480 pixels that can sufficiently cover “sky”, whereas the information in the horizontal axis direction (640 pixels) of “sky” has a high degree of redundancy. FIG.
FIG. 9 is a diagram schematically showing a function of a fourth selection command.

The selection writing means 12 outputs a fourth selection command C
, The image pickup signal obtained by thinning out the pixels in the vertical axis direction of the image pickup surface 26 among the plurality of pixels constituting the image pickup surface 26 is set as a selected image pickup signal S ′. That is, 640 ×
Of the 480 pixels, the pixels on the vertical axis are decimated to −640 ×
Only 240 pixels are stored in the memory 13.

This fourth selection command can also eliminate the redundancy in the vertical direction of “empty” as in the third selection command. FIG. 10 is a diagram schematically showing the function of the fifth selection command. Based on the fifth selection command C, the selective writing unit 12 corresponds to the left and right areas a1 and a2 when the imaging surface 26 is divided into the central area a0 and the left and right areas a1 and a2 located on the left and right sides thereof. The image pickup signals of the portions are respectively selected, and these are set as a selected image pickup signal S '.

That is, of the 640 × 480 pixels, only the areas a1 and a2 (160 × 2 × 480 pixels) where the lane marks on the horizontal axis are present are stored in the memory 13.
When the vehicle V starts running off the lane mark (white line) 35 of the road and issues an alarm (OUT) from the CPU 14 to the driver, it is sufficient to have information on only the lane mark 35. is there.

FIG. 11 is a diagram schematically showing the function of the sixth selection command. Based on the sixth selection command C, the selective writing means 12 corresponds to an upper image signal of a portion corresponding to a predetermined area a3 in the upper half of the imaging surface 26 and a predetermined area a4 in the lower half thereof. And the lower image pickup signal to be selected, and these are used as a selected image pickup signal S '.

That is, out of 640 × 480 pixels,
The brightness is determined using the 20 pixels of the "sky" portion, and the actual recognition of the vehicle V is set to 100 pixels.
Only 512 × 120 pixels are stored in the memory 13.
To determine the night / day mentioned above from the "sky" part,
It is based on the experimental fact that about 20 pixels in the "sky" may be used, and about 100 pixels may be used to recognize the presence of the vehicle V.

Here, the third and fifth selection commands C (FIGS. 8 and 10) will be supplemented. The video camera 11 usually employs an interlaced system, and the imaging surface 2
It is relatively easy to thin out the pixels in the vertical axis direction of No. 6.
For example, the signal of only the signal S _d or even rows of the odd rows only S
_Just select _d . However, some means is further required to thin out the pixels in the horizontal axis direction as in the third and fifth selection commands C. For example, a counter. The details are as follows.

A counter function (not shown) is provided in the arithmetic processing unit 20 (FIG. 2), and every time when a predetermined count value is reached among the imaging signals S from the imaging device 22 in a row unit (vertical direction). The selected imaging signal is read out from the memory 13 as an imaging signal S ′ in column units (horizontal axis direction).
The read operation is performed by, for example, the memory I / F (interface) control circuit 25 in FIG. For example, in the case of FIG. 8, the predetermined count value is “2 → 4 → 6 →... → 640” (even number).
In order to widen the thinning interval, the interval of the predetermined count value may be increased, for example, every three or four.

[0034]

According to an image processing apparatus for capturing a road condition such as a front and a back by a video camera and processing the image, the distance between the vehicle and a preceding vehicle is measured.
The present apparatus performs an operation based on an input signal from a video camera. In this case, the video camera signal needs to be temporarily stored in a memory. In this case, for example, the number of data is 6
In order to store the entire area of 40 × 480 (pixels) × 8 (bits), a large-capacity memory of a little over 2M bits is required.

However, with low-cost hardware, only a memory having a relatively small capacity can be mounted, so that a minimum necessary area is extracted from the video camera signal. Thus, by storing only the area necessary for image processing in the memory, the performance can be sufficiently satisfied while the hardware cost is kept low. As one application example, by using for correcting the optical axis deviation of a stereo camera,
A special effect of increasing the processing speed of the CPU can also be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of the present invention.

FIG. 2 is a diagram showing a specific example of the basic configuration shown in FIG.

FIG. 3 is a diagram schematically showing an outline of an operation of a selective writing means 12.

FIG. 4 is a perspective view for explaining a deviation of an optical axis of the stereo camera.

FIG. 5 is a graph showing a distance measurement error due to an optical axis shift in an easily understandable manner.

FIG. 6 is a diagram showing a state where an optical axis shift is corrected by shifting a position of an area a.

FIG. 7 is a diagram schematically showing a function of a second selection command.

FIG. 8 is a diagram schematically showing a function of a third selection command.

FIG. 9 is a diagram schematically showing a function of a fourth selection command.

FIG. 10 is a diagram schematically showing a function of a fifth selection command.

FIG. 11 is a diagram schematically showing a function of a sixth selection command.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus 11 ... Video camera 12 ... Selective writing means 13 ... Memory 14 ... CPU 15 ... ROM 20 ... Arithmetic processing part 21 ... Lens 25 ... Memory I / F control circuit 26 ... Imaging surface 31 ... Stereo camera 33 ... Light Axis S: Image signal S ': Selected image signal C: Selection command

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) // H04N 13/02 H04N 13/02

Claims (8)

[Claims] 1. An image processing apparatus comprising: a video camera that captures an image of an imaging target; a memory that stores an imaging signal from the video camera; and a CPU that performs predetermined processing on the imaging signal read from the memory. In the above, further comprising a selection writing means for selecting only a predetermined portion of the imaging signal from the video camera in response to a selection command from the CPU, and writing only the selected imaging signal to the memory. Image processing device. 2. In accordance with the selection command, an image signal of a portion corresponding to an area in an image plane defined by the image signal and required for the predetermined processing is selected.
2. The image processing apparatus according to claim 1, wherein the selected signal is used as the selected image pickup signal. 3. An optical axis shift amount obtained by measuring a shift of an optical axis of a stereo camera constituting the video camera in advance, and an area necessary for the predetermined processing is determined according to the optical axis shift amount. The image processing apparatus according to claim 2, wherein the shift is performed within the imaging plane. 4. The image processing apparatus according to claim 2, wherein an image pickup signal of a portion corresponding to a lower half area of the image pickup surface is selected, and the selected image pickup signal is used as the selected image pickup signal. 5. The selected image pickup signal according to claim 2, wherein an image pickup signal obtained by thinning out pixels in a horizontal axis direction of the image pickup plane among a plurality of pixels constituting the image pickup plane is used as the selected image pickup signal. The image processing apparatus according to any one of the preceding claims. 6. The selected image pickup signal according to claim 2, wherein, among the plurality of pixels constituting the image pickup plane, the image pickup signal obtained by thinning out pixels in the vertical direction of the image pickup plane is used as the selected image pickup signal. The image processing apparatus according to any one of the preceding claims. 7. When the imaging surface is divided into a central area and a left area and a right area located on the left and right sides thereof, an imaging signal of a portion corresponding to the left and right areas is selected, and these are selected by the selected imaging signal. The image processing apparatus according to claim 2, wherein 8. An upper imaging signal of a portion corresponding to a predetermined area in an upper half of the imaging surface and a lower imaging signal of a portion corresponding to a predetermined area in a lower half thereof are respectively selected. The image processing apparatus according to claim 2, wherein these are used as the selected image pickup signal.