JP6694695B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to a technique for processing an image.

  2. Description of the Related Art Conventionally, there has been known an image processing device that photographs the rear of the vehicle with a camera installed in the vehicle and displays the photographed image on a display in the vehicle. The driver can drive the vehicle in the reverse direction while referring to the image showing the rear of the vehicle displayed on the display and checking obstacles and the like in the rear. When an abnormality occurs in a microcomputer that performs image processing, such an image processing apparatus immediately performs reset processing and quickly recovers the processing (for example, Patent Document 1). If reset processing is not performed for the microcomputer in which an abnormality has occurred, the image displayed on the display is not updated, and the driver does not notice that the obstacle and the vehicle are approaching, This is because there is a risk of doing it.

JP, 2014-197370, A

  However, if the microcomputer that performs image processing is reset, no image signal is generated during the reset processing, and there is no image displayed on the display. If the microcomputer is reset and the image suddenly disappears from the display while the vehicle is in the reverse drive, the driver needs a new safety confirmation, which hinders the reverse drive.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for resetting a microcomputer in which an abnormality has occurred, without impairing driving of a vehicle.

In order to solve the above-mentioned problems, the invention of claim 1 is an image processing apparatus for processing an image, comprising: shift detecting means for detecting a shift position of a vehicle; The installed camera acquires an imaged image of an object around the vehicle, outputs the imaged image to a display device, and controls the image processing means, and the display device. Synthesizing means for synthesizing a figure with the captured image before output to the synthesizing means, abnormality detecting means for detecting an abnormality in the synthesizing means, and reset processing for the synthesizing means in response to the abnormality detection by the abnormality detecting means. and a reset means for performing said image processing means, the abnormality if the detection means has detect the abnormality is displayed by outputting front of the captured image synthesizing graphic on the display device, wherein When the abnormality detecting unit detects the abnormality while the shift position is the predetermined position, the reset unit does not perform the reset process while the shift position is the predetermined position, and the shift position is An image processing apparatus, characterized in that the reset process is performed after the device is displaced from a predetermined position.

  The invention according to claim 2 is the image processing device according to claim 1, wherein the predetermined position of the shift position is reverse.

  According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect, the reset is performed after the abnormality detection unit detects the abnormality and before the reset unit performs the reset process. An image processing apparatus, further comprising a notification unit that notifies a user in advance that processing will be performed.

  According to a fourth aspect of the invention, in the image processing apparatus according to any one of the first to third aspects, the shift detecting means uses the shift for control after a predetermined time has elapsed after the shift position is changed. When the control shift position that is the position is determined and the shift position changes during the reset process, the control shift position is determined at the end of the reset process regardless of the elapse of the predetermined time. Image processing device.

According to a fifth aspect of the present invention, there is provided an image processing method for processing an image, comprising: (a) detecting a shift position of the vehicle; and (b) the image processing means, wherein the shift position is a predetermined position. a camera installed in the vehicle obtains a captured image obtained by capturing an image of a subject around the said vehicle, to control the process of displaying and outputs the captured image to the display device, (c) a said image processing means The synthesizing means synthesizes a graphic on the captured image before output to the display device, (d) detects an abnormality in the synthesizing means, and (e) detects the abnormality in the step (d). in response to, and performing a reset process to said synthesis means, (f) if the step (d) was detect the abnormality, the imaging before the image processing means to synthesize the graphic on the display device A step of outputting and displaying an image, wherein the step (e) is such that if the step (d) detects the abnormality while the shift position is the predetermined position, the shift position is the predetermined position. The image processing method is characterized in that the reset process is not performed during the position, and the reset process is performed after the shift position deviates from the predetermined position.

  According to the inventions of claims 1 to 5, since the synthesizing means is reset after the shift position deviates from the predetermined position, the synthesizing means and the synthesizing means are controlled while the vehicle is in the same shift state. It is possible to prevent the means and the device from being reset and continuously display the captured image.

  Further, according to the invention of claim 2, in particular, since the resetting process is performed on the synthesizing means after the shift position deviates from the position for moving the vehicle backward, the image processing means controlled by the synthesizing means and the synthesizing means while the vehicle is moving backward You can prevent and from being reset.

  Further, according to the invention of claim 3, the user can understand that the resetting process is performed on the synthesizing unit before the resetting process is started, so that even if the resetting process is started, there is no confusion.

  Further, in particular, according to the invention of claim 4, since part or all of the predetermined time has elapsed during the reset process, it can be estimated that the shift position has changed after the reset process is completed. By displaying the captured image in which the graphic is combined on the display device at the end of the reset process, the user can immediately refer to the captured image in which the graphic is combined after the reset process.

FIG. 1 shows an outline of an image processing system. FIG. 2 is a block diagram showing the configuration of the image processing system. FIG. 3 shows a display example of the display. FIG. 4 is a flowchart showing the processing steps of the video decoder. FIG. 5 is a flowchart showing the processing steps of the multimedia microcomputer. FIG. 6 is a flowchart showing the processing steps of the system control microcomputer. FIG. 7 is a time chart showing the transition of the display on the display. FIG. 8 is a time chart showing the transition of the display on the display. FIG. 9 is a block diagram showing the configuration of the image processing system according to the second embodiment. FIG. 10 is a flowchart showing the processing steps of the video decoder according to the second embodiment. FIG. 11 is a time chart showing a transition of display on the display according to the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. Overview>
FIG. 1 shows an outline of an image processing system 1 according to the first embodiment. The image processing system 1 is a system in which a camera 4 mounted on a vehicle 2 photographs the periphery of the vehicle 2, for example, the rear side, and an image obtained by the photographing is displayed on a display 5 inside the vehicle. A guide line GL indicating the vehicle width and the distance from the rear end of the vehicle is combined with the image obtained by photographing, so that the driver can grasp the distance and the possibility of contact with the obstacle. When driving the vehicle 2 in the backward traveling direction DR while referring to such a composite image CP, the driver accurately grasps the position of the obstacle around the vehicle 2, especially in the blind spot area. , You can drive more safely.

  Since a microcomputer that generates such a composite image CP requires complicated arithmetic processing, the processing may be abnormal. The abnormal microcomputer cannot perform image processing anymore and cannot output an accurate image. For example, the same image may be continuously output. Therefore, the image processing system 1 performs a reset process on the microcomputer having an abnormality in the process to restore the normal state.

  However, if the microcomputer is reset during driving, the image on the display 5 suddenly disappears, and the driver may be confused and hinder driving operation. Further, if the same image is continuously displayed on the display 5 without performing the reset process, the driver does not notice the approach of the obstacle 2 to the vehicle 2 or the entry in the traveling direction DR, and the obstacle with the vehicle 2 is not noticed. There is a risk of contact with objects.

  Therefore, the image processing system 1 according to the present invention does not immediately perform the reset process even if an abnormality occurs in the microcomputer that generates the composite image CP in this manner, and replaces the composite image CP until the shift position is changed. The captured image PP before combining the guide lines GL is displayed on the display 5. As a result, even if the composite image CP cannot be displayed, the display 5 does not enter the image disappearance state during driving, and the same image is not continuously displayed on the display 5. Therefore, the driver can continue the driving operation without being confused by the sudden disappearance of the image while accurately referring to the obstacle around the vehicle 2 with the captured image PP. Hereinafter, such an image processing system 1 will be described in detail.

<1-2. Composition>
FIG. 2 is a block diagram showing the configuration of the image processing system 1, particularly the configuration of the image processing apparatus 3. The image processing system 1 includes an image processing device 3 mounted on a vehicle 2, a camera 4, a display 5, a shift position sensor 6, and a peripheral device 7.

  The vehicle 2 is a transportation device such as an automobile. The vehicle 2 is, in addition to an automobile, a railroad, a ship, an aircraft, or the like. Any device operated by the driver may be used.

  The image processing device 3 is an electronic control device that is installed in the vehicle 2 and causes the display 5 to display images captured by the camera 4 and images for various devices mounted on the vehicle. The electronic control device includes an antenna that wirelessly communicates with an external device. The image processing device 3 includes a video decoder 32, a multimedia microcomputer 31, and a system control microcomputer 33. Functions of the video decoder 32, the multimedia microcomputer 31, and the system control microcomputer 33 will be described later.

  The camera 4 is, for example, a video camera that is installed in the rear of the vehicle 2 so as to face rearward and captures a subject behind the vehicle to generate an image signal.

  The display 5 is an image display device that displays an image signal of a still image or a moving image output from the image processing device 3. For example, a liquid crystal display (LCD) or a plasma display (PDP). The display 5 functions as a display device in the present invention.

  The shift position sensor 6 is a sensor that detects the position of a shift (not shown) that changes gears. The shift position is, for example, parking (P), reverse (R), neutral (N), drive (D), or the like. When the shift position sensor 6 detects the shift position, the shift position sensor 6 transmits a shift signal SS indicating the detected shift position to the system control microcomputer 33.

  The peripheral device 7 is various devices (multimedia) installed in the vehicle 2. The peripheral device 7 is, for example, a radio tuner 71, a CD player 72, a navigation 73, a non-volatile memory 74, or the like. The peripheral device 7 is controlled by the multimedia microcomputer 31. The multimedia microcomputer 31 generates, for example, an audio operation screen for the radio tuner 71 or the CD player 72, sends it to the video decoder, and displays it on the display. The multimedia microcomputer 31 also generates a map image for navigation and executes route guidance. The multimedia microcomputer 31 also stores various data input by the user in the nonvolatile memory 74. For example, the frequency of the radio station selected by the user operating the radio tuner 71, the place of the destination input to the navigation 73, and the like.

  Next, the functions of the multimedia microcomputer 31, the video decoder 32, and the system control microcomputer 33 will be described.

  The multimedia microcomputer 31 is a microcomputer that includes a CPU, a RAM, and a ROM (not shown) and controls the video decoder 32 and the peripheral device 7. By concentrating the control of the video decoder 32 and the peripheral device 7 in the multimedia microcomputer 31, the image processing device 3 can be configured with a simple mechanism. Further, the multimedia microcomputer 31 synthesizes the guide line GL for driving support with the image captured by the camera 4 transmitted from the video decoder 32. The multimedia microcomputer 31 includes a media control unit 31a and a guide line synthesizing unit 31b. The multimedia microcomputer 31 functions as a synthesizing means in the present invention. Further, the guide line GL functions as a figure in the present invention. In the present invention, the composite image functions as a captured image in which graphics are composited.

  The media control unit 31a controls the video decoder 32 and the peripheral device 7. For example, the frequency and volume of the radio tuner 71 are adjusted based on the user's operation. The media control unit 31a also generates a display screen for controlling the peripheral device 7. For example, the music selection screen of the CD player 72 or the map screen of the navigation 73. The media control unit 31a transmits the generated display screen to the video decoder 32.

  The guide line synthesizing unit 31b synthesizes the guide line GL for driving assistance with the captured image of the rear of the vehicle 2 captured by the camera 4. The guide line GL is previously stored as image data in a memory (not shown). When the captured image is received from the video decoder 32, the guide line synthesis unit 31b synthesizes the guide line GL and sends it back to the video decoder 32.

  The video decoder 32 is an integrated circuit that converts an image signal transmitted from the camera 4 and the multimedia microcomputer 31 into an image signal that can be displayed on the display 5. The video decoder 32 is built in and arranged below the multimedia microcomputer 31. Therefore, when the multimedia microcomputer 31 is reset, the video decoder 32 is reset together. The video decoder 32 includes a transmission / reception unit 32a, a decoding unit 32b, a display unit 32c, a sticking detection unit 32d, a caution unit 32e, and a shift detection unit 32f. The video decoder 32 functions as an image processing means in the present invention.

  The transmission / reception unit 32a has a reception function of receiving an image signal transmitted from the camera 4 and the multimedia microcomputer 31.

  The decoding unit 32b is a decoder that decodes the image signal transmitted from the camera 4 and converts it into a signal that can be displayed on the display 5.

  The display unit 32c transmits the composite image signal CS and the normal image signal OS transmitted from the multimedia microcomputer 31, and the captured image signal PS converted by the decoding unit 32b to the display 5, and causes the display 5 to display each image. This is a display control function.

  The display unit 32c causes the display 5 to display an image corresponding to the shift position. That is, the display unit 32c displays an image showing the rear of the vehicle if the shift position is reverse, displays a map image if it is a drive, and displays an audio screen if it is parking.

  The photographed image signal PS is an image signal generated by the camera 4 photographing a subject behind the vehicle 2. When the captured image signal PS is transmitted to the display 5, the display 5 displays a captured image PP that is an image of a subject behind the vehicle 2. However, the captured image signal PS includes the image signal converted by the decoding unit 32b.

  Further, the composite image signal CS is an image signal generated by the multimedia microcomputer 31 by combining an image showing the guide line GL with the captured image signal PS. When the composite image signal CS is transmitted to the display 5, the display 5 displays a composite image CP in which the image of the subject behind the vehicle 2 is combined with the image of the guide line GL.

  The normal image signal OS is an image signal for operating the peripheral device 7 generated by the multimedia microcomputer 31 for the driver to operate the peripheral device 7. For example, it is an audio operation image for operating the radio tuner 71 and the CD player 72. It is also a map image for navigation. When the normal image signal OS is transmitted to the display 5, the display 5 displays the normal image OP such as the operation screen of the peripheral device 7.

  The sticking detection unit 32d detects an abnormality of the multimedia microcomputer 31. That is, the sticking detection unit 32d detects whether or not there is an abnormality in the composite image CP based on the composite image signal CS transmitted from the multimedia microcomputer 31. The sticking detection unit 32d functions as an abnormality detection unit in the present invention.

  Since the multimedia microcomputer 31 controls a plurality of peripheral devices 7 at the same time, data processing becomes complicated. Therefore, the multimedia microcomputer 31 is more likely to cause an abnormality than other microcomputers. The multimedia microcomputer 31 in which the abnormality has occurred may output a fixed image (the same image with no movement).

  The driver who drives the vehicle 2 in the reverse direction generally does not stare at the image on the display 5 and visually confirms the safety in various directions. Therefore, even if the multimedia microcomputer 31 outputs the fixed image, the driver does not immediately notice that the image is fixed. In this case, the driver cannot recognize the approaching or approaching of the obstacle behind the vehicle 2 from the image on the display 5, and the vehicle 2 may come into contact with the obstacle.

  The sticking detection unit 32d determines whether or not the image is a stuck image by judging a change in color of pixel data at predetermined nine points in the image. If the color of the pixel data does not change, it can be determined that the image is a fixed image with no movement. When the sticking detection unit 32d determines that there is sticking in the composite image CP, it sends a sticking signal AS indicating that the composite image CP has sticking to the system control microcomputer 33.

  The caution unit 32e notifies the driver of the vehicle 2 of the occurrence of the abnormality and the reset processing of the system when the sticking detection unit 32d detects the abnormality of the multimedia microcomputer 31 based on the composite image CP. The caution unit 32e combines the notification image indicating that the abnormality occurs and the reset process is performed with the shadow image captured by the camera 4, and gives the notification. The caution unit 32e functions as a notification unit in the present invention.

  FIG. 3 is an example of the notification image CA generated by the caution unit 32e. The notification image CA is image data showing characters that inform the driver that an abnormality has occurred in the system and that the system will be reset. For example, "CAUTION! An error has occurred in the camera system. The system will be reset after the shift position is released from reverse."

  The notification image CA is stored in a memory (not shown) and is read by the caution unit 32e when the sticking detection unit 32d detects an abnormality of the multimedia microcomputer 31. When reading the notification image CA, the caution unit 32e superimposes the notification image CA on the captured image PP and combines them. The captured image PP combined with the notification image CA is displayed on the display 5 by the display unit. As a result, the driver can understand the occurrence of the system abnormality and the fact that the system is reset while referring to the state behind the vehicle 2.

  With reference to FIG. 2 again, the shift detector 32f and the subsequent parts will be described. The shift detector 32f receives the shift signal SS indicating the shift position transmitted from the shift position sensor 6 and detects the input shift position. For example, when the shift signal SS indicates reverse, the shift detector 32f detects the input shift position as reverse. Further, the shift detection unit 32f determines that the shift position has moved after a predetermined period (chattering prevention period) has elapsed since the shift position moved. When the shift position is moved, the shift detection unit 32f may instantaneously detect the other shift position (so-called chattering) because it may pass through the other shift position before moving to the desired shift position. Becomes Therefore, even if the shift position is detected, it is not immediately determined that the shift position has moved. If the shift position continues for a predetermined chattering prevention period, it is determined that the shift position has moved. That is, the shift position used for control (control shift position) is determined. The chattering prevention period is, for example, 500 msec. The shift detection unit 32f functions as a shift detection unit that detects the shift position of the vehicle 2 in the present invention.

  The system control microcomputer 33 is a microcomputer that includes a CPU, a RAM, and a ROM (not shown) and manages the power supply of the image processing device 3 and the peripheral device 7. The system control microcomputer 33 includes a shift detection unit 33b and a reset unit 33a.

  The shift detector 33b receives the shift signal SS indicating the shift position transmitted from the shift position sensor 6, and detects the input shift position. For example, when the shift signal SS indicates reverse, the shift detector 33b detects the input shift position as reverse.

  The reset unit 33a transmits a reset signal RS to the multimedia microcomputer 31 and resets the multimedia microcomputer 31. Further, the reset unit 33a performs a reset process by transmitting a reset signal RS to the peripheral device 7 via a signal line (not shown). The reset process is forcibly returning the registers of the CPU of the multimedia microcomputer 31 to the initial state. The reset processing period is about 1.5 sec. Note that the reset unit 33a may perform the reset process by turning off and on the power to the multimedia microcomputer 31 and the peripheral device 7 instead of transmitting the reset signal RS. The reset unit 33a functions as a reset unit in the present invention.

<1-3. Process>
Next, the processing steps of the image processing apparatus 3 will be described. FIG. 4 is a flowchart showing processing steps of the video decoder 32 included in the image processing apparatus 3. The processing steps of the video decoder 32 are repeatedly executed at a predetermined cycle.

  When the process is started, the transmission / reception unit 32a acquires the captured image signal PS captured and transmitted by the camera 4 (step S101). When the transmitting / receiving unit 32a acquires the captured image signal PS, the decoding unit 32b converts the captured image signal PS into a format that can be displayed on the display 5.

  Next, the shift detector 32f determines whether the shift position is reverse (step S102). As described above, the shift detection unit 32f determines the shift position based on the shift signal SS transmitted from the shift position sensor 6.

  When the shift detection unit 32f determines that the shift position is not reverse (No in step S102), the transmission / reception unit 32a acquires the normal image signal OS transmitted by the multimedia microcomputer 31. When the transmission / reception unit 32a acquires the normal image signal OS, the display unit 32c transmits the normal image signal OS to the display 5 for display (step S112). As a result, the driver can refer to the radio channel selection screen or the map screen when the shift position is set to parking or driving.

  On the other hand, when the shift detector 32f determines that the shift position is reverse (Yes in step S102), the transmitter / receiver 32a transmits the captured image signal PS to the multimedia microcomputer 31 (step S103).

  When the transmission / reception unit 32a receives the composite image signal CS obtained by combining the captured image returned from the multimedia microcomputer 31 with the guide line, the sticking detection unit 32d causes an abnormality of the multimedia microcomputer 31, that is, the composite image CP is stuck. It is determined whether or not there is any by using the method described above (step S104).

  When the sticking detection unit 32d determines that sticking has not occurred in the combined image CP (No in step S104), the display unit 32c transmits the combined image signal CS to the display 5 to display the combined image CP (step S105). ). As a result, the driver can perform the reverse drive of the vehicle 2 while referring to the combined image CP in which the guide line GL is combined with the image behind the vehicle. When the display unit 32c causes the display 5 to display the composite image CP, the processing step of the video decoder 32 ends. In this case, after the predetermined period, the processes in and after step S101 are executed again.

  On the other hand, when the sticking detection unit 32d determines that sticking has occurred in the composite image CP (Yes in step S104), it sends the sticking signal AS to the system control microcomputer 33 (step S106).

  When the sticking detection unit 32d transmits the sticking signal AS, the caution unit 32e combines a notification image CA that notifies that the system has an abnormality and the reset process is performed (step S107).

  When the caution unit 32e combines the notification image CA with the captured image PP, the display unit 32c transmits a captured image signal PS indicating the captured image PP combined with the notification image CA to the display 5 (step S108). As a result, the driver can recognize the occurrence of the abnormality and the execution of the reset process by referring to the captured image PP on which the notification image CA is combined.

  When the display unit 32c transmits the captured image signal PS to the display 5, the video decoder 32 determines whether the reset signal RS is received from the multimedia microcomputer 31 (step S109).

  When the video decoder 32 does not receive the reset signal RS (No in step S109), the video decoder 32 repeatedly executes step S109 and waits until the reset signal RS is received. Since the reset signal RS is always transmitted as long as the fixation detection unit 32d is transmitting the fixation signal AS, the video decoder 32 may wait until the reset signal RS is transmitted.

  Upon receiving the reset signal RS (Yes in step S109), the video decoder 32 performs reset processing (step S110). That is, the video decoder 32 forcibly resets the register included in the CPU of the video decoder 32 to the initial state and performs the reset process.

  When the video decoder 32 finishes the reset process, the shift detector 32f determines whether the shift position is reverse (step S111).

  When the shift detection unit 32f determines that the shift position is reverse (Yes in step S111), the process returns to step S103. In this case, the processing of step S103 and thereafter is executed again.

  On the other hand, when the shift detection unit 32f determines that the shift position is not reverse (No in step S111), the display unit 32c transmits the normal image signal OS transmitted from the multimedia microcomputer 31 to the display 5, and the display 5 normally displays. The image OP is displayed (step S112). When the display unit 32c causes the display 5 to display the normal image OP, the processing process of the video decoder 32 ends.

  Next, the processing steps of the multimedia microcomputer 31 will be described. FIG. 5 is a flowchart showing the processing steps of the multimedia microcomputer 31. The processing steps of the multimedia microcomputer 31 are repeatedly executed at a predetermined cycle.

  When the process is started, the media control unit 31a controls the peripheral device 7 based on the operation of the driver (step S201).

  When the media control unit 31a controls the peripheral device 7, the guide line synthesizing unit 31b determines whether the captured image signal PS is received from the video decoder 32 (step S202).

  The process when the guide line synthesis unit 31b determines that the captured image signal PS is not received (No in step S202) will be described later. On the other hand, when determining that the captured image signal PS is received (Yes in step S202), the guide line synthesis unit 31b synthesizes the guide line GL with the captured image PP based on the captured image signal PS (step S203).

  When the guide line synthesizing unit 31b synthesizes the guide line GL with the captured image PP, the guide line synthesizing unit 31b transmits the synthesized image signal CS to the video decoder 32 (step S204).

  When the guide line composition unit 31b transmits the composite image signal CS to the video decoder 32, the multimedia microcomputer 31 determines whether or not the reset signal RS has been received from the system control microcomputer 33 (step S205).

  On the other hand, when the multimedia microcomputer 31 determines that the reset signal RS is received (Yes in step S205), the multimedia microcomputer 31 performs the reset process (step S206). That is, the multimedia microcomputer 31 forcibly resets the registers of the CPU of the multimedia microcomputer 31 to the initial state and performs the reset process. In addition, the multimedia microcomputer 31 transmits a reset signal RS to the video decoder 32 after performing the reset process.

  When the multimedia microcomputer 31 performs the reset process, when the guide line synthesizing unit 31b determines not to receive the captured image signal PS (No in step S202), and the multimedia microcomputer 31 receives the reset signal RS. When determining not to do so (No in step S205), the media control unit 31a generates the normal image OP for controlling the peripheral device 7 (step S207).

  After generating the normal image OP, the media control unit 31a transmits the normal image signal OS indicating the normal image OP to the video decoder 32 (step S208). When the media control unit 31a transmits the normal image signal OS, the processing process of the multimedia microcomputer 31 ends.

  Next, the processing steps of the system control microcomputer 33 will be described. FIG. 6 is a flowchart showing the processing steps of the system control microcomputer 33. The processing steps of the system control microcomputer 33 are repeatedly executed at a predetermined cycle.

  When the process is started, the system control microcomputer 33 supplies power to the multimedia microcomputer 31 and the peripheral device 7 (step S301).

  Next, it is determined whether the reset unit 33a has received the sticking signal AS transmitted from the video decoder 32 (step S302).

  When it is determined that the reset unit 33a does not receive the sticking signal AS (No in step S302), the processing process of the system control microcomputer 33 ends. However, the power supply executed in step S301 continues, and the multimedia microcomputer 31 and the peripheral device 7 can continue the processing.

  On the other hand, when the reset unit 33a determines that the sticking signal AS is received (Yes in step S302), it resets the peripheral device 7 (step S303). That is, the reset unit 33a shuts off the power supply to the peripheral device 7. By maintaining the reset state of the peripheral device 7, the peripheral device 7 can be restarted early when the shift position is out of reverse.

  When the reset unit 33a resets the peripheral device 7, the system control microcomputer 33 determines whether interrupt processing has occurred (step S304). The interrupt processing is, for example, memory management or the like accompanying the reset processing of the peripheral device 7.

  When the system control microcomputer 33 determines that the interrupt process has occurred (Yes in step S304), the system control microcomputer 33 executes the interrupt process (step S305).

  When the system control microcomputer 33 executes the interrupt process and when it is determined that the interrupt process does not occur (No in step S304), the shift detection unit 33b determines whether the shift position is reverse (step S306). ).

  When the shift detection unit 33b determines that the shift position is reverse (Yes in step S306), the reset unit 33a maintains the reset state of the peripheral device 7 (step S303), and the system control microcomputer 33 restarts the interrupt processing. It is determined (step S304). After that, the processing from step S303 to step S306 is repeatedly executed until the shift detection unit 33b determines that the shift position is not reverse.

  On the other hand, when the shift detection unit 33b determines that the shift position is not the reverse position (No in step S306), the reset unit 33a transmits the reset signal RS to the multimedia microcomputer 31 (step S307). As a result, the multimedia microcomputer 31 is reset after the shift position is out of reverse. Therefore, the video decoder 32 interlocked with the multimedia microcomputer 31 is also reset after the shift position is out of reverse. That is, while the shift position is reverse, the video decoder 32 can display the captured image PP on the display 5.

  When the reset unit 33a transmits the reset signal RS to the multimedia microcomputer 31, the reset unit 33a turns on the power of the camera 4 and the devices related to the camera 4 to restart them (step S308). The device related to the camera 4 is, for example, a device that stores the setting state of the operation of the camera 4.

  When the reset unit 33a restarts the camera 4 or the like, the peripheral unit 7 powers on and restarts. At this time, since the reset state of the peripheral device 7 is maintained by the reset unit 33a, the peripheral device 7 can be quickly started. Also, by preferentially restarting the camera 4 and the like for the peripheral device 7, it is possible to display images quickly. When the reset unit 33a turns on the peripheral device 7 and restarts it, the processing steps of the system control microcomputer 33 are completed.

<1-4. Image transition>
Next, the transition of the image displayed on the display 5 will be described. FIG. 7 is a time chart showing transitions of images displayed on the display 5, and shows a case where the reset process is immediately performed when an abnormality occurs in the multimedia microcomputer 31. The horizontal axis a represents the passage of time.

  If the reverse of the shift position is off, that is, the shift position is a position other than reverse and the multimedia microcomputer 31 is operating normally, the image displayed on the display 5 is an operation image of the peripheral device 7 or the like. This is the image OP (time a0).

  When the reverse of the shift position is turned on, that is, when the shift position is the reverse position, the image displayed on the display 5 is the combined image CP in which the guide line GL is combined with the image behind the vehicle 2 from the normal image OP (time). a1). As a result, the driver having the shift position in the reverse position can perform the reverse drive while referring to the rear of the vehicle 2 and the guide line GL. After the time a1, the sticking detection unit 32d determines whether the composite image CP is stuck.

  After that, when an abnormality occurs in the multimedia microcomputer 31, the sticking detection unit 32d transmits the sticking signal AS to the system control microcomputer 33 (time a2).

  The system control microcomputer 33 which has received the sticking signal AS transmits a reset signal RS to the multimedia microcomputer 31 (time a3). As a result, the multimedia microcomputer 31 and the video decoder 32 are reset. When the video decoder 32 is reset, the image signal is not transmitted to the display 5, and the display 5 enters the image disappearance state RP. At the time point of time a3, the shift position is still in the reverse position, and therefore the driver is in the middle of performing reverse driving. At this time, if the display 5 suddenly becomes the image disappearance state RP, the driver may be confused by an unexpected image state, which may hinder the reverse driving.

  On the other hand, when the abnormal condition of the multimedia microcomputer 31 is left as it is with the composite image CP fixed and the reset process is performed after the shift position is out of reverse, the backward operation may be hindered. That is, since the composite image CP is fixed while the vehicle 2 is moving backward, the driver may not notice the approach of the obstacle behind the vehicle 2. Further, an obstacle that has entered the rear of the vehicle 2 after the composite image CP is fixed is not reflected in the fixed composite image CP, so that the driver may not notice the entrance of the obstacle. Generally, a driver needs to confirm safety in various directions during reverse driving. For this reason, it is not easy for the driver to detect the fixation of the composite image CP during the reverse drive.

  When the reset process ends and the multimedia microcomputer 31 and the video decoder 32 return to the normal state, the composite image is displayed again on the display 5 (time a4).

  When the reverse drive of the vehicle 2 ends, the shift position becomes a position other than reverse, and the normal image OP is displayed on the display 5 instead of the composite image CP (time a5).

  FIG. 8 is a time chart showing the transition of images displayed on the display 5. The captured image PP is displayed while the multimedia microcomputer 31 is abnormal, and the reset processing is performed after the shift position is out of reverse. The following shows the case. The horizontal axis b represents the passage of time.

  Since the time b0 and the time b1 are the same as the time a0 and the time a1 described above, the description thereof will be omitted.

  After the shift position becomes reverse (time b1), if an abnormality occurs in the multimedia microcomputer 31, the video decoder 32 sends a sticking signal AS to the system control microcomputer 33. However, the reset process is not executed at this point (time b2). When the sticking detection unit 32d detects the sticking of the composite image CP, the display unit 32c switches the image to be transmitted to the display 5 from the composite image CP to the captured image PP in which the notification image CA is composited (time b2). As a result, the driver can refer to the rear of the vehicle 2 and can know the approaching or approaching of the obstacle, and there is no hindrance to the reverse driving.

  When the reverse operation of the vehicle 2 is completed, the shift position becomes a position other than reverse (time b3). At this point, the sticking detection unit 32d transmits a sticking signal AS to the system control microcomputer 33. The system control microcomputer 33 that has received the sticking signal AS resets the multimedia microcomputer 31, and further resets the video decoder 32. When the video decoder 32 is reset, the display 5 enters the image disappearance state RP (time b3), but since the notification image CA is displayed in advance to give notice of the abnormality occurrence and the reset processing, the driver: Even if the display on the display 5 is in the image disappearance state RP, there is no confusion. Further, since the shift position is out of reverse, there is no problem in reverse driving.

  When the reset process of the multimedia microcomputer 31 and the video decoder 32 is completed, the shift position is already out of reverse, so the normal image OP is displayed on the display 5 (time b4).

  As described above, in the image processing device 3 according to the first embodiment, when the multimedia microcomputer 31 becomes abnormal while the shift position of the vehicle 2 is the predetermined position such as reverse, the shift position is the predetermined position, For example, during the reverse, the reset process to the multimedia microcomputer 31 is not performed, and the reset process is performed after the shift position is out of the predetermined position. Further, the image taken by the camera 4 is displayed on the display 5 without the intervention of the multimedia microcomputer 31 until the reset process is performed after the multimedia microcomputer 31 becomes abnormal. Thereby, it is possible to prevent the image from suddenly disappearing from the display 5 during the driving operation. Although the driver cannot refer to the composite image generated by the multimedia microcomputer 31, the driver can continuously perform the driving operation by referring to the image captured by the camera 4.

<2. Second Embodiment>
<2-1. Overview>
In the first embodiment, when the reset process ends after the shift position is removed from reverse, the normal image OP is displayed on the display 5 (time b4 in FIG. 8). On the other hand, in the second embodiment, after the shift position is removed from reverse, the process is performed when the shift position is set to reverse again during the reset process.

  As described above, the shift detection unit 32f of the video decoder 32 determines that the shift position has moved after a predetermined time (chattering prevention period) has elapsed since the shift position moved. This prevents erroneous detection of the shift position due to so-called chattering.

  In the second embodiment, the chattering prevention period measured by the shift detection unit 32f after the reset process is completed is shortened. When the shift position is set to the reverse position again during the reset process, the reset process period and the chattering prevention period overlap. In this case, all or part of the chattering prevention period elapses during the reset process. Therefore, when the shift position is set to reverse again during the reset process, it can be estimated that the shift position has changed after the reset process is completed. By displaying the composite image CP in which the guide line GL is composited on the display 5 immediately after the reset process is completed, the driver can perform the reverse driving by referring to the composite image CP immediately after the reset process is completed. The second embodiment includes the same configuration and processing as the first embodiment. Therefore, the differences from the first embodiment will be mainly described below.

<2-2. Composition>
FIG. 9 shows the configuration of the image processing device 3 according to the second embodiment. The image processing device 3 according to the second embodiment includes a period reduction unit 32g in the video decoder 32.

  The period shortening unit 32g shortens the chattering prevention period, which is measured immediately after the reset process ends. When the period shortening unit 32g shortens the chattering prevention period, the shift detection unit 32f determines the shift position movement based on the shortened chattering prevention period. Note that, in the chattering prevention period, when the shift position is changed during the reset process, the shift detection unit 32f starts measurement immediately after the reset process ends. This is because the shift detection unit 32f is not activated during the reset process, and the measurement of the chattering prevention period cannot be started during the reset process.

  When the chattering prevention period is set to 500 msec, for example, the period shortening unit 32g sets a period of less than 500 msec. For example, it is 250 msec. However, it is preferable that the period shortening unit 32g shortens the chattering prevention period to 0 (zero) msec. This is because, in this case, the driver can drive backward by referring to the composite image CP immediately after the reset process is completed. As a result, when the shift position changes during the reset process, the shift detection unit 32f determines the shift position at the end of the reset process regardless of the chattering prevention time when measurement is started after the reset process ends. The shift position determined here is the control shift position in the present invention.

<2-3. Process>
FIG. 10 is a flowchart showing the processing steps of the video decoder 32 according to the second embodiment. The processing step according to the second embodiment includes step S110a after step S110 in the processing step according to the first embodiment. Therefore, the description will focus on step S110a.

  When the reset process ends in step S109, the period shortening unit 32g shortens the chattering prevention period.

  When the period shortening unit 32g shortens the chattering prevention period, the shift detection unit 32f waits for the shortened chattering prevention period to elapse and determines whether the shift position is reverse (step S111). If the chattering prevention period is shortened to 0 (zero) msec, the shift detection unit 32f fixes the shift position when the reset process is completed. The processing based on the determination result of the shift detection unit 32f is as described above.

<2-4. Image transition>
Next, the transition of the image displayed on the display 5 according to the second embodiment will be described. FIG. 11 is a time chart showing the transition of the image displayed on the display 5, and shows the case where the shift position is in reverse during the reset process. The horizontal axis c represents the passage of time.

  Since the times c0 to c3 are the same as the above times b0 to b3, the description thereof will be omitted.

  Even if the shift positions of the multimedia microcomputer 31 and the video decoder 32 are reversed during the reset period (time c4), the reset process is continuously executed. Further, at this time point, the shift detector 32f does not operate because the video decoder 32 is in the process of resetting.

  When the reset process ends, the period shortening unit 32g shortens the chattering prevention period (time c5). In this embodiment, 500 msec is shortened to 0 (zero) msec.

  Since the chattering prevention period is shortened to 0 (zero) msec, the shift detection unit 32f determines the shift position change to the reverse performed during the reset process almost simultaneously with the end of the reset process (time c5). ..

  When the shift detector 32f confirms that the shift position has been changed to reverse, the video decoder 32 causes the display 5 to display a composite image CP showing the rear of the vehicle 2. As a result, even if the driver changes the shift position to reverse during the reset process, the driver refers to the composite image CP showing the rear of the vehicle 2 immediately after the reset process ends without waiting for the chattering prevention period. Reverse driving can be performed.

  As described above, in the image processing device 3 according to the second embodiment, the period shortening unit 32g included in the video decoder 32 shortens the chattering prevention period, which is a period until the movement of the shift position is confirmed. In particular, the period shortening unit 32g shortens the chattering prevention period to 0 (zero) msec. As a result, when the driver sets the shift position to, for example, reverse during the reset process, the driver can immediately refer to the composite image CP that combines the guide lines GL immediately after the reset process, and can perform the reverse drive without delay. .. Further, since the shift position is changed during the reset processing period, the chattering prevention period overlaps with the reset processing period. Therefore, even if the chattering prevention period in which the measurement is started after the reset process is completed is shortened to 0 (zero) msec, substantially or part of the chattering prevention period has elapsed during the reset process period. However, there is almost no problem in determining the shift position change.

<3. Modification>
The present invention is not limited to the above embodiment and can be modified. Hereinafter, modified examples of the present invention will be described. The embodiments described above and below can be combined as appropriate.

  In the above embodiment, the case where the shift position is reverse has been described. However, the shift position may be another shift position. For example, it may be a drive. In this case, when the shift position is the drive, it is preferable to display not the image of the rear of the vehicle but the front of the vehicle, in particular, just below the front or just below the left and right on the display. This is because it is a spot that is likely to be a blind spot for a driver who is driving forward. Even in this case, when an abnormality occurs in the microcomputer, the reset process should not be performed immediately, but the reset process should be performed when the shift position deviates from another position, that is, the drive. While the microcomputer is abnormal, the image taken by the camera may be displayed on the display as it is. As a result, while the driver is driving forward, the image for safety confirmation is not suddenly erased, and driving safety can be improved.

  Further, in the above embodiment, the shift position detected by the shift position sensor 6 has been described by taking an automatic transmission (automatic transmission) as an example. However, the shift position detected by the shift position sensor 6 may be that of a manual transmission (manual transmission). In this case, the shift position sensor 6 detects shift positions of neutral, low, second, third, top, and reverse.

  Further, the functions described as one block do not necessarily have to be realized by a single physical element. It may be realized by distributed physical elements. Further, the functions described as a plurality of blocks in the above embodiments may be realized by a single physical element. Further, even if the device inside the vehicle and the device outside the vehicle share the processing related to any one function and information is exchanged by communication between these devices, the one function may be realized as a whole. Good.

  The configuration described as hardware may be realized by software. On the other hand, the functions described as software may be realized by hardware. Also, hardware or software may be realized by a combination of hardware and software.

1 image processing system 2 vehicle 3 image processing device 4 camera 5 display 6 shift position sensor 7 peripheral device

Claims (5)

An image processing device for processing an image,
Shift detecting means for detecting the shift position of the vehicle,
If the shift position is in a predetermined position, and image processing means for a camera mounted on the vehicle acquires a captured image obtained by capturing an image of a subject around the said vehicle, and displays and outputs the captured image to the display device ,
Combining means for controlling the image processing means and combining a figure with the captured image before output to the display device,
An abnormality detecting means for detecting an abnormality of the synthesizing means,
Resetting means for resetting the synthesizing means in response to the detection of the abnormality by the abnormality detecting means,
Equipped with
Wherein the image processing means, the abnormality if the detection means has detect the abnormality is displayed by outputting front of the captured image synthesizing graphic on the display device,
The reset means is
When the abnormality detecting means detects the abnormality while the shift position is the predetermined position,
While the shift position is the predetermined position, the reset process is not performed,
An image processing apparatus, wherein the reset process is performed after the shift position deviates from the predetermined position. The image processing apparatus according to claim 1,
The image processing apparatus, wherein the predetermined position of the shift position is reverse. The image processing device according to claim 1,
Notifying means for notifying the user in advance that the reset processing will be performed between the time when the abnormality detecting means detects the abnormality and the time when the reset means performs the reset processing,
An image processing apparatus further comprising: The image processing apparatus according to any one of claims 1 to 3,
The shift detection means,
After a lapse of a predetermined time after the shift position is changed, the control shift position which is the shift position used for control is determined,
An image processing apparatus, wherein when the shift position changes during the reset process, the control shift position is fixed at the end of the reset process regardless of the elapse of the predetermined time. An image processing method for processing an image,
(A) a step of detecting the shift position of the vehicle,
(B) image processing means, when the shift position is in a predetermined position, a camera mounted on the vehicle acquires a captured image obtained by capturing an image of a subject around the said vehicle, it outputs the captured image to the display device And the process of displaying
(C) a step of synthesizing a figure on the captured image before output to the display device by a synthesizing means for controlling the image processing means,
(D) a step of detecting an abnormality in the synthesizing means,
(E) a step of resetting the synthesizing means in response to the detection of the abnormality in the step (d);
If (f) said step (d) was detect the abnormality, the steps of the image processing means to display and output the captured image before combining the graphic on the display device,
Equipped with
The step (e) includes
When the step (d) detects the abnormality while the shift position is the predetermined position,
While the shift position is the predetermined position, the reset process is not performed,
The image processing method, wherein the reset process is performed after the shift position deviates from the predetermined position.

Images