JP7504327B2 - Display control device and display control method - Google Patents

Description

The present disclosure relates to a display control device and a display control method.

One system that uses an onboard camera is the electronic mirror system. An electronic mirror system displays on an in-vehicle monitor side and rear images captured by an onboard camera of the conditions of the areas from the sides to the rear of the vehicle that drivers would previously have checked using side mirrors or with their eyes. By looking at the images displayed on the in-vehicle monitor, the driver of the vehicle can recognize the conditions of the areas from the sides to the rear of the vehicle.

As an example of conventional technology related to electronic mirror systems, Patent Document 1 describes an attention ventilation device that acquires the relative position of a vehicle to the side and rear, captured by an on-board camera, generates display information for calling attention to the vehicle to the side and rear as a vehicle to be alerted, and displays the information on a display. The driver can recognize the presence of the vehicle to be alerted by this display.

JP 2018-116516 A

An electronic mirror system displays images captured by imaging units installed in the position of the left and right side mirrors of a vehicle on an in-vehicle monitor installed inside the vehicle. For this reason, a driver viewing an image displayed on the in-vehicle monitor needs to match the situation behind the vehicle in the captured image with the situation behind the vehicle in real space, which makes it difficult to intuitively grasp the situation behind the vehicle in real space compared to when actually looking at the side mirrors.

The device described in Patent Document 1 uses planar images created separately from images captured by the vehicle-mounted camera to show the driver information about vehicles on either side of the vehicle. For this reason, the planar images do not provide a sense of realism while driving, and furthermore, it is difficult to intuitively grasp the situation behind the vehicle in real space.

The present disclosure is intended to solve the above-mentioned problems, and aims to provide a display control device and a display control method that can display information that helps understand the situation behind the vehicle in real space.

The display control device according to the present disclosure includes a side rear vehicle detection unit that detects a second vehicle traveling in the area to the side or rear of a first vehicle using a side rear captured image in which the areas to the side and rear of the first vehicle are captured, a side rear vehicle measurement unit that uses detection information of the second vehicle to measure the speed of the second vehicle, distance information between the second vehicle and the first vehicle, and an estimated arrival time for the second vehicle to arrive at the first vehicle, a vehicle speed information acquisition unit that acquires the speed of the first vehicle, and a vehicle speed information acquisition unit that, when the speed of the first vehicle and the speed of the second vehicle are less than a threshold value, measures the speed of the first vehicle. The vehicle control device includes a driving state determination unit that determines that both the first vehicle and the second vehicle are in a first driving state, and that, if the speed of the first vehicle and the speed of the second vehicle are equal to or greater than a threshold value, determines that the first vehicle and the second vehicle are in a second driving state; and a display control unit that generates display control information for superimposing and displaying distance information on a side rear captured image when the first vehicle and the second vehicle are in the first driving state, and generates display control information for superimposing and displaying an estimated arrival time on a side rear captured image when the first vehicle and the second vehicle are in the second driving state.

According to the present disclosure, when the speed of a first vehicle and the speed of a second vehicle traveling in an area to the side or rear of the first vehicle are less than a threshold, it is determined that the first vehicle and the second vehicle are in a first traveling state, and an image in which distance information between the first vehicle and the second vehicle is superimposed on a side rear captured image is displayed. When the speed of the first vehicle and the speed of the second vehicle are equal to or greater than a threshold, it is determined that the first vehicle and the second vehicle are in a second traveling state, and an image in which an estimated arrival time for the second vehicle to arrive at the first vehicle is superimposed on a side rear captured image is displayed.
In this way, the display control device according to the present disclosure can display information that helps to grasp the situation behind the vehicle in real space.

1 is a block diagram showing a configuration of a display control system according to a first embodiment. 1 is a block diagram showing a configuration of a display control device according to a first embodiment; 4 is a flowchart showing a display control method according to the first embodiment. FIG. 11 is a screen diagram showing an example (1) of a display screen in a traffic jam driving mode. FIG. 11 is a screen diagram showing an example (1) of a display screen in a normal driving mode. 10 is a flowchart showing a modification of the display control method according to the first embodiment. FIG. 2 is a schematic diagram showing an example (1) of a positional relationship between vehicles. FIG. 11 is a screen diagram showing an example (2) of a display screen in the traffic jam driving mode. 9A and 9B are block diagrams showing a hardware configuration for realizing the functions of the display control device according to the first embodiment. FIG. 11 is a block diagram showing a configuration of a display control system according to a second embodiment. 10 is a flowchart showing a display control method according to a second embodiment. FIG. 13 is a schematic diagram showing an example (2) of the positional relationship between vehicles. FIG. 11 is a screen diagram showing an example (2) of a display screen in a normal driving mode. 13 is a flowchart showing a modified example of the display control method according to the second embodiment. FIG. 11 is a schematic diagram showing an example (3) of the positional relationship between vehicles. FIG. 11 is a screen diagram showing an example (3) of a display screen in the traffic jam driving mode.

Embodiment 1.
FIG. 1 is a block diagram showing a configuration of a display control system 1 according to a first embodiment. In FIG. 1, the display control system 1 is a system that displays an image (video) captured by an imaging unit 2A or 2B provided at the position of the left and right side mirrors of a vehicle 10 on a display unit 3A or 3B provided in a vehicle cabin. The imaging unit 2A is provided at the position of the right side mirror and captures the right side and rear areas of the vehicle 10. The image (video) captured by the imaging unit 2A is displayed on the display unit 3A. The imaging unit 2B is provided at the position of the left side mirror and captures the left side and rear areas of the vehicle 10. The image (video) captured by the imaging unit 2B is displayed on the display unit 3B. The display unit 3A displays an image of the right side area of the vehicle 10, and the display unit 3B displays an image of the left side area of the vehicle 10.

The display control device 4 generates display control information for displaying captured images of the left and right sides and rear of the vehicle 10 captured by the imaging unit 2A or 2B (hereinafter referred to as side-rear captured images). The display unit 3A or 3B displays the side-rear captured images of the vehicle 10 based on the display control information. In this way, in the display control system 1, the imaging units 2A and 2B function as electronic side mirrors.

However, it is difficult to get a sense of distance from the image displayed on display unit 3A or 3B compared to looking directly at the side mirror, and it is difficult for the driver of vehicle 10 to intuitively grasp the situation on at least one side or rear of vehicle 10 (hereinafter referred to as the side or rear) in real space.
Therefore, in the display control system 1, information that helps the driver to grasp the situation on the sides and rear of the vehicle 10 in real space is added to the side and rear captured image displayed on the display unit 3A or 3B.

For example, when vehicle traffic is congested in real space, the movement of vehicle 10 and the vehicles traveling to the side and rear of vehicle 10 (hereinafter referred to as rear vehicles) changes slowly, and the driver of vehicle 10 needs to understand the positional relationship between the vehicles in real space. In this case, display control device 4 generates display control information for displaying an image in which distance information between vehicle 10 and the rear vehicles is superimposed on the rear captured image. Display unit 3A or 3B displays the rear captured image on which distance information between vehicle 10 and the rear vehicles is superimposed based on the display control information.

On the other hand, when the vehicle is traveling normally in real space, the movement of the vehicle 10 and the vehicle at the rear changes quickly, and the driver of the vehicle 10 must grasp the movement of the vehicle at the rear approaching the vehicle 10 in real space. In this case, the display control device 4 generates display control information for displaying an image in which the estimated arrival time of the vehicle at the rear reaches the vehicle 10 is superimposed on the vehicle at the rear captured image. The display unit 3A or 3B displays the vehicle at the rear captured image on which the estimated arrival time of the vehicle at the rear reaches the vehicle 10 is superimposed based on the display control information. In this way, the display control device 4 presents the driver of the vehicle 10 with information (distance information or estimated arrival time) that helps the driver to grasp the situation behind the vehicle in real space.

Figure 2 is a block diagram showing the configuration of the display control device 4 according to the first embodiment. For example, the vehicle 10 shown in Figure 1 is a first vehicle equipped with an on-board sensor 5 and an on-board ECU 6 in addition to the imaging unit 2A, the imaging unit 2B, the display unit 3A, and the display unit 3B. The display control device 4 is mounted on the vehicle 10. The on-board sensor 5 is a sensor group consisting of various sensors, and includes a GPS (Global Positioning System), a millimeter wave radar, a direction sensor, and the like. The on-board ECU 6 is an electronic control unit that controls the running of the vehicle 10. For example, the on-board ECU 6 controls the speed of the vehicle 10 in response to the accelerator or brake operation by the driver.

The display control device 4 includes a side rear vehicle detection unit 41, a side rear vehicle measurement unit 42, a vehicle speed information acquisition unit 43, a driving state determination unit 44, and a display control unit 45. The side rear vehicle detection unit 41 detects a side rear vehicle traveling in the side or rear area of the vehicle 10 using a side rear captured image in which the side and rear areas of the vehicle 10 are captured. For example, the side rear vehicle detection unit 41 performs image analysis on the side rear captured image captured by the imaging unit 2A or 2B for each frame to recognize a side rear vehicle of the vehicle 10. For example, an image analysis technique such as pattern matching is used for the image analysis. The side rear vehicle is a second vehicle traveling to the side or rear of the vehicle 10.

The side rear vehicle measurement unit 42 uses the detection information of the side rear vehicle by the on-board sensor 5 to measure the speed of the side rear vehicle, the distance information between the side rear vehicle and vehicle 10, and the estimated arrival time for the side rear vehicle to reach vehicle 10. For example, the side rear vehicle measurement unit 42 measures the relative speed of the side rear vehicle to vehicle 10 based on the change over time in the distance to the surrounding vehicles detected by the millimeter wave radar, which is the on-board sensor 5. When the relative speed of the side rear vehicle to vehicle 10 is a positive value, the side rear vehicle measurement unit 42 uses the relative speed of the side rear vehicle to calculate the estimated arrival time for the side rear vehicle to reach vehicle 10.

In addition, the side rear vehicle measurement unit 42 may obtain at least one of the following information from the side rear vehicle via vehicle-to-vehicle communication: the speed (ground speed) of the side rear vehicle, distance information between the side rear vehicle and vehicle 10, and an estimated arrival time for the side rear vehicle to reach vehicle 10.
Furthermore, the side rear vehicle measurement unit 42 measures the distance information between the side rear vehicle and vehicle 10 from the point in time when the side rear vehicle for which distance information or estimated arrival time information is displayed is determined by the display control unit 45, and measures the estimated arrival time for the side rear vehicle to arrive at vehicle 10.

The vehicle speed information acquisition unit 43 acquires the speed of the vehicle 10. For example, the vehicle speed information acquisition unit 43 acquires the speed of the vehicle 10 from the on-board ECU 6 that controls the driving of the vehicle 10. The vehicle speed information of the vehicle 10 acquired by the vehicle speed information acquisition unit 43 is output to the driving state determination unit 44. In addition, the vehicle speed information acquisition unit 43 may acquire the speed of the vehicle 10 from a vehicle speed sensor mounted on the vehicle 10, instead of from the on-board ECU 6.

The driving state determination unit 44 determines that the driving mode is in the traffic jam driving mode when the speed of the vehicle 10 and the speed of the vehicle on the side are less than a threshold value, and determines that the driving mode is in the normal driving mode when the speed of the vehicle 10 and the speed of the vehicle on the side are equal to or greater than the threshold value. The traffic jam driving mode is a first driving state in which the speed of the vehicle 10 and the speed of the vehicle on the side are less than a threshold value. Note that the speed of the vehicle 10 and the speed of the vehicle on the side are absolute speeds, i.e., ground speeds. The normal driving mode is a second driving state in which the speed of the vehicle 10 and the speed of the vehicle on the side are equal to or greater than a threshold value.

For example, when the driving state determination unit 44 acquires vehicle speed information indicating the speed of the vehicle 10 from the vehicle speed information acquisition unit 43 and acquires measurement information indicating the speed of the side rear vehicle from the side rear vehicle measurement unit 42, it compares these speeds with a threshold value (e.g., 20 km/h). If both of these speeds are less than the threshold value, the driving state determination unit 44 determines that the driving mode is a traffic jam driving mode, and if they are equal to or greater than the threshold value, it determines that the driving mode is a normal driving mode. Information indicating the determination result of this driving state is output from the driving state determination unit 44 to the display control unit 45. Furthermore, if the speed of the side rear vehicle measured by the side rear vehicle measurement unit 42 is the relative speed of the side rear vehicle to the vehicle 10, the driving state determination unit 44 calculates the ground speed of the side rear vehicle using the vehicle speed information of the vehicle 10.

The display control unit 45 generates display control information for displaying distance information from the vehicle 10 superimposed on the side rear captured image when in the traffic jam driving mode, and generates display control information for displaying the estimated arrival time superimposed on the side rear captured image when in the normal driving mode. For example, the display control unit 45 generates display control information for displaying an image in which distance information or estimated arrival time information is superimposed near the side rear vehicle in the side rear captured image by identifying the side rear vehicle detected from the side rear captured image by the side rear vehicle detection unit 41 based on the measurement information of the side rear vehicle measured by the side rear vehicle measurement unit 42. The display unit 3A or 3B displays the side rear captured image on which the distance information or estimated arrival time information is superimposed based on the display control information.

In this way, the display control device 4 displays auxiliary information, which is at least one of text and an image that indicates distance or estimated arrival time, superimposed on the side rear captured image, which is a real-life image. This allows the driver who sees the side rear captured image displayed on the display unit 3A or 3B to intuitively grasp the situation behind the vehicle in real space without losing the sense of realism while driving.

FIG. 3 is a flowchart showing a display control method according to the first embodiment.
While the vehicle 10 is traveling, the side rear vehicle detection unit 41 detects a vehicle at the side rear of the vehicle 10 using the side rear captured images continuously captured by the imaging unit 2A or 2B. At this time, the side rear vehicle detection unit 41 is in a state of determining whether or not a vehicle at the side rear exists (step ST1). The side rear captured images captured by the imaging unit 2A or 2B are displayed sequentially on the display unit 3A or 3B. If a vehicle at the side rear of the vehicle 10 is not detected (step ST1; NO), the series of processes in FIG. 3 ends, and the processes from step ST1 are executed again.

When a vehicle is detected at the rear of the vehicle 10 (step ST1; YES), the rear vehicle measurement unit 42 measures the speed of the vehicle using the detection information of the vehicle. The vehicle speed information acquisition unit 43 acquires the speed information of the vehicle 10. The driving state determination unit 44 uses the measurement information of the vehicle at the rear by the rear vehicle measurement unit 42 and the vehicle speed information of the vehicle 10 acquired by the vehicle speed information acquisition unit 43 to determine whether the speed of the vehicle 10 and the speed of the vehicle at the rear are both less than a certain speed (threshold) (step ST2).

If the speed of the vehicle 10 and the speed of the vehicle at the rear are both less than a certain speed (step ST2; YES), the driving state determination unit 44 determines that the vehicle 10 and the vehicle at the rear are in a traffic jam driving mode. In the traffic jam driving mode, the driving state determination unit 44 acquires distance information between the vehicle 10 and the vehicle at the rear measured by the vehicle at the rear measurement unit 42 (step ST3) and outputs the distance information to the display control unit 45.

When the display control unit 45 acquires the determination result information indicating that the vehicle is in the traffic jam driving mode and the distance information between the vehicle 10 and the vehicle at the rear, the display control unit 45 generates display control information for displaying an image in which the distance information is superimposed on the rear side captured image being captured by the image capturing unit 2A or 2B, and outputs the generated display control information to the display unit 3A or 3B. The display unit 3A or 3B displays the rear side captured image with the distance information superimposed based on the display control information (step ST4).

In addition, the display control unit 45 may sequentially acquire measurement information of the side rear vehicle by the side rear vehicle measurement unit 42 from the point in time when the distance information is superimposed and displayed on the side rear captured image, and superimpose the ever-changing distance information on the side rear captured image. By viewing this side rear captured image, the driver of the vehicle 10 can grasp the change over time in the distance between the side rear vehicle and the vehicle 10 without losing the sense of realism of the driving.

Figure 4 is a screen diagram showing an example (1) of a display screen in traffic jam driving mode. The side rear captured image 50A shown in Figure 4 is an image captured by the imaging unit 2B and displayed on the display unit 3B. A plurality of side rear vehicles including vehicle 10A and vehicle 10B are captured in the side rear captured image 50A. When the distance information satisfies the first display condition, the display control unit 45 generates display control information for superimposing and displaying the distance information on the side rear captured image 50A. For example, when the distance indicated by the distance information is less than a threshold value, the display control unit 45 generates display control information for superimposing and displaying the distance information on the side rear captured image 50A.

When the threshold value is "3 m", as shown in Fig. 4, in the side rear captured image 50A, distance information 51A is displayed near vehicle 10A whose distance from vehicle 10 is "1 m", and distance information 51B is displayed near vehicle 10B whose distance from vehicle 10 is "2 m". This prevents distance information from being displayed for a large number of side rear vehicles, causing the side rear captured image 50A to become cluttered.

The display control unit 45 may highlight the vehicle that displays the distance information to distinguish it from the vehicle that does not display the distance information. For example, as shown in FIG. 4, a frame image 20 that surrounds the vehicle 10A and the vehicle 10B is displayed. The driver of the vehicle 10 can intuitively understand that the vehicles 10A and 10B are traveling near the vehicle 10 by looking at the side rear captured image 50A. For highlighting, other display processing such as changing the color of the vehicle, superimposing a blinking pattern on the vehicle, or blinking the frame image 20 may be adopted in addition to the frame image 20.
In addition, in the traffic jam driving mode, the display control unit 45 may superimpose distance information from the vehicle 10 for all side rear vehicles captured in the side rear captured image 50A.

On the other hand, if the speed of the vehicle 10 and the speed of the rear vehicle are both equal to or greater than a certain speed (step ST2; NO), the driving state determination unit 44 determines that the vehicle 10 and the rear vehicle are in the normal driving mode. In the normal driving mode, the driving state determination unit 44 acquires estimated arrival time information for the rear vehicle to reach the vehicle 10 measured by the rear vehicle measurement unit 42 (step ST5), and outputs the estimated arrival time information to the display control unit 45.

When the display control unit 45 acquires the determination result information indicating that the normal driving mode is being used and the estimated arrival time information for the side rear vehicle to reach vehicle 10, the display control unit 45 generates display control information for displaying an image in which the estimated arrival time information is superimposed on the side rear captured image being captured by the imaging unit 2A or 2B, and outputs the display control information to the display unit 3A or 3B.
The display unit 3A or 3B displays the side rear captured image on which the estimated arrival time information is superimposed based on the display control information (step ST6).

In addition, the display control unit 45 may sequentially acquire measurement information of the side rear vehicle by the side rear vehicle measurement unit 42 from the point in time when the estimated arrival time information is superimposed and displayed on the side rear captured image, and superimpose and display the ever-changing estimated arrival time on the side rear captured image. By viewing this side rear captured image, the driver of the vehicle 10 can grasp the change in the time it takes for the side rear vehicle to reach the vehicle 10 without losing the sense of realism of the drive.

Figure 5 is a screen diagram showing an example (1) of a display screen in normal driving mode. The side rear captured image 50B shown in Figure 5 is an image captured by the imaging unit 2B and displayed on the display unit 3B. Vehicle 10C, which is a side rear vehicle, is captured in the side rear captured image 50B. When the estimated arrival time satisfies the second display condition, the display control unit 45 generates display control information for superimposing and displaying the estimated arrival time on the side rear captured image 50B. For example, when the time indicated by the estimated arrival information is less than a threshold value, the display control unit 45 generates display control information for superimposing and displaying the estimated arrival time information on the side rear captured image 50B.

When the threshold value is "15 sec", as shown in Fig. 5, in the side rear captured image 50B, estimated arrival time information 52 is displayed near vehicle 10C, whose estimated arrival time is "10 sec". This prevents the side rear captured image 50B from becoming cluttered with estimated arrival time information being displayed for a large number of side rear vehicles.

The display control unit 45 may highlight the vehicle that displays the estimated arrival time information to distinguish it from the vehicle that does not display the estimated arrival time information. For example, as shown in Fig. 5, a frame image 20 surrounding the vehicle 10C is displayed. The driver of the vehicle 10 can intuitively know that the vehicle 10C is approaching the vehicle 10 by looking at the side rear captured image 50B.
In addition, for highlighting, other display processes may be used besides the frame image 20, such as changing the color of the vehicle, superimposing a blinking pattern on the vehicle, or making the frame image 20 blink.
Furthermore, in the normal driving mode, the display control unit 45 may superimpose the estimated arrival time information for all side-rear vehicles captured in the side-rear captured image 50B.

The second display condition may also be to identify a side rear vehicle whose estimated arrival time information indicates a time equal to or greater than the threshold value. In this case, the display control unit 45 generates display control information for displaying the estimated arrival time information in the side rear captured image 50B near the side rear vehicle whose estimated arrival time information indicates a time equal to or greater than the threshold value.

Next, a modification of the display control method according to the first embodiment will be described.
FIG. 6 is a flowchart showing a modification of the display control method according to the first embodiment.
While the vehicle 10 is traveling, the side rear vehicle detection unit 41 detects a vehicle behind the vehicle 10 using side rear captured images continuously captured by the imaging unit 2A or 2B. At this time, the side rear vehicle detection unit 41 is in a state of determining whether or not a rear vehicle exists in the same lane as the vehicle 10 (step ST1A). The side rear captured images captured by the imaging unit 2A or 2B are displayed sequentially on the display unit 3A or 3B. If no rear vehicle in the same lane is detected (step ST1A; NO), the series of processes in FIG. 6 ends, and the processes from step ST1A are executed again.

If a rear vehicle is detected in the same lane as vehicle 10 (step ST1A; YES), the rear side vehicle measurement unit 42 uses the detection information of the rear side vehicle to measure the speed of the rear vehicle traveling in the same lane as vehicle 10. The vehicle speed information acquisition unit 43 acquires speed information of vehicle 10. The driving state determination unit 44 uses the measurement information of the rear vehicle by the rear side vehicle measurement unit 42 and the vehicle speed information of vehicle 10 acquired by the vehicle speed information acquisition unit 43 to determine whether the speed of vehicle 10 and the speed of the rear vehicle are both less than a certain speed (threshold) (step ST2A).

If the speed of the vehicle 10 and the speed of the rear vehicle are both less than a certain speed (step ST2A; YES), the driving state determination unit 44 determines that the vehicle 10 and the rear vehicle are in a traffic jam driving mode. In the traffic jam driving mode, the driving state determination unit 44 acquires distance information between the vehicle 10 and the rear vehicle traveling in the same lane, and outputs the distance information to the display control unit 45.

When the display control unit 45 acquires the determination result information indicating that the vehicle is in the traffic jam driving mode and the distance information between the rear vehicle and the vehicle 10, the display control unit 45 generates display control information for displaying an image with distance information superimposed only on the rear captured image captured by the imaging unit 2A or 2B that is traveling rearward in the same lane and closest to the vehicle 10. The display unit 3A or 3B displays the side rear captured image with the distance information superimposed based on the display control information (step ST3A).

In addition, the display control unit 45 may sequentially acquire measurement information of the side rear vehicle by the side rear vehicle measurement unit 42 from the point in time when the distance information is superimposed and displayed on the side rear captured image, and superimpose the ever-changing distance information on the side rear captured image. By viewing this side rear captured image, the driver of the vehicle 10 can grasp the change over time in the distance between the side rear vehicle and the vehicle 10 without losing the sense of realism of the driving.

Figure 7 is a schematic diagram showing an example (1) of the positional relationship between vehicles. As shown in Figure 7, when the vehicle is in the congestion driving mode, the driver of the vehicle 10 wants to know whether the nearest vehicle 10D traveling in the same lane is traveling at an appropriate vehicle interval. Therefore, in the congestion driving mode, distance information is displayed near the vehicle 10D traveling in the same lane as the vehicle 10.

Figure 8 is a screen diagram showing an example (2) of a display screen in the traffic jam driving mode. The side rear captured image 50C shown in Figure 8 is an image captured by the imaging unit 2B and displayed on the display unit 3B. A plurality of side rear vehicles including vehicle 10D are captured in the side rear captured image 50C. The display control unit 45 generates display control information for displaying distance information for vehicle 10D, which is traveling in the same lane as vehicle 10, among the plurality of side rear vehicles. For example, in the side rear captured image 50C, distance information 51 is displayed in the vicinity of vehicle 10D, which is traveling in the same lane as vehicle 10 and is closest to vehicle 10.

The display control unit 45 may highlight the vehicle that displays the distance information to distinguish it from the vehicle that does not display the distance information. For example, as shown in Fig. 8, a frame image 20 surrounding the vehicle 10D is displayed. The driver of the vehicle 10 can intuitively grasp the distance between the vehicle 10 and the vehicle 10D that is following the vehicle 10 by looking at the side rear captured image 50C.
In addition, for highlighting, other display processes may be used besides the frame image 20, such as changing the color of the vehicle, superimposing a blinking pattern on the vehicle, or making the frame image 20 blink.

On the other hand, if the speed of the vehicle 10 and the speed of the rear vehicle are both equal to or greater than a certain speed (step ST2A; NO), the driving state determination unit 44 determines that the vehicle 10 and the rear vehicle are in the normal driving mode. In the normal driving mode, the driving state determination unit 44 acquires estimated arrival time information for the rear vehicle to reach the vehicle 10 measured by the rear vehicle measurement unit 42 (step ST4A), and outputs the estimated arrival time information to the display control unit 45.

When the display control unit 45 acquires the determination result information indicating that the normal driving mode is selected and the estimated arrival time information indicating that the vehicle at the rear will reach the vehicle 10, the display control unit 45 generates display control information for displaying an image in which the estimated arrival time information is superimposed on the rear side image being captured by the image capturing unit 2A or 2B, and outputs the generated display control information to the display unit 3A or 3B. The display unit 3A or 3B displays the rear side image on which the estimated arrival time information is superimposed based on the display control information (step ST5A).

In addition, the display control unit 45 may sequentially acquire measurement information of the side rear vehicle by the side rear vehicle measurement unit 42 from the point in time when the estimated arrival time information is superimposed and displayed on the side rear captured image, and superimpose and display the ever-changing estimated arrival time on the side rear captured image. By viewing this side rear captured image, the driver of the vehicle 10 can grasp the change in the time it takes for the side rear vehicle to reach the vehicle 10 without losing the sense of realism of the drive.

Next, a hardware configuration for realizing the functions of the display control device will be described.
The functions of the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43, the driving state determination unit 44, and the display control unit 45 included in the display control device 4 are realized by a processing circuit. That is, the display control device 4 includes a processing circuit for executing the processes from step ST1 to step ST6 shown in Fig. 3. The processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.

Figure 9A is a block diagram showing a hardware configuration that realizes the functions of the display control device 4. Figure 9B is a block diagram showing a hardware configuration that executes software that realizes the functions of the display control device 4. In Figures 9A and 9B, the input interface 100 is an interface that relays information input to the display control device 4 from the imaging units 2A and 2B, the on-board sensor 5 and the on-board ECU 6. The output interface 101 is an interface that relays display control information output from the display control device 4 to the display units 3A and 3B.

When the processing circuit is the dedicated hardware processing circuit 102 shown in FIG. 9A, the processing circuit 102 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.
The functions of the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43, the driving condition determination unit 44 and the display control unit 45 provided in the display control device 4 may be realized by separate processing circuits, or these functions may be realized together by a single processing circuit.

When the processing circuit is the processor 103 shown in Fig. 9B, the functions of the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43, the driving state determination unit 44 and the display control unit 45 provided in the display control device 4 are realized by software, firmware or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 104.

The processor 103 reads out and executes the programs stored in the memory 104, thereby realizing the functions of the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43, the driving condition determination unit 44 and the display control unit 45 provided in the display control device 4.
For example, the display control device 4 includes a memory 104 for storing a program which, when executed by the processor 103, results in the processing of steps ST1 to ST6 shown in FIG.
These programs cause the computer to execute the processing procedures or methods performed by the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43, the driving state determination unit 44, and the display control unit 45.
The memory 104 may be a computer-readable storage medium that stores programs for causing the computer to function as a side rear vehicle detection unit 41, a side rear vehicle measurement unit 42, a vehicle speed information acquisition unit 43, a driving condition determination unit 44, and a display control unit 45.

Memory 104 may be, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD, etc.

Some of the functions of the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43, the driving condition determination unit 44 and the display control unit 45 provided in the display control device 4 may be realized by dedicated hardware, and some of the other functions may be realized by software or firmware.
For example, the side rear vehicle detection unit 41, the side rear vehicle measurement unit 42, the vehicle speed information acquisition unit 43 and the driving state determination unit 44 realize their functions by a processing circuit 102 which is dedicated hardware, and the display control unit 45 realizes its function by a processor 103 reading and executing a program stored in a memory 104. In this way, the processing circuit can realize the above functions by hardware, software, firmware or a combination of these.

As described above, the display control device 4 according to the first embodiment includes a side rear vehicle detection unit 41 that detects side rear vehicles traveling on the side rear of the vehicle 10 using a side rear captured image of the vehicle 10, a side rear vehicle measurement unit 42 that uses the detection information of the side rear vehicle to measure the speed of the side rear vehicle, distance information from the vehicle 10, and an estimated arrival time to reach the vehicle 10, a vehicle speed information acquisition unit 43 that acquires the speed of the vehicle 10, a driving state determination unit 44 that determines that the vehicle is in a traffic jam driving mode if the speed of the vehicle 10 and the speed of the side rear vehicle are less than a threshold value, and determines that the vehicle is in a normal driving mode if the speed of the vehicle 10 and the speed of the side rear vehicle are equal to or greater than the threshold value, and a display control unit 45 that generates display control information for superimposing and displaying the distance information from the vehicle 10 on the side rear captured image when the traffic jam driving mode is selected, and generates display control information for superimposing and displaying the estimated arrival time on the side rear captured image when the normal driving mode is selected.
In the traffic jam driving mode in which the speed of the vehicle 10 and the speed of the vehicle behind the vehicle are below the threshold, the movement of the vehicle 10 and the surrounding vehicles changes slowly, and the driver of the vehicle 10 must understand the positional relationship between the vehicles in real space. In this case, the display control device 4 generates display control information for displaying an image in which distance information between the vehicle 10 and the surrounding vehicles is superimposed on the side rear captured image. The display unit 3A or 3B displays the side rear captured image on which the distance information between the vehicle 10 and the surrounding vehicles is superimposed based on the display control information.
On the other hand, in the normal driving mode in which the speed of the vehicle 10 and the speed of the vehicle behind the vehicle are equal to or higher than the threshold value, the movement of the vehicle 10 and the surrounding vehicles changes quickly, and the driver of the vehicle 10 must grasp the movement of the vehicles approaching the vehicle 10 in real space. In this case, the display control device 4 generates display control information for displaying an image in which the estimated arrival time for the surrounding vehicles to reach the vehicle 10 is superimposed on the side rear captured image. The display unit 3A or 3B displays the side rear captured image on which the estimated arrival time for the surrounding vehicles to reach the vehicle 10 is superimposed based on the display control information. In this way, the display control device 4 can display information (distance information or estimated arrival time) that helps the driver of the vehicle 10 grasp the situation behind the vehicle in real space.

In the display control device 4 according to the first embodiment, when the vehicle 10D following behind the vehicle 10 in the same lane as the vehicle 10 and the vehicle 10 are in a congestion driving mode, the display control unit 45 generates display control information for superimposing and displaying distance information between the vehicle 10 and the vehicle 10D that is closest to the vehicle 10 and following behind the vehicle 10 in the same lane as the vehicle 10 on the side rear captured image 50C. By looking at the side rear captured image 50C, the driver of the vehicle 10 can intuitively grasp the distance between the vehicle 10D and the vehicle 10D that is following behind the vehicle 10.

In the display control method according to the first embodiment, the side rear vehicle detection unit 41 detects a side rear vehicle using a side rear captured image, the side rear vehicle measurement unit 42 measures the speed of the side rear vehicle, the distance information from the vehicle 10, and the estimated arrival time using the detection information of the side rear vehicle, the vehicle speed information acquisition unit 43 acquires the speed information of the vehicle 10, the driving state determination unit 44 determines that the vehicle is in a traffic jam driving mode when the speed of the vehicle 10 and the speed of the side rear vehicle are less than a threshold value, and determines that the vehicle is in a normal driving mode when the speed of the vehicle 10 and the speed of the side rear vehicle are equal to or greater than a threshold value, and the display control unit 45 generates display control information for superimposing and displaying the distance information between the vehicle 10 and the side rear vehicle on the side rear captured image in the traffic jam driving mode, and generates display control information for superimposing and displaying the estimated arrival time for the side rear vehicle to reach the vehicle 10 on the side rear captured image in the normal driving mode. By the display control device 4 executing the display control method, information that helps to grasp the situation behind the vehicle in real space can be displayed.

In the display control device 4 according to the first embodiment, the display control unit 45 generates display control information for displaying the distance information superimposed on the side rear captured image when the distance information satisfies the first display condition, and generates display control information for displaying the estimated arrival time superimposed on the side rear captured image when the estimated arrival time satisfies the second display condition. Since only information that satisfies the first or second display condition is displayed, the amount of information superimposed on the side rear captured image is reduced, and a cluttered display can be prevented.

Embodiment 2.
Fig. 10 is a block diagram showing a configuration of a display control system 1A according to embodiment 2. In Fig. 10, the same components as those shown in Fig. 1 are denoted by the same reference numerals, and the description thereof will be omitted. The display control device 4A includes a side rear vehicle detection unit 41A, a side rear vehicle measurement unit 42, a vehicle speed information acquisition unit 43, a driving state determination unit 44A, a display control unit 45A, and a turn signal direction information acquisition unit 46.

The side rear vehicle detection unit 41A detects a side rear vehicle traveling in the side or rear area of the vehicle 10 using a side rear captured image in which the side and rear areas of the vehicle 10 are captured, as in the first embodiment. Furthermore, the side rear vehicle detection unit 41A detects the direction in which a turn signal mounted on the side rear vehicle is turned on. For example, the side rear vehicle detection unit 41A performs image analysis on the side rear captured image captured by the imaging unit 2A or 2B for each frame, and recognizes the side rear vehicle of the vehicle 10 and the turn signal of the vehicle. For example, an image analysis technique such as pattern matching is used for the image analysis.

When it is determined that the vehicle 10 and the vehicle at the rear are changing lanes to the same lane based on the lighting direction of the turn signal of the vehicle 10 and the turn signal of the vehicle at the rear, the driving state determination unit 44A determines whether the vehicle 10 and the vehicle at the rear are in a traffic jam driving mode or a normal driving mode based on whether the speed of the vehicle 10 and the vehicle at the rear are below a threshold. The traffic jam driving mode is a first driving state in which the speed of the vehicle 10 and the vehicle at the rear are below a threshold, and the normal driving mode is a second driving state in which the speed of the vehicle 10 and the vehicle at the rear are equal to or above a threshold.

The display control unit 45A generates display control information for displaying distance information from the vehicle 10 superimposed on the side rear captured image in the traffic jam driving mode, and generates display control information for displaying the estimated arrival time superimposed on the side rear captured image in the normal driving mode. For example, when there is a side rear vehicle that is about to change lanes into the same lane as the lane change destination of the vehicle 10 based on the turn signal direction information of the side rear vehicle detected by the side rear vehicle detection unit 41A, the display control unit 45A generates display control information for displaying an image with distance information or estimated arrival time information superimposed near the side rear vehicle in the side rear captured image. The display unit 3A or 3B displays the side rear captured image with the distance information or estimated arrival time information superimposed based on the display control information.

The blinker direction information acquisition unit 46 acquires blinker direction information indicating the lighting direction of the blinkers mounted on the vehicle 10. For example, the blinker direction information acquisition unit 46 acquires the blinker direction information of the vehicle 10 from the on-board ECU 6 that controls the driving of the vehicle 10. The blinker direction information acquired by the blinker direction information acquisition unit 46 is output to the driving state determination unit 44A and the display control unit 45A.

In this way, the display control device 4A displays auxiliary information, either in text or an image, indicating distance or estimated arrival time, superimposed on the side rear captured image only for the side rear vehicle that is about to change lanes into the same lane as the vehicle 10's lane change destination. This allows the driver who sees the side rear captured image displayed on the display unit 3A or 3B to intuitively grasp the situation behind the vehicle in real space, without losing the sense of realism while driving.

FIG. 11 is a flowchart showing a display control method according to the second embodiment.
The driving state determination unit 44A determines whether the vehicle 10 is emitting a blinker, i.e., whether the vehicle 10 is turning on the blinker, based on the blinker direction information acquired by the blinker direction information acquisition unit 46 (step ST1B). If the vehicle 10 is not emitting a blinker (step ST1B; NO), the display control device 4A ends the series of processes in FIG.

If the vehicle 10 is emitting a blinker (step ST1B; YES), the side rear vehicle detection unit 41A uses the side rear captured images continuously captured by the imaging unit 2A or 2B to detect a side rear vehicle traveling in a lane two lanes away from the lane in which the vehicle 10 is traveling and emitting a blinker. At this time, the side rear vehicle detection unit 41A is in a state of determining whether or not the side rear vehicle exists (step ST2B). The side rear captured images captured by the imaging unit 2A or 2B are displayed sequentially on the display unit 3A or 3B. If a side rear vehicle traveling in a lane two lanes away from the lane in which the vehicle 10 is traveling and emitting a blinker is not detected (step ST2B; NO), the display control device 4A temporarily ends the series of processes in FIG. 11 and repeats the processes from step ST1B again.

If a rear vehicle traveling in the lane two lanes away from the lane in which the vehicle 10 is traveling and activating a blinker is detected (step ST2B; YES), the rear vehicle measurement unit 42 measures the speed of the rear vehicle using the detection information of the rear vehicle. The vehicle speed information acquisition unit 43 acquires speed information of the vehicle 10. The driving state determination unit 44A uses the measurement information of the rear vehicle by the rear vehicle measurement unit 42 and the vehicle speed information of the vehicle 10 acquired by the vehicle speed information acquisition unit 43 to determine whether the speed of the vehicle 10 and the speed of the rear vehicle are both less than a certain speed (threshold) (step ST3B).

Figure 12 is a schematic diagram showing an example (2) of the positional relationship between vehicles. When vehicle 10E is attempting to change lanes into the same lane as the lane to which vehicle 10 is attempting to change lanes, the driver of vehicle 10 wants to know whether vehicle 10E is traveling in a manner that may cause a collision with vehicle 10. For example, in Figure 12, vehicle 10 is attempting to change into the center lane by turning on the left turn signal 53A. Vehicle 10F, which is the vehicle behind vehicle 10, is traveling in the center lane at a position far from vehicle 10. Vehicle 10E is attempting to change into the center lane by turning on the right turn signal 53B.

In a situation where vehicle 10 is about to change to the center lane, if vehicle 10F is traveling far from vehicle 10 in the center lane, vehicle 10E traveling in the lane two lanes away from the lane in which vehicle 10 is traveling across the center lane may collide with vehicle 10 if it changes lanes to the center lane. Therefore, when vehicle 10 is about to change lanes, display control device 4A displays distance information or estimated arrival time for vehicle 10E that is about to change lanes to the same lane as vehicle 10's lane change destination. By looking at this display, the driver of vehicle 10 can intuitively grasp the presence of a vehicle on the side rear that may collide when changing lanes.

If the speed of vehicle 10 and the speed of vehicle 10E are both less than a certain speed (step ST3B; YES), the driving state determination unit 44A determines that vehicle 10 and vehicle 10E are in a traffic jam driving mode. In the traffic jam driving mode, the driving state determination unit 44A acquires distance information between vehicle 10E and vehicle 10 measured by the side rear vehicle measurement unit 42 (step ST4B) and outputs the distance information to the display control unit 45A.

When the display control unit 45A acquires the determination result information indicating that the vehicle is in the traffic jam driving mode and the distance information between the vehicle 10E and the vehicle 10, the display control unit 45A generates display control information for displaying an image in which the distance information is superimposed on the side rear captured image being captured by the imaging unit 2A or 2B, and outputs the display control information to the display unit 3A or 3B. The display unit 3A or 3B displays the side rear captured image with the distance information superimposed based on the display control information (step ST5B).

The display control unit 45A may sequentially acquire measurement information of the side rear vehicle by the side rear vehicle measurement unit 42 from the point in time when the distance information is superimposed and displayed on the side rear captured image, and may superimpose and display the distance information that changes from moment to moment on the side rear captured image. By viewing this side rear captured image, the driver of the vehicle 10 can grasp the change over time in the distance between the vehicle 10E and the vehicle 10 without losing the sense of realism of the driving.

On the other hand, if the speeds of the vehicles 10 and 10E are both equal to or greater than a certain speed (step ST3B; NO), the driving state determination unit 44A determines that the vehicles 10 and 10E are in the normal driving mode. In the normal driving mode, the driving state determination unit 44A acquires estimated arrival time information measured by the side rear vehicle measurement unit 42 for the vehicle 10E to reach the vehicle 10 (step ST6B), and outputs the estimated arrival time information to the display control unit 45A.

When the display control unit 45A acquires the determination result information indicating that the normal driving mode is selected and the estimated arrival time information for the vehicle 10E to reach the vehicle 10, the display control unit 45A generates display control information for displaying an image in which the estimated arrival time information is superimposed on the side rear captured image being captured by the imaging unit 2A or 2B, and outputs the display control information to the display unit 3A or 3B. The display unit 3A or 3B displays the side rear captured image on which the estimated arrival time information is superimposed based on the display control information (step ST7B).

In addition, the display control unit 45A may sequentially acquire measurement information of the side rear vehicle by the side rear vehicle measurement unit 42 from the point in time when the estimated arrival time information is superimposed and displayed on the side rear captured image, and superimpose the ever-changing estimated arrival time on the side rear captured image. By viewing this side rear captured image, the driver of the vehicle 10 can grasp the change in the time it takes for the vehicle 10E to reach the vehicle 10 without losing the sense of realism of the drive.

In addition, the display control unit 45A may generate display control information for superimposing and displaying the distance information on the side rear captured image when the distance information satisfies the first display condition. For example, when the distance indicated by the distance information is less than a threshold, the display control unit 45A generates display control information for superimposing and displaying the distance information on the side rear captured image. Based on this display control information, the display units 3A and 3B superimpose and display the distance information only for the side rear vehicle whose distance information satisfies the first display condition in the side rear captured image.

Fig. 13 is a screen diagram showing an example (2) of a display screen in the normal driving mode. The side rear captured image 50D shown in Fig. 13 is an image captured by the imaging unit 2B and displayed on the display unit 3B. The side rear captured image 50D includes images of the vehicles 10E and 10F.
The display control unit 45A generates display control information for displaying, in the side rear captured image 50D, the estimated arrival time information 52 in the vicinity of the vehicle 10E that is about to change lanes into the same lane as the lane to which the vehicle 10 is changing lanes. Based on this display control information, the display units 3A and 3B display the estimated arrival time information 52 superimposed near the vehicle 10E in the side rear captured image 50D.

When the estimated arrival time satisfies the second display condition, the display control unit 45A may generate display control information for superimposing and displaying the estimated arrival time on the side rear captured image 50D.
For example, the display control unit 45A generates display control information for superimposing and displaying estimated arrival time information less than the threshold on the side rear captured image 50D.

The display control unit 45A may highlight the vehicle that displays the estimated arrival time information to distinguish it from the vehicle that does not display the estimated arrival time information. For example, as shown in Fig. 13, a frame line image 20 surrounding the vehicle 10E is displayed. By looking at the side rear captured image 50D, the driver of the vehicle 10 can intuitively understand that there is a vehicle 10E that is about to change lanes into the same lane as the lane into which the vehicle 10 is about to change lanes.
In addition, for highlighting, other display processes may be used besides the frame image 20, such as changing the color of the vehicle, superimposing a blinking pattern on the vehicle, or making the frame image 20 blink.

Furthermore, in the normal driving mode, the display control unit 45A may superimpose the estimated arrival time information for all side rear vehicles captured in the side rear captured image 50D.
Furthermore, the second display condition may specify a side rear vehicle whose estimated arrival time information indicates a time equal to or longer than the threshold value. In this case, the display control unit 45A generates display control information for displaying the estimated arrival time information in the side rear captured image 50D near the side rear vehicle whose estimated arrival time information indicates a time equal to or longer than the threshold value.

Next, a modification of the display control method according to the second embodiment will be described.
14 is a flowchart showing a modified example of the display control method according to the second embodiment, and shows a case where the vehicle 10 and the vehicle behind it are both traveling at less than a certain speed, i.e., in a traffic jam driving mode. The driving state determination unit 44A uses the measurement information of the vehicle behind it by the vehicle behind it measurement unit 42 and the vehicle speed information of the vehicle 10 acquired by the vehicle speed information acquisition unit 43 to determine that the speed of the vehicle 10 and the speed of the vehicle behind it are both less than a certain speed (threshold) (step ST1C).

Based on the turn signal direction information acquired by the turn signal direction information acquisition unit 46, the driving state determination unit 44A determines whether the vehicle 10 is emitting a turn signal, i.e., whether the vehicle 10 has turned on the turn signal (step ST2C). If the vehicle 10 is not emitting a turn signal (step ST2C; NO), the display control device 4A ends the series of processes in FIG.

If the vehicle 10 is using a blinker (step ST2C; YES), the side rear vehicle detection unit 41A uses the side rear captured images continuously captured by the imaging unit 2A or 2B to detect a side rear vehicle traveling in an adjacent lane to the lane in which the vehicle 10 is traveling. At this time, the side rear vehicle detection unit 41A is in a state of determining whether or not the side rear vehicle exists (step ST3C). The side rear captured images captured by the imaging unit 2A or 2B are displayed sequentially on the display unit 3A or 3B. If a side rear vehicle traveling in an adjacent lane is not detected (step ST3C; NO), the display control device 4A temporarily ends the series of processes in FIG. 14 and repeats the processes from step ST1C again.

If a rear vehicle traveling in the adjacent lane is detected (step ST3C; YES), the driving condition determination unit 44A identifies the rear vehicle closest to vehicle 10 among the rear vehicles traveling in the adjacent lane measured by the rear vehicle measurement unit 42, obtains distance information between the identified rear vehicle and vehicle 10 (step ST4C), and outputs the distance information to the display control unit 45A.

When the display control unit 45A acquires the determination result information indicating that the vehicle is in the traffic jam driving mode and the distance information, the display control unit 45A generates display control information for displaying an image in which the distance information is superimposed near the side rear vehicle closest to the vehicle 10 in the side rear captured image being captured by the imaging unit 2A or 2B, and outputs the display control information to the display unit 3A or 3B. The display unit 3A or 3B displays the side rear captured image in which the distance information is superimposed near the side rear vehicle closest to the vehicle 10 based on the display control information (step ST5C).

Figure 15 is a schematic diagram showing an example (3) of the positional relationship between vehicles. When vehicle 10G traveling in an adjacent lane is present, the driver of vehicle 10 wants to know whether there is enough space between the vehicles to avoid a collision with vehicle 10G when vehicle 10 changes lanes. For example, in Figure 15, vehicle 10 turns on the left turn signal 53A and is attempting to change to the center lane. Vehicle 10G, which is a vehicle on the side behind vehicle 10 in the adjacent lane, is traveling close to vehicle 10.

In a situation where vehicle 10 is about to change to the center lane, if vehicle 10G is traveling close to vehicle 10 in the center lane, there is a possibility that vehicle 10G and vehicle 10 may collide when vehicle 10 changes to the center lane. Therefore, when vehicle 10 is about to change to an adjacent lane in the congestion driving mode, display control device 4A displays distance information for vehicle 10G traveling in the adjacent lane and closest to vehicle 10. By looking at this display, the driver of vehicle 10 can intuitively grasp the presence of a vehicle on the side rear that may collide when changing lanes.

Figure 16 is a screen diagram showing an example (3) of a display screen in the traffic jam driving mode. The side rear captured image 50E shown in Figure 16 is an image captured by the imaging unit 2B and displayed on the display unit 3B. A plurality of side rear vehicles including vehicle 10G are captured in the side rear captured image 50E. The display control unit 45A identifies vehicle 10G, of the plurality of side rear vehicles, that is traveling in the adjacent lane to which vehicle 10 is about to change lanes and is closest to vehicle 10, and generates display control information for displaying distance information 51 in the vicinity of vehicle 10G. The display unit 3B displays the side rear captured image 50E based on this display control information.

The display control unit 45A may highlight the vehicle that displays the distance information to distinguish it from the vehicle that does not display the distance information. For example, as shown in Fig. 16, a frame line image 20 surrounding the vehicle 10G is displayed. By looking at the side rear captured image 50E, the driver of the vehicle 10 can intuitively grasp the vehicle distance between the vehicle 10G that is traveling in the adjacent lane and is closest to the vehicle 10 and about to change lanes in the congestion driving mode.
In addition, for highlighting, other display processes may be used besides the frame image 20, such as changing the color of the vehicle, superimposing a blinking pattern on the vehicle, or making the frame image 20 blink.

As described above, the display control device 4A according to the second embodiment includes the blinker direction information acquisition unit 46 that acquires blinker direction information indicating the lighting direction of the blinker mounted on the vehicle 10.
The side rear vehicle detection unit 41A detects the lighting direction of a turn signal mounted on a side rear vehicle.
When it is determined that the vehicle 10 and the vehicle at the rear are changing lanes to the same lane based on the lighting direction of the turn signal of the vehicle 10 and the lighting direction of the turn signal mounted on the vehicle at the rear, the driving state determination unit 44A determines whether the vehicle 10 and the vehicle at the rear are in a traffic jam driving mode or a normal driving mode based on whether the speed of the vehicle 10 and the speed of the vehicle at the rear are less than a threshold value. This allows the driver of the vehicle 10 to intuitively grasp the presence of a vehicle at the rear with which there is a possibility of collision when changing lanes.

In the display control device 4A according to the second embodiment, when it is determined that the vehicle 10E traveling in the lane two lanes away from the lane in which the vehicle 10 is traveling is about to change lanes to the same lane based on the turn signal direction information of the vehicle 10 and the turn signal direction mounted on the vehicle 10E, the driving state determination unit 44A determines whether the vehicle 10 and the vehicle 10E are in a traffic jam driving mode or a normal driving mode based on whether the speed of the vehicle 10 and the speed of the vehicle 10E are less than a threshold value. This allows the driver of the vehicle 10 to intuitively grasp the presence of the vehicle 10E with which there is a possibility of collision when changing lanes.

Although the display control device 4 according to the first embodiment and the display control device 4A according to the second embodiment are mounted on the vehicle 10, the present invention is not limited to this configuration.
For example, a server, which is an external device, may include the display control device 4 or 4A, and the vehicle 10 may include a communication unit that communicates with the server instead of the display control device 4 or 4A. In the vehicle 10, the communication unit acquires display control information generated by the server, and the display units 3A and 3B display the side rear captured images based on the display control information acquired by the communication unit.
Also, an in-vehicle terminal mounted on the vehicle 10 may include some of the components of the display control device 4 or 4A, and the in-vehicle terminal and the display control device 4 or 4A may cooperate to display the side rear captured image.

In addition, it is possible to combine the embodiments or modify any of the components of each embodiment, or omit any of the components of each embodiment.

The display control device disclosed herein can be used, for example, in a vehicle electronic mirror system.

1, 1A display control system, 2A, 2B imaging unit, 3A, 3B display unit, 4, 4A display control device, 5 on-board sensor, 6 on-board ECU, 10, 10A to 10G vehicle, 20 frame line image, 41, 41A side rear vehicle detection unit, 42 side rear vehicle measurement unit, 43 vehicle speed information acquisition unit, 44, 44A driving state determination unit, 45, 45A display control unit, 46 turn signal direction information acquisition unit, 50A to 50E side rear captured image, 51, 51A, 51B distance information, 52 estimated arrival time information, 53A, 53B turn signal, 100 input interface, 101 output interface, 102 processing circuit, 103 processor, 104 memory.

Claims (7)

a side rear vehicle detection unit that detects a second vehicle traveling in a side or rear area of a first vehicle by using a side rear captured image in which a side and rear area of the first vehicle is captured;
a side rear vehicle measurement unit that measures a speed of the second vehicle, distance information between the second vehicle and the first vehicle, and an estimated arrival time for the second vehicle to arrive at the first vehicle using detection information of the second vehicle;
A vehicle speed information acquisition unit that acquires a speed of the first vehicle;
a running state determination unit that determines that the first vehicle and the second vehicle are in a first running state when the speed of the first vehicle and the speed of the second vehicle are less than a threshold value, and that determines that the first vehicle and the second vehicle are in a second running state when the speed of the first vehicle and the speed of the second vehicle are equal to or greater than the threshold value;
a display control unit that generates display control information for superimposing the distance information on the side rear captured image when the first vehicle and the second vehicle are in the first driving state, and generates display control information for superimposing the estimated arrival time on the side rear captured image when the first vehicle and the second vehicle are in the second driving state. 2. The display control device according to claim 1, wherein, when the second vehicle following behind the first vehicle in the same lane as the first vehicle and the first vehicle are in the first driving state, the display control unit generates display control information for superimposing and displaying the distance information between the second vehicle, which is closest to the first vehicle and following behind the first vehicle in the same lane as the first vehicle, on the side rear captured image. a turn signal direction information acquisition unit that acquires turn signal direction information indicating a turn signal direction of a turn signal mounted on the first vehicle;
the side rear vehicle detection unit detects a lighting direction of a turn signal mounted on the second vehicle,
The display control device according to claim 1, characterized in that when it is determined that the first vehicle and the second vehicle are attempting to change lanes into the same lane based on the lighting direction of the blinker of the first vehicle and the lighting direction of the blinker mounted on the second vehicle indicated by the blinker direction information, the driving state determination unit determines whether the first vehicle and the second vehicle are in the first driving state or the second driving state based on whether the speed of the first vehicle and the speed of the second vehicle are less than the threshold value. The display control device according to claim 3, characterized in that when it is determined that the second vehicle traveling in a lane two lanes away from the lane in which the first vehicle is traveling is attempting to change lanes to the same lane based on the lighting direction of the blinker of the first vehicle indicated by the blinker direction information and the lighting direction of the blinker mounted on the second vehicle, the driving state determination unit determines whether the first vehicle and the second vehicle are in the first driving state or the second driving state based on whether the speed of the first vehicle and the speed of the second vehicle are less than the threshold value. The display control device according to claim 3, characterized in that the driving state determination unit determines whether the first vehicle and the second vehicle are in the first driving state or the second driving state based on whether a speed of the second vehicle that is closest to the first vehicle and traveling in the lane to which the first vehicle is trying to change and a speed of the first vehicle are less than the threshold value. 6. The display control device according to claim 1, wherein the display control unit generates display control information for superimposing and displaying the distance information on the side rear captured image when the distance information satisfies a first display condition, and generates display control information for superimposing and displaying the estimated arrival time on the side rear captured image when the estimated arrival time satisfies a second display condition. A display control method by a display control device, comprising:
a side rear vehicle detection unit detecting a second vehicle traveling in a side or rear area of the first vehicle by using a side rear captured image in which the side and rear areas of the first vehicle are captured;
a side rear vehicle measurement unit measuring a speed of the second vehicle, distance information between the second vehicle and the first vehicle, and an estimated arrival time for the second vehicle to arrive at the first vehicle using detection information of the second vehicle;
A vehicle speed information acquisition unit acquires a speed of the first vehicle;
a running state determination unit determining that the first vehicle and the second vehicle are in a first running state when the speed of the first vehicle and the speed of the second vehicle are less than a threshold value, and determining that the first vehicle and the second vehicle are in a second running state when the speed of the first vehicle and the speed of the second vehicle are equal to or greater than the threshold value;
a display control unit generating display control information for superimposing the distance information on the side rear captured image when the first vehicle and the second vehicle are in the first driving state, and generating display control information for superimposing the estimated arrival time on the side rear captured image when the first vehicle and the second vehicle are in the second driving state.

Images