JP7571565B2 - Overlay image display device - Google Patents

Description

The present invention relates to a superimposed image display device that provides driving support for vehicles.

Conventionally, various means have been used as information providing means for providing vehicle occupants with various information for driving support such as route guidance and warnings of obstacles. For example, this can be a display on a liquid crystal display installed in the vehicle, or audio output from a speaker. In recent years, one such information providing means is a device that provides information by displaying an image superimposed on the occupant's surrounding environment (landscape, real scene). For example, this includes head-up displays, windshield displays, and methods of displaying an image superimposed on a captured image of the vehicle's surroundings displayed on a liquid crystal display.

Here, when guiding a vehicle to change lanes, it is important to change the image displayed according to the progress of the lane change by displaying an image superimposed on the surrounding environment. For example, JP 2020-118545 A discloses a technology in which, at the timing when a lane change is recommended, an image of an arrow that crosses from the lane in which the vehicle is currently traveling to the lane to which the vehicle is moving is superimposed on the road surface ahead in the direction of travel, and the overall shape of the arrow is then continuously changed according to the progress of the lane change, and finally, when the lane change is completed, a straight arrow image is displayed.

JP 2020-118545 A (pages 13-14, FIG. 6)

Here, in order to continuously change the overall shape of the arrow according to the progress of the vehicle's lane change as in Patent Document 1, it is necessary to accurately identify the progress of the lane change in real time. In other words, there is a problem that the arrow shape will be unnatural unless the position and orientation of the vehicle relative to the lane before the lane change and the lane to be moved to are accurately identified based on those lanes. Therefore, Patent Document 1 requires detailed information to identify the progress of the lane change, such as the vehicle's current position and orientation in addition to the road shape around the vehicle, and the burden of calculation processing using that information is also large.

The present invention has been made to solve the above-mentioned problems in the conventional technology, and aims to provide a superimposed image display device that, when a guidance object that guides a vehicle to change lanes is superimposed on the scenery around the vehicle and made visible, can provide guidance that is effective and does not cause the user to feel uncomfortable, even in a situation where there is no detailed information to identify the progress of the lane change, as compared to the conventional technology.

In order to achieve the above-mentioned object, a first superimposed image display device according to the present invention is an superimposed image display device that is mounted on a vehicle and allows a guide object that provides information to an occupant of the vehicle to be visually superimposed on the scenery around the vehicle, and has an object display means for displaying the guide object that guides the vehicle to change lanes, and a progress acquisition means for acquiring a progress status of the vehicle's lane change, wherein the progress status of the vehicle's lane change is divided into a plurality of stages, the guide objects are composed of a plurality of objects spaced apart from each other, and objects whose display positions are to be moved and a movement pattern are set for each of the plurality of stages on an object-by-object basis, and the object display means moves and displays objects whose display positions are set as targets for movement in the current progress status of the vehicle's lane change in the movement pattern set for the object-by-object basis.
In addition, a second superimposed image display device according to the present invention is a superimposed image display device that is mounted on a vehicle and allows a guide object that provides information to an occupant of the vehicle to be visually recognized by superimposing it on the scenery around the vehicle, and has an object display means for displaying the guide object that guides the vehicle to change lanes, and a progress acquisition means for acquiring a progress status of the vehicle's lane change, wherein the guide object consists of a plurality of objects spaced apart from each other, and the object display means arranges the plurality of objects along a path of the vehicle's lane change, and changes and displays the display mode by performing a first step of moving the plurality of objects, on an object-by-object basis, a predetermined distance in a direction opposite to the direction of the lane change as the progress status of the vehicle's lane change progresses, and a second step of moving the objects after being moved in the first step toward the vehicle.
The term "scenery" includes not only the scenery actually viewed from the vehicle (actual scenery), but also an image of the scenery, a reproduced image of the scenery, and the like.

According to the first and second superimposed image display devices of the present invention having the above configuration, when a guidance object for guiding a vehicle to change lanes is visually superimposed on the scenery around the vehicle, the guidance objects are displayed as a plurality of objects spaced apart from each other, and the display mode is changed and displayed on an object-by-object basis according to the progress of the vehicle's lane change, so that even if there is some discomfort in the display position or shape of each object, guidance can be provided that does not give a sense of discomfort in the progress of the lane change as a whole of the objects. Therefore, detailed information for identifying the progress of the lane change is not required compared to the conventional art, and the processing load can be reduced.

1 is a block diagram showing a navigation device according to a first embodiment. 5 is a flowchart of a driving support processing program according to the first embodiment. FIG. 4 is a diagram showing an example of a driving guidance screen displayed on a liquid crystal display. 13 is a flowchart of a sub-processing program of a progress status identification process. 11 is a diagram illustrating a method for identifying the relative positional relationship between a vehicle and a marking line that is to be crossed by a lane change. FIG. 13 is a diagram illustrating classification of progress status for a lane change of a vehicle. FIG. 13 is a diagram illustrating classification of progress status for a lane change of a vehicle. 13 is a flowchart of a sub-processing program of a guide object display position determination process. 13 is a diagram showing the arrangement of guide objects when the progress of a lane change by a vehicle is in phase 0. FIG. 13 is a diagram showing the arrangement of guide objects when the progress of a lane change by a vehicle is in Phase 1. FIG. 13 is a diagram showing the arrangement of guide objects when the progress of a lane change by a vehicle is in Phase 2. FIG. 13 is a diagram showing the arrangement of guide objects when the progress of a lane change by a vehicle is in Phase 3. FIG. 13 is a diagram showing the arrangement of guide objects when the progress of a lane change by a vehicle is in Phase 4. FIG. FIG. 13 is a diagram showing the shape of a guide object. FIG. 11 is a diagram showing an example of a visually recognized image in which a guide object arranged in a three-dimensional space is visually recognized. 13 is a diagram showing a driving guidance screen that is displayed on the liquid crystal display as the vehicle travels when the progress of the vehicle's lane change is from phase 0 to phase 1. FIG. 13 is a diagram showing a driving guidance screen that is displayed on the liquid crystal display as the vehicle travels when the progress of the vehicle's lane change has progressed from Phase 1 to Phase 2. FIG. 13 is a diagram showing a driving guidance screen that is displayed on the liquid crystal display as the vehicle travels when the progress of the vehicle's lane change has progressed from phase 2 to phase 3. FIG. 13 is a diagram showing a driving guidance screen that is displayed on the liquid crystal display as the vehicle travels when the progress of the vehicle's lane change has progressed from Phase 3 to Phase 4. FIG. FIG. 11 is a schematic configuration diagram of a superimposed image display device according to a second embodiment. 13A and 13B are diagrams showing a display example of a guide object in the superimposed image display device according to the second embodiment.

Below, a first embodiment and a second embodiment in which the superimposed image display device according to the present invention is embodied in a navigation device will be described in detail with reference to the drawings.

[First embodiment]
First, a schematic configuration of a navigation device 1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the navigation device 1 according to the first embodiment.

As shown in FIG. 1, the navigation device 1 according to the first embodiment includes a current position detection unit 11 that detects the current position of the vehicle in which the navigation device 1 is mounted, a data recording unit 12 in which various data is recorded, a navigation ECU 13 that performs various calculation processes based on input information, an operation unit 14 that accepts operations from the user, a liquid crystal display 15 that displays to the user an actual image of the area ahead in the direction of travel, a speaker 16 that outputs voice guidance regarding route guidance, a DVD drive 17 that reads a DVD, which is a storage medium, and a communication module 18 that communicates with an information center such as a probe center or a VICS (registered trademark: Vehicle Information and Communication System) center. The navigation device 1 is also connected to a front camera 19 and various sensors installed in the vehicle in which the navigation device 1 is mounted, via an in-vehicle network such as a CAN.

Each of the components of the navigation device 1 will be described below in order.
The current position detection unit 11 is composed of a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and is capable of detecting the current vehicle position, direction, vehicle running speed, current time, etc. Here, the vehicle speed sensor 22 in particular is a sensor for detecting the travel distance and vehicle speed of the vehicle, and generates pulses in response to the rotation of the drive wheels of the vehicle and outputs the pulse signal to the navigation ECU 13. The navigation ECU 13 then calculates the rotation speed and travel distance of the drive wheels by counting the generated pulses. It is not necessary for the navigation device 1 to be equipped with all of the above four types of sensors, and the navigation device 1 may be configured to be equipped with only one or a plurality of these types of sensors.

The data recording unit 12 also includes a hard disk (not shown) as an external storage device and recording medium, and a recording head (not shown) which is a driver for reading the map information DB 31 and predetermined programs recorded on the hard disk and writing predetermined data to the hard disk. The data recording unit 12 may be configured with a flash memory, a memory card, or an optical disk such as a CD or DVD instead of a hard disk. The map information DB 31 may also be stored on an external server and acquired by the navigation device 1 through communication.

Here, the map information DB 31 is a storage means that stores, for example, link data 32 relating to roads (links), node data 33 relating to node points, branch point data 34 relating to branch points, point data relating to points such as facilities, map display data for displaying a map, search data for searching for routes, search data for searching for points, etc.

The link data 32 includes data representing the width, gradient, cant, bank, road surface condition, number of lanes, traffic divisions for each lane according to the direction of travel, points where the number of lanes decreases, points where the road width narrows, and railroad crossings for each link that makes up the road; data representing the radius of curvature, intersections, T-junctions, corner entrances and exits for corners; data representing downhill roads, uphill roads, and other road attributes; and data representing expressways and general roads (national highways, prefectural roads, narrow streets, and other road types).

The node data 33 also includes data on the coordinates (positions) of actual road branching points (including intersections, T-junctions, etc.) and node points set at a predetermined distance on each road according to the radius of curvature, etc., node attributes indicating whether a node corresponds to an intersection, etc., a connecting link number list which is a list of link numbers of links connecting to the node, an adjacent node number list which is a list of node numbers of nodes adjacent to the node via links, and the height (altitude) of each node point.

The branch point data 34 also includes the intersection name of the branch point, relevant node information that identifies the node that forms the branch point, connecting link information that identifies the link connected to the branch point, the destination name corresponding to the link connected to the branch point, information that identifies the shape of the branch point, etc. Also stored are structures that can serve as landmarks when providing right/left turn guidance at the branch point.

Meanwhile, the navigation ECU (electronic control unit) 13 is an electronic control unit that controls the entire navigation device 1, and includes internal storage devices such as a CPU 41 as an arithmetic device and control device, a RAM 42 that is used as a working memory when the CPU 41 performs various arithmetic processing and stores route data and the like when a route is searched, a ROM 43 that stores a driving support processing program (FIG. 2) described below as well as control programs, and a flash memory 44 that stores programs read from the ROM 43. The navigation ECU 13 has various means as processing algorithms. For example, the object display means displays a guidance object that guides the vehicle to change lanes. The progress status acquisition means acquires the progress status of the vehicle's lane change.

The operation unit 14 is operated when inputting a departure point as the starting point of the journey and a destination point as the end point of the journey, and has a number of operation switches (not shown) such as various keys and buttons. The navigation ECU 13 controls the execution of various corresponding operations based on switch signals output by pressing each switch. The operation unit 14 may be configured to have a touch panel provided on the front side of the liquid crystal display 15. It may also be configured to have a microphone and a voice recognition device.

The LCD display 15 also displays map images including roads, traffic information, operation guidance, operation menus, key guidance, guided route from the departure point to the destination, guidance information along the guided route, news, weather forecasts, time, emails, television programs, etc. In particular, in the first embodiment, during normal driving, the LCD display 15 displays an image captured by the front camera 19, i.e., the scenery (actual image) around the vehicle at the current time (particularly in front of the vehicle), and further displays a guidance object superimposed on the scenery as necessary.

Here, the guidance objects displayed superimposed on the scenery include information about the vehicle and various information used to assist the driver. For example, the guidance objects include warnings for objects (other vehicles, pedestrians, guide signs) that should be warned to the driver, the guide route set by the navigation device 1 and guidance information based on the guide route (arrows indicating the direction of right and left turns, icons indicating the markers of the guidance branch point, distance to the guidance branch point, etc.), warnings displayed on the road surface (watch for rear-end collisions, speed limits, etc.), lane markings on which the vehicle is traveling, current vehicle speed, shift position, remaining energy, advertising images, facility information, guide signs, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, etc. In the first embodiment described below, the guidance objects are displayed at the timing when it is recommended to change lanes, and serve as guidance information for guiding the vehicle to change lanes (i.e., encouraging the vehicle to change lanes). More specifically, the guidance objects are displayed superimposed on the road surface ahead of the vehicle in the traveling direction, and are multiple arrows indicating the direction of movement of the vehicle when changing lanes.

The speaker 16 also outputs voice guidance to guide the driver along the guide route based on instructions from the navigation ECU 13, as well as traffic information.

The DVD drive 17 is a drive capable of reading data recorded on recording media such as DVDs and CDs. Based on the read data, music and video are played, and the map information DB 31 is updated. Note that instead of the DVD drive 17, a card slot for reading and writing to a memory card may be provided.

The communication module 18 is a communication device for receiving traffic information, such as congestion information, regulation information, and traffic accident information, transmitted from a traffic information center, such as a VICS center or a probe center, and is, for example, a mobile phone or DCM.

The front camera 19 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis oriented forward in the direction of travel of the vehicle. The image captured by the front camera 19 is displayed on the liquid crystal display 15 as the scenery (actual image) around the vehicle (particularly in front of the vehicle) as described above. When the vehicle changes lanes, the navigation ECU 13 also performs image recognition processing on the captured image to identify the relative position of the vehicle and the marking line that will be crossed when changing lanes.

Next, the driving support processing program executed by the navigation ECU 13 in the navigation device 1 having the above configuration will be described with reference to FIG. 2. FIG. 2 is a flowchart of the driving support processing program according to the first embodiment. Here, the driving support processing program is executed after the vehicle's ACC power (accessory power supply) is turned on, and is a program that provides driving support for the vehicle by allowing the driver to visually recognize a guide object superimposed on the scenery around the vehicle displayed on the liquid crystal display 15. The programs shown in the flowcharts in FIG. 2, FIG. 4, and FIG. 8 below are stored in the RAM 42 and ROM 43 of the navigation device 1, and are executed by the CPU 41.

In the following description, an example of vehicle driving assistance using a guide object is described, in which the vehicle is guided to change lanes (i.e., encouraged to change lanes) at the recommended timing for lane changing. The guide object to be displayed is superimposed on the road surface ahead of the vehicle in the traveling direction, and a process in which multiple arrows indicating the direction of movement of the vehicle when changing lanes are displayed as guide objects is described as an example. However, the navigation device 1 can also use guide objects to provide guidance and information other than the above driving assistance. The guide object to be displayed can also be information other than the arrows. For example, it is possible to display, as guide objects, warnings for objects to be warned to the occupant (other vehicles, pedestrians, guide signs), warnings to be displayed on the road surface (beware of rear-end collisions, speed limits, etc.), lane markings on which the vehicle is traveling, current vehicle speed, gear shift position, remaining energy, advertising images, facility information, guide signs, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, etc.

First, in step (hereinafter abbreviated as S) 1 of the driving assistance processing program, the CPU 41 determines whether it is the recommended timing for a lane change. The recommended timing for a lane change is, for example, a timing when it is necessary to change lanes in order to travel along the guidance route set in the navigation device 1 (for example, when traveling in a lane other than the left lane before an intersection where a left turn is required, or when exiting a highway). The CPU 41 makes the determination based on the guidance route set in the navigation device 1, map information acquired from the map information DB 31 or an external server, and the current position of the vehicle detected by the current position detection unit 11 (the driving lane is also identified using the front camera 19).

However, even if a guidance route has not been set in the navigation device 1, it is desirable to determine that it is time to recommend a lane change if the lane on which the vehicle is currently traveling will become impassable due to a reduction in the number of lanes or construction work, or if congestion occurs ahead in the direction of travel of the lane on which the vehicle is currently traveling.

If it is determined that it is a recommended time to change lanes (S1: YES), the process proceeds to S2. On the other hand, if it is determined that it is not a recommended time to change lanes (S1: NO), the driving support processing program is terminated without displaying the guidance object. However, instead of not displaying the guidance object when it is not a recommended time to change lanes, a guidance object in the form of an arrow indicating the direction to go straight may be displayed.

In S2, the CPU 41 performs a progress status determination process ( FIG. 4 ) to be described later. The progress status determination process is a process for determining the progress status of the vehicle's lane change based on the relative positional relationship between the vehicle and the marking line that is to be crossed by the lane change and the elapsed time. In particular, in the first embodiment, the progress status of the vehicle's lane change is determined by dividing it into the following five phases.
"Phase 0": The stage in which the vehicle is traveling in the lane before changing lanes. "Phase 1": The stage from when the vehicle starts crossing the lane marking until the lane marking is in the center of the vehicle. "Phase 2": The stage from when the lane marking passes the center of the vehicle until the vehicle crosses the lane marking. "Phase 3": The stage from when the vehicle crosses the lane marking until the lane change is completed. "Phase 4": The stage in which the vehicle has completed the lane change and is traveling in the lane after the lane change.

However, the progress of a vehicle's lane change does not necessarily have to be divided into the above five phases, and may be divided into more detailed or coarse phases. However, it is desirable to divide it into at least three phases: the phase in which the vehicle is traveling in the lane before the lane change (corresponding to phase 0 in the above example), the phase during the lane change (corresponding to phases 1 to 3 in the above example), and the phase in which the vehicle is traveling in the lane after the lane change (corresponding to phase 4 in the above example).

Next, in S3, the CPU 41 performs the guide object display position determination process (FIG. 8) described below. The guide object display position determination process is a process for specifically determining the shape of the guide object to be displayed on the liquid crystal display 15 and the position (range) in which the guide object is to be displayed, based on the progress of the lane change of the vehicle identified in S2 above.

Then, in S4, the CPU 41 generates an image of the guide object having the shape determined in S3, and further transmits a control signal to the LCD display 15, which causes the LCD display 15 to draw the image of the generated guide object in the position (range) determined in S3. Note that the LCD display 15 displays an image captured in advance by the front camera 19, i.e., the scenery (actual scenery image) around the vehicle (particularly in front of the vehicle) at the current time. As a result, it becomes possible for the vehicle occupants to visually recognize the guide object superimposed on the scenery.

3 is a diagram showing an example of a driving guidance screen 51 displayed on the liquid crystal display 15 in S4. As shown in FIG. 3, the liquid crystal display 15 displays a current view 52 in front of the vehicle captured by the front camera 19. Then, an image 53 of a guide object is displayed superimposed on the view 52 in front of the vehicle. The image 53 of the guide object includes images of a plurality of arrow-shaped objects spaced apart from each other, and the images of the plurality of objects are displayed at a predetermined interval so as to straddle the current lane to the target lane along the lane change path. The direction of each arrow basically indicates the direction of lane change, but the direction is different for each arrow. The arrow located at the frontmost position indicates the direction of lane change at an angle of 90 degrees to the traveling direction of the road, the arrow located second from the front indicates the direction of lane change at an angle of 60 degrees to the traveling direction of the road, the arrow located third from the front indicates the direction of lane change at an angle of 30 degrees to the traveling direction of the road, and the arrow located furthest from the front indicates the traveling direction of the road, i.e., the straight direction. Therefore, when a vehicle occupant views the driving guidance screen 51, they can understand the direction of movement and route of the vehicle when changing lanes from the arrangement and orientation of each arrow.

As described later, the display mode of the image 53 of the guidance object changes for each individual object according to the progress of the lane change of the vehicle. Specifically, the display position of the object corresponding to the current progress of the lane change of the vehicle is moved along the road width direction (horizontal direction) or the traveling direction (vertical direction) of the road. When moving toward the vehicle along the traveling direction, the size of the image is enlarged, and further, the objects on the front side are moved out of the screen in order and hidden. As a result, even if there is some discomfort in the display position of each object or some error occurs in identifying the progress of the lane change of the vehicle in S2, the guidance object as a whole can provide effective and natural guidance according to the progress of the lane change in real time, and by visually checking the image 53 of the guidance object, the vehicle occupant can also know whether the lane change is before, during, or completed. Details will be described later.

Then, in S5, the CPU 41 determines whether or not to end the lane change guidance. The CPU 41 determines to end the lane change guidance when a predetermined time has elapsed since the progress of the lane change of the vehicle was first determined to be "Phase 4" in S2. The predetermined time is, for example, the time until all of the guidance objects that move toward the vehicle according to the progress of the lane change as described above move off the screen and are hidden.

If it is determined that the lane change guidance should be ended (S5: YES), the process proceeds to S6. On the other hand, if it is determined that the lane change guidance should not be ended (S5: NO), the process returns to S1 and the guidance object continues to be displayed.

At S6, the CPU 41 sends a control signal to the LCD display 15, and the guidance object that was displayed on the LCD display 15 is hidden. Note that the image captured by the front camera 19, i.e., the scenery (actual scenery image) around the vehicle at the current time (particularly in front of the vehicle), continues to be displayed. However, the scenery image may also be hidden, and the display screen of the LCD display 15 may be switched to displaying a map image, etc.

Furthermore, when the vehicle subsequently reaches the timing when the next lane change is recommended, the processing from S2 onwards is executed and the image of the guidance object is displayed again on the LCD display 15.

Next, the sub-processing of the progress status determination process executed in S2 will be described with reference to FIG. 4. FIG. 4 is a flowchart of the sub-processing program of the progress status determination process.

First, in S11, the CPU 41 performs image recognition processing on the captured image captured by the front camera 19 to identify the marking line that is to be crossed by the lane change, and obtains the distance between the vehicle and the marking line that is to be crossed by the lane change (hereinafter, referred to as the marking line distance). For example, as shown in FIG. 5, when the vehicle 60 changes lanes from the right lane 61 to the left lane 62, the marking line that is to be crossed by the lane change is the lane boundary line 63 between the lanes 61 and 62, and the distance L from the center of the vehicle 60 to the center of the lane boundary line 63 is obtained as the marking line distance. Note that FIG. 5 shows a case where the vehicle is located to the right of the lane boundary line 63, but even when the vehicle crosses the lane boundary line 63 and is located to the left of the lane boundary line 63, the distance L from the center of the vehicle 60 to the center of the lane boundary line 63 is similarly obtained as the marking line distance.

Next, in S12, the CPU 41 determines whether the distance between the lane lines acquired in S11 is less than 50 cm. The distance that is the determination condition in S12 is a threshold value for determining the timing when the vehicle starts or finishes crossing the lane line, but the value is not limited to 50 cm and can be changed as appropriate, and may be, for example, 40 cm or 60 cm.

If it is determined in S11 that the distance between the lane lines is less than 50 cm (S12: YES), the process proceeds to S13. On the other hand, if it is determined in S11 that the distance between the lane lines is 50 cm or more (S12: NO), the process proceeds to S17.

Then, in S13, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and determines whether the read parameter value is "phase 0." The parameter value stored in the flash memory 44 is initially set to "phase 0 (the stage in which the vehicle is traveling in the lane before the lane change)," and is updated appropriately based on the relative positional relationship between the vehicle and the lane marking that will be crossed by the lane change and the elapsed time, as described below (S14, S16, S18, S20).

If it is determined that the parameter value is "phase 0" (S13: YES), the process proceeds to S14. On the other hand, if it is determined that the parameter value is other than "phase 0" (S13: NO), the process proceeds to S15.

In S14, the CPU 41 reads from the flash memory 44 a parameter value that specifies the current stage (phase) of the progress of the vehicle's lane change, and updates the read parameter value to "Phase 1 (the stage from when the vehicle starts crossing the lane line until the lane line is positioned at the center of the vehicle)." As a result, from S3 onwards, a guidance object is displayed according to the progress of the lane change in Phase 1. Details will be described later.

Meanwhile, in S15, the CPU 41 determines whether 0.5 seconds have passed since the parameter value was updated to "phase 1" in S14. Here, in the first embodiment, the moving speed in the road width direction when the vehicle moves between lanes is assumed to be 1 m/s based on empirical rules. Therefore, as shown in FIG. 6, the time when the parameter value was updated to "phase 1" in S14, that is, the time when 0.5 seconds have passed from the time when the distance between the center of the vehicle 60 and the center of the lane boundary line 63 to be crossed by the lane change is 50 cm, is estimated to be the time when the lane boundary line 63 is located at the center of the vehicle 60.

The elapsed time that serves as the judgment condition in S15 is the time that elapses until the lane marking is located at the center of the vehicle, assuming that the speed of movement in the road width direction when the vehicle moves in the lane is 1 m/s. Therefore, if a different moving speed is assumed, the elapsed time will change accordingly. Also, if the distance that serves as the judgment condition in S12 is set to a value other than 50 cm, the elapsed time will change in the same way.

If it is determined in S14 that 0.5 seconds have passed since the parameter value was updated to "Phase 1" (S15: YES), the process proceeds to S16. On the other hand, if it is determined in S14 that 0.5 seconds have not passed since the parameter value was updated to "Phase 1" (S15: NO), the process ends the progress status determination process without updating the parameter value that identifies the stage (phase) of the progress status for the current lane change of the vehicle, and proceeds to S3.

In S16, the CPU 41 reads from the flash memory 44 a parameter value that specifies the current stage (phase) of the progress of the vehicle's lane change, and updates the read parameter value to "Phase 2 (the stage from when the lane marking passes the center of the vehicle until it crosses the lane marking)." As a result, from S3 onwards, a guidance object is displayed according to the progress of the lane change in Phase 2. Details will be described later.

On the other hand, in S17, which is executed if it is determined in S11 that the distance between the lane markings is 50 cm or more, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and determines whether the read parameter value is "phase 2."

If it is determined that the parameter value is "phase 2" (S17: YES), the process proceeds to S18. On the other hand, if it is determined that the parameter value is other than "phase 2" (S17: NO), the process proceeds to S19.

In S18, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and updates the read parameter value to "Phase 3 (the stage from crossing the dividing line to completing the lane change)." As a result, from S3 onwards, a guidance object is displayed according to the progress of the lane change in Phase 3. Details will be described later.

Meanwhile, in S19, the CPU 41 determines whether 1.3 seconds have passed since the parameter value was updated to "phase 3" in S18. Here, in the first embodiment, the moving speed in the road width direction when the vehicle moves between lanes is assumed to be 1 m/s based on empirical rules. Also, it is assumed that the lane width is 3.5 m, which is generally used for national roads and expressways. Therefore, as shown in FIG. 7, the time when the parameter value was updated to "phase 3" in S18, that is, the time when 1.3 seconds have passed since the vehicle 60 crossed the lane boundary line 63 and the distance between the center of the vehicle 60 and the center of the lane boundary line 63 became 50 cm, is estimated to be the time when the vehicle 60 is located near the center of the lane after the lane change.

The elapsed time, which is the judgment condition in S19, is the time elapsed until the vehicle is positioned at the center of the lane, assuming that the moving speed in the road width direction when the vehicle moves in the lane is 1 m/s and the lane width is 3.5 m. Therefore, if the moving speed and lane width are assumed to be different values, the elapsed time will change accordingly. For example, the elapsed time may be set to a different value depending on the type of road the vehicle is traveling on. Also, the same change will occur if the distance, which is the judgment condition in S12, is set to a value other than 50 cm.

If it is determined in S18 that 1.3 seconds have passed since the parameter value was updated to "Phase 3" (S19: YES), the process proceeds to S20. On the other hand, if it is determined in S18 that 1.3 seconds have not passed since the parameter value was updated to "Phase 3" (S19: NO), the process ends the progress status determination process without updating the parameter value that identifies the stage (phase) of the progress status for the current lane change of the vehicle, and proceeds to S3.

In S20, the CPU 41 reads from the flash memory 44 a parameter value that specifies the current stage (phase) of the progress of the vehicle's lane change, and updates the read parameter value to "Phase 4 (the stage where the lane change has been completed and the vehicle is traveling in the lane after the lane change)." As a result, from S3 onwards, a guidance object is displayed according to the progress of the lane change in Phase 4. Details will be described later.

Next, the sub-processing of the guide object display position determination process executed in S3 will be described with reference to FIG. 8. FIG. 8 is a flowchart of the sub-processing program of the guide object display position determination process.

First, in S31, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and determines whether the read parameter value is "phase 0." "Phase 0" is a stage in which the vehicle is traveling in the lane before the lane change, and is far away from the marking line that will be crossed by the lane change. The parameter value stored in the flash memory 44 is initially set to "phase 0," and is updated as appropriate based on the relative positional relationship between the vehicle and the marking line that will be crossed by the lane change and the elapsed time, as described above (S14, S16, S18, S20).

If it is determined that the parameter value is "phase 0" (S31: YES), the process proceeds to S32. On the other hand, if it is determined that the parameter value is other than "phase 0" (S31: NO), the process proceeds to S33.

In S32, the CPU 41 sets the "position where the guide object is superimposed on the scenery" to a position (initial position) on the road surface a predetermined distance ahead (for example, 10 m ahead) in the direction of travel from the current position of the vehicle. Here, in the first embodiment, the guide objects to be displayed are a plurality of arrows arranged at a predetermined interval so as to straddle the current lane 61 along the lane change path of the vehicle 60 from the lane 61 currently travelling to the target lane 62, as shown in FIG. 9. In the following description, the arrows are referred to as the first object 65, the second object 66, the third object 67, and the fourth object 68 in order from the one closest to the vehicle 60. In S32, the position where the first object 65, which is located closest to the vehicle 60, is superimposed on the scenery is set to a position on the road surface a predetermined distance ahead (for example, 10 m ahead) in the direction of travel from the current position of the vehicle 60, and slightly to the right of the center of the lane in which the vehicle is travelling (for example, 1 m to the right of the center of the vehicle). The position where the second object 66 is superimposed on the scenery is a position on the road surface a predetermined distance forward from the first object 65, which is slightly left of the center of the lane on which the vehicle runs (for example, 1 m to the left of the center of the vehicle). The position where the third object 67 is superimposed on the scenery is a position on the road surface a predetermined distance forward from the second object 66, which is near the dividing line that is crossed by lane change (for example, 1.75 m to the left of the center of the vehicle). The position where the fourth object 68 is superimposed on the scenery is a position on the road surface a predetermined distance forward from the third object 67, which is near the center of the target lane 62 (for example, 3.5 m to the left of the center of the vehicle). On the other hand, the vertical position of the guide object may be a position where the lower end is in contact with the road surface, or a position spaced a predetermined distance above the road surface. The placement interval of each object is, for example, 10 m. However, the number of objects and the placement interval can be set appropriately. During phase 0, the relative positions from the vehicle at which the first object 65, second object 66, third object 67, and fourth object 68 are superimposed on the scenery are fixed at the positions shown in FIG. 9. Then, the process proceeds to S42.

On the other hand, in S33, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and determines whether the read parameter value is "phase 1." "Phase 1" is the stage from when the vehicle starts crossing the lane marking until the lane marking is positioned at the center of the vehicle.

If it is determined that the parameter value is "phase 1" (S33: YES), the process proceeds to S34. On the other hand, if it is determined that the parameter value is other than "phase 1" (S33: NO), the process proceeds to S35.

In S34, the CPU 41 updates the "position where the guide object is superimposed on the scenery" to a position where the first object 65 is moved by v1×t toward the vehicle side from the currently set position. Note that v1 is the current vehicle speed of the vehicle acquired by the vehicle speed sensor 22. t is the interval at which the guide object display position determination process (FIG. 8) of S3 is executed in the driving support processing program, and is set to, for example, 0.03 sec. As a result of performing the process of S34, the relative positions from the vehicle 60 where the second object 66, the third object 67, and the fourth object 68 are superimposed on the scenery are fixed at the positions shown in FIG. 10 (the same initial positions as in S32) during phase 1. On the other hand, for the first object 65, the relative position from the vehicle where the first object 65 is superimposed on the scenery approaches the vehicle 60 side according to the traveling speed of the vehicle 60, and more specifically, the first object 65 is visually recognized by the occupant as if it is stationary on the road surface and the distance from the vehicle is gradually reduced. Note that the first object 65 is removed from the display list once it has passed the vehicle 60. Then, the process proceeds to S42.

Next, in S35, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and determines whether the read parameter value is "phase 2." "Phase 2" is the stage from when the lane marking passes the center of the vehicle to when the lane marking is crossed.

If it is determined that the parameter value is "phase 2" (S35: YES), the process proceeds to S36. On the other hand, if it is determined that the parameter value is other than "phase 2" (S35: NO), the process proceeds to S39.

In S36, the CPU 41 updates the position of the second object 66 to a position that is moved by v1×t toward the vehicle side with respect to the currently set position for the "position where the guide object is superimposed on the scenery". Note that v1 is the current vehicle speed of the vehicle acquired by the vehicle speed sensor 22. t is the interval at which the guide object display position determination process (FIG. 8) of S3 is executed in the driving support processing program, and is set to, for example, 0.03 sec. Also, the position of the third object 67 is updated to a position that is moved by v2×t along the road width direction in the opposite direction to the lane change direction (to the right if moving to the left lane) with respect to the currently set position. Note that v2 is the moving speed in the road width direction when the vehicle moves in the lane, and is assumed to be 1 m/s based on empirical rules. Similarly, the position of the fourth object 68 is updated to a position that is moved by v2×t along the road width direction in the opposite direction to the lane change direction (to the right if moving to the left lane) with respect to the currently set position. As a result of the processing of S36 to S38, as shown in FIG. 11, during phase 2, the relative position of the second object 66 from the vehicle superimposed on the scenery approaches the vehicle 60 according to the traveling speed of the vehicle 60, more specifically, the second object 66 is visually recognized by the occupant as if it were stationary on the road surface and the distance from the vehicle gradually decreases. Also, the relative positions of the third object 67 and the fourth object 68 from the vehicle superimposed on the scenery move in the opposite direction to the moving direction of the vehicle 60 according to the lateral moving speed of the vehicle 60, more specifically, the third object 67 is visually recognized by the occupant as always being located near the dividing line without changing the distance from the vehicle, and the fourth object 68 is visually recognized as always being located near the center of the target lane 62 without changing the distance from the vehicle. Note that the second object 66 is removed from the display target when it passes the vehicle 60. Then, the process proceeds to S42.

Next, in S39, the CPU 41 reads from the flash memory 44 a parameter value that specifies the stage (phase) of the progress of the vehicle's current lane change, and determines whether the read parameter value is "Phase 3." "Phase 3" is the stage from crossing the dividing line to completing the lane change.

If it is determined that the parameter value is "phase 3" (S39: YES), the process proceeds to S40. On the other hand, if it is determined that the parameter value is "phase 4" (S39: NO), the process proceeds to S41.

In S40, the CPU 41 updates the "position where the guide object is superimposed on the scenery" to a position where the position of the third object 67 is moved by v1×t toward the vehicle side from the currently set position. Note that v1 is the current vehicle speed of the vehicle acquired by the vehicle speed sensor 22. t is the interval at which the guide object display position determination process (FIG. 8) of S3 is executed in the driving support processing program, and is set to, for example, 0.03 sec. As a result of performing the process of S40, the relative position of the fourth object 68 from the vehicle 60 where the fourth object 68 is superimposed on the scenery during phase 3 is fixed at the position shown in FIG. 12 (the position after the lateral movement in S38). On the other hand, for the third object 67, the relative position from the vehicle where the third object 67 is superimposed on the scenery approaches the vehicle 60 side according to the traveling speed of the vehicle 60, more specifically, the third object 67 is visually recognized by the occupant as if it is stationary on the road surface and the distance from the vehicle is gradually reduced. Note that the third object 67 is removed from the display target when it passes the vehicle 60. Then proceed to S42.

On the other hand, in S41, the CPU 41 updates the "position where the guide object is superimposed on the scenery" to a position where the position of the fourth object 68 is moved by v1×t toward the vehicle from the currently set position. Here, v1 is the current vehicle speed of the vehicle acquired by the vehicle speed sensor 22. t is the interval at which the guide object display position determination process (FIG. 8) of S3 is executed in the driving support processing program, and is set to, for example, 0.03 sec. As a result of performing the process of S41, as shown in FIG. 13, during phase 4, the relative position of the fourth object 68 from the vehicle superimposed on the scenery approaches the vehicle 60 according to the traveling speed of the vehicle 60, and more specifically, the fourth object 68 is visually recognized by the occupant as if it is stationary on the road surface and the distance from the vehicle is gradually shortened. Here, the fourth object 68 is removed from the display target when it passes the vehicle 60.

Next, in S42, the CPU 41 generates a three-dimensional space corresponding to the surroundings of the current position of the vehicle (particularly forward in the direction of travel of the vehicle). In addition to roads, buildings, road signs, etc. may be modeled in the three-dimensional space, or only roads may be modeled. Alternatively, the three-dimensional space may simply be a blank three-dimensional space with only the ground without modeling roads. The three-dimensional space may be stored in advance as three-dimensional map information in the map information DB 31, and in S42, the three-dimensional map information of the surroundings of the vehicle position may be read from the map information DB 31. The three-dimensional space may also be generated based on an image captured by the front camera 19. For example, by performing point cloud matching on the image captured by the front camera 19, it is possible to detect roads and structures around the roads and generate a three-dimensional space.

In addition, in S42, the CPU 41 also determines the current position and orientation of the vehicle in the generated three-dimensional space based on the parameters detected by the current position detection unit 11. In particular, the position of the front camera 19 installed in the vehicle is set as the current position of the vehicle, and the direction of the optical axis of the front camera 19 is set as the orientation of the vehicle. The position of the front camera 19 also corresponds to the position of the vehicle occupant, and the direction of the optical axis of the front camera 19 also corresponds to the line of sight of the vehicle occupant.

Next, in S43, the CPU 41 generates a guide object to be displayed on the liquid crystal display 15. The shape of the generated guide object is an arrow shape of the same size for the first object 65, the second object 66, the third object 67, and the fourth object 68 (however, even if the objects are the same size, their apparent size changes depending on the distance from the vehicle in the three-dimensional space in which they are placed, as described later). The guide object is a two-dimensional polygon that does not have any thickness. However, it may be a three-dimensional polygon with thickness. As shown in FIG. 14, the first object 65, the second object 66, and the third object 67 are inclined rather than parallel to the road surface ahead of the vehicle in the traveling direction. The inclination direction is a direction corresponding to the moving direction of the lane change, and more specifically, the inclination direction is such that the outer periphery side (the left side when changing lanes to the right, and the right side when changing lanes to the left) is higher in the turning direction during the lane change. The specific values of the inclination angle θ are 90 degrees for the first object 65, 60 degrees for the second object 66, and 30 degrees for the third object 67. Meanwhile, the bottom surface of the fourth object 68 is parallel to the road surface. The arrow directions of the objects are also different: the first object 65 is inclined 90 degrees with respect to the traveling direction of the road to indicate the direction of lane change, the second object 66 is inclined 60 degrees with respect to the traveling direction of the road to indicate the direction of lane change, the third object 67 is inclined 30 degrees with respect to the traveling direction of the road to indicate the direction of lane change, and the fourth object 68 indicates the traveling direction of the road, i.e., the straight ahead direction.

The shape of the guidance object generated in S43 can be changed as appropriate, and can be any shape other than an arrow as long as it can indicate the direction in which a lane change is to be made.

Next, in S44, the CPU 41 places the guide object generated in S43 in the three-dimensional space generated in S42. The position at which the guide object is placed in the three-dimensional space is determined based on the "position at which the guide object is to be superimposed on the landscape" set in S32, S34, S36 to S38, S40, and S41.

For example, when the progress of the current lane change of the vehicle is "phase 0", the arrows of the first object 65, the second object 66, the third object 67, and the fourth object 68 are placed at a predetermined interval on the road surface ahead of the vehicle 60 in the direction of travel in three-dimensional space and at a predetermined distance ahead of the current position of the vehicle 60, as shown in FIG. 9. In particular, they are placed at a position where the distance from the vehicle 60 to the first object 65 is a predetermined distance (for example, 10 m).

Next, in S45, the CPU 41 first acquires an image (hereinafter referred to as a visual image) of the three-dimensional space in which the guide object is arranged, viewed in the direction of travel of the vehicle from the vehicle position (corresponding to the viewpoint) identified in S42. For example, FIG. 15 is a diagram showing a visual image 71 acquired when the vehicle and guide object are arranged in the manner shown in FIG. 9. In particular, since the position of the vehicle is the position of the front camera 19, the acquired visual image 71 is an image that can be viewed when the guide object arranged in the three-dimensional space is viewed in the direction of travel of the vehicle from the viewpoint of the front camera 19, but also corresponds to the field of view of the vehicle occupant.

Then, the CPU 41 stores the shape and position of the guide object contained in the visual image 71 as the shape and position of the guide object to be displayed by the liquid crystal display 15. Here, the shape of the guide object stored in S45 is the shape of the guide object that can be seen when the guide object arranged in three-dimensional space is viewed from the viewpoint of the vehicle (more precisely, the front camera 19). In addition, the position of the guide object stored in S45 is the position of the guide object that can be seen when the guide object arranged in three-dimensional space is viewed from the viewpoint of the vehicle (more precisely, the front camera 19).

Then, in S46, the CPU 41 determines the range in which the guide object is to be displayed on the liquid crystal display 15, based on the shape and position of the guide object stored in S45. The display range of the guide object is the range in which the guide object having the shape stored in S45 is displayed at the same position as the guide object arranged in three-dimensional space. Then, the process proceeds to S4, where a control signal is sent to the liquid crystal display 15 as described above, and an image of the guide object having the shape stored in S45 is displayed on the liquid crystal display 15 in the display range determined in S46.

As a result, the driving guidance screen 51 displayed on the liquid crystal display 15 as the vehicle travels becomes a screen as shown in Figs. 16 to 19. First, when the progress status of the lane change of the vehicle is in "phase 0", as shown in Fig. 16, the images of the first object 65, the second object 66, the third object 67, and the fourth object 68 are displayed superimposed on the scenery 52 in front of the vehicle in the traveling direction captured by the front camera 19. The position where the image of the first object 65 is superimposed is, for example, a position corresponding to 10 m ahead of the vehicle. The images of the first object 65, the second object 66, the third object 67, and the fourth object 68 are viewed in a state where they are arranged along the path of the lane change. The direction of the arrow of each object indicates the direction of the lane change. Therefore, when the vehicle occupant views the driving guidance screen 51, the moving direction and the path of the vehicle during the lane change can be understood from the arrangement and direction of each arrow. During Phase 0, the relative position of each object to the vehicle is fixed, so the display position and size of the image of each displayed object on the screen are also fixed.

After that, when the vehicle starts to change lanes and the distance from the vehicle to the lane marking becomes less than the threshold, the progress of the vehicle's lane change transitions to "Phase 1". In Phase 1, the relative positions of the second object 66, the third object 67, and the fourth object 68 to the vehicle are fixed, so that the display positions and display sizes on the screen of the images of the second object 66, the third object 67, and the fourth object 68 are also fixed, as shown in FIG. 16. On the other hand, the relative position of the first object 65 from the vehicle moves toward the vehicle in accordance with the vehicle's traveling speed, so that the image of the first object 65 moves toward the bottom of the screen so as to approach the vehicle, and the display size gradually increases, as shown in FIG. 16. Then, the image of the first object 65 finally moves off the screen and is excluded from the display targets.

After that, when a predetermined time has elapsed since the transition to Phase 1, the progress status of the lane change of the vehicle transitions to the state of "Phase 2". In the state of Phase 2, the relative position of the second object 66 from the vehicle moves toward the vehicle according to the traveling speed of the vehicle, so that the image of the second object 66 moves downward on the screen so as to approach the vehicle, and the display size also gradually increases, as shown in FIG. 17. Then, the image of the second object 66 finally moves off the screen and is excluded from the display target. On the other hand, the relative positions of the third object 67 and the fourth object 68 from the vehicle move along the road width direction in the opposite direction to the direction of the lane change (to the right if moving to the left lane), so that the images of the third object 67 and the fourth object 68 move to the right side of the screen without changing the display size, as shown in FIG. 17. More specifically, the display position moves so that the image of the third object 67 is always located near the dividing line and the image of the fourth object 68 is always located near the center of the target lane in accordance with the change in the scenery 52.

After that, when the distance from the vehicle to the lane marking becomes equal to or greater than the threshold, the progress of the vehicle's lane change transitions to "Phase 3". In Phase 3, the position of the fourth object 68 relative to the vehicle is fixed, so the display position and display size on the screen of the image of the fourth object 68 are also fixed, as shown in FIG. 18. On the other hand, the position of the third object 67 relative to the vehicle moves toward the vehicle in accordance with the vehicle's traveling speed, so the image of the third object 67 moves toward the bottom of the screen so as to approach the vehicle, and the display size gradually increases, as shown in FIG. 18. Then, the image of the third object 67 finally moves off the screen and is excluded from the display.

After that, when a predetermined time has elapsed since the transition to Phase 3, the progress of the vehicle's lane change transitions to "Phase 4". In Phase 4, the relative position of the fourth object 68 from the vehicle moves toward the vehicle in accordance with the vehicle's traveling speed, so that the image of the fourth object 68 moves toward the bottom of the screen so as to approach the vehicle, and the display size gradually increases, as shown in FIG. 19. Then, the image of the fourth object 68 finally moves off the screen and is removed from the display.

As a result, when the vehicle occupant visually checks the driving guidance screen 51, the vehicle can grasp the moving direction and the course of the vehicle in the lane change from the arrangement of the image of the guidance object consisting of multiple arrows and the direction of each arrow before the lane change. In addition, after the lane change is started, the display position of the image of the guidance object corresponding to the current lane change progress of the vehicle is moved in a moving manner corresponding to the current lane change progress of the vehicle, for each object. Note that the lane change progress identified in the first embodiment is not detected accurately, but is predicted using the relative position with respect to the dividing line and the elapsed time, so there is some error in identifying the lane change progress of the vehicle, which creates some discomfort in the display position and shape of each object, but the guidance object as a whole does not cause discomfort in the correspondence with the scenery, and effective guidance is possible. In addition, by visually checking the image of the guidance object, the vehicle occupant can also grasp whether the lane change is before the lane change, the lane change is in progress, or the lane change is completed.

As described above in detail, according to the navigation device 1, the method for drawing an image of a guidance object, and the computer program executed by the navigation device 1 in the first embodiment, when a guidance object for guiding a vehicle to change lanes is displayed superimposed on the scenery around the vehicle, the guidance object is made into a plurality of objects spaced apart from each other, the progress of the vehicle's lane change is acquired (S2), and the display mode of the plurality of objects is changed and displayed according to the progress of the vehicle's lane change for each object (S3, S4), so that even if there is some discomfort in the display position or shape of each object, guidance that does not give a sense of discomfort in the progress of the lane change for the objects as a whole is possible. Therefore, detailed information for identifying the progress of the lane change is not required compared to the conventional method, and the processing load can be reduced.
In addition, the progress of the vehicle's lane change is identified and acquired based on the relative positional relationship between the vehicle and the lane marking that is to be crossed by the lane change and the elapsed time (S11 to S20), so it is possible to estimate the progress of the vehicle's lane change while reducing the amount of information required to identify the progress of the lane change compared to the conventional method. Therefore, the processing load related to identifying the progress of the vehicle's lane change can be reduced.
In addition, the progress of a vehicle's lane change is divided into at least three stages: the stage of traveling in the lane before the lane change, the stage of changing the lane, and the stage of traveling in the lane after the lane change. Therefore, even if the progress of the lane change is not divided into detailed stages, by changing the display mode of the guidance object according to the minimum necessary division, it is possible to provide guidance that does not create a sense of incongruity regarding the progress of the lane change.
In addition, the progress of the vehicle's lane change is divided into a plurality of stages, and the object to be moved in display position and the movement pattern are set for each of the plurality of stages on an object-by-object basis, and the display position of the object corresponding to the progress of the current vehicle's lane change is moved in the movement pattern corresponding to the progress of the current vehicle's lane change (S31 to S41), making it possible to provide guidance that does not create a sense of incongruity regarding the progress of the lane change.
In addition, a plurality of objects are arranged along the path of the vehicle's lane change, and the display mode is changed and displayed through a first step of moving the objects a predetermined distance in the direction opposite to the lane change as the vehicle progresses in the lane change, and a second step of moving the objects moved in the first step toward the vehicle (S31 to S41).By moving the objects horizontally and vertically according to the progress of the vehicle's lane change, it is possible to provide guidance that does not create an awkward feeling about the correspondence between the guidance objects and the scenery as viewed by the occupants of the vehicle moving due to the lane change.

[Second embodiment]
Next, a superimposed image display device according to a second embodiment will be described with reference to Fig. 20 and Fig. 21. In the following description, the same reference numerals as those in the configuration of the superimposed image display device according to the first embodiment shown in Figs. 1 to 19 indicate the same or corresponding parts as those in the configuration of the superimposed image display device according to the first embodiment.

The schematic configuration of the superimposed image display device according to the second embodiment is almost the same as that of the superimposed image display device according to the first embodiment. In addition, various control processes are also almost the same as those of the superimposed image display device according to the first embodiment.
However, the superimposed image display device of the first embodiment differs in that it displays an image captured by the front camera 19 on the liquid crystal display 15 of the navigation device 1, and further displays a guide object on the liquid crystal display 15, thereby superimposing the guide object on the scenery around the vehicle, whereas the superimposed image display device of the second embodiment uses a head-up display system as a means for displaying an image to be superimposed on the scenery around the vehicle.

A schematic configuration of the superimposed image display device according to the second embodiment will be described below with reference to Fig. 20. Fig. 20 is a schematic configuration diagram of a superimposed image display device 101 according to the second embodiment.
20 , the superimposed image display device 101 basically includes a navigation device 103 mounted on a vehicle 102, and a front display 104 also mounted on the vehicle 102 and connected to the navigation device 103. The front display 104 functions as a head-up display together with a windshield 105 of the vehicle 102, and serves as an information providing means for providing various information to an occupant 106 of the vehicle 102.

Here, the front display 104 is a liquid crystal display that is installed inside the dashboard 107 of the vehicle 102 and has the function of displaying images on an image display surface provided on the front. For example, a CCFL (cold cathode fluorescent lamp) or a white LED is used as the backlight. Note that, in addition to a liquid crystal display, the front display 104 may also be an organic electroluminescence display or a combination of a liquid crystal projector and a screen.

The front display 104 functions as a head-up display together with the windshield 105 of the vehicle 102, and is configured to reflect the image output from the front display 104 on the windshield 105 in front of the driver's seat so that it can be viewed by the occupants 106 of the vehicle 102. Note that the front display 104 displays a guide object as necessary. Note that in the second embodiment described below, the guide object is displayed at a timing when it is recommended to change lanes, as in the first embodiment, and serves as guide information to guide the vehicle to change lanes (i.e., to encourage the vehicle to change lanes). More specifically, it is displayed superimposed on the road surface ahead of the vehicle in the direction of travel, and is a plurality of arrows that indicate the direction of movement of the vehicle when changing lanes.

When the occupant 106 sees the image displayed on the front display 104 reflected by the windshield 105, the image displayed on the front display 104 is perceived by the occupant 106 not at the position of the windshield 105, but at a distant position beyond the windshield 105, as a virtual image 110. The virtual image 110 is displayed superimposed on the surrounding environment (landscape, real scene) in front of the vehicle, and can be displayed superimposed on any object (road surface, building, object that is the subject of a warning, etc.) located in front of the vehicle.

The position where the virtual image 110 is generated, more specifically the distance L from the occupant 106 to the virtual image 110 (hereinafter referred to as the imaging distance), is determined by the position of the front display 104. For example, the imaging distance L is determined by the distance (optical path length) along the optical path from the position where the image is displayed on the front display 104 to the windshield 105. For example, the optical path length is set so that the imaging distance L is 1.5 m.

A front camera 111 is also installed above the vehicle's front bumper or behind the rearview mirror. The front camera 111 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis direction facing forward in the direction of travel of the vehicle. Image processing is then performed on the image captured by the front camera 111 to detect the condition of the forward environment (i.e. the environment on which the virtual image 110 is superimposed) as viewed by the occupant 106 through the windshield. Note that a sensor such as a millimeter wave radar may be used instead of the front camera 111.

An in-vehicle camera 112 is also installed on the top surface of the vehicle's instrument panel. The in-vehicle camera 112 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis direction facing the driver's seat. The range in the vehicle where the face of an occupant is generally expected to be located is set as the detection range (imaging range of the in-vehicle camera 112), and an image of the face of the occupant 106 sitting in the driver's seat is captured. Then, image processing is performed on the captured image captured by the in-vehicle camera 112 to detect the eye position (starting point of gaze) and gaze direction of the occupant 106.

The superimposed image display device according to the second embodiment displays an image 120 of a guide object on the front display 104 as shown in FIG. 21 in S4 of the driving support processing program (FIG. 2) described above. As a result, when a vehicle occupant visually recognizes the image 120 of a guide object displayed on the front display 104 as shown in FIG. 21, a virtual image 121 of the image 120 of the guide object is visually recognized superimposed on the scenery seen through the windshield 105.

As a result, it becomes possible to grasp the direction of movement and course of the vehicle when changing lanes, just like the superimposed image display device according to the first embodiment. In the superimposed image display device according to the second embodiment, the shape of the guide object to be displayed on the front display 104 and the position (range) in which the guide object is to be displayed are determined in the guide object display position determination process of S3. In addition, it is preferable that the current position and orientation of the vehicle identified in three-dimensional space in S42 be the position of the vehicle occupant and the line of sight of the occupant detected using the in-vehicle camera 112.

Incidentally, the present invention is not limited to the above-described embodiment, and it is needless to say that various improvements and modifications are possible without departing from the spirit and scope of the present invention.
For example, as a means for displaying an image to be superimposed on the scenery around the vehicle, the first embodiment uses the liquid crystal display 15 that displays an image of the actual scenery, and the second embodiment uses a head-up display system, but a windshield display (WSD) that displays an image on the windshield may also be used. In the WSD, the windshield may be used as a screen to display an image from a projector, or the windshield may be a transparent liquid crystal display. The image displayed on the windshield by the WSD becomes an image to be superimposed on the scenery around the vehicle.

In the first and second embodiments, the guide objects are four arrow-shaped objects arranged along the lane change path, but the object shapes may be other than arrows. For example, they may be circular or rectangular images. The number of objects is not limited to four, and may be, for example, three or five.

In the first and second embodiments, when the progress of the lane change of the vehicle is "phase 1", only the display position of the first object 65 is moved toward the vehicle, and the display positions of the second object 66, the third object 67, and the fourth object 68 are fixed. However, the display positions of the second object 66, the third object 67, and the fourth object 68 may be moved in the opposite direction to the lane change along the road width direction. In the first and second embodiments, the display position of the guidance object is changed according to the progress of the lane change of the vehicle. However, in addition to changing the display position, the display size or the display color may be changed.

In the first embodiment, the real-world image captured by the front camera 19 and the guide objects are displayed on the LCD display 15 of the navigation device 1, but the display that displays the real-world image and the guide objects may be a display other than the LCD display 15 as long as it is a display disposed within the vehicle.

In the second embodiment, the front display 104 is configured to generate a virtual image in front of the windshield 105 of the vehicle 102, but the virtual image may be generated in front of a window other than the windshield 105. In addition, the object onto which the image is reflected by the front display 104 may not be the windshield 105 itself, but a visor (combiner) installed around the windshield 105.

In the first and second embodiments, the navigation ECU 13 of the navigation device 1 is configured to execute the processing of the driving assistance processing program (Figure 2), but the execution entity can be changed as appropriate. For example, the processing may be executed by the control unit of the liquid crystal display 15, the vehicle control ECU, or other in-vehicle device.

1...navigation device, 15...liquid crystal display, 19...front camera, 41...CPU, 42...RAM, 43...ROM, 51...travel guidance screen, 52...landscape, 53...image of guidance object, 60...vehicle, 61, 62...lane, 63...lane boundary line (dividing line), 65...first object, 66...second object, 67...third object, 68...fourth object

Claims (4)

A superimposed image display device is mounted on a vehicle, and displays a guide object for providing information to a vehicle occupant by superimposing the guide object on a landscape around the vehicle, the superimposed image display device comprising:
an object display means for displaying the guidance object for guiding the vehicle to change lanes;
a progress status acquisition means for acquiring a progress status of a lane change of a vehicle,
The progress of the vehicle in changing lanes is divided into a plurality of stages;
the guide object is composed of a plurality of objects spaced apart from each other, and an object to be moved as a display position and a manner of movement are set for each of the plurality of steps on an object-by-object basis;
The object display means is a superimposed image display device that displays objects set as targets for moving their display positions in the progress of the current vehicle's lane change by moving the display positions of the objects in a set movement manner on an object-by-object basis. A superimposed image display device is mounted on a vehicle, and displays a guide object for providing information to a vehicle occupant by superimposing the guide object on a landscape around the vehicle, the superimposed image display device comprising:
an object display means for displaying the guidance object for guiding the vehicle to change lanes;
a progress status acquisition means for acquiring a progress status of a lane change of a vehicle,
The guide object is made up of a plurality of objects spaced apart from one another,
The object display means includes:
arranging the plurality of objects along a lane change path of the vehicle;
A superimposed image display device that changes and displays the display mode by performing a first step of moving the plurality of objects, on an object-by-object basis, a predetermined distance in a direction opposite to the direction of the lane change as the vehicle 's lane change progresses, and a second step of moving the objects that have been moved in the first step toward the vehicle . The superimposed image display device according to claim 1 or claim 2, wherein the progress acquisition means identifies and acquires the progress of the vehicle's lane change based on the relative positional relationship between the vehicle and the dividing line that will be crossed by the lane change and the elapsed time . 4. The superimposed image display device according to claim 1, wherein the progress of the vehicle in changing lanes is divided into at least three stages: a stage in which the vehicle is traveling in the lane before the lane change, a stage in which the lane is being changed, and a stage in which the vehicle is traveling in the lane after the lane change.

Images