KR102339522B1 - Integrated vehicle and driving information display method and apparatus - Google Patents

Abstract

A multi-purpose display that provides vehicle information and driving information by integrating HUD, Cluster, and AVN functions according to mode selection without obstruction of the driver's line of sight without designing a light device using only one display panel. Apparatus and method. According to the present invention, a procedure for acquiring an image of a driver by a second camera, a procedure for acquiring an image in front of a vehicle by a first camera, a procedure for detecting a forward gaze area of the driver, and a vehicle acquired by the first camera There is provided a vehicle information or driving information integrated display method including a procedure of extracting a portion of the front image corresponding to the detected forward gaze region of the driver from the front image and displaying the extracted part of the front image on a display panel for displaying vehicle information or driving information.

Description

Vehicle/driving information integrated display method and apparatus {Integrated vehicle and driving information display method and apparatus}

The present invention relates to a multi-purpose vehicle/driving information integrated display method and apparatus for performing HUD, cluster, and AVN functions according to display modes using a single display panel.

A cluster is a device such as an instrument panel that shows vehicle information such as SCC and LDWS or operation information necessary for driving such as the driving state of the vehicle and the operation status of various devices, and is a device such as an engine speed, speed, direction indication, trip computer, It is designed so that various warning indicators can be easily and quickly recognized at a glance.

AVN (audio video navigation system) is a vehicle device that integrates in-vehicle audio, multimedia device, and navigation device to provide a multimedia service and vehicle information or driving information as one system. The multimedia system and the navigation system, which were independently provided to the vehicle, are integrated into one system, and it is to guide the driving route or driving information when driving the vehicle, or to provide broadcasting and multimedia elements.

A head up display (HUD) is a device that displays vehicle information or driving information on the windshield (windshield) side of the front sight to match the driver's eye level, not the conventional instrument panel behind the steering wheel. Conventionally, there is a HUD device that displays driving information by installing a general flat panel on the windshield side at the driver's eye level, but this has a fatal disadvantage of obscuring the front sight while driving. A recent HUD is a virtual image optical-based display that forms an image on a windshield, and the display panel has a light-transmitting property to secure a forward view.

Recently, there is a combiner type HUD using a transmissive OLED panel, and there is a cluster/HUD integrated display using a folding type transmissive single panel.

The prior art has the following problems. In order to display an image in a projection type, a bulky and difficult to design light fixture must be included, and correction is required for each vehicle model due to the difference in windshields between vehicle models. And since it emits strong brightness, it generates high heat and there is a problem of heat dissipation.

In addition, in the case of conventional flat panel panels such as LCD and OLED, the transmittance is low or light does not pass through, so it blocks the front sight, making it difficult to function as a HUD. In addition, an independent panel and controller must be used for each cluster, HUD, and AVN, and in the case of a foldable cluster/HUD system, only a transparent display panel can be used. And since the display image is displayed fixed at the driver's eye level, the display area is not automatically moved according to the driver's eye level movement. In the case of the conventional optical-based HUD, image defects such as brightness degradation, distortion, and double image of the displayed image occur.

A multi-purpose display that provides vehicle information and driving information by integrating HUD, Cluster, and AVN functions according to mode selection without obstruction of the driver's line of sight without designing a light device using only one display panel. We would like to propose a device.

In order to solve the above problems, according to one aspect of the present invention, a procedure for acquiring an image of a driver by a second camera, a procedure for acquiring an image in front of a vehicle by a first camera, and a method for detecting a forward gaze area of the driver Procedure, vehicle information or driving including a procedure of extracting a portion of a front image corresponding to the detected front sight area of the driver from the vehicle front image acquired with the first camera and displaying it on a vehicle information or driving information display panel An information integrated display method is provided.

Hereinafter, the driver's image is referred to as 'driver image', the image in front of the vehicle is referred to as 'vehicle front image', the driver's forward gaze area is referred to as 'driver forward lined area', and the portion of the front image is referred to as 'front image portion'. to be abbreviated The distinction of each term will become clear in the description of the specification below.

In the process of displaying the front image portion corresponding to the driver's forward gaze area on the display panel, the front image portion may be overlapped with vehicle information or driving information displayed on the display.

In an embodiment, the procedure for detecting the forward gaze area of the driver may be performed as a procedure of calculating the size and position of the forward gaze area based on the tilt angle of the display panel or the horizontal position, tilt angle, and height of the driver's seat seat. . In another embodiment, in the procedure for detecting the driver's forward gaze region, the driver's eyes are detected from the driver's image acquired by the second camera, and the region around the driver's gaze looking forward based on the height of the eyes and the left and right positions. can be detected.

In the above method, the first camera is installed to face the front of the vehicle from the rear of the display panel, and the second camera is installed to face the driver from the front of the display panel.

On the other hand, according to another aspect of the present invention for solving the above problems, a display panel for displaying vehicle information or driving information, a first camera for acquiring a vehicle front image viewed from the rear of the display panel to the front of the vehicle, the driver's means for detecting a current forward gaze region; and means for extracting a portion of a forward image corresponding to the detected forward gaze region of the driver from the vehicle front image acquired with the first camera and displaying the extracted portion on the display panel.

Here, the display panel is movable from a first position, which is the position of the cluster of the vehicle, to a second position vertically upward, so that it functions as a cluster at the first position and functions as a HUD or AVN at the second position. have.

In addition, when the front image portion is displayed on the display panel, the front image portion may be displayed in an overlapping manner with vehicle information or driving information displayed on the display.

In addition, the second camera is preferably an infrared camera. This makes it possible to acquire the driver's image with the camera regardless of day or night.

In an exemplary embodiment, the driver's forward gaze region detecting unit may calculate the size and position of the forward gaze region based on a tilt angle of the display panel or a horizontal position, tilt angle, and height of the driver's seat. In another embodiment, a second camera for acquiring an image of the driver on the front of the display panel is further included, and by detecting the driver's eyes from the driver's image acquired by the second camera, the height and left and right positions of the eyes are determined. As a reference, it is possible to detect an area around the line of sight in which the driver gazes forward.

In addition, the driver's forward gaze region detecting means and the forward image portion extracting means corresponding to the driver forward gaze region may operate when the display panel is in the second position.

In the device, the first camera is installed to face the front of the vehicle from the rear of the display panel, and the second camera is installed to face the driver from the front of the display panel. In addition, the driver's forward gaze region detecting means, the forward image portion extracting means corresponding to the driver's forward gaze region, and the display display means may be integrally implemented as one processor.

It is a panel display HUD, not an optical projection HUD, and at the same time, it can serve as a cluster and AVN functions, so there is an effect of reducing product cost and increasing utility.

Performance improvement: A HUD image with less distortion can be provided to the driver regardless of changes or differences in the vehicle windshield. Compared to the optical-based transmissive HUD, the screen contrast ratio is excellent.

Convenience improvement: In addition to driving, it is possible to replace the function of the existing AVN device by displaying the multimedia display image.

Reduced number of parts and reduced failure rate: The number of parts can be reduced because the optics of the existing system are not required. In addition, AVN panels and HUD optical systems become unnecessary. This reduces the failure rate of the product.

1 and 2 are schematic diagrams for explaining the concept of the first embodiment of the present invention;
3 and 4 are schematic diagrams of a device according to the present invention;
5 is an exemplary view of the panel rear camera front image 27 acquired by the panel rear camera 30 and the obscured scene image 28 extracted therefrom
6 is an exemplary view of the installation position of the panel front camera 20 and the panel rear camera 30
7 is an exemplary view showing a gap 40 formed by a bezel region.
8 illustrates one hardware implementation form of the vehicle/driving information integrated display device according to the present invention.
9 is an operation flowchart of an integrated display method and apparatus according to the present invention;

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings.

1 and 2 are schematic diagrams for explaining the concept of the first embodiment of the present invention. The display panel 10 is moved between the lower cluster position and the upper HUD or AVN function position in a sliding manner by the driving unit (or manually) (10'). That is, when the display panel 10 is positioned below, it functions as a general cluster instrument (cluster mode M1), and when the display panel 10 is moved upward (10'), it serves as a display of HUD or AVN (HUD or AVN mode M2). After moving the display panel 10 to the HUD or AVN mode M2 position in FIG. 2 , it is possible to tilt the panel at a predetermined angle for convenience of use (tilting function).

However, when the display panel 10 is in the cluster mode, there is no problem in securing the front view, but in the HUD or AVN mode, the driver's line of sight 12 is blocked by the display panel 10 ′, thereby blocking the front view of the vehicle. The area of the front field of view that is covered will be slightly different depending on the tilting position of the display panel 10'.

Since the present invention uses a flat panel display panel instead of a windshield projection HUD, in order to solve the problem that the driver's view is blocked by the display panel 10' in the HUD or AVN mode, in the first embodiment of the present invention, Two cameras were used.

3 and 4 conceptually show the principle of the device according to the present invention.

It consists of a display panel 10 positioned in front of the driver 14 , a panel front camera 20 positioned in front of the panel to acquire a driver image, and a panel rear camera 30 positioned in front of the panel to acquire a front image of the rear panel.

When the driver 14 sits in the driver's seat and looks forward in the HUD or AVN mode state of FIG. 1 , the front scene 26 entering the front view 16 of the driver 14 by the display panel 10 moved upwards ), a scene 28 that is obscured in front by the field of view 18 obscured by the display panel 10 occurs. Accordingly, the driver cannot see the scene 28 in which the display panel 10 covers the front view 18 among the front scenes 26 .

The front panel camera 20 detects the driver's eyes 15 from the image by acquiring the driver's image regardless of day or night by using an infrared LED. The processor (eg, GPU) calculates the forward gaze 22 of the driver 14 based on the height and left and right positions of the eyes 15 from the detected information (at this time, the image through eye tracking) matching algorithm can be used). And the panel rear camera 30 acquires the front image 27 of the panel rear camera by photographing the rear surface of the display panel 10 , that is, the vehicle front region 24 .

The processor extracts (separates) the image corresponding to the calculated driver's forward gaze, ie, the image of the obscured scene 28, from the thus-obtained panel rear camera front image 27 and displays it on the display panel 10 . In the display on the panel, vehicle information or driving information to be originally displayed on the display panel 10 and the extracted image of the scene 28 are superimposed and displayed.

Accordingly, the driver can receive the original vehicle information or driving information through the display panel 10 and at the same time, view the front scene portion corresponding to the portion covered by the display panel 10 auxiliary through the display panel 10 . Therefore, it is possible to safely drive without obstructing the forward view.

Since the front panel camera 20 detects the driver 14's eyes 15 from the acquired driver image and calculates the driver's front field of view 18, the display panel 10 is moved upward to enter the HUD or AVN mode. When using the tilting function that adjusts the angle when in use, when the driver's position changes while driving (bend or lift the head, lean forward or lean back, etc.), and when the driver's seat It is possible to automatically calculate the driver's front field of view 18 immediately in response to a change in position, so that an image covered by the display panel 10 can be accurately separated and extracted and displayed overlaid on the display panel 10 .

5 shows a panel rear camera front image 27 acquired by the panel rear camera 30 and an obscured scene image 28 extracted therefrom. It is a basic concept of the present invention to display the extracted obscured scene image 28 on the display panel 10 so that the driver can see all the scenes ahead without missing it.

As shown in FIG. 6 , the front panel camera 20 may be installed in a non-display area, for example, a bezel area 34 , in which the image panel 32 is not located on the front side of the display panel 10 , and may be installed in front of the rear panel of the display panel 10 . The panel rear camera 30 for acquiring an image can be installed on the rear surface 38 of the display panel 10 . It can be seen that the infrared LED 36 is installed together around the front panel camera 20 .

Of course, FIG. 6 is only an exemplary diagram, and in actual implementation, various other forms of implementation will be possible. For example, the front panel camera 20 may not necessarily be installed on the front of the display panel 10 , but may be installed on the dashboard, A-pillar, windshield, rearview mirror, or the like. The rear panel camera 30 may likewise be installed on a windshield, dashboard, rear of the rearview mirror, or any other location on the front of the vehicle.

In addition, in FIG. 7 , a bezel area (not shown in FIG. 6 ) of the display panel 10 is used by using an image matching algorithm to detect the front view 22 of the driver 14 and calculate the front view 18 . 34), indicating that the complete front image can be reproduced without the image gap 40 formed by the bezel.

8 illustrates one hardware implementation form of the vehicle/driving information integrated display device according to the present invention. And Figure 9 is an operation flowchart of the integrated display method and apparatus according to the present invention. An operation of an embodiment according to the present invention will be described with reference to FIGS. 8 and 9 .

First, referring to FIG. 8 , the memory controller stores vehicle information 81 such as SCC and LDWS and external images 82 such as rear camera and mobile device images in the image storage memory 85 included in the operating system 84 . 83, and a graphic processor (GPU) 86 that retrieves an image from the image storage memory 85 through the operating system 84 and processes the image to display the image on the display panel to output an overlaid image, and the GPU ( 86), there is a panel image display unit 87 to display the superimposed image output on the display panel (10).

The above configuration is a hardware configuration for displaying a typical panel image. In addition to this, in the present invention, an image acquired from the front panel camera 20 and the rear panel camera 30 is received to recognize the driver's eye position (ie, gaze). And from this, the panel rear image output from the gaze recognition and image processing unit 50 that performs image processing according to the present invention (that is, the hidden scene image finally extracted from the front image 27 of the panel rear camera shown in FIGS. 3 to 5 ) 28) is input to the memory control unit 83.

In FIG. 9 , when the operation of the image panel and the camera starts ( 51 ), the front panel camera 20 acquires an image of the driver sitting in the driver's seat ( 52 ). From this image, the gaze recognition and image processing unit 50 recognizes the driver's eye position (line of sight) (53) and calculates the forward gaze area the driver is looking at (54). Meanwhile, the panel rear camera 30 acquires a front image of the vehicle (ie, a rear image of the display panel 10) (55). From the acquired front image of the vehicle, the driver's forward gaze region calculated in step 54 is cut out (56), and the cut image is output as a panel rear image 80 as shown in FIG. The panel rear image 80 is stored in the memory 85 frame by frame by the memory controller 83 and the operating system 84 (57). After that, the GPU 86 reads the memory 85 for each image line in order to bring the panel rear image 80 from the memory 85 (58), and vehicle information or driving information to be originally displayed on the display panel 10 (or other images) and superimposed (59). The panel image display unit 87 displays the superimposed image 88 on the display panel 10 (60). Accordingly, the driver can see a scene covered by the display panel 10 in the forward gaze area through the display panel 10 by the processing technology of the present invention.

Meanwhile, in the second embodiment of the present invention, the object of the invention can be achieved without the second camera, that is, the front panel camera 20 . For example, by receiving the tilt angle data and seat position (horizontal position, tilting, height) data of the display panel 10 arbitrarily adjusted by the driver, the processor calculates the driver's forward gaze area and performs image matching processing. . Since the area and position of the forward gaze region that the driver looks at will vary according to the tilting angle of the display panel 10 , the size and position of the forward gaze region that is covered according to the tilting angle of the display panel 10 is calculated (or tilted). By reading the size and position information of the forward gaze area stored in advance for each angle), image matching processing can be performed.

Similarly, since the area and position of the forward-looking area that the driver is looking at will vary depending on the horizontal position, tilting angle, and height of the driver's seat, calculate the size and position of the forward-looking area that is covered according to these seat positions (or each By reading the size and position information of the forward gaze area stored in advance for each sheet data), image matching processing will be possible.

Some specific embodiments of the present invention have been described above. However, the technical scope of the present invention is not limited only to these examples. The technical scope or scope of the present invention is determined by a rational interpretation of the claims made below.

Display panel (10) in cluster mode, display panel (10') in HUD or AVN mode, driver's line of sight (12), driver (14), driver's front sight (16), driver's front view (18) , panel front camera 20, driver's front sight 22, panel rear camera shooting range 24, front scene 26, panel rear camera acquisition front image 27, obscured scene image 28, Back Panel Camera(30), Image Panel(32), Bezel Area(34), Infrared LED(36), Display Panel Back(38)

Claims (11)

A procedure of acquiring an image in front of the vehicle by the first camera and displaying it on a display panel;
a procedure of detecting the driver's forward gaze region from the driver's image acquired using the second camera in order to detect a region where the driver's forward gaze is obscured by the display panel; and
and displaying a vehicle front image portion corresponding to the detected forward gaze area of the driver on the display panel. The method according to claim 1, wherein the procedure for detecting the driver's forward gaze area comprises:
and a procedure for calculating the size and position of the forward gaze area based on the tilt angle of the display panel or the horizontal position, tilt angle, and height of the driver's seat. According to claim 1, wherein the vehicle front image portion corresponding to the detected driver's forward gaze region
Vehicle information or driving information integrated display method, characterized in that extracted from the vehicle front image acquired with the first camera. The method of claim 1, wherein the process of displaying the front image portion of the vehicle corresponding to the driver's forward gaze area on the display panel comprises:
and displaying the front image of the vehicle by overlapping the vehicle information or operation information of the vehicle displayed on the display panel. A display panel that displays vehicle information or driving information;
A first camera for acquiring a vehicle front image viewed from the rear of the display panel to the front of the vehicle,
a second camera that acquires an image of the driver in order to detect an area in which the driver's front gaze is obscured by the display panel;
means for detecting the driver's forward gaze region from the driver's image acquired using the second camera; and
and means for displaying, on the display panel, a portion of the front image of the vehicle corresponding to the forward-looking area of the driver detected by the means for detecting the forward-looking area of the driver. The method of claim 5, wherein the display panel
Vehicle information or Driving information integrated display device. The method of claim 5, wherein the driver's forward gaze area detecting means comprises:
and means for calculating the size and position of the forward gaze area based on the tilt angle of the display panel or the horizontal position, tilt angle, and height of the driver's seat. The method of claim 5, wherein the driver's forward gaze area detecting means comprises:
and means for detecting the driver's eyes from the driver's image acquired by the second camera and detecting a forward gaze area based on the height and left and right positions of the eyes. The vehicle information or driving information according to claim 6, wherein the driver's forward gaze region detecting means and the forward image part extraction means corresponding to the driver's forward gaze region operate when the display panel is in the second position. Integrated display unit. According to claim 5, wherein the means for displaying the vehicle front image portion on the display panel,
The vehicle information or operation information integrated display apparatus further comprising a means for overlapping the vehicle information or operation information displayed on the display panel to display the front image of the vehicle. The method according to claim 5, wherein the means for displaying the front image of the vehicle on the display panel
The vehicle information or driving information integrated display device, characterized in that the portion of the vehicle front image corresponding to the detected forward gaze area of the driver is extracted from the vehicle front image acquired with the first camera.

Images