DE102018000859A1 - Depicting augmented reality - Google Patents

Abstract

The invention relates to a method and display device (100) for presenting augmented reality. According to the invention, it is determined whether a user of the display device (100) is color-defective. If the user is color-misguided, an object (304) located in a field of view of the user is identified which has a color whose perceptibility is impaired by the user due to color vision deficiency, and a designation (301) of the color is seen by the user is displayed, so that the name (301) is assigned to the object (304).

Description

The invention relates to a method and a display device for displaying augmented reality.

Augmented reality is understood to mean the extension of a real situation by a virtual additional content. The additional content can be displayed in a direct view or in a representation of the situation. The additional content is for example a text or a graphic that is visually detectable for a viewer of the situation.

The additional content is usually generated by means of a computer and output by means of a display device. It is possible that the real situation is directly viewed by the viewer and only the supplemental content is viewed by the viewer on the display device, which is located in the field of view of the viewer. The field of view of a viewer is understood here to mean a spatial area which contains all visual objects that can be fixed with the eyes of the beholder.

As an alternative to directly viewing the real situation, the situation and the additional content can be displayed together by means of the display device. Thus, a picture of the real situation can be shown, in which the additional content is displayed. In this case, the additional content can completely or partially overlay a displayed image of the real situation.

Color-blind persons typically can not perceive some or all of the colors or can only distinguish between them to a limited extent. Color vision problems affect especially males. A common form of color vision deficiency is the red-green visual impairment, in which a person can not distinguish the colors red and green or worse than normal-sighted persons from each other. A color vision deficiency can impair the perception of a real situation and in particular make it difficult to recognize certain aspects of the situation.

Colors are often used to label objects. For example, flags or warning symbols are often colored to visually highlight them. Among other things, colored signs and warning symbols are used for traffic regulation and guidance, for example in the form of colored traffic signs and colored light signals. Color-blind persons may have difficulty perceiving such signs and warning symbols or interpreting them correctly.

A color vision deficiency can restrict a person's ability to drive. There is a risk that this person may overlook or misinterpret a sign or a warning symbol, so that traffic safety is impaired.

The invention has for its object to support a color vision viewer of a real situation.

• - Ermitteln, ob ein Nutzer der Anzeigevorrichtung farbenfehlsichtig ist, und, wenn der Nutzer farbenfehlsichtig ist,
• - Identifizieren eines in einem Blickfeld des Nutzers angeordneten Objektes, welches eine Farbe aufweist, deren Wahrnehmbarkeit durch den Nutzer infolge der Farbenfehlsichtigkeit beeinträchtigt ist, und
• - Anzeigen einer Bezeichnung der Farbe im Blickfeld des Nutzers, so dass die Bezeichnung dem Objekt zugeordnet wird.

The object is achieved with the method according to the invention for displaying augmented reality by means of a display device. The method comprises the following steps:

• Determining whether a user of the display device is color-defective and, if the user is color-defective,
• Identifying an object arranged in a field of view of the user, which has a color whose perceptibility is impaired by the user as a result of color vision deficiency, and
• - Display a name of the color in the field of view of the user, so that the name is assigned to the object.

Consequently, the invention provides a display device with which a user with a color vision is indicated a designation of the color of an object whose perceptibility is impaired by the user as a result of color vision deficiency.

According to the invention, it is determined whether the user is color-defective. In this case, an object arranged in a field of view of the user is identified which has a color whose perceptibility is impaired by the user as a result of color vision deficiency. Furthermore, a designation of the color of the object is displayed in the field of view of the user, so that the designation is assigned to the object.

Thus, the invention provides a representation of augmented reality for a color misguided user of a display device, wherein the user's perception of reality is enhanced by the designation of the color of an object whose perceptibility is impaired by the user due to color vision deficiency.

In one embodiment of the invention, to determine whether the user is color misguided, personal data of the user is transmitted from a user's portable electronic device to the display device, for example from an external data source.

This is advantageous because a color deficiency of the user of the display device in personal data of the user can be noted. By transmitting the data to the display device can be determined by means of the display device based on the data, whether the user is color defective. For transmitting the data to the display device, a portable electronic device of the user is provided. As a result, the data can be transmitted to the display device in a particularly simple and / or automated manner. As an alternative to the user, the portable electronic device can also be assigned to another person.

The portable electronic device is, for example, a mobile phone, a portable computer, a smart card, an implant, or a vehicle key that may be provided with a biometric sensor. The external data source is, for example, another portable electronic device, a stationary computer, a server, or a data cloud service.

By transferring the personal data from the external data source to the portable electronic device, the data does not need to be permanently stored in the portable electronic device. However, the portable electronic device may also include a storage unit in which the data is permanently stored.

In a further embodiment of the invention, the object is identified taking into account its color, its shape and / or its position.

This ensures that only one relevant to the user object is identified, so that denoting the color of an irrelevant to the user object is omitted. An object that is relevant for the user is, for example, a traffic light system, such as a traffic light or a light signal at a railway crossing, or another traffic sign in a traffic route used by the user.

The consideration of the color makes it possible to identify an object whose perceptibility is impaired by the user due to its color vision deficiency. For example, it is a red object or a green object. The object can also be present as a single element of a complex object. However, for the user, not every object in his field of vision is usually relevant which has a color whose perceptibility is impaired by the user as a result of color vision deficiency. Therefore, as an alternative or in addition to the color, further features can serve to identify the object.

In particular, the consideration of the shape makes it possible to identify an object that is recognizable as being relevant on the basis of its shape, that is, on the basis of its external shape and size. These are, for example, a traffic light, a stop sign or a St. Andrew's Cross.

In addition, the consideration of the position makes it possible to identify an object that is recognizable as being relevant on the basis of its arrangement in space. This is, for example, an object that is located in a traffic route used by the user and / or to which the user is currently moving purposefully.

In a further embodiment of the invention, the object belongs to a traffic signal system.

This takes into account that colored light signals, such as red and green light signals, are often used to output alerts, warnings and all-clears, or to regulate traffic. Since color vision deficiency often occurs as red-green vision weakness, the display of a designation of the color of a light signal is particularly advantageous.

In a further embodiment of the invention, the designation of the color of the object is displayed as a text or as a graphic symbol.

This allows a particularly easy detection of the name of the color by the user.

In a further embodiment of the invention, a rendered representation of the object is displayed in the field of view of the user. In other words, the object is represented as a graphic created from an image of the object.

This visually highlights the object vis-à-vis its surroundings, so that the user becomes aware of the object.

In a further embodiment of the invention, information about the color of the object is received by the display device.

Thus, the color of the object can be detected even if the color with the display device alone, for example, using an image pickup unit of the display device, not or can not be determined with certainty. Such a case can occur, for example, in poor visibility.

• - eine Ermittlungseinheit, die zum Ermitteln, ob ein Nutzer der Anzeigevorrichtung farbenfehlsichtig ist, eingerichtet ist,
• - eine Identifizierungseinheit, die zum Identifizieren eines in einem Blickfeld des Nutzers angeordneten Objektes, welches eine Farbe aufweist, deren Wahrnehmbarkeit durch den Nutzer infolge der Farbenfehlsichtigkeit beeinträchtigt ist, wenn der Nutzer farbenfehlsichtig ist, eingerichtet ist, und
• - eine Anzeigeeinheit, die zum Anzeigen einer Bezeichnung der Farbe im Blickfeld des Nutzers, so dass die Bezeichnung dem Objekt zugeordnet wird, eingerichtet ist.

The object underlying the invention is achieved in addition to the method with a display device for displaying augmented reality. The display device has the following functional units:

• - a determination unit configured to determine whether a user of the display device is color-defective;
• an identification unit adapted to identify an object located in a field of view of the user which has a color whose perceptibility is impaired by the user as a result of color vision deficiency when the user is color-defective, and
• a display unit arranged to display a designation of the color in the field of view of the user so that the designation is assigned to the object.

The display device according to the invention makes it possible to carry out the method according to the invention. Therefore, the advantages of the display device meet the above-mentioned advantages of the method according to the invention.

In one embodiment of the invention, the display device is arranged to display the designation of the color of the object on a windshield of a vehicle.

This takes into account that colored signs and warning symbols are often used for traffic control and guidance, for example in the form of colored traffic signs and colored light signals. An overlook or misinterpretation of a sign or warning symbol due to a color deficiency of a driver of a vehicle is dangerous because it may affect the safety of the traffic.

Therefore, the display device according to the invention is particularly suitable for assisting a driver of a vehicle with color vision in the perception of a traffic situation. The display of the designation of the color of the object on a windshield of a vehicle allows the driver to simultaneously perceive the designation of the color and the traffic situation, so that a distraction of the driver from the traffic situation is avoided.

• 1 zeigt ein Blockdiagramm einer Anzeigevorrichtung gemäß einer Ausführungsform der Erfindung.
• 2 zeigt ein Ablaufdiagramm einer Ausführungsform des erfindungsmäßen Verfahrens.
• 3 zeigt schematisch einen Blick durch eine Frontscheibe eines Fahrzeug mit der Anzeigevorrichtung aus 1.

The invention will now be explained with reference to the drawings.

• 1 shows a block diagram of a display device according to an embodiment of the invention.
• 2 shows a flowchart of an embodiment of the inventive method.
• 3 schematically shows a view through a windscreen of a vehicle with the display device 1 ,

1 shows a block diagram of a display device 100 according to an embodiment of the invention. The display device 100 is set up to show augmented reality. For this purpose, the display device 100 a determination unit 102 , an identification unit 104 and a display unit 106 on. In addition, the display device 100 have further functional units that in 1 are not shown.

The determination unit 102 is to determine if a user of the display device 100 is color defective, furnished. For example, the determination unit 102 determine if the user has a red-green vision deficiency.

For this purpose, the determination unit 102 receive personal data of the user from a portable electronic device of the user and gain information from the data about a possible color deficiency of the user. The discovery unit has to receive the data 102 via a data receiving unit, for example via a Bluetooth receiver and / or a receiver for near-field communication (abbreviated to NFC). The discovery unit has access to the information from the data 102 via a computing unit, for example in the form of a microprocessor. In addition, the determination unit 102 a memory unit in which the data and / or the information can be stored, for example an EEPROM or a flash memory.

The portable electronic device is, for example, a mobile phone, a portable computer, a car key, an implant, or a smart card. To send the user's personal information to the investigator 102 For example, the portable electronic device is equipped with a data transmission unit, for example with a Bluetooth transmitter and / or a transmitter for near-field communication. Besides, the portable electronic device may include a storage unit in which the data is storable, for example, an EEPROM or a flash memory.

Optionally, the portable electronic device may be configured to receive the user's personal data from an external data source, such as another portable electronic device, a data cloud service, a stationary computer, or a server. For this purpose, the portable electronic device may comprise a receiving unit, for example a mobile radio receiving unit.

Alternatively to receiving the personal data of the user by means of the determination unit 102 For example, the data or the information about a possible color deficiency of the user may already be in a storage unit of the display device 100 stored, for example in an EEPROM or in a flash memory. The storage of the data or the information can, for example, by means of their manual input to the display device 100 have been effected.

The identification unit 104 is arranged for identifying an object arranged in a field of view of the user, which has a color whose perceptibility is impaired by the user as a result of color vision deficiency. To identify the object, known methods of image recognition are applicable.

For example, the identification unit 104 identify the object taking into account its color, shape and / or position. If by means of the determination unit 102 A red-green visual impairment of the user was determined, can be determined by the identification unit 104 identify an object that has the color red or the color green.

For this purpose, the identification unit 104 an image pickup unit and an image recognition unit. The image acquisition unit is, for example, a digital camera or an infrared camera. The image acquisition unit is set up to take an image of a real situation in a user's environment, in particular a region lying in front of the user, and to transmit it to the image recognition unit in the form of image data. In addition, the image acquisition unit may comprise an optical spectrometer, by means of which the image data is extended by additional information, for example an exact color of a specific object.

The image recognition unit is, for example, a microprocessor. The image recognition unit is configured to find the object using the image data in the image, for example, based on a color of the object.

Optionally, the image recognition unit may be designed to determine a shape and / or a position of the object and to determine based on whether the object is relevant for the user. It is possible that only in this case the name of the color of the display unit 106 is displayed - that is otherwise dispense with the display.

As another option, the identification unit 104 a receiving unit adapted to receive information about the color of the object, for example a WLAN receiving unit. Inter alia, information about a color currently displayed by a light signal can be received by the receiving unit. This is particularly advantageous when the color with the image recording unit or the image recognition unit is not or can not be determined with certainty, for example because of poor visibility.

It is also possible that the name of the color only from the display unit 106 is displayed when a detected with the image pickup unit or the image recognition unit color of an object matches the color of the object according to an information received by the receiving unit. This avoids displaying a wrong label.

The display unit 106 is for displaying a designation of the color of one of the identification unit 104 identified object that is in the field of view of the user. In particular, the designation is displayed such that it is assigned to the object. This allows the user to recognize that the label belongs to the object.

For this purpose, the display unit 106 display the label so that the user perceives the label near the item, for example, above, below, to the right or to the left of the item. To do so, the label may be displayed at a position that is near a position of the object from the user's perspective or from a user's perspective. The designation can be displayed as a text and / or as a graphical symbol.

In addition, the display unit 106 be provided to display in the field of view of the user a superimposed on the object rendered representation of the object. For example, the rendered representation is displayed at a user perceived position of the object. In other words, the designation is displayed at a position where the object is located from the viewpoint of the user or from a perspective of the user.

In particular, the display unit 106 be designed to display the name of the color on a windshield of a vehicle. In this case, the display unit 106 belong to a windscreen display (English head-up display). Alternatively, the display unit 106 be designed to display the name on a lens of a user's glasses. In yet another alternative, the display unit is 106 designed to display the label by projecting it on any area of the user's field of view, for example using laser technology.

The display unit 106 may also be able to present a real situation in the user's field of vision, as a picture of the situation. In this case, the designation of the color of the object can be faded in the image, so that the designation is assigned to the object. To display the image, the display unit has 106 for example via a screen.

Optionally, the image can be displayed as a rendered representation of the real situation, ie abstracted. This can reduce the amount of information in the image and focus the user's attention on aspects of the situation that are essential to him. Alternatively or additionally, the rendered representation can have a particularly high contrast, whereby the perception of the situation by the user can be further improved. In a further alternative, an object can be highlighted and / or displayed larger than in reality by means of the rendered representation of the real situation, whereby the perception of the situation by the user can be additionally improved.

As another option, the display unit 106 in addition to the name of the color, display another virtual supplemental content associated with the object or other object in the user's field of view. The further virtual additional content may include a marking of the respective object, a designation of the respective object or a distance to the respective object.

2 shows a flowchart 200 an embodiment of the method according to the invention with the method steps 201 to 203 , The method is used with the display device 100 out 1 executed. It will also open 3 referenced, which schematically and partially a view through a windscreen 302 of a vehicle, which with the display device 100 out 1 Is provided. The user of the display device 100 is also the driver of the vehicle.

In the first process step 201 is determined by means of the determination unit 102 the display device 100 Determines if the user is color-defective. For this purpose, personal data of the user are evaluated by the portable electronic device 303 the user to the display device 100 be transmitted. The transmission may be wired or wireless, for example by means of Bluetooth or near field communication.

In the portable electronic device 303 it is a mobile phone that the user carries in the vehicle. The portable electronic device 303 but can also be a portable computer, a car key, an implant or a smart card.

The personal data of the user is obtained from a storage unit of the portable electronic device 303 to the display device 100 or their determination unit 102 transfer. Optionally, the data may be transferred from an external data source to the portable electronic device 303 transferred and from this to the display device 100 to get redirected. The external data source may be, inter alia, another portable electronic device, a data cloud service, a stationary computer, or a server. Alternatively, the determination unit 102 be set up, the personal data of the user using existing in the vehicle user interface or on the portable electronic device 303 Interrogate existing user interfaces, for example by means of a graphically guided dialogue on a screen or a voice control with access to the speaker and microphone.

Furthermore, from the personal data of the user by means of the arithmetic unit of the determination unit 102 obtained the information that the user is color-defective, namely due to a red-green-visual weakness. Therefore, the method becomes the second process step 202 continued. If, on the other hand, it were determined that the user is not color-defective, the procedure would be terminated, but could be restarted if another user controls the vehicle.

In the second process step 202 is by means of the identification unit 104 the display device 100 an object located in the field of view of the user 304 identified, which has a color whose visibility is impaired by the user due to the color vision deficiency. As in the first step 201 A red-green visual weakness of the user was determined, an object is identified, which has the color red or the color green.

For this purpose, with the image acquisition unit of the identification unit 104 a picture of one through the windscreen 302 visible real situation. In other words, an image of an area of the vehicle surroundings lying ahead of the vehicle is recorded.

Further, by means of the image recognition unit of the identification unit 104 in the picture the object 304 found, which is a red light signal 305 having. Furthermore, by means of the image recognition unit based on the shape and the position of the object 304 found that it was at the object 304 is a traffic light, which is arranged in a traffic route used by the user. Thus, the object 304 relevant to the user, which leads to his identification.

If, on the other hand, it is not possible to identify an object which has a color whose perceptibility is impaired by the user as a result of color vision deficiency, the method can be ended or interrupted. In such an interruption, it would be possible for the method after a certain period of time with the second method step 202 to resume.

After carrying out the second method step 202 becomes the process with the third process step 203 continued, in which by means of the display unit 106 the display device 100 a name 301 the color of the object 304 is displayed in the user's field of vision. The display is made such that the name 301 the object 304 is assigned.

This is the name 301 the color is displayed so that the user's name 301 near the object 304 perceives, namely to the right of the object 304 , To do this, the display unit projects 106 the name 301 on the windscreen 302 wherein a head position and / or an eye position of the user can be taken into account. Alternatively, to the right of the object 304 can the label 301 before, on, above, below, to the right or to the left of the object 304 are displayed.

In the present case, the name is displayed as a text, namely as the word "red". But it is also possible, the name 301 instead or in addition to display as a graphic symbol. For example, a glow of the red light signal 305 graphically highlighted (not shown).

Notwithstanding the indication of the name 301 The color near the object may be the name 301 be assigned to the object by passing between the object 304 and the name 301 an arrow is displayed which points to the object 304 has.

Furthermore, the object can 304 a rendered representation of the object 304 be superimposed by the rendered representation at the user perceived position of the object 304 projected onto the windscreen in the user's field of vision. This way, the object becomes 304 visually highlighted to the user.

Alternatively to projecting on the windscreen 302 can the label 301 the color projected onto another projection screen, which is separate from the front screen 302 is arranged, for example between the user and the windscreen 302 ,

As another alternative to projecting on the windscreen 302 can the label 301 be displayed on a lens of the user's glasses.

As yet another alternative, the term 301 be displayed in an image of the real situation, which is displayed on a screen arranged in the vehicle, the image may have a rendered representation of the situation.

Optionally, with the display unit 106 another additional virtual content will be displayed, which expands the situation. The additional content becomes the object 304 or assigned to another object in the field of view of the user and may include, inter alia, a name of the respective object, a state description of the respective object or a distance to the respective object. A state description of an object may include text describing a state of the object, for example, "flashing light active" if the object has a flashing light, such as a St. Andrew's cross. For this purpose, the text can be displayed pulsating analogous to the visual light.

After performing the third process step 203 can the process with the second process step 202 be continued by identifying another object. Alternatively, the process may be terminated.

LIST OF REFERENCE NUMBERS

100

display device

102

determining unit

104

identification unit

106

display unit

200

flow chart

201 to 203

steps

301

designation

302

windscreen

303

portable electronic device

304

object

305

light signal

Claims (10)

A method of presenting augmented reality using a display device (100), comprising the steps of: Determining whether a user of the display device (100) is color-defective and, if the user is color-defective, Identifying an object (304) arranged in a field of view of the user, which has a color whose perceptibility is impaired by the user as a result of color vision deficiency, and - displaying a designation (301) of the color in the field of view of the user, so that the designation (301) is assigned to the object (304). Method according to Claim 1 in which, in order to determine whether the user is color-defective, personal data of the user is transmitted from a user's portable electronic device (303) to the display device (100). Method according to Claim 2 in which the data has been transferred from an external data source to the portable electronic device (303). Method according to one of the preceding claims, wherein the object (304) is identified taking into account its color, its shape and / or its position. Method according to one of the preceding claims, in which the object (304) belongs to a traffic light system. Method according to one of the preceding claims, in which the designation (301) is displayed as a text or as a graphic symbol. Method according to one of the preceding claims, in which a rendered representation of the object (304) is displayed in the field of vision of the user. A method according to any one of the preceding claims, wherein information about the color of the object (304) is received by the display device (100). An augmented reality display device (100) comprising: a determination unit (102) which is set up to determine whether a user of the display device (100) is color-defective; an identification unit (104) arranged to identify an object (304) arranged in a user's field of vision which has a color whose perceptibility is impaired by the user as a result of color vision deficiency when the user is color-defective; - A display unit (106) adapted to display a designation (301) of the color in the field of view of the user, so that the name (301) is associated with the object (304). Display device (100) according to Claim 9 which is adapted to display the designation (301) on a windshield (302) of a vehicle (300).

Images