CN118269656A - Display method, display system and storage medium for display information - Google Patents

Abstract

The application provides a display method, a display system and a storage medium for display information, and relates to the technical field of vehicle display. The display method is applied to a virtual image display device and an operation device, a communication connection is established between the virtual image display device and the operation device, and the display method includes: displaying real-time rendering road condition models corresponding to a plurality of concerned objects in a real scene in a display interface of the operation equipment; the operation device acquires an operation instruction and sends the operation instruction to the virtual image display device; the virtual image display apparatus controls display of information corresponding to the plurality of objects of interest in the projected virtual image according to the operation instruction. The virtual image display device is thereby able to control the display of information in the projected virtual image in accordance with the operation instruction, thereby enabling interaction with the virtual image display device.

Description

Display method, display system and storage medium for display information

Technical Field

The present application belongs to the field of vehicle display technology, and in particular, relates to a display method, a display system and a storage medium for displaying information.

Background technique

A Head Up Display (HUD) is a display device that projects images into the driver's field of vision. The Augmented Reality HUD (AR-HUD) is a type of HUD technology that can reasonably overlay and display some information in the driver's field of vision and combine it with actual traffic conditions. Through AR-HUD technology, the driver can expand and enhance his or her perception of the driving environment.

Summary of the invention

According to the first aspect of the present application, the present application provides a method for displaying display information, which is applied to a virtual image display device and an operating device, and a communication connection is established between the virtual image display device and the operating device. The display method includes: displaying real-time rendered road condition models corresponding to multiple objects of interest in a real scene in a display interface of the operating device; the operating device obtains an operation instruction and sends the operation instruction to the virtual image display device; the virtual image display device controls the display of information corresponding to the multiple objects of interest in the projected virtual image according to the operation instruction.

In a second aspect, the present application provides a display device, which is applied to a virtual image display device and an operating device, and the display device includes:

A display unit, used to display real-time rendered road condition models corresponding to multiple objects of interest in a real scene in a display interface of an operating device;

A sending unit, used for operating the device to obtain an operation instruction and sending the operation instruction to the virtual image display device;

The control unit is used for the virtual image display device to control the display of information corresponding to multiple objects of interest in the projected virtual image according to the operation instruction.

In a third aspect, the present application provides a display system, the display system comprising a virtual image display device and an operating device, a communication connection is established between the virtual image display device and the operating device,

The operating device is used to display the real-time rendered road condition models corresponding to the multiple objects of interest in the real scene in the display interface of the operating device;

The operating device is also used to obtain operating instructions and send the operating instructions to the virtual image display device;

The virtual image display device is used to control the display of information corresponding to multiple objects of interest in the projected virtual image according to the operation instruction.

In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the method described in any one of the above-mentioned first aspects.

In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in any one of the above-mentioned first aspects.

In a sixth aspect, an embodiment of the present application provides a chip, including a processor, for calling and running a computer program from a memory, so that a display system equipped with the chip executes the method described in any of the above-mentioned first aspects.

For example, in the embodiments of the first to sixth aspects of the present application, a real-time rendered road condition model corresponding to each of the multiple objects of interest in the real scene is displayed in the display interface of the operating device, the operating device obtains an operating instruction, and sends the operating instruction to the virtual image display device, and the virtual image display device controls the display of information corresponding to the multiple objects of interest in the projected virtual image according to the operating instruction. Thus, the virtual image display device can control the display of information in the projected virtual image according to the operating instruction, thereby realizing interaction with the virtual image display device. And the virtual image display device can control the amount of information corresponding to the multiple objects of interest displayed in the projected virtual image according to the operating instruction, avoid display information overload, reduce the burden of users reading information, will not interfere with the user's line of sight, increase the time for users to make judgments on driving operations, and improve driving safety and comfort.

For example, in the embodiments of the first to sixth aspects of the present application, the operating device obtains an operating instruction, including: the operating device obtains an input display condition, the display condition including the point of interest display area in the virtual image and/or the distance range between the object of interest in the real scene and the reference object; the operating device filters out target point of interest information that meets the display conditions from multiple point of interest information, and sends the target point of interest information to the virtual image display device, wherein each point of interest information in the multiple point of interest information corresponds one-to-one to the object of interest, wherein the operating instruction includes the target point of interest information; the virtual image display device controls the display of information corresponding to the multiple objects of interest in the projected virtual image according to the operating instruction, including: the virtual image display device displays the target point of interest information in the projected virtual image.

For example, in the embodiments of the first to sixth aspects of the present application, the operating device obtains the display conditions of the input, including: the operating device obtains a first operation performed by the user on the operating device through a preset interaction method, and the interaction method includes one or any combination of touch interaction, voice interaction, and gesture interaction; the operating device determines the display conditions based on the first operation.

For example, in the embodiments of the first to sixth aspects of the present application, it also includes: when the first operation is an area selection operation in a touch interaction mode, the operating device determines the operating area of the first operation in the display interface of the operating device; the operating device determines the corresponding area of the operating area in the virtual image, and determines the corresponding area as the point of interest display area, wherein the virtual image is divided into at least one point of interest display area, the display interface is divided into at least one sub-display area, and each point of interest display area in the at least one point of interest display area corresponds one-to-one to each sub-display area.

For example, in the embodiments of the first to sixth aspects of the present application, the operating device filters out target interest point information that meets the display conditions from multiple interest point information, including: determining the respective positions of the multiple interest point information in the virtual image, and/or the distance between the object of interest corresponding to each interest point in the multiple interest point information and the reference object; when the display conditions include an interest point display area, the interest point information located in the interest point display area is determined as the target interest point information; when the display conditions include a distance range, the interest point information whose distance between the object of interest and the reference object is within the distance range is determined as the target interest point information; when the display conditions include an interest point display area and a distance range, the interest point information located in the interest point display area and whose distance between the object of interest and the reference object is within the distance range is determined as the target interest point information.

For example, in the embodiments of the first to sixth aspects of the present application, the virtual image display device controls the display of information corresponding to multiple objects of interest in the projected virtual image according to the operating instructions, including: the virtual image display device displays the target interest point information indicated by the operating instructions in the projected virtual image, wherein the display method also includes: synchronously displaying the target interest point information displayed in the virtual image in the display interface of the operating device.

For example, in the embodiments of the first to sixth aspects of the present application, the method further includes: after the virtual image display device displays the target interest point information in the projected virtual image, the display method further includes at least one of the following:

When the operating device receives a first instruction for the first target point of interest information, the operating device sends the first instruction to the virtual image display device; the virtual image display device adjusts the display mode of the first target point of interest information in the virtual image according to the first instruction, the first instruction is issued by the user through a preset interactive method, and the first instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction; when the operating device receives a second instruction for the first target point of interest information, the operating device determines the next level of information corresponding to the first target point of interest information, and sends the next level of information to the virtual image display device; the virtual image display device displays the next level of information in the virtual image; when the operating device receives a third instruction for the real-time rendering of the road condition model, the operating device adjusts the display mode of the real-time rendering of the road condition model on the display interface according to the third instruction, wherein the third instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction;

When the operating device receives the fourth instruction for the operation display interface, it adjusts the display mode of the real-time rendered road condition model and the target point of interest information in the display interface according to the fourth instruction, wherein the fourth includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction and a rotation instruction.

For example, in the embodiments of the first to sixth aspects of the present application, it also includes: synchronously displaying the target point of interest information displayed in the virtual image in the display interface of the operating device, wherein the display interface is used to control the display of the virtual image.

For example, in the embodiments of the first to sixth aspects of the present application, it also includes: determining an area in the virtual image other than the interest point display area; and stopping displaying the content in the area in the virtual image other than the interest point display area.

For example, in the embodiments of the first to sixth aspects of the present application, it also includes: when switching to the target mode, when a selection instruction for the first real-time rendering road condition model is received in the display interface, the object of interest corresponding to the first real-time rendering road condition model in the real scene is determined as the target object; the associated information of the target object is obtained through the operating device, the associated information includes information for guiding the purchase of the target object; the virtual image display device displays the associated information in the virtual image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow chart of a method for displaying information provided by an exemplary embodiment of the present application;

FIG2 is a schematic diagram showing a real-time rendered road condition model according to an exemplary embodiment of the present application;

FIG3 is a schematic diagram showing a display of associated information according to an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of a display system provided by an exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

A Head Up Display (HUD) is a display device that projects images into the driver's field of vision. The Augmented Reality HUD (AR-HUD) is a type of HUD technology that can reasonably overlay and display some information in the driver's field of vision and combine it with actual traffic conditions. Through AR-HUD technology, the driver can expand and enhance his or her perception of the driving environment.

The inventors of this application discovered during their research that as the information carried by the display interface of AR-HUD becomes more and more diverse, if too much information is displayed on the same level interface, it will cause information overload, increase the user's burden of reading information, easily interfere with the user's line of sight, and require the user to spend more time making judgments on driving operations, thereby reducing driving safety and comfort.

In view of this, the embodiments of the present application provide a display method, a display system and a storage medium for displaying information. A real-time rendered road condition model corresponding to each of multiple objects of interest in a real scene is displayed in the display interface of an operating device; the operating device obtains an operating instruction and sends the operating instruction to a virtual image display device; the virtual image display device controls the display of information corresponding to multiple objects of interest in the projected virtual image according to the operating instruction. Thus, the virtual image display device can control the display of information in the projected virtual image according to the operating instruction, thereby realizing interaction with the virtual image display device. Furthermore, in some embodiments, the virtual image display device can control the amount of information corresponding to multiple objects of interest displayed in the projected virtual image according to the operating instruction, thereby avoiding overload of displayed information, reducing the burden of users reading information, not interfering with the user's line of sight, increasing the time for users to make judgments on driving operations, and improving driving safety and comfort.

The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

The display method of display information provided in the present application is applied to a virtual image display device and an operating device, and a communication connection is established between the virtual image display device and the operating device. For example, the virtual image display device and the operating device are connected wirelessly in advance (such as a Bluetooth connection, a wireless network connection, etc.). For another example, the virtual image display device and the operating device are connected wirelessly in advance (such as a data cable connection). This is only an exemplary description and is not limited to this.

In one possible implementation, the virtual image display device may be an AR-HUD device, and the operating device may be an onboard computer, an onboard central control device, a computer, a tablet computer, a computer, a smart phone, a personal digital assistant (PDA), or the like.

Please refer to Figure 1, which is a schematic flow chart of a method for displaying information provided by an exemplary embodiment of the present application. The method shown in Figure 1 may include: S101 to S103, which are as follows:

S101: Displaying real-time rendered road condition models corresponding to a plurality of objects of interest in a real scene on a display interface of an operating device.

For example, the object of interest in a real scene can be a point of interest (POI) in a geographic information system. The POI in the embodiments of the present application generally refers to point data in an Internet electronic map. For example, in a geographic information system, a POI can be a house, a shop, a mailbox, a bus stop, etc. The point of interest information can include multiple attribute information such as the name, address, coordinates, and category of the point of interest. In some embodiments,

Exemplarily, the real-time rendered road condition model is obtained by scanning the objects of interest in the real scene (such as POI in the geographic information system). Exemplarily, based on the positioning results of the Simultaneous Localization and Mapping (SLAM) technology, combined with the vehicle's Global Positioning System (GPS), and matched with a high-precision map, the real information of the positioning surroundings, the positioning vehicle position, the current vehicle scene and the surrounding environment can be obtained on the map. At the same time, the objects of interest are rendered in real time through cameras, laser scanning vehicle software, etc., and the real-time rendered road condition models corresponding to multiple objects of interest are obtained, and these real-time rendered road condition models are presented in the display interface of the operating device. Optionally, the camera can also be used to identify pictures, text, objects, languages and other information.

For ease of understanding, please refer to Figure 2, which is a schematic diagram of a real-time rendered road condition model shown in an exemplary embodiment of the present application. As shown in Figure 2, the real-time rendered road condition model corresponding to multiple objects of interest such as the vehicle, the vehicles around the vehicle (such as the front vehicle as shown in Figure 2), the road, and the various buildings on both sides of the road, pedestrians, etc. is displayed in the display interface of the operating device. Specifically, a 3D rendering model corresponding to these objects of interest can be rendered and displayed in the display interface. At the same time, navigation information, map information, vehicle driving parameters, etc. can also be displayed in the display interface of the operating device. Driving parameters may include vehicle speed, fuel consumption, driving route, safe distance from the vehicle in front, gear position, remaining power, etc. The user can adjust the virtual image projected by the virtual image display device by operating the operating device.

In this implementation, the object of interest in the real scene is scanned, reconstructed and rendered to obtain a real-time rendered road condition model, which simplifies the object of interest and occupies less memory. The real-time rendered road condition model is displayed on the operating device, which is convenient for the user to intuitively check the vehicle and the surrounding environment, so that the user can control the displayed information according to their own needs, and then adjust the virtual image projected by the virtual image display device in real time.

Optionally, in a possible implementation, the operating device can be installed in the steering wheel of the vehicle. For example, the operating device is integrated into the steering wheel of the vehicle. The display interface of the operating device can display real-time rendered road condition models corresponding to multiple objects of interest, and synchronize the information displayed in the virtual image projected by the virtual image display device. The user can manipulate the control interface to manipulate the information displayed in the virtual image projected by the virtual image display device. This implementation makes it easy for the user to manipulate the operating device, thereby quickly adjusting the virtual image projected by the virtual image display device.

S102: The operating device obtains an operating instruction and sends the operating instruction to the virtual image display device.

Exemplarily, when the operating device receives the operating instruction from the user, it sends the operating instruction to the virtual image display device, wherein the operating instruction may include the target point of interest information.

Exemplarily, the operating device obtaining the operating instruction may include: the operating device obtaining input display conditions, the display conditions including the point of interest display area in the virtual image and/or the distance range between the object of interest and the reference object in the real scene; the operating device filtering out target point of interest information that meets the display conditions from multiple point of interest information, and sending the target point of interest information to the virtual image display device, wherein each point of interest information in the multiple point of interest information corresponds one-to-one to the object of interest.

Exemplarily, the display condition may also be understood as a filtering condition, which is used to filter out target interest point information meeting the condition from a plurality of interest point information.

Optionally, in a possible implementation, the display condition may include a point of interest display area in a virtual image of a virtual image display device. The virtual image of the virtual image display device refers to a virtual image formed by the virtual image display device on the outside of a car windshield (such as a front windshield). The point of interest display area in the embodiment of the present application can be specified by a user (such as a driver of the vehicle, a passenger, etc.) according to their own needs. For example, the point of interest display area can be one or more combinations of the left area, the middle area, the right area, the upper area, the lower area, etc. in the virtual image. This is only an exemplary description and is not limited to this.

For example, the display interface of the virtual image display device (i.e., the interface presenting the virtual image) can be pre-divided in a variety of ways, such as left, middle, and right horizontal divisions, so as to obtain different display areas (such as the left area, the middle area, the right area, etc.), which can be used to display the point of interest information. After the user specifies the point of interest display area, the virtual image display device can only display the point of interest information in the specified area. In other words, the user can independently select the area where the point of interest is to be displayed, and the area that the user does not want to see can be not displayed.

Optionally, in another possible implementation, the display condition may include a distance range between the object of interest and the reference object in the real scene. Among them, the object of interest may include movable objects and immovable objects. Exemplarily, movable objects may include external means of transportation (cars, bicycles, electric vehicles, buses, etc.), pedestrians, animals, etc. Immovable objects may include traffic indicators, buildings (such as buildings, bridges, viaducts, overpasses, etc.), roadblocks, etc. The reference object may be the vehicle, AR-HUD equipment, the eyes of the driver of the vehicle, etc. The distance range in the embodiment of the present application may be specified by the user (such as the driver of the vehicle, passengers, etc.) according to their own needs. For example, the distance range may be 0 to 20 meters, 20 to 50 meters, greater than 50 meters, etc. This is only an exemplary description and is not limited to this.

Exemplarily, the virtual image display device can project multiple layers of virtual images with AR display effects and different imaging distances, such as near-layer virtual images, middle-layer virtual images, and far-layer virtual images. By displaying the point of interest information in these virtual images, the displayed different point of interest information can present the display effects of close-up, middle-range, and far-range. For example, when the distance range is 0 to 7 meters, a close-up display effect is presented; when the distance range is 7 to 50 meters, a middle-range display effect is presented; when the distance range is greater than 50 meters, a far-range display effect is presented. This is only an exemplary description and is not limited to this. After filtering out the target point of interest information, the virtual image display device can display the target point of interest information in a virtual image adapted to the distance according to the distance between the object of interest and the reference object corresponding to each target point of interest information. Among them, the division method of the point of interest display area in the multi-layer virtual image is the same.

In some embodiments, the virtual image display device may project a single layer of virtual image with an AR display effect, and after filtering out the target interest point information, the virtual image display device displays the filtered target interest point information in the virtual image.

Optionally, in yet another possible implementation, the display condition may include a display area of a point of interest in a virtual image displayed by the virtual image display device, and a distance range between an object of interest and a reference object in a real scene.

For example, the display condition inputted by the operating device may be that a user (such as the driver of the vehicle, a passenger, etc.) inputs the display condition on the operating device (such as the vehicle-mounted central control device) through a preset interaction method. The interaction method may include one or any combination of touch interaction, voice interaction, and gesture interaction.

For example, the operating device includes a touch screen, and the user can perform touch operations on the touch screen through fingers (finger pulps, knuckles, etc.) or touch objects such as styluses, thereby inputting display conditions. For another example, the user controls the operating device with voice to input display conditions. For another example, the user controls the operating device with gestures to input display conditions. It is understandable that in a possible implementation, any combination of these methods of inputting display conditions can also be used. This is only an exemplary description and is not limited to this.

For example, according to different display conditions, different target interest point information can be screened out from the multiple interest point information, wherein each interest point information in the multiple interest point information corresponds to the object of interest one by one.

Optionally, in a possible implementation, the display condition includes a point of interest display area in a virtual image of the virtual image display device. For example, when a user is driving on the right side of a lane, he or she inputs the display condition: the right side area in the virtual image of the virtual image display device through touch interaction or voice interaction. Specifically, the user clicks (double-clicks) the right side of the touch screen of the operating device, or the user voice inputs the keyword information "right side", and the right side of the touch screen is displayed as selected. The operating device selects the point of interest information located in the right area from multiple point of interest information as the target point of interest information.

Optionally, in a possible implementation, the display condition includes a point of interest display area and a point of interest category in a virtual image of a virtual image display device. The point of interest category may include a cafe, a restaurant, a shopping mall, a hospital, a scenic spot, a company, a gas station, a charging station, a park, etc. For example, a user inputs a display condition: a right area in a virtual image of a virtual image display device and a point of interest of a cafe category. The operating device selects the point of interest information located in the right area and having a cafe category from the multiple point of interest information as the target point of interest information.

In this implementation, users can set different display conditions according to their own needs (such as driving direction, destination, etc.), so as to filter out different target points of interest information. When the information is subsequently displayed in the virtual image projected by the virtual image display device, the burden of reading information for users is reduced, the user's line of sight is not disturbed, the time for users to make judgments on driving operations is increased, and the driving safety and comfort are improved. For example, when the user is driving on the right, this method can be used to select the closest, most convenient, and most desired point of interest, which improves the user experience.

Optionally, in a possible implementation, the display condition includes a distance range between the object of interest and the reference object in the real scene. For example, the user inputs the display condition through touch interaction or voice interaction: the distance range between the object of interest and the reference object in the real scene is 0 to 20 meters. Specifically, the user can select different distance ranges by sliding up and down on the touch screen, or the user can voice input the keyword information "XX distance", and the touch screen will display different display effects (such as close view, medium view, and long view) accordingly. The operating device filters out the point of interest information that meets the distance range selected by the user from multiple point of interest information as the target point of interest information.

Optionally, in a possible implementation, the display condition may further include a display quantity and/or a display mode. The display mode may include a distance priority display mode, a good review priority display mode, and a comprehensive evaluation priority display mode.

For example, the operating device may be pre-installed with applications such as maps, shopping, services, and food. The operating device may sort multiple points of interest information in different display modes based on the information obtained from these applications to obtain sorting results, and use the points of interest information corresponding to each result in the sorting results as the target points of interest information. For example, the user inputs the display conditions through touch interaction or voice interaction: favorable comment priority display mode and coffee shop. When the operating device obtains the search results according to the favorable comment priority display mode, the operating device uses the points of interest information corresponding to each result in the search results as the target points of interest information. This cross-analysis method provides users with controllable and targeted information, making the target points of interest information finally presented to users more accurate.

For another example, the operating device may pre-install applications such as maps, shopping, services, and food. The operating device may sort multiple points of interest information in different display modes according to the information obtained from these applications to obtain sorting results, select the specified number of points of interest information in the sorting results, and determine the selected points of interest information as the target points of interest information. Among them, the specified number of displays is set by the user according to actual needs. For example, the specified number of displays may be 1 to 5, 6 to 10, 10 to 15, etc. For example, the user enters the display conditions through touch interaction or voice interaction: favorable comment priority display mode, display number of 5, and coffee shop. Specifically, the user clicks on the touch screen of the operating device to select the display number of 5 and the point of interest category of coffee shop, and selects the favorable comment priority display mode. When the operating device obtains the search results according to the favorable comment priority display mode, the operating device selects the first 5 in the search results as the target points of interest information. This is only an exemplary description and is not limited to this. The operating device sends the filtered target points of interest to the virtual image display device.

S103: The virtual image display device controls the display of information corresponding to the multiple objects of interest in the projected virtual image according to the operation instruction.

Exemplarily, the virtual image display device can control the display of information corresponding to multiple objects of interest in the projected virtual image according to the operation instruction, for example, controlling the quantity, position, size, etc. of information corresponding to multiple objects of interest displayed in the projected virtual image.

For example, the virtual image display device receives the target interest point information sent by the operating device, and displays the target interest point information in the projected virtual image.

It is understandable that when determining the target point of interest information, it is screened out from multiple points of interest information according to the display conditions. Therefore, when displayed in the virtual image projected by the virtual image display device, it will be displayed in the corresponding point of interest display area (such as one or more combinations of the left area, the middle area, the right area, etc.) and/or in the corresponding display quantity (specified display quantity).

In this implementation, the real-time rendered road condition models corresponding to the multiple objects of interest in the real scene are displayed in the display interface of the operating device; the operating device obtains the operating instruction and sends the operating instruction to the virtual image display device; the virtual image display device controls the display of the information corresponding to the multiple objects of interest in the projected virtual image according to the operating instruction. Thus, the virtual image display device can control the amount of information corresponding to the multiple objects of interest displayed in the projected virtual image according to the operating instruction, avoid display information overload, reduce the burden of reading information for users, will not interfere with the user's line of sight, increase the time for users to make judgments on driving operations, and improve driving safety and comfort.

For ease of understanding, the following embodiments of the present application take the virtual image display device as an AR-HUD device as an example to illustrate different display scenarios. In one example, the virtual image projected by the AR-HUD device is located in front of the front windshield of the vehicle, and can be located in front of the field of vision of the main driver, or in front of the field of vision of the co-driver, or both the main driver and the co-driver can view the virtual image located in front of the front windshield of the vehicle.

It is worth noting that the virtual image can display various information required during driving. In some embodiments, such as displaying information about points of interest, navigation information, vehicle driving parameters, road signs, lane departure warning information, obstacle collision warning information, etc. Among them, driving parameters may include vehicle speed, fuel consumption, driving route, safe distance from the vehicle in front, gear position, remaining power, etc. It can also display advertising videos, news information, event promotion videos, shopping mall discount recommendations, oil price discounts, congested road section countdown and other intelligent multimedia information.

Optionally, in a possible implementation, the operation device in S102 above filters out target POI information that meets the display condition from multiple POI information, which may include at least one of the following methods, as follows:

In one example, the positions of the plurality of interest point information in the virtual image and/or the distance between the object of interest corresponding to each interest point in the plurality of interest point information and the reference object are determined.

Exemplarily, the position of each point of interest information in the virtual image projected by the virtual image display device is determined. The virtual image projected by the virtual image display device is divided into at least one point of interest display area, each point of interest information corresponds to an interest point display area in the virtual image, and the interest point display area can be used to indicate the area where the point of interest information is located in the virtual image projected by the virtual image display device.

Exemplarily, each of the plurality of interest point information corresponds to an object of interest, and the distance between the object of interest corresponding to each of the plurality of interest point information and the reference object can also be determined. In this embodiment, when positioning is performed using SLAM positioning technology and GPS positioning technology, the distance between each object of interest and the reference object can be obtained.

In another example, when the display condition includes an interest point display area, the interest point information located in the interest point display area is determined as the target interest point information.

It is worth noting that the interest point display area in the virtual image of the virtual image display device included in the above display conditions is the interest point display area specified by the user according to his own needs. When the area where a certain interest point information is located in the virtual image projected by the virtual image display device is the same as the interest point display area specified by the user, that is, when the interest point display area corresponding to a certain interest point information in the virtual image is the same as the interest point display area specified by the user, the interest point information is determined as the target interest point information.

For example, if a coffee shop is in the right area of the virtual image, and the display condition includes that the POI display area is the right area of the virtual image, then the information of the coffee shop is determined as the target POI information.

In yet another example, when the display condition includes a distance range, the interest point information whose distance between the focus object and the reference object is within the distance range is determined as the target interest point information.

Exemplarily, when the display condition includes a distance range between the object of interest and the reference object in the real scene, it is determined whether the distance between the object of interest and the reference object is within the distance range. If the judgment result is that the distance between the object of interest and the reference object is within the distance range, the information of the object of interest is determined as the target point of interest information. If the judgment result is that the distance between the object of interest and the reference object is not within the distance range, no processing is performed.

In this implementation, when the virtual image display device carries excessive information, it can be personalized according to the user's wishes, realizing the primary and secondary hierarchical display of information, and effectively solving the problem of information overload.

In another example, when the display conditions include an interest point display area and a distance range, interest point information located in the interest point display area and whose distance between the focus object and the reference object is within the distance range is determined as target interest point information.

Exemplarily, when the display conditions include a point of interest display area in a virtual image of a virtual image display device, and a distance range between an object of interest and a reference object in a real scene, in one possible implementation, the point of interest information that matches the point of interest display area may be first filtered, and the point of interest information that matches the distance range may be filtered from the initially filtered point of interest information, and the last filtered point of interest information may be determined as the target point of interest information.

Optionally, in another possible implementation, the POI information that meets the distance range is first filtered, the POI information that meets the POI display area is filtered from the initially filtered POI information, and the finally filtered POI information is determined as the target POI information.

In this implementation, the target point of interest information of the user is screened out according to different display conditions, and when it is subsequently displayed in the virtual image projected by the virtual image display device, it is avoided that too much information is displayed on the same display interface, and the display information overload is avoided, thereby reducing the user's burden of reading information, not interfering with the user's line of sight, and increasing the time for the user to make judgments on driving operations, thereby improving driving safety and comfort.

Optionally, in a possible implementation, the display condition for the above-mentioned operating device to obtain input may include: the operating device obtains a first operation performed by the user on the operating device through a preset interaction method, the interaction method including one or any combination of touch interaction, voice interaction, and gesture interaction; the operating device determines the display condition based on the first operation.

For example, the interaction method may include one or any combination of touch interaction, voice interaction, and gesture interaction. In this implementation, multiple interaction methods are provided to users, which enriches the interaction methods and facilitates users to choose their favorite interaction methods, thereby improving user experience and favorability.

For example, the user is driving on the right side of the lane, and operates the real-time rendered road condition model displayed in the operating device through touch interaction. For example, the user clicks on the area where the real-time rendered road condition model is located on the right side of the touch screen of the operating device, and the right side of the touch screen is displayed as selected. At this time, the display condition is determined that the point of interest display area in the virtual image of the virtual image display device is the right area.

For another example, the user slides up and down in the area where the real-time rendering road condition model is located on the touch screen of the operating device to select a distance range. At this time, the display condition is determined to be that the distance range between each object of interest and the reference object in the real scene is the distance range selected by the user.

In this implementation mode, users can operate the real-time rendered road condition model according to their own needs to determine different display conditions, so as to facilitate the subsequent screening of different target interest point information according to different display conditions. When it is subsequently displayed in the virtual image projected by the virtual image display device, the user's burden of reading information is reduced, it will not interfere with the user's line of sight, and the time for the user to make judgments on driving operations is increased, thereby improving driving safety and comfort.

Optionally, in a possible implementation, the display information display method provided by the present application also includes, when the first operation is an area selection operation in a touch interaction method, the operating device determines an operation area of the first operation in a display interface of the operating device; the operating device determines a corresponding area of the operation area in the virtual image, and determines the corresponding area as a point of interest display area.

Exemplarily, the first operation is an area selection operation in a touch interaction manner, for example, the user operates the display interface of the operating device by clicking. The operating device determines the operation area of the first operation in the display interface of the operating device. For example, if the user clicks the area on the right side of the display interface, the area on the right side of the display interface is determined as the operation area, and the operation area is displayed as selected. At this time, the operating device determines the area corresponding to the operation area in the virtual image projected by the virtual image display device, and determines the corresponding area as the point of interest display area.

The virtual image projected by the virtual image display device is divided into at least one point of interest display area, the display interface is divided into at least one sub-display area, and each point of interest display area in the at least one point of interest display area corresponds to each sub-display area one by one.

For example, the area on the right side of the display interface is determined as the operation area, and the operation area can be understood as a sub-display area. In the virtual image projected by the virtual image display device, there is an interest point display area corresponding to the sub-display area. In other words, the area on the right side of the display interface is determined as the operation area, and the right side area in the virtual image projected by the virtual image display device corresponds to the operation area. That is, the final interest point information is displayed in the right side area of the virtual image projected by the virtual image display device.

In this implementation, after the operation area is determined according to the user's operation, the area corresponding to the operation area is determined in the virtual image as the point of interest display area, thereby achieving synchronization between the display interface of the operating device and the virtual image projected by the virtual image display device, making it convenient for the user to control the virtual image projected by the virtual image display device in real time by manipulating the display interface.

Optionally, in a possible implementation, the operating device can be installed in the steering wheel of the vehicle. For example, the operating device is integrated into the steering wheel of the vehicle. The display interface of the operating device can display real-time rendered road condition models corresponding to multiple objects of interest, and realize synchronous display with the display content in the virtual image projected by the virtual image display device. The user can control the control interface to realize the control of each display content in the virtual image projected by the virtual image display device. This implementation makes it easy for the user to control the operating device, thereby quickly adjusting the virtual image projected by the virtual image display device.

Optionally, in a possible implementation, a camera may be built into the vehicle, such as in the steering wheel, to implement gesture interaction between the user and the operating device. That is, the camera may be used to recognize the user's gestures, and the user may use gestures such as "swipe up", "swipe down", "swipe left", and "swipe right" to turn pages up, down, left, and right on the display content of the display interface of the operating device. Since the display content of the display interface of the operating device may be synchronized with the display content in the virtual image projected by the virtual image display device, when the user manipulates the display content of the display interface of the operating device through gestures, the display content in the virtual image projected by the virtual image display device also gives corresponding feedback.

Optionally, the user can also confirm the selected content through a "grab" gesture, and can also zoom in, zoom out, rotate, etc. on the selected content. Similarly, when the user performs these operations on the display content of the display interface of the operating device, the display content in the virtual image projected by the virtual image display device also gives corresponding feedback. For example, the display content in the virtual image projected by the virtual image display device is also zoomed in, zoomed out, rotated, etc. accordingly. This is only an exemplary description and is not limited to this.

Optionally, in a possible implementation, the virtual image display device controls display of information corresponding to multiple objects of interest in the projected virtual image according to an operation instruction, including: the virtual image display device displays the target interest point information indicated by the operation instruction in the projected virtual image.

Exemplarily, the virtual image display device can control the display of information corresponding to multiple objects of interest in the projected virtual image according to the operation instruction. For example, the virtual image display device receives the target interest point information sent by the operation device and displays the target interest point information in the projected virtual image.

It is understandable that when determining the target point of interest information, it is screened out from multiple points of interest information according to the display conditions. Therefore, when displayed in the virtual image projected by the virtual image display device, it will be displayed in the corresponding point of interest display area (such as one or more combinations of the left area, the middle area, the right area, etc.) and/or in the corresponding display quantity (specified display quantity).

In this implementation, the virtual image display device can control the amount of information corresponding to multiple objects of interest displayed in the projected virtual image according to operating instructions, thereby avoiding information overload, reducing the user's burden of reading information, not interfering with the user's line of sight, increasing the time for the user to make judgments on driving operations, and improving driving safety and comfort.

Optionally, in a possible implementation, the display method provided by the present application further includes: synchronously displaying the target point of interest information displayed in the virtual image in the display interface of the operating device, that is, realizing synchronization of the content displayed on the display interface of the operating device with the display content in the virtual image projected by the virtual image display device. Exemplarily, the target point of interest information displayed in the virtual image projected by the virtual image display device is also synchronously displayed in the display interface of the operating device. Among them, the display interface is used to control the display of the virtual image. For example, when the user manipulates the display interface through a preset interactive method, the manipulation result will also act on the virtual image projected by the virtual image display device. In this implementation, the synchronous display of the display interface of the operating device and the virtual image projected by the virtual image display device is realized.

Optionally, in a possible implementation manner, the display method of display information provided by the present application may further include at least one of the following methods after the virtual image display device displays the target interest point information in the projected virtual image, as follows:

In one example, when receiving the first instruction for the first target point of interest information, the operating device sends the first instruction to the virtual image display device.

Exemplarily, the first target point of interest information represents any one of the multiple target point of interest information. The first instruction is issued by the user through a preset interaction method, and the first instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction. For example, the user operates the first target point of interest information through one or any combination of touch interaction, voice interaction, and gesture interaction, and the operating device receives the user's operation on the first target point of interest information, generates a first instruction accordingly, and sends the first instruction to the virtual image display device.

For example, the point of interest corresponding to the first target point of interest information is a shopping mall. The user zooms in on the point of interest information of the shopping mall (such as the shopping mall name, trademark identifier, etc.) displayed on the operating device through touch interaction. The operating device receives the user's zoom-in operation on the point of interest information of the shopping mall, and generates a corresponding zoom-in instruction, which is sent to the virtual image display device.

In another example, the virtual image display device adjusts the display manner of the first target interest point information in the virtual image according to the first instruction.

The virtual image display device receives the first instruction for the first target interest point information sent by the operating device, and adjusts the display mode of the first target interest point information in the virtual image according to the first instruction. For example, when the first instruction is a zoom-in instruction, the virtual image display device zooms in the first target interest point information displayed in the virtual image; when the first instruction is a zoom-out instruction, the virtual image display device zooms out the first target interest point information displayed in the virtual image; when the first instruction is a move instruction, the virtual image display device moves the first target interest point information displayed in the virtual image; when the first instruction is a rotation instruction, the virtual image display device rotates the first target interest point information displayed in the virtual image.

Optionally, in a possible implementation, when the operating device receives the first instruction for the first real-time rendered road condition model, the operating device sends the first instruction to the virtual image display device, and the virtual image display device adjusts the display mode of the first real-time rendered road condition model in the virtual image according to the first instruction. The first real-time rendered road condition model represents any real-time rendered road condition model among multiple real-time rendered road condition models.

In this implementation, when the operating device receives the user's operating instruction, it responds accordingly and sends the response result to the virtual image display device in a timely manner, so that the virtual image display device can synchronously display the response result made according to the user's operating instruction, and realize the synchronization of the display content of the operating device display interface and the virtual image projected by the virtual image display device. And this targeted enhanced display of a certain target point of interest information improves the display effect.

In yet another example, when the operating device receives the second instruction for the first target interest point information, it determines the next level information corresponding to the first target interest point information, and sends the next level information to the virtual image display device.

Exemplarily, the second instruction may include a next-level operation (such as determining an operation, expanding and displaying) instruction, etc. According to the content of the first target point of interest information, when the operating device receives a next-level operation (such as expanding and displaying) instruction for the first target point of interest information, the next-level information corresponding to the first target point of interest information is determined, and the next-level information is sent to the virtual image display device.

For example, the current first target point of interest information displays the icon, name, distance, recommendation index, etc. of a restaurant. After the user issues a second instruction for the first target point of interest information through a preset interactive method, the next level of information can be displayed, that is, the virtual image and the display interface both jump to display the next level of information at the location of the first target point of interest, and the first target point of interest information is no longer displayed. The next level of information can display the selected restaurant's positive reviews, average price, waiting information, navigation here, etc. for user reference. If the user clicks the return control on the display interface through a preset interactive method, the display interface jumps to the previous level interface, and the user can reselect another restaurant.

For example, if the user clicks "Navigate here" in the next level of information, it means that the user finally chooses this restaurant and can enter the next level of information display, which can display the route to the restaurant, specific navigation information, and the congested road section ahead prompt information. In the congested road section ahead prompt information, the congestion icon, congested road section length, estimated congestion time, and the countdown to the last time period of congestion can be displayed. If the user clicks on the congested road section ahead prompt information, the next level of information corresponding to the congested road section ahead prompt information can be entered. The next level of information corresponding to the congested road section ahead prompt information can display the cause of the congestion, such as school dismissal, vehicle accidents, red light congestion, etc.

The virtual image display device displays the next level information in a virtual image.

The virtual image display device receives the next level of information sent by the operating device, and displays the next level of information in the virtual image in real time. The HUD device in the prior art only provides the user with a display function, and fails to provide the next level of information interaction function. The user cannot truly interact with the HUD device, and the user's visible information is limited, which restricts the user's further interaction and experience. In this application, the operating device and the virtual image display device support a multi-level information display function, so that the virtual image display device can display multi-level information, which is convenient for the user to view and select information of different levels, realizes a variety of interactive functions between the user and the virtual image display device, and enriches the interactive experience between the user and the virtual image display device.

In another example, when the operating device receives the third instruction for the real-time rendering of the road condition model, the operating device adjusts the display mode of the real-time rendering of the road condition model on the display interface according to the third instruction.

Exemplarily, the third instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction. It is worth noting that, in this case, only the display of the real-time rendered road condition model is adjusted by the third instruction, while the display interface of the operating device and the display mode of the target point of interest information in the virtual image projected by the virtual image display device are not changed.

For example, when the third instruction is a zoom-in instruction, the real-time rendered road condition model is zoomed in on the display interface; when the third instruction is a zoom-out instruction, the real-time rendered road condition model is zoomed out on the display interface; when the third instruction is a move instruction, the real-time rendered road condition model is moved on the display interface; when the third instruction is a rotation instruction, the real-time rendered road condition model is rotated on the display interface. This is only an exemplary description and is not limited to this.

In another example, when the operating device receives the fourth instruction to operate the display interface, the operating device simultaneously adjusts the display mode of the real-time rendered road condition model and the target point of interest information in the display interface according to the fourth instruction.

Exemplarily, the fourth instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction. The fourth instruction is used to simultaneously adjust the display mode of the real-time rendered road condition model and the target point of interest information in the display interface. That is, the real-time rendered road condition model and the target point of interest information in the display interface are zoomed in, zoomed out, moved, and rotated.

In this implementation, when the operating device receives the user's operating instruction, it responds accordingly and sends the response result to the virtual image display device in a timely manner, so that the virtual image display device can synchronously display the response result made according to the user's operating instruction, and realize the synchronization of the display content of the operating device display interface and the virtual image projected by the virtual image display device. And this targeted enhanced display of a certain target point of interest information improves the display effect.

Optionally, in a possible implementation, the display information display method provided in the present application also includes determining an area in the virtual image excluding the point of interest display area; and stopping displaying content in the area in the virtual image excluding the point of interest display area.

Exemplarily, the virtual image display device displays the target interest point information in the projected virtual image, and can determine the area other than the interest point display area in the virtual image, and stop displaying the content in the area other than the interest point display area in the virtual image. For example, the display of the content in the area other than the interest point display area in the virtual image can be stopped by turning off the backlight source corresponding to the area other than the interest point display area in the virtual image display device.

In this implementation, only the target point of interest information is displayed in the projected virtual image, which greatly reduces the amount of displayed information and improves the quality of displayed information. It can effectively avoid display information overload, reduce the user's burden of reading information, will not interfere with the user's line of sight, and increase the time for users to make judgments on driving operations, thereby improving driving safety and comfort.

Optionally, in a possible implementation, the display information display method provided by the present application also includes displaying real-time rendered road condition models corresponding to multiple objects of interest in the display interface of the operating device; when switching to the target mode, when a selection instruction for the first real-time rendered road condition model is received in the display interface, the object of interest corresponding to the first real-time rendered road condition model in the real scene is determined as the target object; the associated information of the target object is obtained through the operating device; and the virtual image display device displays the associated information in the virtual image.

Exemplarily, the target mode may be a shopping mode, in which the operating device can identify the target object, and the first real-time rendering traffic condition model is any one of a plurality of real-time rendering traffic condition models. The display interface of the operating device displays the real-time rendering traffic condition models corresponding to the plurality of objects of interest, that is, the display interface of the operating device displays the plurality of real-time rendering traffic condition models. If the user switches the target mode and selects the first real-time rendering traffic condition model through a preset interactive method, the operating device receives the user's selection operation on the first real-time rendering traffic condition model, that is, the operating device receives the selection instruction on the first real-time rendering traffic condition model in the display interface. At this time, the object of interest corresponding to the first real-time rendering traffic condition model in the real scene is determined as the target object.

The operating device can analyze the target object through big data to obtain the associated information of the target object. The associated information may include information for guiding the purchase of the target object. Optionally, the operating device can also be connected to various shopping platforms currently on the market, and these shopping platforms can be used to identify and analyze the target object to obtain the associated information of the target object. The virtual image display device displays the associated information in the virtual image.

For example, when switching to the target mode, the vehicle system can classify and identify vehicles, pedestrians, buildings, etc. around the vehicle. When the user clicks to confirm the first real-time rendered road condition model of interest in the display interface, the operating device determines the object of interest corresponding to the first real-time rendered road condition model in the real scene as the target object. At the same time, the camera will immediately capture the image of the target object and focus on and magnify the target object. After that, the image is analyzed to obtain the associated information of the target object corresponding to the image.

Optionally, the associated information may initially display basic information such as item pictures, prices, and purchase platforms. If the user triggers the next level of associated information through a preset interactive method, detailed information such as comprehensive evaluation, immediate purchase, and adding to shopping cart may be displayed. Optionally, the user may also zoom in, zoom out, move, and rotate the associated information and the next level of associated information through a preset interactive method. It is understandable that the user's operations in the display interface may be synchronized in the virtual image projected by the virtual image display device.

For example, when a user is driving and is interested in a certain vehicle in front of him, he switches to the target mode and clicks on the real-time rendered road condition model of the vehicle he is interested in in the display interface, and the operating device identifies the vehicle in the real scene as the target object. At this time, the image of the vehicle in front can be obtained by scanning with the camera and transmitted to the operating device. The operating device uses the shopping platform to scan and identify the image of the vehicle and obtain the relevant information of the vehicle, such as the vehicle brand, model, price, year, color, test drive location and other information. The virtual image display device displays this information in the virtual image.

For ease of understanding, please refer to FIG. 3, which is a schematic diagram of displaying associated information shown in an exemplary embodiment of the present application. As shown in FIG. 3, in addition to displaying the vehicle, the virtual image projected by the virtual image display device can also display associated information of the vehicle that the user is interested in. For example, the vehicle brand, model, price, color, and other information are displayed in the right area of the displayed vehicle. It should be understood that FIG. 3 is only an exemplary description. In the actual implementation process, the display content and display method of the associated information can be adjusted according to the needs of the user.

For another example, when a user is driving and is interested in the clothing of a pedestrian, the user clicks on the real-time rendered road condition model of the pedestrian of interest in the display interface, and the operating device determines the pedestrian in the real scene as the target object. At this time, the image of the pedestrian can be obtained by scanning with a camera and transmitted to the operating device. The operating device uses the shopping platform to scan and identify the image of the pedestrian and obtain the associated information of the pedestrian's clothing, such as clothing brand, price, color, purchase method, discount and other information. The virtual image display device displays this information in the virtual image. Optionally, the target object of interest to the user can also be collected, so that the user can share the target object or the associated information of the target object with others.

In some embodiments, after receiving a purchase instruction for a target object (for example, receiving a purchase instruction for a target object by operating a device), facial recognition of the user is performed through a camera in the vehicle to perform facial payment and complete the purchase.

Optionally, in a possible implementation, the user can choose to exit the target mode. In this way, the user can search, collect, and purchase items of interest encountered during driving, enriching the functions of the virtual image display device and bringing convenience to the user. If it cooperates with shopping platforms, it can also improve economic benefits.

Optionally, in a possible implementation, when receiving a play operation, the operating device acquires multimedia data, synchronizes the multimedia data to the virtual image display device, and the virtual image display device plays the multimedia data.

For example, when a user plays a video on the display interface of an operating device, he or she can pause, continue playing, fast forward, rewind, etc. the video through touch interaction, and the virtual image display device also synchronizes these operations, that is, the video played in the virtual image projected by the virtual image display device will also be paused, continued, fast forward, rewind, etc.

Optionally, when a user plays a video on the display interface of the operating device, he can pause, continue playing, fast forward, rewind, and other operations on the video through gesture interaction. For example, according to different preset gesture operations, the user makes different gesture operations in front of the camera set on the steering wheel to achieve operations such as pausing, continuing playing, fast forward, and rewinding the video. The virtual image display device also synchronizes these operations, that is, the video played in the virtual image projected by the virtual image display device will also pause, continue playing, fast forward, and rewind. In this implementation, the functions of the virtual image display device are enriched, bringing a better experience to the user.

The display method of display information provided by the present application enhances the information of interest to the user, effectively controls the amount of information displayed by the virtual image display device, and avoids information overload. The user can select the content of interest to interact with, which increases the interactivity between the user and the virtual image display device. In addition, the display method of display information provided by the present application expands the display application of the virtual image display device through the combination of the Internet, multimedia information and the virtual image display device. Real-time information such as entertainment, sales, news, services, tourism, medical treatment, advertising, education, technology, and real estate can be presented in the virtual image display device to enable users to query and share in real time.

Please refer to Figure 4, which is a schematic diagram of a display system provided by an exemplary embodiment of the present application. As shown in Figure 4, the display system 200 includes a virtual image display device 210 and an operating device 220, and a communication connection is established between the virtual image display device 210 and the operating device 220. The display system 200 can implement the steps in the above-mentioned various display method embodiments.

The embodiment of the present application further provides a display device, which is applied to a virtual image display device and an operating device, and the display device includes:

A display unit, used to display real-time rendered road condition models corresponding to multiple objects of interest in a real scene in a display interface of an operating device;

A sending unit, used for operating the device to obtain an operation instruction and sending the operation instruction to the virtual image display device;

The control unit is used for the virtual image display device to control the display of information corresponding to multiple objects of interest in the projected virtual image according to the operation instruction.

In one possible implementation, the sending unit is also used to operate the device to obtain input display conditions, which display conditions include the interest point display area in the virtual image and/or the distance range between the object of interest in the real scene and the reference object; the operating device filters out target interest point information that meets the display conditions from multiple interest point information, and sends the target interest point information to the virtual image display device, wherein each interest point information in the multiple interest point information corresponds one-to-one to the object of interest, wherein the operation instruction includes the target interest point information; the virtual image display device controls the display of information corresponding to the multiple objects of interest in the projected virtual image according to the operation instruction, including: the virtual image display device displays the target interest point information in the projected virtual image.

In a possible implementation, the sending unit is also used for the operating device to obtain a first operation performed by the user on the operating device through a preset interaction method, where the interaction method includes one or any combination of touch interaction, voice interaction, and gesture interaction; the operating device determines the display condition based on the first operation.

In a possible implementation, the sending unit is also used to, when the first operation is an area selection operation in a touch interaction mode, the operating device determines an operating area of the first operation in a display interface of the operating device; the operating device determines a corresponding area of the operating area in a virtual image, and determines the corresponding area as a point of interest display area, wherein the virtual image is divided into at least one point of interest display area, the display interface is divided into at least one sub-display area, and each point of interest display area in the at least one point of interest display area corresponds one-to-one to each sub-display area.

In one possible implementation, the sending unit is also used to determine the respective positions of multiple points of interest information in the virtual image, and/or the distance between the object of interest corresponding to each point of interest in the multiple points of interest information and the reference object; when the display conditions include an interest point display area, the interest point information located in the interest point display area is determined as the target interest point information; when the display conditions include a distance range, the interest point information whose distance between the object of interest and the reference object is within the distance range is determined as the target interest point information; when the display conditions include an interest point display area and a distance range, the interest point information located in the interest point display area and whose distance between the object of interest and the reference object is within the distance range is determined as the target interest point information.

In a possible implementation, the control unit is also used to enable the virtual image display device to display the target interest point information indicated by the operation instruction in the projected virtual image, wherein the display method also includes: synchronously displaying the target interest point information displayed in the virtual image in the display interface of the operating device.

In a possible implementation, the sending unit is further used for, when the operating device receives a first instruction for the first target point of interest information, sending the first instruction to the virtual image display device; the virtual image display device adjusts the display mode of the first target point of interest information in the virtual image according to the first instruction, the first instruction is issued by the user through a preset interaction method, and the first instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction; when the operating device receives a second instruction for the first target point of interest information, determining the next level information corresponding to the first target point of interest information, and sending the next level information to the virtual image display device; the virtual image display device displays the next level information in the virtual image; when the operating device receives a third instruction for the real-time rendering of the road condition model, the operating device adjusts the display mode of the real-time rendering of the road condition model on the display interface according to the third instruction, wherein the third instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction;

When the operating device receives the fourth instruction for the operation display interface, it adjusts the display mode of the real-time rendered road condition model and the target point of interest information in the display interface according to the fourth instruction, wherein the fourth includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction and a rotation instruction.

In a possible implementation, the display device provided by the present application further includes: a synchronization unit, configured to synchronously display the target point of interest information displayed in the virtual image in the display interface of the operating device, wherein the display interface is used to control the display of the virtual image.

In a possible implementation, the display device provided in the present application further includes: a determination unit, configured to determine an area in the virtual image other than the area where the points of interest are displayed; and to stop displaying content in the area in the virtual image other than the area where the points of interest are displayed.

In one possible implementation, the display device provided in the present application also includes a processing unit, which is used to, when switched to the target mode, determine the object of interest corresponding to the first real-time rendering road condition model in the real scene as the target object when a selection instruction for the first real-time rendering road condition model is received in the display interface; obtain associated information of the target object through an operating device, the associated information including information for guiding the purchase of the target object; and the virtual image display device displays the associated information in the virtual image.

The embodiment of the present application further provides a computer storage medium, which may be non-volatile or volatile. The computer storage medium stores a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

The present application also provides a computer program product. When the computer program product runs on a virtual image display device or an operating device, the virtual image display device or the operating device executes the steps in the above-mentioned various display method embodiments.

An embodiment of the present application further provides a chip or integrated circuit, which includes: a processor for calling and running a computer program from a memory, so that a virtual image display device or an operating device equipped with the chip or integrated circuit executes the steps in the above-mentioned display method embodiments.

The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims (10)

1. A method for displaying display information, characterized in that it is applied to a virtual image display device and an operating device, a communication connection is established between the virtual image display device and the operating device, and the display method comprises: Displaying real-time rendered road condition models corresponding to multiple objects of interest in the real scene in the display interface of the operating device; The operating device acquires an operating instruction and sends the operating instruction to the virtual image display device; The virtual image display device controls display of information corresponding to the plurality of objects of interest in the projected virtual image according to the operation instruction. 2. The display method according to claim 1, wherein the operation device obtains the operation instruction, comprising: The operating device acquires input display conditions, wherein the display conditions include a display area of a point of interest in the virtual image and/or a distance range between an object of interest and a reference object in a real scene; The operating device selects target interest point information that meets the display condition from a plurality of interest point information, and sends the target interest point information to the virtual image display device, wherein each interest point information in the plurality of interest point information corresponds to an object of interest one by one, and wherein the operating instruction includes the target interest point information; The virtual image display device controls display of information corresponding to the multiple objects of interest in the projected virtual image according to the operation instruction, including: the virtual image display device displays the target interest point information in the projected virtual image. 3. The display method according to claim 2, wherein the operation device obtains the input display condition, comprising: The operating device acquires a first operation performed by a user on the operating device in a preset interaction manner, where the interaction manner includes one or any combination of touch interaction, voice interaction, and gesture interaction; The operation device determines the display condition according to the first operation. 4. The display method according to claim 3, characterized in that the display method further comprises: When the first operation is an area selection operation in a touch interaction manner, the operating device determines an operation area of the first operation in a display interface of the operating device; The operating device determines a corresponding area of the operating area in the virtual image, and determines the corresponding area as the point of interest display area, wherein the virtual image is divided into at least one point of interest display area, the display interface is divided into at least one sub-display area, and each point of interest display area in the at least one point of interest display area corresponds one-to-one to each sub-display area. 5. The display method according to claim 1, wherein the virtual image display device controls the display of information corresponding to the plurality of objects of interest in the projected virtual image according to the operation instruction, comprising: The virtual image display device displays the target point of interest information indicated by the operation instruction in the projected virtual image, wherein the display method further comprises: The target point of interest information displayed in the virtual image is synchronously displayed in the display interface of the operating device. 6. The display method according to claim 5, characterized in that after the virtual image display device displays the target point of interest information in the projected virtual image, the display method further comprises at least one of the following: When receiving a first instruction for the first target point of interest information, the operating device sends the first instruction to the virtual image display device; The virtual image display device adjusts the display mode of the first target interest point information in the virtual image according to the first instruction, wherein the first instruction is issued by the user in a preset interactive mode, and the first instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction; When receiving the second instruction for the first target interest point information, the operating device determines the next level information corresponding to the first target interest point information, and sends the next level information to the virtual image display device; The virtual image display device displays the next level information in the virtual image; When the operating device receives a third instruction for the real-time rendered road condition model, the operating device adjusts the display mode of the real-time rendered road condition model on the display interface according to the third instruction, wherein the third instruction includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction, and a rotation instruction; When the operating device receives the fourth instruction for the operation display interface, it simultaneously adjusts the display mode of the real-time rendered road condition model and the target point of interest information in the display interface according to the fourth instruction, wherein the fourth includes at least one of a zoom-in instruction, a zoom-out instruction, a move instruction and a rotation instruction. 7. The display method according to any one of claims 1 to 6, characterized in that the display method further comprises: Determine an area of the virtual image other than the interest point display area; The display of the content in the area other than the interest point display area in the virtual image is stopped. 8. The display method according to claim 1, characterized in that the display method further comprises: In the case of switching to the target mode, when a selection instruction for the first real-time rendered road condition model is received in the display interface, an object of interest corresponding to the first real-time rendered road condition model in the real scene is determined as a target object; Acquiring, by means of the operating device, associated information of the target object, wherein the associated information includes information for guiding purchase of the target object; The virtual image display device displays the associated information in the virtual image. 9. A display system for displaying information, characterized in that the display system comprises a virtual image display device and an operating device, wherein a communication connection is established between the virtual image display device and the operating device. The operating device is used to display the real-time rendered road condition models corresponding to the multiple objects of interest in the real scene in the display interface of the operating device; The operating device is also used to obtain an operating instruction and send the operating instruction to the virtual image display device; The virtual image display device is used to control the display of information corresponding to the multiple objects of interest in the projected virtual image according to the operation instruction. 10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the display method according to any one of claims 1 to 8.