WO2022252688A1 - Augmented reality data presentation method and apparatus, electronic device, and storage medium - Google Patents

Abstract

Embodiments of the present disclosure disclose an augmented reality data presentation method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring positioning information of an augmented reality (AR) device; on the basis of the positioning information, if it is determined that a preset position condition is satisfied between the AR device and an explanation area corresponding to any preset knowledge point, acquiring audio explanation data corresponding to the any preset knowledge point, as well as AR special effect data matching the audio explanation data; and controlling the AR device to play back the audio explanation data and to display the AR special effect data that matches the currently playing audio presentation data.

Description

Augmented reality data presentation method, device, electronic device and storage medium

Cross References to Related Applications

This disclosure is based on a Chinese patent application with application number 202110619445.8 and a filing date of June 3, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated into this disclosure by reference.

technical field

The present disclosure relates to the field of augmented reality technology, and in particular to a method, device, electronic device and storage medium for presenting augmented reality data.

Background technique

Augmented Reality (Augmented Reality, AR) is a new technology that integrates real world information and virtual world information. The goal of this technology is to put the virtual world on the screen and interact with the real world. With the development of science and technology, AR technology is applied in various scenarios, such as travel scenarios, shopping scenarios, education scenarios and so on.

Therefore, it is particularly important to propose a display method for AR data.

Contents of the invention

Embodiments of the present disclosure at least provide an augmented reality data presentation method, device, electronic device, and storage medium.

In the first aspect, an embodiment of the present disclosure provides a method for presenting augmented reality data, including: acquiring positioning information of an AR device; When the preset location conditions are satisfied, obtain the audio commentary data corresponding to any preset knowledge point and the AR special effect data matching the audio commentary data; control the AR device to play the audio commentary data, and display AR special effect data matching the currently playing audio commentary data.

In the embodiment of the present disclosure, by controlling the AR device to play the audio commentary data corresponding to any preset knowledge point, while playing the audio commentary data, the AR special effect data matching the currently played audio commentary data is displayed, and the audio commentary is explained by using the AR special effect data The auxiliary explanation of the data makes the explanation process of the preset knowledge points clearer and more intuitive, and improves the efficiency of the explanation of the preset knowledge points.

In some possible implementation manners, the controlling the AR device to play audio commentary data and presenting AR special effect data matching the currently played audio commentary data includes: determining that the AR device meets the requirements based on the positioning information. In the case of triggering display conditions of the AR special effect data, the AR device is controlled to play the audio commentary data and display the AR special effect data.

In some possible implementation manners, the method also includes:

In a case where it is determined based on the positioning information that the AR device does not meet the triggering display condition of the AR special effect data, the AR device is controlled to display first guidance information for guiding adjustment of the positioning information of the AR device.

In some possible implementation manners, the method further includes: when the audio commentary data corresponding to any preset knowledge point is played, and/or when the presentation of the AR special effect data ends, controlling the AR The device displays at least one of the following information:

Second guidance information for instructing the AR device to move to the explanation area corresponding to the next preset knowledge point;

List information used to indicate the preset knowledge points that the AR device has not visited;

The navigation end information is used to indicate that the AR device has played the audio commentary data corresponding to each preset knowledge point.

In some possible implementation manners, the displaying the AR special effect data corresponding to the currently played audio commentary data includes: during the process of playing the audio commentary data, switching to display various AR special effect data corresponding to the currently played audio commentary data. AR special effect data.

In some possible implementation manners, the switching display of various AR special effect data corresponding to the currently played audio commentary data includes: responding to a target trigger operation, selecting from a variety of AR special effect data corresponding to the currently played audio commentary data , determining the AR special effect data to be displayed; controlling the AR device to display the AR special effect data to be displayed.

In some possible implementation manners, the obtaining the positioning information of the augmented reality AR device includes: obtaining the real-time scene image collected by the AR device; determining the AR device based on the real-time scene image and the constructed 3D scene model location information.

In some possible implementations, the audio commentary data corresponding to each preset knowledge point is pre-determined according to the following steps: acquiring the audio commentary data to be processed corresponding to the target area; Determine target keywords that match a plurality of preset knowledge points in the data; according to the corresponding playback time positions of each of the target keywords in the audio commentary data to be processed, from the audio commentary data to be processed, respectively Determine the audio commentary data corresponding to each of the preset knowledge points.

In some possible implementation manners, the method is applied to a client application platform, and the client application platform is a web page (Web) application platform or a small program application platform.

In a second aspect, an embodiment of the present disclosure provides an augmented reality data presentation device, including:

The first obtaining module is configured to obtain the positioning information of the AR device;

The second acquiring module is configured to acquire the corresponding position of any preset knowledge point when it is determined that the explanation area corresponding to the AR device and any preset knowledge point satisfies the preset location condition based on the positioning information. audio commentary data, and AR special effect data matching the audio commentary data;

The control module is configured to control the AR device to play audio commentary data, and display AR special effect data matching the currently played audio commentary data.

In some possible implementation manners, the control module is configured to: control the AR device to play the audio when it is determined based on the positioning information that the AR device satisfies the trigger display condition of the AR special effect data Explain the data and display the AR special effect data.

In some possible implementation manners, the apparatus further includes: a first presentation module configured to: control the The AR device displays first guide information for guiding and adjusting the positioning information of the AR device.

In some possible implementation manners, the device further includes: a second display module configured to:

When the audio explanation data corresponding to any preset knowledge point is played, and/or the AR special effect data display is completed, the AR device is controlled to display at least one of the following information:

Second guidance information for instructing the AR device to move to the explanation area corresponding to the next preset knowledge point;

List information used to indicate the preset knowledge points that the AR device has not visited;

The navigation end information is used to indicate that the AR device has played the audio commentary data corresponding to each preset knowledge point.

In some possible implementation manners, the control module is configured to: during the process of playing the audio commentary data, switch and display various AR special effect data corresponding to the currently played audio commentary data.

In some possible implementation manners, the control module is configured to: determine the AR special effect data to be displayed from a variety of AR special effect data corresponding to the currently played audio commentary data in response to a target trigger operation; control the AR The device displays the AR special effect data to be displayed.

In some possible implementation manners, the first obtaining module is configured to: obtain a real-time scene image collected by the AR device; determine the positioning information of the AR device based on the real-time scene image and the constructed 3D scene model .

In some possible implementation manners, the device further includes a determination module configured to predetermine the audio commentary data corresponding to each preset knowledge point according to the following steps: acquire the audio commentary data to be processed corresponding to the target area; A point library, determining target keywords respectively matching with a plurality of preset knowledge points from the audio commentary data to be processed; according to the corresponding playing time position of each target keyword in the audio commentary data to be processed, from Among the audio commentary data to be processed, audio commentary data corresponding to each of the preset knowledge points are respectively determined.

In some possible implementation manners, the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor The memory communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the steps of the method for presenting augmented reality data as described in the above first aspect or any implementation manner are executed.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor as described in the above-mentioned first aspect or any implementation manner. The steps of the augmented reality data presentation method.

In order to make the above-mentioned purpose, features and advantages of the embodiments of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 shows a schematic flowchart of a method for presenting augmented reality data provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of triggering display conditions corresponding to AR special effect data in a method for presenting augmented reality data provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic interface diagram of an AR device provided by an embodiment of the present disclosure;

Fig. 4a shows a schematic interface diagram of an AR device provided by an embodiment of the present disclosure;

Fig. 4b shows a schematic interface diagram of an AR device provided by an embodiment of the present disclosure;

Fig. 4c shows a schematic interface diagram of an AR device provided by an embodiment of the present disclosure;

Fig. 4d shows a schematic interface diagram of an AR device provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic flowchart of another augmented reality data presentation method provided by an embodiment of the present disclosure;

FIG. 6 shows a schematic structural diagram of an augmented reality data presentation device provided by an embodiment of the present disclosure;

Fig. 7 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments It is only a part of the embodiments of the present disclosure, but not all the embodiments. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of the present disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the embodiments of the present disclosure.

AR is a new technology that integrates real world information and virtual world information. The goal of this technology is to put the virtual world on the screen and interact with the real world. With the development of science and technology, AR technology is applied in various scenarios, such as travel scenarios, shopping scenarios, education scenarios and so on. Based on this, embodiments of the present disclosure provide an augmented reality data presentation method, device, electronic device, and storage medium.

The defects in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure below for the above problems should be the result of the inventor Contributions made to this disclosure during the course of this disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In order to facilitate understanding of the embodiments of the present disclosure, a method for presenting augmented reality data disclosed in the embodiments of the present disclosure is first introduced in detail. The execution subject of the augmented reality data display method provided by the embodiments of the present disclosure may be an AR device, and the AR device is a smart device capable of supporting AR functions. For example, the AR device includes but is not limited to a mobile phone, a tablet, and AR glasses.

Referring to FIG. 1 , it is a schematic flowchart of a method for presenting augmented reality data provided by an embodiment of the present disclosure. The method includes S101-S103, wherein:

S101, acquiring positioning information of an AR device;

S102. Based on the positioning information, when it is determined that a preset location condition is satisfied between the AR device and the explaining area corresponding to any preset knowledge point, acquire audio explaining data corresponding to any preset knowledge point, And AR special effect data matching with the audio commentary data;

S103. Control the AR device to play audio commentary data, and display AR special effect data matching the currently played audio commentary data.

In the above method, by controlling the AR device to play the audio explanation data corresponding to any preset knowledge point, while playing the audio explanation data, display the AR special effect data matching the currently played audio explanation data, and use the AR special effect data to explain the audio The auxiliary explanation of the data makes the explanation process of the preset knowledge points clearer and more intuitive, and improves the efficiency of the explanation of the preset knowledge points.

S101-S103 will be specifically described below.

For S101: After the AR device reaches the target area, you can scan the information code set in the target area to enter the AR navigation process, that is, you can match the corresponding audio explanation for the AR device through the obtained positioning information of the AR device The data, as well as the AR special effect data matched with the audio commentary data, provide audio guides and AR special effect guides for AR devices. Wherein, for example, in the cultural tourism scene, the target area may be an area corresponding to any target scenic spot, or may be various exhibition areas in the target scenic spot, etc.

In some optional embodiments, the positioning information of the AR device can be determined according to the sensors set on the AR device. For example, the sensor can be a Global Positioning System (Global Positioning System, GPS), an inertial sensor (Inertial Measurement Unit, IMU), etc. .

In an optional implementation manner, in S101, obtaining the positioning information of the AR device may include: obtaining a real-time scene image collected by the AR device; determining the AR device based on the real-time scene image and the constructed 3D scene model. Device location information.

After acquiring the real-time scene image collected by the AR device, you can extract the feature points in the real-time scene image, match the feature point with the feature point cloud included in the 3D scene model, and determine the positioning information when the AR device collects the real-time scene image. Wherein, the positioning information may include position information and/or orientation information, for example, the position information may be the coordinate information of the AR device in the coordinate system corresponding to the 3D scene model; the orientation information may be the Euler angle corresponding to the AR device.

In some optional embodiments, the three-dimensional scene model can be constructed according to the following steps: collecting multiple frames of scene images at different positions, different angles, and different times in the target area, and extracting feature points from each frame of the scene image to obtain each frame The point cloud set corresponding to the scene image; the feature point cloud corresponding to the target area is obtained by using the point cloud set corresponding to the multi-frame scene image, and the feature point cloud corresponding to the target area constitutes a 3D scene model.

In some other optional embodiments, it is also possible to collect scene videos at different positions, different angles, and different times, obtain multiple frames of video frames from the collected scene videos, and extract feature points from each frame of video frames to obtain each frame. The point cloud set corresponding to the video frame; the 3D scene model corresponding to the target area is obtained by using the point cloud set corresponding to the multi-frame video frame.

Here, by using the real-time scene image collected by the AR device and the constructed three-dimensional scene model, the positioning information of the AR device can be determined more accurately.

For S102: preset knowledge points can be set as any explainable knowledge points in the target area. For example, when the target area is a calligraphy and painting exhibition hall, the preset knowledge point can be the name of any painting and calligraphy in the exhibition; for example, when the target area is the Forbidden City Museum, the preset knowledge point can be the name of any building in the exhibition, etc. .

In some optional embodiments, the spatial position of each preset knowledge point and the explanation area corresponding to each preset knowledge point can be determined in the constructed three-dimensional scene model. For example, the position of the real object in the target area can be determined as the spatial position of the preset knowledge point corresponding to the real object; the surrounding area of the real object in the target area can also be set as the preset knowledge corresponding to the real object Points of explanation area.

Exemplarily, the preset location condition can be that the distance between the AR device and the explanation area corresponding to the preset knowledge point is less than the set distance threshold, or the preset location condition can also be that the AR device is located in the explanation area corresponding to the preset knowledge point area etc.

In some optional embodiments, the distance between the AR device and the explaining area corresponding to each preset knowledge point may be determined based on the position information indicated by the positioning information of the AR device. For example, the distance between the AR device and the central point of the explaining area may be determined. When the distance between the AR device and the explanation area corresponding to the preset knowledge point is less than the preset distance threshold, the audio explanation data corresponding to the preset knowledge point and the AR special effect data matching the audio explanation data are acquired.

Alternatively, based on the explanation area corresponding to the preset knowledge point and the position information indicated by the positioning information of the AR device, determine whether the AR device is located in the explanation area of the preset knowledge point, if it is determined that the AR device is located in the explanation area of the preset knowledge point , the audio commentary data corresponding to the preset knowledge point and the AR special effect data matching the audio commentary data are acquired.

The audio explanation data can be preset audio data explaining the preset knowledge point; the AR special effect data can be AR data including one or more of the following contents: video content, image content, and text content. For example, when the preset knowledge point is the Hall of Supreme Harmony, the audio explanation data can be the audio data explaining the purpose, history, culture, architectural style and other information of the Hall of Supreme Harmony; AR data; or, the AR special effect data can also be the AR data of the overall picture of the Hall of Supreme Harmony in the historical time period, etc. Wherein, the AR special effect data matched with the audio commentary data can be set as required; and the audio commentary data can correspond to one kind of AR special effect data, or can correspond to multiple kinds of AR special effect data.

Among them, the display pose of the AR special effect data can be set in the 3D scene model as required. For example, the display pose of AR special effect data can be located in the explanation area of the preset knowledge points. Alternatively, the display pose of the AR special effect data may also be determined based on the spatial position corresponding to the preset knowledge point.

In some optional embodiments, each audio commentary data can be matched with corresponding AR special effect data in advance, and after the audio commentary data is acquired, the AR special effect data matching the audio commentary data can be determined.

Alternatively, after acquiring the audio commentary data corresponding to any preset knowledge point, the AR special effect data matching the audio commentary data may be determined in real time. For example, after obtaining the audio commentary data corresponding to any preset knowledge point, the target keywords corresponding to the audio commentary data can be identified; based on the target keywords corresponding to the audio commentary data and the keywords corresponding to each AR special effect data, Determine the AR special effect data matching the audio commentary data.

Optionally, after obtaining the audio commentary data corresponding to any preset knowledge point, the audio commentary data can be identified to determine the target keywords included in the audio commentary data; or, the text content of the audio commentary data can also be determined , identifying the text content corresponding to the audio commentary data, and determining a target keyword included in the audio commentary data; the target keyword may be a word related to a preset knowledge point. Then, the AR special effect data matching the audio commentary data may be determined based on the target keywords corresponding to the audio commentary data and the keywords corresponding to each AR special effect data.

In this embodiment, when controlling the AR device to play audio commentary data, the AR device can be controlled to display AR special effect data related to the target keyword according to the target keyword of the currently played audio commentary data, so that the played AR special effect data and The content of the played audio explanation data is consistent, and the preset knowledge points can be clearly explained and displayed.

In an optional implementation manner, the audio commentary data corresponding to each preset knowledge point can be pre-determined according to the following steps:

Step A1, obtaining the audio commentary data to be processed corresponding to the target area;

Step A2, according to the preset knowledge point library, determine the target keywords respectively matching with multiple preset knowledge points from the audio commentary data to be processed;

Step A3, according to the corresponding playing time position of each target keyword in the audio commentary data to be processed, from the audio commentary data to be processed, respectively determine the audio commentary data corresponding to each of the preset knowledge points .

Exemplarily, the to-be-processed audio commentary data corresponding to the target area can be generated according to the multiple knowledge point information included in the target area, wherein the to-be-processed audio commentary data can be complete audio explanations for each knowledge point respectively data. Then, according to the preset knowledge point library, the target keywords that match the multiple preset knowledge points can be determined from the audio explanation data to be processed; Set the target keywords that the knowledge points match respectively. Finally, according to the playback time position corresponding to each target keyword in the audio commentary data to be processed, the audio commentary data to be processed is segmented, and from the multiple audio commentary data obtained after segmentation, it is determined that each preset knowledge Click the corresponding audio explanation data.

Here, the audio commentary data to be processed can be identified according to the preset knowledge point database, and the target keywords in the audio data to be processed that match the preset knowledge points can be determined; From the audio commentary data to be processed, determine the audio commentary data corresponding to each preset knowledge point more accurately and quickly.

For S103: the audio commentary data can be played (or output) through the voice playback device of the AR device, and the AR device can be controlled to display the AR special effect data matching the currently played audio commentary data.

In an optional implementation manner, the controlling the AR device to play audio commentary data and display AR special effect data matching the currently played audio commentary data may include:

Method 1: When it is determined based on the positioning information that the AR device satisfies the trigger display condition of the AR special effect data, control the AR device to play the audio commentary data and display the AR special effect data.

Method 2: When it is determined based on the positioning information that the AR device does not meet the trigger display condition of the AR special effect data, control the AR device to display the first guide for guiding the adjustment of the positioning information of the AR device information.

Here, when the AR device satisfies the triggering display conditions of the AR special effect data, the AR device is controlled to play the audio explanation data and display the AR special effect data, so as to avoid poor playback effect of the AR special effect data when the positioning information of the AR device is inappropriate. If the situation occurs, the display effect of AR special effect data will be improved.

When the AR device does not meet the trigger display condition of the AR special effect data, the AR device can be controlled to display the first guidance information for guiding the adjustment of the positioning information of the AR device, so that the positioning information of the AR device can be adjusted according to the first guidance information , so that the adjusted AR device can display AR special effect data more clearly and intuitively, avoiding the omission of preset knowledge points, and improving the efficiency of explaining preset knowledge points.

In some optional embodiments, a corresponding trigger display condition may be set for each AR special effect data. For example, the triggering display condition may include a triggering display direction and/or a triggering display distance. When the triggering display condition includes a triggering display direction, and when the positioning information of the AR device indicates that the orientation of the AR device is within the set triggering display direction, it is determined that the AR device satisfies the triggering display condition of the AR special effect data. When the trigger display condition includes the trigger display distance, when the positioning information of the AR device indicates that the distance between the AR device and the display pose of the AR special effect data is less than the set first distance value, and/or greater than the set second distance value , to determine that the AR device satisfies the trigger display condition of the AR special effect data.

Referring to FIG. 2 , in FIG. 2 , for the AR special effect data, the trigger display direction and trigger display distance corresponding to the AR special effect data may be generated based on the display pose of the AR special effect data. Exemplarily, when the orientation of the AR device is within the direction range corresponding to the trigger display direction, it is determined that the AR device satisfies the trigger display condition, for example, for the AR device 21 in FIG. 2 , when the orientation of the AR device 21 is direction 211, If the orientation of the AR device 21 is within the direction range corresponding to the set trigger display direction, the AR device 21 satisfies the trigger display condition; when the AR device 21 is facing in direction 212, the AR device 21 does not meet the trigger display condition.

For another example, if the distance between the AR device 21 in FIG. 2 and the display location of the AR special effect data is less than the set first distance value and greater than the set second distance value, then the AR device 21 meets the set trigger display distance. The distance between the AR device 22 in FIG. 2 and the display position of the AR special effect data is greater than the set first distance value, so the AR device 22 does not meet the set trigger display distance.

When the triggering display conditions include the triggering display direction and the triggering display distance, then when the orientation indicated by the positioning information of the AR device satisfies the triggering display direction condition, and the location information indicated by the AR device’s positioning information satisfies the triggering display distance condition, determine the AR The device meets the trigger display conditions for AR special effect data. When the orientation indicated by the positioning information of the AR device does not meet the conditions for triggering the display direction, and/or the position information indicated by the positioning information of the AR device does not meet the conditions for triggering the display distance, it is determined that the AR device does not meet the conditions for triggering the display of AR special effect data .

When the AR device satisfies the trigger display condition corresponding to the AR special effect data, the AR device is controlled to play the audio explanation data and display the AR special effect data. If the AR device does not meet the trigger display condition corresponding to the AR special effect data, generate the first guide information for guiding the adjustment of the positioning information of the AR device, and display the first guide information through the AR device, so as to pass the first guide information The guidance information adjusts the pose of the AR device, so that the AR device can display the AR special effect data more clearly.

Refer to the schematic diagram of an AR device interface shown in Figure 3, which shows the first guide information, for example, the first guide information can be "Please move the AR device to the right to view the AR corresponding to the preset knowledge point A special effects data".

In an optional implementation manner, the method further includes: when the audio commentary data corresponding to any preset knowledge point is played, and/or when the display of the AR special effect data ends, controlling the AR The device displays at least one of the following information: second guidance information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point; list information for indicating that the AR device has not visited the preset knowledge point ; Navigation end information used to indicate that the AR device has played the audio explanation data corresponding to each preset knowledge point.

In this embodiment, a variety of information types to be displayed are set, so that after the audio explanation data corresponding to any preset knowledge point is played and/or the AR special effect data is displayed, the AR device can be controlled to display at least one type of information to improve Demonstrated diversity and flexibility.

In one embodiment, after the audio commentary data corresponding to any preset knowledge point is played and/or the display of AR special effect data is finished, when there are preset knowledge points that have not been visited on the AR device, the AR device display user can be controlled. The second guide information for instructing the AR device to move to the explaining area corresponding to the next preset knowledge point. Referring to Fig. 4a, the second guidance information included in Fig. 4a may be "please move forward and view preset knowledge point B". Exemplarily, Fig. 4a may also include navigation information from the current position to the next preset knowledge point (ie, preset knowledge point B).

In another embodiment, the AR device can also be controlled to display a list of preset knowledge points that the AR device has not visited; see FIG. 4b.

In yet another embodiment, the AR device can also be controlled to display the list information of all preset knowledge points, and mark the visited preset knowledge points and unvisited preset knowledge points among all the preset knowledge points. See Figure 4c, where the preset knowledge points with checkmark "√" can be browsed preset knowledge points, and the preset knowledge points without checkmark "√" can be unbrowsed preset knowledge points ; and then in response to the trigger operation, determine the preset knowledge point of the next tour of the AR device, and control the AR device to display the second guide for instructing the AR device to move to the explanation area corresponding to the determined preset knowledge point of the next tour information. Wherein, the triggering operation may be voice data, touch screen operation, and the like.

In another embodiment, when the AR device has no preset knowledge points that have not been visited, the AR device can also be controlled to display the navigation end indicating that the AR device has finished playing the audio explanation data corresponding to each preset knowledge point. Information, as shown in FIG. 4d, the navigation end information may be "this navigation is over, welcome to use next time". It is also possible to control the AR device to display a navigation end mark, for example, the smiley face mark in FIG. 4d may be a navigation end mark.

In an optional implementation manner, the displaying the AR special effect data corresponding to the currently played audio commentary data includes: during the process of playing the audio commentary data, switching and displaying various AR special effect data corresponding to the currently played audio commentary data. AR special effect data.

When the audio commentary data matches multiple AR special effect data, in the process of playing the audio commentary data, various AR special effect data corresponding to the currently played audio commentary data can be switched and displayed. For example, when the audio commentary data is audio data explaining historical figures, multiple AR special effect data may include AR special effects of historical figures in different postures, for example, multiple AR special effect data may include AR special effects of historical figures in learning postures Special effects, AR special effects of historical figures in a resting posture, etc. When the audio commentary data is audio data explaining buildings, multiple AR special effect data may include AR special effects of buildings under different lighting conditions, or the AR special effect data may include AR special effects of buildings located in different historical periods Wait.

For example, various AR special effect data can be switched and displayed according to the target keywords being explained in the currently played audio explanation data, so that the displayed AR special effect data matches the target keywords being explained. Or, during the process of playing the audio commentary data, the user can switch and display various AR special effect data corresponding to the currently played audio commentary data by adjusting the position of the AR device. For example, in the process of playing audio commentary data, the user can switch and display various AR special effect data corresponding to the currently played audio commentary data by moving the AR device left and right, or turning the AR device clockwise.

In this embodiment, the audio commentary data can correspond to various AR special effect data, so that in the process of playing the audio commentary data, various AR special effect data corresponding to the currently played audio commentary data can be switched and displayed, thereby improving the display of the AR special effect data Flexibility and variety.

In an optional implementation manner, the switching displays a variety of AR special effect data corresponding to the currently played audio commentary data, including:

Step B1, in response to the target trigger operation, determine the AR special effect data to be displayed from the various AR special effect data corresponding to the currently played audio commentary data;

Step B2, controlling the AR device to display the AR special effect data to be displayed.

When the AR device plays the audio commentary data, the AR device can be synchronously controlled to play the AR special effect data corresponding to the audio commentary data. For example, one can be randomly selected from a variety of AR special effect data to play. If the user wants to view other AR special effect data, the target trigger operation can be performed. Among the various AR special effect data, determine the AR special effect data to be displayed; control the AR device to display the AR special effect data to be displayed.

The target trigger operation can be set according to actual needs. For example, the target trigger operation can be voice data, touch screen operation, pose change status of the AR device, etc. Exemplarily, voice data can be input into the AR device. For example, the input voice data can be "special effect one". AR special effect data to be displayed. Alternatively, a touch screen operation may be performed on the AR device, and in response to the touch screen operation, the AR special effect data to be displayed is determined from various AR special effect data corresponding to the currently played audio commentary data. Alternatively, the AR special effect data to be displayed may be determined from a variety of AR special effect data corresponding to the currently played audio commentary data through the pose change state of the AR device. For example, if the pose transformation state of the AR device is moving left and right, the next AR special effect data behind the currently played AR special effect data among various special effect data may be determined as the AR special effect data to be displayed. Furthermore, the AR device can be controlled to display the determined AR special effect data to be displayed.

Here, when the audio commentary data corresponds to a variety of AR special effect data, the AR special effect data to be displayed may be determined from the various AR special effect data corresponding to the currently playing audio commentary data in response to the target trigger operation, so that the determined AR special effect data to be displayed AR special effect data can meet user needs and improve the flexibility of AR special effect data display.

In an optional implementation manner, the method may be applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

In this embodiment, the method can be applied to a client application platform, and the client can be a web application platform on the AR device, or a small program application platform on the AR device. Alternatively, the client application platform may also be an application program for AR navigation on the AR device.

Referring to Fig. 5, the augmented reality data presentation method may include:

S501. Acquire positioning information of the AR device after the AR device arrives at a target area.

S502. Based on the positioning information of the AR device, it is judged whether a preset location condition is satisfied between the AR device and an explanation area corresponding to any preset knowledge point.

S503. If the preset position condition is met, acquire the audio commentary data corresponding to any preset knowledge point meeting the preset position condition, and the AR special effect data matching the audio commentary data. For example, when the distance between the AR device and the explaining area corresponding to any preset knowledge point is less than 2 meters, it is confirmed that the AR device satisfies the preset location condition. If the preset position condition is not satisfied between the AR device and the explaining area corresponding to any preset knowledge point, the positioning information of the AR device can continue to be acquired.

S504. Based on the positioning information, determine whether the AR device satisfies a condition for triggering display of the AR special effect data.

S505. If the trigger display condition of the AR special effect data is satisfied, control the AR device to play the audio explanation data and display the AR special effect data.

S506. If the trigger display condition of the AR special effect data is not met, control the AR device to display first guidance information for guiding and adjusting the positioning information of the AR device, so as to guide the AR device to view the AR special effect data. For example, when the orientation of the AR device is within the trigger display direction corresponding to the AR special effect data, and/or, the distance between the AR device and the display pose of the AR device and the AR special effect data is within 0.5 meters to 5 meters (including 0.5 meters) and 5 meters), it is determined that the AR device satisfies the trigger display condition for AR special effect data.

S507. After the audio explanation data of any preset knowledge point is played, and/or, after the AR special effect data is displayed, the AR device may be prompted to view the next preset knowledge point until all the preset knowledge points in the target area correspond to If all the audio commentary data of the audio guide is played, it is confirmed that the navigation is finished.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same idea, the embodiment of the present disclosure also provides an augmented reality data presentation device, as shown in FIG. Two acquisition module 602 and control module 603, specifically:

The first obtaining module 601 is configured to obtain positioning information of the AR device;

The second obtaining module 602 is configured to obtain any preset location condition when it is determined that the explanation area corresponding to the AR device and any preset knowledge point satisfies a preset location condition based on the positioning information. Audio commentary data corresponding to the knowledge point, and AR special effect data matching the audio commentary data;

The control module 603 is configured to control the AR device to play audio commentary data, and display AR special effect data matching the currently played audio commentary data.

In a possible implementation manner, the control module 603 is configured to: control the AR device to play the The audio explains the data and displays the AR special effect data.

In a possible implementation manner, the device further includes: a first display module 604, configured to: when it is determined based on the positioning information that the AR device does not meet the trigger display condition of the AR special effect data, control The AR device displays first guide information for guiding adjustment of positioning information of the AR device.

In a possible implementation manner, the device further includes: a second display module 605 configured to:

When the audio explanation data corresponding to any preset knowledge point is played, and/or the AR special effect data display is completed, the AR device is controlled to display at least one of the following information:

Second guidance information for instructing the AR device to move to the explanation area corresponding to the next preset knowledge point;

List information used to indicate the preset knowledge points that the AR device has not visited;

The navigation end information is used to indicate that the AR device has played the audio commentary data corresponding to each preset knowledge point.

In a possible implementation manner, the control module 603 is configured to: during the process of playing the audio commentary data, switch and display various AR special effect data corresponding to the currently played audio commentary data.

In a possible implementation manner, the control module 603 is configured to: determine the AR special effect data to be displayed from various AR special effect data corresponding to the currently played audio commentary data in response to a target trigger operation; control the The AR device displays the AR special effect data to be displayed.

In a possible implementation manner, the first acquisition module 601 is configured to: acquire a real-time scene image collected by the AR device; determine the positioning of the AR device based on the real-time scene image and the constructed 3D scene model information.

In a possible implementation manner, the device further includes a determination module 606 configured to predetermine the audio commentary data corresponding to each preset knowledge point according to the following steps: acquire the audio commentary data to be processed corresponding to the target area; A knowledge point library, determining target keywords that match a plurality of preset knowledge points from the audio commentary data to be processed; according to the corresponding playing time positions of each target keyword in the audio commentary data to be processed, From the audio commentary data to be processed, audio commentary data corresponding to each of the preset knowledge points are respectively determined.

In a possible implementation manner, the method is applied to a client application platform, and the client application platform is a Web application platform or an applet application platform.

The functions of the device provided by the embodiments of the present disclosure or the included templates can be used to execute the methods described in the above method embodiments, and its specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, details are not repeated here.

Based on the same technical idea, an embodiment of the present disclosure also provides an electronic device. Referring to FIG. 7 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure, an electronic device 700 includes a processor 701 , a memory 702 and a bus 703 . Among them, the memory 702 is used to store execution instructions, including a memory 7021 and an external memory 7022; the memory 7021 here is also called an internal memory, and is used to temporarily store calculation data in the processor 701 and exchange data with an external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the memory 7021. When the electronic device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 is executing the augmented reality data presentation method of the embodiment of the present disclosure. A step of.

In addition, an embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the method for presenting augmented reality data described in the foregoing method embodiments is executed. A step of. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

An embodiment of the present disclosure also provides a computer program product, the computer program product carries a program code, and the instructions included in the program code can be used to execute the steps of the method for presenting augmented reality data described in the method embodiment above, for details, please refer to The foregoing method embodiments are not described in detail here.

Wherein, the above-mentioned computer program product may be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. Wait.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, and details are not repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

The above is only a specific implementation of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be determined by the protection scope of the claims.

Claims (12)

A method for presenting augmented reality data, comprising: Obtain the positioning information of the augmented reality AR device; Based on the positioning information, when it is determined that a preset location condition is satisfied between the AR device and the explaining area corresponding to any preset knowledge point, acquire the audio explaining data corresponding to the any preset knowledge point, and AR special effect data matched by the audio commentary data; Controlling the AR device to play audio commentary data, and displaying AR special effect data matching the currently played audio commentary data. The method according to claim 1, wherein the controlling the AR device to play audio commentary data and display AR special effect data matching the currently played audio commentary data comprises: When it is determined based on the positioning information that the AR device satisfies the triggering display condition of the AR special effect data, the AR device is controlled to play the audio commentary data and display the AR special effect data. The method according to claim 1 or 2, wherein the method further comprises: In a case where it is determined based on the positioning information that the AR device does not meet the triggering display condition of the AR special effect data, the AR device is controlled to display first guidance information for guiding adjustment of the positioning information of the AR device. The method according to any one of claims 1 to 3, wherein the method further comprises: When the audio explanation data corresponding to any preset knowledge point is played, and/or the AR special effect data display is completed, the AR device is controlled to display at least one of the following information: Second guidance information for instructing the AR device to move to the explanation area corresponding to the next preset knowledge point; List information used to indicate the preset knowledge points that the AR device has not visited; The navigation end information is used to indicate that the AR device has played the audio commentary data corresponding to each preset knowledge point. The method according to any one of claims 1 to 4, wherein the displaying the AR special effect data corresponding to the currently played audio commentary data includes: During the process of playing the audio commentary data, various AR special effect data corresponding to the currently played audio commentary data are switched and displayed. The method according to claim 5, wherein the switching display of various AR special effect data corresponding to the currently played audio commentary data includes: In response to the target trigger operation, determine the AR special effect data to be displayed from among the various AR special effect data corresponding to the currently played audio commentary data; Controlling the AR device to display the AR special effect data to be displayed. The method according to any one of claims 1 to 6, wherein said acquiring the positioning information of the AR device includes: Obtaining real-time scene images collected by the AR device; Based on the real-time scene image and the constructed 3D scene model, determine the positioning information of the AR device. The method according to any one of claims 1 to 7, wherein the audio explanation data corresponding to each preset knowledge point is predetermined according to the following steps: Obtain the audio commentary data to be processed corresponding to the target area; According to the preset knowledge point library, determine target keywords respectively matching with multiple preset knowledge points from the audio commentary data to be processed; According to the corresponding playing time position of each target keyword in the audio commentary data to be processed, the audio commentary data corresponding to each preset knowledge point is respectively determined from the audio commentary data to be processed. The method according to any one of claims 1 to 8, wherein the method is applied to a client application platform, and the client application platform is a web page application platform or an applet application platform. An augmented reality data presentation device, comprising: The first obtaining module is configured to obtain the positioning information of the augmented reality AR device; The second acquiring module is configured to acquire the corresponding position of any preset knowledge point when it is determined that the explanation area corresponding to the AR device and any preset knowledge point satisfies the preset location condition based on the positioning information. audio commentary data, and AR special effect data matching the audio commentary data; The control module is configured to control the AR device to play audio commentary data, and display AR special effect data matching the currently played audio commentary data. An electronic device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus When the machine-readable instructions are executed by the processor, the steps of the method for presenting augmented reality data according to any one of claims 1 to 9 are executed. A computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the method for presenting augmented reality data according to any one of claims 1 to 9 are executed.

Images