CN106203286B - Augmented reality content acquisition method and device and mobile terminal - Google Patents

Abstract

The embodiment of the present invention discloses a method, device and mobile terminal for obtaining content of augmented reality. The method comprises: obtaining a target picture selected by a user; determining a target content in the target picture; and obtaining enhanced content corresponding to the target content. The embodiment of the present invention adopts the above technical solution, so that a user can select an existing picture as a target picture according to his or her own preferences, and automatically obtain the enhanced content by obtaining the enhanced content corresponding to a certain content in the existing picture, thereby meeting the user's need to quickly obtain the expected enhanced content.

Description

A kind of augmented reality content acquisition method, device and mobile terminal

technical field

Embodiments of the present invention relate to the technical field of mobile terminals, and in particular, to an augmented reality content acquisition method, device, and mobile terminal.

Background technique

Augmented Reality (AR) technology is a technology that increases the user's perception of the real world through the information provided by the computer system, applies virtual information to the real world, and superimposes computer-generated virtual objects, scenes or system prompt information on the real world. scene, so as to realize the augmentation of reality.

At present, the camera function has become a standard configuration of mobile terminals, and mobile terminal users can easily record the scenery and wonderful moments around them in their daily life. The photos taken by the ordinary camera mode are basically the same as what the human eye sees. Many scenes, phenomena or things cannot be captured only by adjusting the camera parameters. Not to the moon and so on. More and more users are not satisfied with the shooting of real scenes, but want to incorporate more elements into their photos when taking pictures to make the photos more beautiful or eye-catching. Augmented reality technology can meet this demand of users, and can superimpose virtual content as augmented content with images obtained by cameras to obtain augmented reality photos. However, there are many types and styles of enhanced content, and it is time-consuming and labor-intensive for users to select enhanced content that meets their needs. In addition, they may not be able to find their favorite enhanced content. Therefore, how to automatically obtain the enhanced content expected by users has become a problem at present. Technical problems to be solved urgently.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide an augmented reality content acquisition method, device and mobile terminal, so as to realize the automatic acquisition of augmented reality content.

In a first aspect, an embodiment of the present invention provides an augmented reality content acquisition method, including:

Get the target image selected by the user;

determining the target content in the target picture;

Acquire enhanced content corresponding to the target content.

In a second aspect, an embodiment of the present invention provides an augmented reality content acquisition device, including:

The target image acquisition module is used to acquire the target image selected by the user;

a target content determination module for determining the target content in the target picture;

An enhanced content acquisition module, configured to acquire enhanced content corresponding to the target content.

In a third aspect, an embodiment of the present invention provides a mobile terminal, where the mobile terminal integrates the augmented reality content acquisition device according to the embodiment of the present invention.

The augmented reality content acquisition solution provided in the embodiment of the present invention acquires the target picture selected by the user, determines the target content in the target picture, and acquires the enhanced content corresponding to the target content, so as to facilitate subsequent use of the enhanced content. By adopting the above technical solutions, users can select an existing picture as a target picture according to their own preferences, and automatically obtain the enhanced content by obtaining the enhanced content corresponding to a certain content in the existing picture, so as to satisfy the user's expectation of rapid acquisition. demand for enhanced content.

Description of drawings

FIG. 1 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 1 of the present invention;

2 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 2 of the present invention;

3 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 3 of the present invention;

4 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 4 of the present invention;

FIG. 5 is a schematic diagram of displaying a target picture according to Embodiment 4 of the present invention;

6 is a schematic diagram of a shooting preview interface according to Embodiment 4 of the present invention;

7 is a schematic diagram of displaying an augmented reality photo according to Embodiment 4 of the present invention;

FIG. 8 is a structural block diagram of an augmented reality content acquisition apparatus according to Embodiment 5 of the present invention.

Detailed ways

The technical solutions of the present invention are further described below with reference to the accompanying drawings and through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Before discussing the exemplary embodiments in greater detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowchart depicts the steps as a sequential process, many of the steps may be performed in parallel, concurrently, or concurrently. Furthermore, the order of the steps can be rearranged. The process may be terminated when its operation is complete, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

Example 1

1 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 1 of the present invention. The method can be executed by an augmented reality content acquisition device, where the device can be implemented by software and/or hardware, and can generally be integrated in mobile in the terminal. As shown in Figure 1, the method includes:

Step 101: Obtain a target image selected by a user.

Exemplarily, the mobile terminal in this embodiment may specifically be a terminal such as a mobile phone, a tablet computer, and a digital camera.

Exemplarily, the target picture may include an augmented reality picture. The augmented reality picture may specifically be a picture containing augmented content. Enhanced content can include physical images (such as trees, moons, beaches, and mascots), special effects (such as smoke effects, steam effects, and motion trail effects, etc.) and natural phenomena (such as rain, snow, rainbows, and sun circles, etc.) ) and other virtual content. The target pictures may also include normal pictures without enhanced content.

In this embodiment, there may be many specific sources of the target image, which are not specifically limited in this embodiment. Exemplarily, when a user browses a web page, he will see a lot of various pictures such as news pictures or promotional pictures; when a user is using an application on a mobile terminal, he will also see pictures from Moments, Weibo or other applications. pictures taken by others or uploaded pictures; in addition, the user may also browse pictures in the local gallery of the mobile terminal, or view pictures sent by friends.

Among the above-mentioned pictures that exist in various forms, if the user sees that a certain picture contains the content he needs, he can obtain the picture as a target picture by downloading or saving as a target picture. For example, if a user sees a beautiful teddy bear in a photo uploaded by a friend in the circle of friends, and wants to put the teddy bear as an enhanced content in his own picture or photo, he can save the friend's photo as the target picture.

Step 102: Determine the target content in the target picture.

Exemplarily, the target content may be specifically understood as content that the user is interested in or content related to augmented reality contained in the target picture. This embodiment does not limit the specific manner of determining the target content, and some implementation manners are given below as schematic descriptions:

For example, the target content in the target picture may be determined according to the user's selection operation. The selection operation can be an operation such as the user's circle selection in the target picture, such as selecting by using a finger to outline the outline of the target content on the screen. For example, the user can select a teddy bear in the target picture as the target content; select The operation may also be to select from multiple candidate options, for example, by identifying the content of the target image (including scenes or special effects, etc.), and providing the user with candidate target content according to the recognition result, prompting the user to select a target from it. Content, as the above example, when identifying the target picture, when it is recognized that the target picture contains scenes such as a teddy bear, a table, and a vase, the user can be prompted to select the target content.

The above-mentioned method of determining the target content in the target picture can be applied to ordinary pictures as well as augmented reality pictures. When determining the target content, it is not necessary to consider whether the target picture contains augmented reality content, but the target content can be directly determined according to the user's wishes.

For another example, the specific manner of determining the target content may also be: identifying the content contained in the target picture, and using the identified content related to augmented reality as the target content. This way of determining the target content in the target picture is especially suitable for augmented reality pictures.

Exemplarily, the content related to augmented reality may include 3D images, cartoon images, hand-painted patterns, images with special artistic effects (such as oil painting images and sketch images, etc.) and images with special image processing effects (such as translucent images and special effects images, etc.) and other virtual content. When the above-mentioned virtual content is identified, the current target picture may be determined as an augmented reality picture, and the above-mentioned virtual content may be determined as the target content. Generally, since it is difficult to obtain virtual content through a camera, when a user sees such an augmented reality picture, it is easier for a user to determine the above-mentioned virtual content as the augmented content to be obtained by himself. In this way of determining the target content, the mobile terminal can automatically determine the target content by means of identification, which is fast and accurate, effectively saves the user's time for selecting the target content, and quickly obtains the enhanced content expected by the user.

Illustratively, the augmented reality-related content may also include the identified content consistent with a certain augmented content stored in the augmented content database. At this time, this method of determining the target content is especially suitable for the target picture having a special format different from the ordinary picture (hereinafter referred to as the enhanced format picture). For example, when generating an augmented reality picture, the pixels or regions corresponding to the augmented content may be encoded in a special format, such as adding a logo for indicating the identity of the augmented content (hereinafter referred to as the augmented logo). When it is recognized that the current target picture is an enhanced format picture, the content related to the augmented reality in the target picture can be determined as the target content according to the enhanced identification after decoding.

Step 103: Acquire enhanced content corresponding to the target content.

Exemplarily, the enhanced content corresponding to the target content may be understood as the enhanced content consistent with the target content. The target content can be understood as part of the image or part of the pattern contained in the target picture, and the enhanced content consistent with the target content can be understood as the same image or pattern corresponding to the target content that can be used for augmented reality applications.

The method for obtaining the content of augmented reality provided by the first embodiment of the present invention obtains the target picture selected by the user, determines the target content in the target picture, and obtains the enhanced content corresponding to the target content, so as to facilitate subsequent use of the enhanced content. By adopting the above technical solutions, users can select an existing picture as a target picture according to their own preferences, and automatically obtain the enhanced content by obtaining the enhanced content corresponding to a certain content in the existing picture, so as to satisfy the user's expectation of rapid acquisition. demand for enhanced content.

Embodiment 2

FIG. 2 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 2 of the present invention. Based on the above-mentioned embodiment, this embodiment optimizes the steps of acquiring the augmented content corresponding to the target content. The method It is especially suitable for the situation where there are enhanced contents corresponding to the target contents in the enhanced contents database.

The method of this embodiment includes the following steps:

Step 201: Obtain a target image selected by a user.

Step 202: Determine the target content in the target picture.

Exemplarily, the target content in this embodiment may specifically be enhanced content included in the target picture.

Step 203: Identify attribute information of the target content.

Exemplarily, the attribute information may specifically include a number or an identifier for distinguishing it from other enhanced content. Therefore, for a target content, its attribute information is unique and definite.

Exemplarily, the attribute information may further include feature information of the target content, such as the presentation theme of the target content (such as what kind of object, what kind of special effect, or what kind of natural phenomenon the target content is specifically), shape, color, size, transparency, etc. and brightness, etc.

Step 204: Acquire enhanced content corresponding to the target content from the enhanced content database according to the attribute information.

Exemplarily, the enhanced content database can be specifically loaded in a local or remote server of the mobile terminal.

Exemplarily, when the attribute information is a number, the enhanced content database can be searched according to the number, and the enhanced content corresponding to the number can be obtained; when the attribute information is the feature information of the target content, the enhanced content can be searched according to the feature information. A matching search is performed in the database, and the searched enhanced content is used as the enhanced content to be acquired.

On the basis of the above embodiments, the second embodiment of the present invention acquires the enhanced content corresponding to the target content from the enhanced content database according to the attribute information of the target content, which can further accelerate the acquisition speed of the enhanced content and improve the user experience.

Embodiment 3

3 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 3 of the present invention. Based on the foregoing embodiment, this embodiment optimizes the steps of acquiring augmented content corresponding to the target content. The method It is especially suitable for the situation where there is no enhanced content corresponding to the target content in the enhanced content database.

The method of this embodiment includes the following steps:

Step 301: Obtain a target image selected by a user.

Step 302: Determine the target content in the target picture.

Step 303: Extract image data corresponding to the target content from the target picture.

Exemplarily, image data of pixels or regions corresponding to the target content may be extracted from the target picture, such as binary image data; the image corresponding to the target content may also be captured from the target picture by means of screenshots.

Step 304: Generate enhanced content corresponding to the target content according to the image data, and acquire the generated enhanced content.

Exemplarily, the enhanced content corresponding to the target content can be generated according to the image information contained in the image data, which can be understood as restoring the image corresponding to the target content in the target picture according to the image data to obtain the enhanced content to be acquired.

Embodiment 3 of the present invention, on the basis of the above embodiments, extracts image data corresponding to the target content from the target picture, and uses this to generate enhanced content corresponding to the target content, which can improve the success rate of acquiring the enhanced content, and further meet user needs.

Based on the above embodiment, after acquiring the generated enhanced content, the method may further include: storing the generated enhanced content in an enhanced content database. The advantage of this setting is that the enhanced content database can be updated in real time, and the enhanced content contained in the enhanced content database can be enriched for subsequent use.

It should be noted that, for the case where the target picture is an augmented reality picture, the enhanced content in the target picture can exist in various forms as the target content, and the various acquisition enhanced content provided above can be selected based on different existing forms. specific way. For example, the enhanced content may be a virtual image (which can be understood as a pre-drawn template image) added on the basis of a picture based on a real scene, and the virtual image may be pre-stored in the enhanced content database, so it can be obtained from the enhanced content database ; for another example, the enhanced content may be the modification or special processing of the scene based on the picture based on the real scene, such as enhancing the brightness of a scene or exaggerating the aperture effect of the sun, etc., which can be extracted from the target picture by extracting The enhanced content can be obtained in the form of image data; for another example, the enhanced content can be a drawing or pattern (such as a heart-shaped cloud hand-painted by the user) added on the basis of a picture based on a real scene, and the enhanced content at this time is more personalized. Generally, the corresponding enhanced content cannot be searched in the enhanced content database, and the enhanced content can also be obtained by extracting image data from the target picture.

Embodiment 4

FIG. 4 is a schematic flowchart of an augmented reality content acquisition method according to Embodiment 4 of the present invention. In this embodiment, optimization is performed on the basis of the foregoing embodiment. After acquiring the augmented content corresponding to the target content, augmented reality is added. Operations related to photo shooting.

Step 401: Obtain a target image selected by a user.

Exemplarily, FIG. 5 is a schematic diagram of displaying a target picture according to Embodiment 4 of the present invention. As shown in FIG. 5 , the picture displayed on the screen of the mobile terminal is the target picture selected by the user, and the target picture includes a house and a cartoon version of the moon.

Step 402: Determine the target content in the target picture.

Exemplarily, as shown in FIG. 5 , the cartoon version of the moon in the target picture is determined as the target content.

Step 403: Acquire enhanced content corresponding to the target content.

Exemplarily, the attribute information of the cartoon version of the moon can be identified, such as the number, according to the number, the mobile terminal is locally searched for the corresponding enhanced content, if the local does not exist, the remote server can be continued to search for the enhanced content corresponding to the number. .

Step 404: Display the enhanced content on the shooting preview interface.

Exemplarily, the acquired enhanced content can be directly used for the shooting of augmented reality photos, and the acquired enhanced content can be displayed in the shooting preview interface for the user to view. The acquired images are superimposed, allowing the user to preview the enhancement. FIG. 6 is a schematic diagram of a shooting preview interface provided in Embodiment 4 of the present invention. As shown in FIG. 6 , the preview image obtained by the camera is the sea, and the cartoon version of the moon obtained in the above steps is displayed in the shooting preview interface, and the user can preview To the enhancement of adding a cartoon version of the moon on top of the sea.

Preferably, the attribute information of the enhanced content can be adjusted according to the received preset adjustment instruction. The attribute information may include at least one of display position, display size, color and transparency on the shooting preview interface. Exemplarily, the preset adjustment instruction can be a user's gesture operation on the screen of the mobile terminal, such as dragging, zooming, etc. on the enhanced content; it can also be that the user clicks on the enhanced content to trigger the pop-up of the attribute list, and then clicks on the enhanced content. Adjustment instructions corresponding to a series of operations such as selecting in the attribute list or inputting attribute content.

Step 405: When receiving the photographing instruction, superimpose the image captured by the camera and the augmented content to generate an augmented reality photo.

Exemplarily, the user can adjust the real shooting content in the shooting preview interface by changing the position of the mobile terminal. When the combined effect of the real shooting content and the enhanced content in the shooting preview interface meets the user's requirements, the user can click the shooting button to take a photo. , at this time, the mobile terminal will superimpose the image captured by the camera and the augmented content to generate an augmented reality photo.

FIG. 7 is a schematic diagram of displaying an augmented reality photo according to Embodiment 4 of the present invention. As shown in Figure 7, the real image containing the sea is synthesized with the cartoon version of the moon, and an augmented reality photo containing the sea and the cartoon version of the moon is obtained.

On the basis of the above-mentioned embodiments, the fourth embodiment of the present invention adds operations related to the shooting of augmented reality photos, and the obtained enhanced content can be directly used for the shooting of augmented reality photos, which improves the shooting experience of users and enhances the augmented reality photos at the same time. Fun to shoot.

Embodiment 5

8 is a structural block diagram of an augmented reality content acquisition device according to Embodiment 5 of the present invention, the device may be implemented by software and/or hardware, generally integrated in a mobile terminal, and may be acquired by executing an augmented reality content acquisition method Enhanced content. As shown in FIG. 8 , the apparatus includes a target picture acquisition module 801 , a target content determination module 802 and an enhanced content acquisition module 803 .

Among them, the target image acquisition module 801 is used to acquire the target image selected by the user; the target content determination module 802 is used to determine the target content in the target image; the enhanced content acquisition module 803 is used to acquire the target content related to the target content. Corresponding enhancements.

The augmented reality content acquisition device provided by the fifth embodiment of the present invention acquires a target image selected by a user, determines the target content in the target image, and acquires enhanced content corresponding to the target content, thereby facilitating subsequent use of the enhanced content. By adopting the above technical solutions, users can select an existing picture as a target picture according to their own preferences, and automatically obtain the enhanced content by obtaining the enhanced content corresponding to a certain content in the existing picture, so as to satisfy the user's expectation of rapid acquisition. demand for enhanced content.

On the basis of the above embodiment, the target content determination module can be used to: determine the target content in the target picture according to the user's selection operation; or identify the content contained in the target picture, and compare the identified and Augmented reality related content as target content.

On the basis of the above embodiment, the enhanced content acquisition module includes an identification and identification unit and an enhanced content acquisition unit. Wherein, the identification and identification unit is used to identify the attribute information of the target content; the enhanced content acquisition unit is used to obtain the enhanced content corresponding to the target content from the enhanced content database according to the attribute information.

Based on the above embodiments, the enhanced content acquisition module includes an image data extraction unit and an enhanced content generation unit. Wherein, the image data extraction unit is used to extract the image data corresponding to the target content from the target picture; the enhanced content generation unit is used to generate the enhanced content corresponding to the target content according to the image data, obtain Generated enhancements.

On the basis of the above embodiment, the apparatus further includes: an enhanced content storage module, configured to store the generated enhanced content in an enhanced content database after acquiring the generated enhanced content.

On the basis of the above-mentioned embodiment, the apparatus further includes a display module and a photo generation module. The display module is configured to display the enhanced content on the shooting preview interface after acquiring the enhanced content corresponding to the target content; the photo generation module is configured to display the enhanced content on the shooting preview interface, after receiving the enhanced content When the photographing instruction is reached, the image captured by the camera and the augmented content are superimposed to generate an augmented reality photo.

Embodiment 6

The sixth embodiment provides a mobile terminal, where the mobile terminal includes the augmented reality content acquisition device described in the embodiments of the present invention, and can acquire augmented reality content by executing the augmented reality content acquisition method.

Exemplarily, the mobile terminal in this embodiment may specifically be a terminal such as a mobile phone, a tablet computer, and a digital camera, and is preferably a smart phone.

When the user uses the mobile terminal in this embodiment, the user can select an existing picture as the target picture according to his own preferences, and automatically obtain the enhanced content by acquiring the enhanced content corresponding to a certain content in the existing picture. , to meet the needs of users to quickly obtain the expected enhanced content.

The augmented reality content acquisition device and mobile terminal provided in the above embodiments can execute the augmented reality content acquisition method provided by any embodiment of the present invention, and have corresponding functional modules and beneficial effects for implementing the method. For technical details that are not described in detail in the foregoing embodiments, reference may be made to the method for acquiring content of augmented reality provided by any embodiment of the present invention.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (9)

1. An augmented reality content acquisition method, comprising:

acquiring an augmented reality picture selected by a user through a downloading mode or an additional storage mode;

determining target content in the augmented reality picture, identifying attribute information of the target content, and acquiring the augmented content consistent with the target content from an augmented content database according to the attribute information, wherein the target content is virtual content;

if the augmented content which is consistent with the target content does not exist in the augmented content database, extracting image data corresponding to the target content from the augmented reality picture, restoring the augmented content which is consistent with the target content according to the image data, and acquiring the augmented content;

displaying the enhanced content on a shooting preview interface;

and when a photographing instruction is received, overlapping the image captured by the camera with the augmented content and generating an augmented reality photo.

2. The method of claim 1, wherein determining the target content in the augmented reality picture comprises:

determining target content in the augmented reality picture according to selection operation of a user; or

And identifying the content contained in the augmented reality picture, and taking the identified content related to augmented reality as target content.

3. The method of claim 1, further comprising, after obtaining the restored enhancement content:

and storing the restored enhanced content into the enhanced content database.

4. The method of claim 1, wherein after the displaying the enhanced content in the capture preview interface, further comprising:

and adjusting the attribute information of the enhanced content according to the received preset adjusting instruction, wherein the attribute information comprises at least one of a display position, a display size, a color and a transparency on the shooting preview interface.

5. An augmented reality content acquisition apparatus, comprising:

the target picture acquisition module is used for acquiring an augmented reality picture selected by a user in a downloading mode or an additional storage mode;

the augmented content acquiring module is used for determining target content in the augmented reality picture, identifying attribute information of the target content, and acquiring augmented content consistent with the target content from an augmented content database according to the attribute information, wherein the target content is virtual content; if the augmented content which is consistent with the target content does not exist in the augmented content database, extracting image data corresponding to the target content from the augmented reality picture, restoring the augmented content which is consistent with the target content according to the image data, and acquiring the augmented content;

the display module is used for displaying the enhanced content on a shooting preview interface after the enhanced content consistent with the target content is obtained;

and the photo generation module is used for performing superposition processing on the image captured by the camera and the enhanced content and generating an augmented reality photo when a photographing instruction is received after the enhanced content is displayed on the photographing preview interface.

6. The apparatus of claim 5, wherein the enhanced content acquisition module is configured to:

determining target content in the augmented reality picture according to selection operation of a user; or

And identifying the content contained in the augmented reality picture, and taking the identified content related to augmented reality as target content.

7. The apparatus of claim 5, further comprising:

and the enhanced content storage module is used for storing the restored enhanced content into the enhanced content database after acquiring the restored enhanced content.

8. The apparatus of claim 5, wherein the display module is further configured to:

and adjusting the attribute information of the enhanced content according to the received preset adjusting instruction, wherein the attribute information comprises at least one of a display position, a display size, a color and a transparency on the shooting preview interface.

9. A mobile terminal characterized in that it integrates an augmented reality content acquisition device according to any one of claims 5-8.

Images