CN110741327B - Mud toy system and method based on augmented reality and digital image processing - Google Patents

Abstract

The invention discloses a clay toy system and method based on augmented reality and digital image processing. The system includes an image acquisition module, an AR identification module, a display, an arithmetic unit, and an I/O interactive control system; wherein, the image acquisition The module is used to collect the image or image data of the physical clay model and transmit it to the AR recognition module; the AR recognition module is used to generate a corresponding three-dimensional clay model for the acquired image data of the physical clay model and transmit it to the operator; The arithmetic unit is used to process special effects, add audio and sound effects, and add actions to the obtained three-dimensional clay model, and transmit it to the display for display; the I/O interactive control system is used to control the arithmetic unit, including selecting Added special effects, audio sound effects and actions, memory: used to store data of special effects, audio sound effects and actions. The invention not only extends the playing methods of traditional clay toys, but also solves the problem of recognizing non-planar entities by augmented reality technology.

Description

Clay toy system and method based on augmented reality and digital image processing

technical field

The invention relates to the application of augmented reality technology in the field of children's toys, in particular to a clay toy system based on augmented reality and digital image processing and a method thereof.

Background technique

Augmented reality technology is a new technology that "seamlessly" integrates real world information and virtual world information. Touch, etc.), through computer and other science and technology, simulate and then superimpose, apply virtual information to the real world, and be perceived by human senses, so as to achieve a sensory experience beyond reality. The real environment and virtual objects are superimposed on the same screen or space in real time. Traditional clay toys have the benefits of improving children's creativity, hands-on ability, and color observation ability. However, due to the strong plasticity and the need to master more skills, without the accompanying education of adults, most children are unable to produce excellent works at the beginning, and do not continue to learn and create after the initial novelty, and cannot grow and improve as they should.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention aims to provide a clay toy system and method based on augmented reality and digital image processing, which can quickly and stably identify three-dimensional physical clay models, and submit corresponding recognition accuracy. , to complete the evaluation of the work, and to track and position synchronously output, combine the virtual 3D model with the real world, and then conduct single or multi-player game interaction, which not only extends the gameplay of traditional clay toys, but also solves the problem of augmented reality Techniques for the recognition of non-planar entities.

In order to achieve the above object, the present invention adopts the following technical solutions:

A clay toy system based on augmented reality and digital image processing, including an image acquisition module, an AR recognition module, a display, an arithmetic unit, and an I/O interactive control system; among them,

The image collection module is used to collect the image data of the physical clay model and transmit it to the AR recognition module;

The AR recognition module is used to generate a corresponding three-dimensional clay model for the acquired image data of the physical clay model and transmit it to the computing unit;

The computing unit is used to process special effects, add audio and sound effects and add actions to the obtained three-dimensional clay model, and transmit it to the display for display;

The I/O interactive control system is used to control the arithmetic unit, including special effects, audio sound effects and actions selected for addition;

Memory: used to store special effects, audio sound effects and motion data, the I/O interactive control system and computing unit are connected to the memory.

It should be noted that it includes a plurality of image acquisition modules and arithmetic units, each image acquisition module corresponds to an arithmetic unit, and all image acquisition modules and arithmetic units are connected to the AR recognition module, and each arithmetic unit is respectively Connected to an I/O interactive control system; different image acquisition modules are used to collect image data of different physical clay models and transmit them to the AR recognition module, and the AR recognition module generates the corresponding 3D clay models and then transmits them to the corresponding In the computing unit, the corresponding computing unit performs special effect processing, adds audio sound effects, and matches actions; the special effects, audio sound effects, and actions used by each computing unit are controlled by the I/O interactive control system connected to it.

It should be further noted that an online battle engine module and a battle scene image module are also arranged between the AR recognition module and one of the computing units, and the AR recognition module, the online battle engine module and the battle scene image module and the computing unit sequentially connected between;

It should be noted that each I/O interactive control system is connected to a gyroscope control module for inputting control instructions to the I/O interactive control system through acceleration in different directions.

It should be noted that it also includes a mode selection module for selecting a multiplayer interactive mode and a stand-alone mode.

The method for using the above-mentioned clay toy system based on enhanced display and digital image processing includes the following steps:

S1 uses the mode selection module to select multi-person interactive mode or stand-alone mode;

S2 When the multi-person interactive mode is selected, the I/O interactive control system loads the battle engine, and each user uses a different image acquisition module to collect the image data of the physical clay model made by himself, and selects his own through the I/O interactive control system. The required special effects, audio sound effects and action processing, the image data of the final physical clay model is processed by the AR recognition module to generate a three-dimensional clay model, and then processed by the special effects, audio sound effects and actions of the calculator and output to the display for display; The three-dimensional clay models processed by the AR recognition module and the calculator of all users' physical clay models are displayed on the monitor, so as to realize the interaction with other three-dimensional clay models;

When the stand-alone mode is selected, after the image acquisition module collects the image of the physical clay model, the AR recognition module generates the corresponding three-dimensional clay model and displays it on the display after being processed by the calculator.

It should be noted that, the control command is input to the I/O interactive control system through the gyroscope control module, and the accelerations in different directions respectively represent a control command.

It should be noted that, in step S2, the specific processing method of the AR recognition module in multi-person interactive mode or stand-alone mode is as follows:

2.1) Use the image acquisition module to perform multi-dimensional data sampling of the physical clay model;

2.2) The AR recognition module conducts coordinate analysis on the collected data to identify the regular or irregular shape of the physical clay model;

2.3) If the physical clay model is regular, call the sampled two-dimensional/three-dimensional model to pick the color at the specified point and perform compensation processing, and then perform color matching processing on the two-dimensional/three-dimensional model to generate a three-dimensional clay model;

If the physical clay model is irregular:

If it is a pure white background map, generate a sample map by supplementing the color difference, and then match the sample map with the sampled 2D/3D model feature points, and finally generate a 3D clay model of the physical clay model;

If it is not a pure white base map, calculate the coordinates and color values of the physical feature points according to the sampling data, and then use the point cloud coordinates to combine the 3D clay model. After the combination is completed, match the feature points of the 3D clay model and the physical clay model;

2.4) The memory outputs the action, sound effect and special effect data selected by the I/O interactive control system to the calculator, and the calculator processes the action, sound effect and special effect of the three-dimensional clay model.

The beneficial effect of the present invention is that it can quickly and stably recognize the three-dimensional physical clay model, submit the corresponding recognition accuracy, complete the evaluation of the work, and continuously track and position synchronously output, and combine the virtual three-dimensional model with the real world Combined with single or multi-player game interaction, it not only extends the gameplay of traditional clay toys, but also solves the problem of recognition of non-planar entities by augmented reality technology.

Description of drawings

Fig. 1 is a schematic diagram of the system principle of the present invention.

Fig. 2 is a schematic diagram of the specific processing flow of the AR identification module of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution, and provides detailed implementation and specific operation process, but the protection scope of the present invention is not limited to the present invention. Example.

As shown in Figure 1, the clay toy system based on augmented reality and digital image processing includes an image acquisition module, an AR recognition module, a display, an arithmetic unit, and an I/O interactive control system; among them,

The image collection module is used to collect the image data of the physical clay model and transmit it to the AR recognition module;

The AR recognition module is used to generate a corresponding three-dimensional clay model for the acquired image data of the physical clay model and transmit it to the computing unit;

The computing unit is used to perform special effect processing, audio sound effects and action matching on the obtained three-dimensional clay model, and transmit it to the display for display;

The I/O interactive control system is used to control the arithmetic unit, including special effects, audio sound effects and actions selected for addition.

Further, the system includes a plurality of image acquisition modules and arithmetic units, each image acquisition module corresponds to an arithmetic unit, and all image acquisition modules and arithmetic units are connected to the AR recognition module, and each arithmetic unit is connected to In an I/O interactive control system; different image acquisition modules are used to collect image data of different physical clay models and transmit them to the AR recognition module. The AR recognition module generates the corresponding 3D clay model and then transmits it to the corresponding calculation In the processor, the corresponding calculator performs special effect processing, adds audio and sound effects, and matches actions; the special effects, audio and sound effects and actions used by each calculator are controlled by the I/O interactive control system connected to it.

Furthermore, an online battle engine module and a battle scene image module are also arranged between the AR recognition module and one of the computing units, and the AR recognition module, the online battle engine module and the battle scene image module are connected to the computing unit. connected in sequence; the online battle engine module and the battle scene image module are used to add and display the battle scene on the display.

Further, each I/O interactive control system is connected to a gyroscope control module for inputting control instructions to the I/O interactive control system through acceleration in different directions.

Further, the system also includes a mode selection module, which is used to select a multiplayer interactive mode and a stand-alone mode.

The method for using the above-mentioned clay toy system based on enhanced display and digital image processing includes the following steps:

S1 uses the mode selection module to select multi-person interactive mode or stand-alone mode;

S2 When the multi-person interactive mode is selected, the I/O interactive control system loads the battle engine, and each user uses a different image acquisition module to collect the image data of the physical clay model made by himself, and selects his own through the I/O interactive control system. The required special effects, audio sound effects and action processing, the image data of the final physical clay model is processed by the AR recognition module to generate a three-dimensional clay model, and then processed by the special effects, audio sound effects and actions of the calculator and output to the display for display; The three-dimensional clay models processed by the AR recognition module and the calculator of all users' physical clay models are displayed on the monitor, so as to realize the interaction with other three-dimensional clay models;

When the stand-alone mode is selected, after the image acquisition module collects the image of the physical clay model, the AR recognition module generates the corresponding three-dimensional clay model and displays it on the display after being processed by the calculator.

The generated 3D clay model can be shared on social networking sites, can be used to develop games, can be photographed or applied to the knowledge popularization system.

The generated 3D clay model can also perform intelligent rating activities. Through the intelligent rating function, the 3D clay model can be intelligently rated according to the preset rules. If the intelligent rating is passed, the next step can be carried out, such as in the simulation management system. Simulate the sale of props or cards, manage currency, interact with other users, etc.; you can also send corresponding rewards according to the rating results through the customs clearance system, and take pictures and share through the communication and sharing system; if the rating is not passed, you need to Transfer to the image acquisition module and continue to make new physical clay models for challenges.

It should be noted that, the control command is input to the I/O interactive control system through the gyroscope control module, and the accelerations in different directions respectively represent a control command.

It should be noted that when the multiplayer interactive mode is selected, the battle scene can also be added through the online battle engine and the battle scene module, and displayed on the display together.

It should be noted that the steps of generating the three-dimensional clay model by the AR recognition module include: analysis, extraction of pattern tracking features for sampling, matching, analysis, feature value screening and real-time tracking.

It should be noted that, as shown in Figure 2, in step S2, the specific processing method of the AR recognition module in multi-person interactive mode or stand-alone mode is as follows:

2.1) Use the image acquisition module to perform multi-dimensional data sampling of the physical clay model;

2.2) The AR recognition module conducts coordinate analysis on the collected data to identify the regular or irregular shape of the physical clay model;

2.3) If the physical clay model is regular, call the sampled two-dimensional/three-dimensional model to pick the color at the specified point and perform compensation processing, and then perform color matching processing on the two-dimensional/three-dimensional model to generate a three-dimensional clay model;

If the physical clay model is irregular:

If it is a pure white background map, generate a sample map by supplementing the color difference, and then match the sample map with the sampled 2D/3D model feature points, and finally generate a 3D clay model of the physical clay model;

If it is not a pure white base map, calculate the coordinates and color values of the physical feature points according to the sampling data, and then use the point cloud coordinates to combine the 3D clay model. After the combination is completed, match the feature points of the 3D clay model and the physical clay model;

2.4) The memory outputs the action, sound effect and special effect data selected by the I/O interactive control system to the calculator, and the calculator processes the action, sound effect and special effect of the three-dimensional clay model.

For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and ideas, and all these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims (4)

1. The application method of the clay toy system based on augmented reality and digital image processing is characterized by comprising the following steps of: the mud toy system based on augmented reality and digital image processing comprises an image acquisition module, an AR identification module, a display, an arithmetic unit, an I/O interaction control system, a memory and a mode selection module; wherein,,

the image acquisition module is used for acquiring image data of the physical clay model and transmitting the image data to the AR identification module;

the AR identification module is used for generating a corresponding three-dimensional mud gel model for the acquired image data of the real mud gel model and transmitting the three-dimensional mud gel model to the arithmetic unit;

the arithmetic unit is used for carrying out special effect processing, adding audio sound effects and matching actions on the obtained three-dimensional clay model, and transmitting the special effect processing, the audio sound effects and the matching actions to the display for display;

the I/O interaction control system is used for controlling the arithmetic unit, wherein the I/O interaction control system comprises a special effect, an audio sound effect and an action which are selected to be added;

the memory is used for storing data of special effects, audio sound effects and actions, and the I/O interaction control system and the arithmetic unit are connected to the memory;

the mode selection module is used for selecting a multi-person interaction mode and a single-machine mode;

the using method comprises the following steps:

s1, selecting a multi-user interaction mode or a single-machine mode by using a mode selection module;

s2, when a multi-person interaction mode is selected, the I/O interaction control system loads an fight engine, each user uses different image acquisition modules to acquire image data of a self-made physical clay model, the I/O interaction control system selects required special effects, audio sound effects and action processing, and finally the image data of the physical clay model is processed by the AR recognition module to generate a three-dimensional clay model, and then is processed by an arithmetic unit and output to a display for display; all the real mud gel models of the users are processed by the AR identification module and the arithmetic unit to obtain three-dimensional mud gel models which are displayed on the display, so that interaction with other three-dimensional mud gel models is realized;

when a single machine mode is selected, the image acquisition module acquires the image of the real object mud gel model, the AR recognition module generates a corresponding three-dimensional mud gel model, and the three-dimensional mud gel model is processed by the arithmetic unit and then displayed on the display;

in step S2, the specific processing method of the AR identification module in the multi-user interaction mode or the single-machine mode is as follows:

2.1 Using the image acquisition module to sample the multi-dimensional data of the physical clay model;

2.2 The AR identification module performs coordinate analysis on the acquired data to identify the shape rule or irregularity of the physical clay model;

2.3 If the physical clay model is regular, calling a two-dimensional/three-dimensional model designated point obtained by sampling to take color and performing compensation treatment, and then performing color matching treatment on the two-dimensional/three-dimensional model to generate a three-dimensional clay model;

if the physical clay model is irregular:

if the real object model is the pure white background image, a sampling image is generated by supplementing chromatic aberration, then the sampling image is matched with the characteristic points of the two-dimensional/three-dimensional model obtained by sampling, and finally a three-dimensional mud gel model of the real object mud gel model is generated;

if the three-dimensional mud gel model is not the pure white background image, calculating the coordinates and the color values of the physical characteristic points according to the sampling data, combining the three-dimensional mud gel model by adopting the point cloud coordinates, and matching the characteristic points of the three-dimensional mud gel model and the physical mud gel model after the combination is completed;

2.4 The memory outputs the action, sound effect and special effect data selected by the I/O interactive control system to the arithmetic unit, and the arithmetic unit processes the action, sound effect and special effect of the three-dimensional clay model.

2. The method of claim 1, wherein each of the I/O interactive control systems is connected to a gyro control module, control commands are input to the I/O interactive control systems through the gyro control module, and accelerations in different directions each represent a control command.

3. The method of claim 1, comprising a plurality of image acquisition modules and operators, each image acquisition module corresponding to an operator, and all image acquisition modules and operators being connected to the AR identification module, each operator being connected to an I/O interactive control system; the different image acquisition modules are used for acquiring image data of different physical clay models and transmitting the image data to the AR identification module, the AR identification module generates a corresponding three-dimensional clay model and then transmits the corresponding three-dimensional clay model to a corresponding arithmetic unit, and the corresponding arithmetic unit carries out special effect processing, audio sound effect adding and matching actions; the special effects, audio effects and actions used by each operator are controlled by the I/O interactive control system to which they are connected.

4. The method according to claim 3, wherein an online combat engine module and an combat scene image module are further arranged between the AR identification module and one of the operators, and the AR identification module, the online combat engine module and the combat scene image module are sequentially connected with the operators.

Images