CN112764548A - AR auxiliary assembly system - Google Patents
AR auxiliary assembly system Download PDFInfo
- Publication number
- CN112764548A CN112764548A CN202110209353.2A CN202110209353A CN112764548A CN 112764548 A CN112764548 A CN 112764548A CN 202110209353 A CN202110209353 A CN 202110209353A CN 112764548 A CN112764548 A CN 112764548A
- Authority
- CN
- China
- Prior art keywords
- assembly
- user
- animation
- assembling
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23P—METAL-WORKING NOT OTHERWISE PROVIDED FOR; COMBINED OPERATIONS; UNIVERSAL MACHINE TOOLS
- B23P19/00—Machines for simply fitting together or separating metal parts or objects, or metal and non-metal parts, whether or not involving some deformation; Tools or devices therefor so far as not provided for in other classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2008—Assembling, disassembling
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Architecture (AREA)
- Mechanical Engineering (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an AR auxiliary assembly system, which comprises a two-dimensional code recognition module, a user operation interface module, a grabbing action recognition module, a gesture recognition module and an assembly animation demonstration module; the following hardware is included: assembling a work bench, a personal computer and AR glasses; the following software functions are included: user login, two-dimension code space anchoring, workpiece grabbing and recognition, judgment of finishing assembly gestures, prompt of assembly steps and assembly animation demonstration; the method comprises the following working procedures: the method comprises a user login verification process, a two-dimensional code scanning and three-dimensional space positioning process, a three-dimensional guide part and tool picking process, an augmented reality animation assembly guide process and a gesture recognition completion assembly step process. The assembly animation is displayed by using the virtual and real combined assembly action, so that the assembly process can be clearly displayed to an operator; by applying the grabbing action recognition and gesture recognition technologies, an operator can conveniently and quickly feed back the operation information to the system.
Description
Technical Field
The invention relates to a workpiece assembling technology, in particular to an AR auxiliary assembling system.
Background
In a scene of workpiece assembly, an assembly worker needs to be familiar with an assembly process as soon as possible, and an assembly skill is generally mastered in a traditional mode by looking up an assembly guide book or in a form of a master with a vain. Learning assembly skills through assembly guides has several drawbacks: the operation guidance on the assembly guidance is not intuitive enough, and the two-dimensional assembly guidance in the assembly drawing comes in and goes out with the real three-dimensional situation; in the process of learning and assembling, a guide book needs to be held by hands, so that double-hand operation cannot be realized, and the operation cannot be performed by two hands under a special environment; assembly learning can not realize paperless, and if the assembly flow is updated, only the instruction book can be printed again; the assembly training process is poorly scalable and cannot be aided by the advantages of computer computing and information storage to improve the training process. When the master learns the assembly skills in a bare-handed manner, the one-to-one learning expert cost and the learning cost are high, and the learning effect is difficult to guarantee under the condition of one-to-many learning.
The first prior art is as follows:
as shown in fig. 1, the intelligent assistive assembly system is an interactive three-dimensional assistive assembly system. Firstly, identifying parts/components by using two-dimensional codes to realize the management of the parts/components in the assembly process; then, a complete intelligent auxiliary assembly knowledge base is established, so that technical data used in the assembly process are more complete and clear; calling SolidWorks software through an API function to complete management and operation of the assembly files, generating corresponding technical files and storing the technical files in an assembly knowledge base for later use; and finally, scanning the two-dimensional code on the part/component through the mobile equipment, and analyzing the scanned data through remote service, so that the auxiliary three-dimensional model assembly drawing and the technical file of the mobile terminal are pushed, and an assembly worker is helped to complete an assembly task.
The first prior art has the following defects:
the technology has the disadvantages that the three-dimensional model assembly drawing is not visual enough when the computer software is used for recognizing the three-dimensional model, the size and the shape of the model on the computer are difficult to correspond to the model in a real scene, errors in the assembly process are easily caused, and risks are increased for the assembly task.
The second prior art is:
as shown in fig. 2, an auxiliary assembly device for aerospace parts relates to the field of aerospace. The auxiliary assembly device for the aerospace parts comprises a movable plate, wherein the upper portion of the movable plate comprises a driving box, an electric telescopic rod, an assembly plate, an anti-skid block, a buffering spring and a button, and the lower portion of the driving box is fixedly connected with the upper portion of the movable plate. This auxiliary assembly device of aerospace part, drive case through setting up, and electric telescopic handle, the assembly plate, the non slipping spur, the cooperation setting of buffering spring and button, thereby the messenger places the part with the front of assembly plate, buffering spring and non slipping spur through setting up can carry out the centre gripping according to different part sizes, carry out the centre gripping after the centre gripping and stretch out and draw back through the electric telescopic handle who sets up in drive incasement portion, upwards deliver through with the part, thereby the manual work is carried out and is installed it, the effect of convenient auxiliary assembly part has been reached, it is troublesome to have solved current aerospace part assembly part, consume the problem of a large amount of manpowers.
The second prior art has the following defects:
the technology has the disadvantages that the assembly assistance can be only carried out on parts and assembly parts of specific models, the requirements on scenes are high, and the reproducibility is poor. And meanwhile, the method also lacks an assembly instruction and is difficult to use alone without the assistance of skilled workers.
Disclosure of Invention
The invention aims to provide an AR auxiliary assembly system.
The purpose of the invention is realized by the following technical scheme:
the AR auxiliary assembly system comprises a two-dimensional code recognition module, a user operation interface module, a grabbing action recognition module, a gesture recognition module and an assembly animation demonstration module;
the two-dimension code recognition module realizes the functions of two-dimension code information recognition and space anchoring of the assembly table through two-dimension code recognition and three-dimension space anchoring technologies;
the user operation interface module realizes the functions of function selection, flow control and operation information display through a UI (user interface) displayed by an augmented reality technology;
the grabbing action recognition module realizes the function of recognizing the action of grabbing parts by obtaining;
the gesture recognition module realizes the function of recognizing the user-defined gesture by acquiring the positions of fingertips, finger joints and wrist joints in real time;
the assembly animation demonstration module realizes the assembly animation demonstration function of combining virtuality and reality.
According to the technical scheme provided by the invention, the AR auxiliary assembly system provided by the embodiment of the invention can clearly show the assembly process to an operator because the assembly animation is displayed by using the assembly action combining the virtuality and the reality; by applying the grabbing action recognition and gesture recognition technologies, an operator can conveniently and quickly feed back the operation information to the system.
Drawings
FIG. 1 is a general architecture of an intelligent assistive assembly system of the prior art one;
FIG. 2 is an auxiliary assembly device for aerospace parts according to a second prior art;
FIG. 3 is a block diagram of an AR auxiliary mounting system according to an embodiment of the present invention;
fig. 4 is a flowchart of an AR auxiliary assembling system according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be described in further detail below. Details which are not described in detail in the embodiments of the invention belong to the prior art which is known to the person skilled in the art.
The preferred embodiments of the AR-assisted fitting system of the present invention are:
the device comprises a two-dimensional code recognition module, a user operation interface module, a grabbing action recognition module, a gesture recognition module and an assembly animation demonstration module;
the two-dimension code recognition module realizes the functions of two-dimension code information recognition and space anchoring of the assembly table through two-dimension code recognition and three-dimension space anchoring technologies;
the user operation interface module realizes the functions of function selection, flow control and operation information display through a UI (user interface) displayed by an augmented reality technology;
the grabbing action recognition module realizes the function of recognizing the action of grabbing parts by obtaining;
the gesture recognition module realizes the function of recognizing the user-defined gesture by acquiring the positions of fingertips, finger joints and wrist joints in real time;
the assembly animation demonstration module realizes the assembly animation demonstration function of combining virtuality and reality.
The following hardware is included: assembling a work bench, a personal computer and AR glasses;
the assembling workbench is a basic platform of an AR auxiliary assembling project and comprises a workbench body, a workpiece library, a tool library, an assembling tool, a man-machine cooperation robot installation interface, a power supply system and an anti-static system;
the personal computer is a platform for software development and is configured as follows: memory: 8GB, hard disk: 2T, CPU: intel i 7-7700;
the AR glasses are equipment for operating augmented reality software, and are configured as follows: memory 4GB, hard disk: 64GB, processor: cellhalon 850, which carries a transparent holographic lens, 4 visible light cameras, 2 infrared cameras, a depth sensor, an accelerometer, a gyroscope, and a magnetometer.
The following software functions are included: user login, two-dimension code space anchoring, workpiece grabbing and recognition, judgment of finishing assembly gestures, prompt of assembly steps and assembly animation demonstration;
the user login function is matched with the augmented reality display technology of the AR glasses, when a user inputs user name and password information in an interface, the system verifies the information with user information prestored in a background, and when the user passes the verification, the user logs in the system;
the two-dimension code space anchoring function acquires three-dimensional space information of the assembly table by scanning the two-dimension code placed at the designated position, generates a virtual three-dimensional model at the designated position of the assembly table, and is matched with an object grabbing action recognition and three-dimensional assembly animation module;
when a specific part needs to be taken in the assembly process, the system can set a virtual three-dimensional collision box at the position of the part box where the part is located, the system can judge the relative position of a human hand and the collision box in real time, and when the human hand contacts the collision box, the system obtains successful grabbing information and feeds back the information to a user in a virtual image and sound mode;
the assembling finishing gesture judging function calls an interface to acquire space positions of fingertips, hand joints and wrist joints in real time, calculates relative distances among the joints to judge whether the gesture is finished or not, can convey information to a system through a user-defined gesture in a using process, and prompts the system to enter the next step through an OK gesture when the current assembling process is finished;
the assembly step prompting function adopts an augmented reality technology, a virtual user interaction interface is superposed in a real scene, through the interface, a user can obtain part and tool information required to be adopted by the process and assembly process guide information, the user can drag and zoom the interface, the range displayed by the interface can be controlled through a scroll bar, and when the current assembly step is completed, the next step can be started by clicking a next button prompting system;
the assembling animation demonstration function is characterized in that in each assembling link, through an augmented reality technology, the AR glasses can generate assembling animation in front of an assembling worker, the animation adopts space position information obtained by two-dimension code space anchoring, animation is superposed at a position to be assembled on a real assembling workpiece, and the animation prompts the shape, the assembling position and the installation mode of a part to be assembled of the assembling worker through reciprocating motion. Through the animation, an assembler can visually and rapidly know the assembly process, the assembly efficiency is improved, the learning cost is reduced, and the assembly error probability is reduced.
The method comprises the following working procedures:
the method comprises a user login verification process, a two-dimensional code scanning and three-dimensional space positioning process, a three-dimensional guide part and tool picking process, an augmented reality animation assembly guide process and a gesture recognition completion assembly step process.
The AR auxiliary assembly system solves the following problems in the conventional auxiliary assembly system:
the problem that learning of assembly drawings in a two-dimensional scene in a computer or mobile equipment is not intuitive enough and both hands are needed for searching and browsing technical files is solved. The system uses an augmented reality technology to superpose a virtual three-dimensional assembly animation and a real assembly part to guide an assembly method, superposes a virtual three-dimensional indication mark in a real space to indicate the position of a part to be assembled, uses a virtual user interaction interface to display assembly guidance information, and uses a user-defined gesture to issue a signal for completing the current process to the system, thereby solving the problems.
Based on the application scenario mentioned in the prior art, in order to make up for the disadvantages of the conventional scheme, the invention applies the augmented reality technology to assist the assembly of workers. The AR augmented reality technology is a relatively new technology content which enables real world information and virtual world information content to be integrated together, and implements analog simulation processing on the basis of computer and other scientific technologies of entity information which is relatively difficult to experience in the space range of the real world originally, and virtual information content is effectively applied in the real world in an overlapping mode and can be perceived by human senses in the process, so that the sensory experience beyond reality is realized. After the real environment and the virtual object are overlapped, the real environment and the virtual object can exist in the same picture and space at the same time. The invention applies the technology to realize that the assembly prompt information, the part box to be assembled, the three-dimensional animation and other virtual information are superposed on the real information in the assembly process, and the assembly process is displayed in a virtual-real combined manner, so that an assembler can quickly and intuitively know the assembly process and complete the assembly task.
The advantages of using the invention are:
by applying the augmented reality technology, the information of the real world can be observed, and meanwhile, the assistance of virtual information can be received, so that the working efficiency is improved; by using the software, the assembly action can be completed according to the guidance of the software on the occasion that experts cannot arrive, and the working cost is reduced; the two hands are liberated, auxiliary assembly software can be used while the assembly parts are operated, and the operation objects do not need to be switched continuously in a computer, paper materials and an assembly tool; the operation is simple and visual, the learning cost is low, and the operation is easy and rapid; the assembly animation is displayed by using the virtual and real combined assembly action, so that the assembly process can be clearly displayed to an operator; by applying the grabbing action recognition and gesture recognition technologies, an operator can conveniently and quickly feed back the operation information to the system.
The specific embodiment is as follows:
as shown in fig. 3, the AR-aided assembly system is an intelligent aid system for factory assembly, and is composed of modules of two-dimensional code recognition, user operation interface, grasping action recognition, gesture recognition, and assembly animation demonstration. The two-dimension code recognition module realizes the functions of two-dimension code information recognition, space anchoring of an assembly table and the like through two-dimension code recognition and three-dimension space anchoring technologies; the user operation interface realizes the functions of function selection, flow control and operation information display through a UI (user interface) displayed by the augmented reality technology; the grabbing action recognition module realizes the function of recognizing the action of grabbing parts by obtaining; the gesture recognition module realizes the function of recognizing the user-defined gesture by acquiring the positions of fingertips, finger joints and wrist joints in real time; and the assembly animation demonstration module realizes the assembly animation demonstration function of combining virtuality and reality.
1 hardware solution
The hardware of the invention is mainly composed of a plurality of parts, namely an assembly workbench, a personal computer and AR glasses. The assembling workbench serves as a basic platform of an AR auxiliary assembling project and provides functions of an assembling operation platform, material storage and the like. The device comprises a workbench body, a workbench workpiece library (for placing standard parts and parts), a tool library (for placing various assembling, detecting and welding tools), an assembling tool, a man-machine cooperation robot mounting interface, a power supply system, an anti-static system and the like. The personal computer is used as a platform for software development and is configured as follows: memory: 8GB, hard disk: 2T, CPU: intel i 7-7700. The AR glasses are used as a device for running augmented reality software and are configured as follows: memory 4GB, hard disk: 64GB, processor: cellhalon 850, which carries a transparent holographic lens, 4 visible light cameras, 2 infrared cameras, a depth sensor, an accelerometer, a gyroscope, and a magnetometer.
2 software scheme
The software functions of the AR auxiliary assembly system comprise user login, two-dimension code space anchoring, workpiece grabbing and recognition, judgment of assembly finishing gestures, assembly step prompting, assembly animation demonstration and the like.
(1) User login
The user login function is matched with the augmented reality display technology of the AR glasses, when a user inputs user name and password information in an interface, the system verifies the information with user information prestored in a background, and when the user passes the verification, the user logs in the system.
(2) Two-dimensional code space anchoring
The two-dimensional code space anchoring acquires the three-dimensional space information of the assembly table by scanning the two-dimensional code placed at the designated position. And generating a virtual three-dimensional model at the designated position of the assembly table, and matching the virtual three-dimensional model with the object grabbing action recognition and three-dimensional assembly animation module.
(3) Workpiece grabbing recognition
When a workpiece grabbing and recognizing function needs to take a specific part in the assembling process, the system can set a virtual three-dimensional collision box at the position of the part box where the part is located. The system can judge the relative position of the human hand and the collision box in real time, and when the human hand contacts the collision box, the system obtains the information of successful grabbing and feeds back the information to the user in a virtual image and sound mode.
(4) End of assembly gesture determination
And finishing assembling the gesture, judging the spatial positions of the fingertips, the hand joints and the wrist joints by using the function calling interface, and calculating the relative distance between the joints to judge whether the gesture is finished. In the process of using the invention, the user can convey information to the system through the custom gesture. And when the current assembly process is finished, prompting the system to enter the next step by using an OK gesture.
(5) Assembly step tips
And the prompt function in the assembling step applies an augmented reality technology and superposes a virtual user interaction interface in a real scene. Through the interface, a user can obtain the information of parts and tools required by the process and the guidance information of the assembly process. The user can drag and zoom the interface, the range displayed by the interface can be controlled through the scroll bar, and when the current assembly step is completed, the system can be prompted to enter the next step by clicking the next button.
(6) Assembly animation demonstration
The assembly animation demonstration function is that in each assembly link, through the augmented reality technology, the AR glasses can generate assembly animation in front of an assembler. The animation adopts space position information obtained by two-dimension code space anchoring, and the animation is superposed at the position to be assembled on the real assembly workpiece. The animation prompts the assembly worker through reciprocating motion about the shape, the assembly position and the installation mode of the part to be assembled. Through the animation, an assembler can visually and rapidly know the assembly process, the assembly efficiency is improved, the learning cost is reduced, and the assembly error probability is reduced.
3 workflow
(1) Correlation system
The system comprises a two-dimensional code recognition system, a user operation interface system, a grabbing action recognition system, a gesture recognition system and an assembly animation demonstration system.
(2) Workflow, as shown in fig. 4:
the work flow can be divided into 5 flows: the method comprises a user login verification process, a two-dimensional code scanning and three-dimensional space positioning process, a three-dimensional guide part and tool picking process, an augmented reality animation assembly guide process and a gesture recognition completion assembly step process.
The invention is designed aiming at the assembly training link in an intelligent factory, and an augmented reality technology is used for assisting an assembly training system, thereby bringing the following beneficial effects:
(1) the invention applies the augmented reality technology, and can check the assembly document and the virtual assembly animation superposed in the real scene while using two hands to operate the real assembly part simultaneously. By using the technology, both hands can be liberated, and the assembly efficiency and the success rate are improved.
(2) The operation method is simple, and the user interface is visual. The virtual three-dimensional operation interface developed based on the AR glasses is simple and intuitive, is not different from the operation mode of computer or mobile phone software commonly used by people, and an assembler can easily get hands in a short time. And the usability of interaction modes such as touch, drag and zoom of the virtual three-dimensional model based on human intuition design is also strong.
(3) The invention designs a grabbing action judging and custom gesture identifying module. When a specific part needs to be taken in the assembling process, the system can set a virtual three-dimensional collision box at the position of the part box where the part is located. The system can judge the relative positions of the human hand and the collision box in real time, and when the human hand contacts the collision box, the system obtains information of successful grabbing and feeds back the information to the user in a virtual image and sound mode; in the invention, the user can use the customized gesture to convey the system information. The gesture is judged whether to be completed or not by acquiring the spatial positions of fingertips, hand joints and wrist joints in real time and calculating the relative distance between the joints. The grabbing judgment and the user-defined gesture recognition module quoted in the system are accurate in recognition and fast in response speed. The two modules can improve the interactive experience between the user and the system, conveniently and quickly transmit information to the system, and assist in completing the assembly task.
The technical key points of the invention are as follows:
(1) the method adopts an augmented reality technology, uses AR glasses to assist workers in completing field assembly, and uses a virtual three-dimensional model and a man-machine interaction page to be superposed in a real scene in the assembly process to help the workers complete the assembly process;
(2) the assembly animation demonstration function of the invention superposes the assembly animation of the virtual assembly part on the real workpiece to be assembled for demonstrating the assembly process;
(3) the invention can recognize the user-defined gesture and transmit the information of the completion of the assembling step to the system.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (4)
1. An AR auxiliary assembly system is characterized by comprising a two-dimensional code recognition module, a user operation interface module, a grabbing action recognition module, a gesture recognition module and an assembly animation demonstration module;
the two-dimension code recognition module realizes the functions of two-dimension code information recognition and space anchoring of the assembly table through two-dimension code recognition and three-dimension space anchoring technologies;
the user operation interface module realizes the functions of function selection, flow control and operation information display through a UI (user interface) displayed by an augmented reality technology;
the grabbing action recognition module realizes the function of recognizing the action of grabbing parts by obtaining;
the gesture recognition module realizes the function of recognizing the user-defined gesture by acquiring the positions of fingertips, finger joints and wrist joints in real time;
the assembly animation demonstration module realizes the assembly animation demonstration function of combining virtuality and reality.
2. The AR assistive assembly system of claim 1, comprising the following hardware: assembling a work bench, a personal computer and AR glasses;
the assembling workbench is a basic platform of an AR auxiliary assembling project and comprises a workbench body, a workpiece library, a tool library, an assembling tool, a man-machine cooperation robot installation interface, a power supply system and an anti-static system;
the personal computer is a platform for software development and is configured as follows: memory: 8GB, hard disk: 2T, CPU: intel i 7-7700;
the AR glasses are equipment for operating augmented reality software, and are configured as follows: memory 4GB, hard disk: 64GB, processor: cellhalon 850, which carries a transparent holographic lens, 4 visible light cameras, 2 infrared cameras, a depth sensor, an accelerometer, a gyroscope, and a magnetometer.
3. The AR assistive assembly system of claim 2, comprising the following software functions: user login, two-dimension code space anchoring, workpiece grabbing and recognition, judgment of finishing assembly gestures, prompt of assembly steps and assembly animation demonstration;
the user login function is matched with the augmented reality display technology of the AR glasses, when a user inputs user name and password information in an interface, the system verifies the information with user information prestored in a background, and when the user passes the verification, the user logs in the system;
the two-dimension code space anchoring function acquires three-dimensional space information of the assembly table by scanning the two-dimension code placed at the designated position, generates a virtual three-dimensional model at the designated position of the assembly table, and is matched with an object grabbing action recognition and three-dimensional assembly animation module;
when a specific part needs to be taken in the assembly process, the system can set a virtual three-dimensional collision box at the position of the part box where the part is located, the system can judge the relative position of a human hand and the collision box in real time, and when the human hand contacts the collision box, the system obtains successful grabbing information and feeds back the information to a user in a virtual image and sound mode;
the assembling finishing gesture judging function calls an interface to acquire space positions of fingertips, hand joints and wrist joints in real time, calculates relative distances among the joints to judge whether the gesture is finished or not, can convey information to a system through a user-defined gesture in a using process, and prompts the system to enter the next step through an OK gesture when the current assembling process is finished;
the assembly step prompting function adopts an augmented reality technology, a virtual user interaction interface is superposed in a real scene, through the interface, a user can obtain part and tool information required to be adopted by the process and assembly process guide information, the user can drag and zoom the interface, the range displayed by the interface can be controlled through a scroll bar, and when the current assembly step is completed, the next step can be started by clicking a next button prompting system;
the assembling animation demonstration function is characterized in that in each assembling link, through an augmented reality technology, the AR glasses can generate assembling animation in front of an assembling worker, the animation adopts space position information obtained by two-dimension code space anchoring, animation is superposed at a position to be assembled on a real assembling workpiece, and the animation prompts the shape, the assembling position and the installation mode of a part to be assembled of the assembling worker through reciprocating motion. Through the animation, an assembler can visually and rapidly know the assembly process, the assembly efficiency is improved, the learning cost is reduced, and the assembly error probability is reduced.
4. The AR assistive assembly system of claim 3, comprising the following workflow:
the method comprises a user login verification process, a two-dimensional code scanning and three-dimensional space positioning process, a three-dimensional guide part and tool picking process, an augmented reality animation assembly guide process and a gesture recognition completion assembly step process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110209353.2A CN112764548A (en) | 2021-02-24 | 2021-02-24 | AR auxiliary assembly system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110209353.2A CN112764548A (en) | 2021-02-24 | 2021-02-24 | AR auxiliary assembly system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112764548A true CN112764548A (en) | 2021-05-07 |
Family
ID=75704148
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110209353.2A Pending CN112764548A (en) | 2021-02-24 | 2021-02-24 | AR auxiliary assembly system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112764548A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113298003A (en) * | 2021-06-03 | 2021-08-24 | 北京安达维尔科技股份有限公司 | AR-based aviation cable assembly system and method |
| CN113673894A (en) * | 2021-08-27 | 2021-11-19 | 东华大学 | A multi-person collaborative AR assembly method and system based on digital twin |
| CN113947053A (en) * | 2021-10-21 | 2022-01-18 | 黄航 | Circuit board layout method and system based on AR |
| CN114131293A (en) * | 2021-11-25 | 2022-03-04 | 南京航空航天大学 | A kind of automatic processing method of aircraft structural reinforcement |
| CN115847428A (en) * | 2023-02-20 | 2023-03-28 | 中国信息通信研究院 | AR technology-based mechanical assembly auxiliary guide system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109448126A (en) * | 2018-09-06 | 2019-03-08 | 国营芜湖机械厂 | A kind of aircraft equipment repairing auxiliary system and its application method based on mixed reality |
| CN109491497A (en) * | 2018-10-19 | 2019-03-19 | 华中科技大学 | A kind of human assistance assembly application system based on augmented reality |
| CN109933194A (en) * | 2019-03-05 | 2019-06-25 | 郑州万特电气股份有限公司 | To the exchange method of virtual target object in a kind of mixed reality environment |
| CN110162179A (en) * | 2019-05-24 | 2019-08-23 | 北京理工大学 | A kind of Intellisense virtual assembly system |
| CN110928418A (en) * | 2019-12-11 | 2020-03-27 | 北京航空航天大学 | A kind of MR-based aviation cable auxiliary assembly method and system |
-
2021
- 2021-02-24 CN CN202110209353.2A patent/CN112764548A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109448126A (en) * | 2018-09-06 | 2019-03-08 | 国营芜湖机械厂 | A kind of aircraft equipment repairing auxiliary system and its application method based on mixed reality |
| CN109491497A (en) * | 2018-10-19 | 2019-03-19 | 华中科技大学 | A kind of human assistance assembly application system based on augmented reality |
| CN109933194A (en) * | 2019-03-05 | 2019-06-25 | 郑州万特电气股份有限公司 | To the exchange method of virtual target object in a kind of mixed reality environment |
| CN110162179A (en) * | 2019-05-24 | 2019-08-23 | 北京理工大学 | A kind of Intellisense virtual assembly system |
| CN110928418A (en) * | 2019-12-11 | 2020-03-27 | 北京航空航天大学 | A kind of MR-based aviation cable auxiliary assembly method and system |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113298003A (en) * | 2021-06-03 | 2021-08-24 | 北京安达维尔科技股份有限公司 | AR-based aviation cable assembly system and method |
| CN113298003B (en) * | 2021-06-03 | 2024-05-03 | 北京安达维尔科技股份有限公司 | AR-based aviation cable assembly system and method |
| CN113673894A (en) * | 2021-08-27 | 2021-11-19 | 东华大学 | A multi-person collaborative AR assembly method and system based on digital twin |
| CN113673894B (en) * | 2021-08-27 | 2024-02-02 | 东华大学 | Multi-person cooperation AR assembly method and system based on digital twinning |
| CN113947053A (en) * | 2021-10-21 | 2022-01-18 | 黄航 | Circuit board layout method and system based on AR |
| CN114131293A (en) * | 2021-11-25 | 2022-03-04 | 南京航空航天大学 | A kind of automatic processing method of aircraft structural reinforcement |
| CN114131293B (en) * | 2021-11-25 | 2022-11-01 | 南京航空航天大学 | A kind of automatic processing method of aircraft structural reinforcement |
| US11787051B2 (en) | 2021-11-25 | 2023-10-17 | Nanjing University Of Aeronautics And Astronautics | Method for automatically processing structure-reinforcing member of aircraft |
| CN115847428A (en) * | 2023-02-20 | 2023-03-28 | 中国信息通信研究院 | AR technology-based mechanical assembly auxiliary guide system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112764548A (en) | AR auxiliary assembly system | |
| Cao et al. | An exploratory study of augmented reality presence for tutoring machine tasks | |
| JP6233314B2 (en) | Information processing apparatus, information processing method, and computer-readable recording medium | |
| JP6116064B2 (en) | Gesture reference control system for vehicle interface | |
| CN106843150A (en) | A kind of industry spot simulation method and device | |
| CN110928418A (en) | A kind of MR-based aviation cable auxiliary assembly method and system | |
| CN109448126B (en) | Mixed reality-based aviation equipment repair auxiliary system and use method thereof | |
| CN102508546A (en) | Three-dimensional (3D) virtual projection and virtual touch user interface and achieving method | |
| CN103019377A (en) | Head-mounted visual display equipment-based input method and device | |
| Lee et al. | Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment | |
| Profanter et al. | Analysis and semantic modeling of modality preferences in industrial human-robot interaction | |
| Shi et al. | Molder: an accessible design tool for tactile maps | |
| Shao et al. | A natural interaction method of multi-sensory channels for virtual assembly system of power transformer control cabinet | |
| CN111580677B (en) | Human-computer interaction method and human-computer interaction system | |
| Qian et al. | Arnnotate: An augmented reality interface for collecting custom dataset of 3d hand-object interaction pose estimation | |
| CN113270006A (en) | HoloLens-based printing machine operation training system and method | |
| CN105786361A (en) | 3D vehicle-mounted terminal man-machine interaction system | |
| CN112732075B (en) | A teaching method and system for teaching experiment-oriented virtual-reality fusion machine teacher | |
| Yu et al. | A multi-sensor gesture interaction system for human-robot cooperation | |
| CN104834410A (en) | Input apparatus and input method | |
| Yang et al. | Interactive augmented reality authoring system using mobile device as input method | |
| Wang et al. | Multi-channel augmented reality interactive framework design for ship outfitting guidance | |
| US20210260753A1 (en) | Training processing device, intermediation device, training system, and training processing method | |
| CN116661592A (en) | A remote experiment system and method for virtual-real interaction | |
| Kirakosian et al. | Immersive simulation and training of person-to-3d character dance in real-time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210507 |
|
| RJ01 | Rejection of invention patent application after publication |