CN114580846B - Real-time processing path correction method combining vision and digital twinning - Google Patents

Abstract

The invention discloses a real-time processing path correction method combining vision and digital twinning, which comprises the following steps: a polishing unit is subjected to digital modeling; recording standard workpiece pose information: scanning and recording a standard workpiece, and marking the pose information of the standard workpiece; correcting the path of the workpiece: scanning and recording a processing workpiece, marking the pose information of the processing workpiece, comparing the pose information of the processing workpiece with the pose information of a standard workpiece to obtain a position deviation value of the same processing point of the processing workpiece and the standard workpiece, and correcting the path of the processing workpiece according to the deviation value; virtual and real synchronization of equipment; the digital twin body real-time analysis processing condition: feeding back and predicting whether to continue polishing at the current contact stress; verifying the path correction effect; the application aims to provide a real-time processing path correcting method combining vision and digital twinning, and solves the problem of processing defects caused by differences of poses, surface roughness and the like of a processing workpiece and a standard workpiece under the condition of a given robot processing path.

Description

A real-time correction method of processing path combining vision and digital twin

technical field

The invention relates to the technical field of mechanical processing, in particular to a method for real-time correction of a processing path combining vision and digital twins.

Background technique

As one of the most common and extensive basic production processes in the manufacturing industry, the polishing and polishing industry has a huge market demand. Robots replace manual polishing and polishing, which not only improves efficiency, precision and product quality uniformity, liberates productivity, but also reduces job risk. However, there are certain limitations in robot grinding and polishing. At present, most robot grinding adopts the "position-speed" control principle, that is, for a certain polished product, the grinding path and movement speed of the robot are manually planned and compared in advance. For the robot loading processing spindle, under the working conditions of fixed workpiece, two kinds of defects may occur:

Defect 1: When the processing path is determined, the position of the workpiece to be polished deviates from the position of the workpiece during robot path planning, resulting in grinding errors;

Defect 2: For workpieces of the same model and batch, the surface roughness of each workpiece to be polished cannot be exactly the same. For some local surfaces with some protrusions or excessive burrs, according to the established movement path, it may be due to processing progress. Excessive dosage may damage the surface of the workpiece.

Contents of the invention

The purpose of the present invention is to propose a real-time correction method for processing paths combining vision and digital twins, so as to solve the problem of processing defects caused by differences in pose and surface roughness between the processed workpiece and the standard workpiece in the case of a given robot processing path The problem.

To achieve this goal, the present invention adopts the following technical solutions: a method for real-time correction of processing paths combined with vision and digital twins, including the following steps:

Digital modeling of the grinding unit: establish a digital model of the grinding unit according to the equipment of the grinding unit;

Record the pose information of the standard workpiece: scan and record the standard workpiece, analyze and process the recorded image information, and mark the pose information of the standard workpiece in the digital model of the grinding unit;

Correct the path of the workpiece: scan and record the processed workpiece, analyze and process the recorded image information, mark the pose information of the processed workpiece in the digital model of the grinding unit, and compare the pose information of the processed workpiece with the pose information of the standard workpiece Compare to get the position deviation value of the same processing point, and correct the path of the workpiece according to the deviation value;

Virtual and real synchronization of equipment: use digital twin technology to build a physical simulation platform for virtual and real synchronization of equipment;

The digital twin analyzes the processing status in real time: assign physical attributes to the digital model of the grinding unit, the digital model of the grinding tool, assign physical attributes to the digital model of the workpiece, and the contact stress between the digital model of the grinding tool and the digital model of the workpiece Make simulation and analysis, feedback and predict whether to continue grinding with the current contact stress;

Verify the path correction effect: compare the action of the grinding unit equipment with the digital model action of the digital twin to judge whether the path is corrected successfully in real time.

Preferably, in the step of digital modeling of the grinding unit, specifically, a digital model with a ratio of 1:1 to the grinding unit equipment is established based on the grinding unit equipment, and the grinding unit equipment is generated by a visual three-dimensional imaging method.

Preferably, in the recording of the position and posture information of the standard workpiece, specifically for the robot processing path of the grinding unit equipment, in the path planning stage, the standard workpiece is scanned and recorded using the visual three-dimensional imaging method, and the generated three-dimensional image information is transmitted to the digital The twin simulation platform performs analysis and processing, and marks the pose information of the standard workpiece in the digital model of the grinding unit.

Preferably, the pose information includes six-axis angles of the standard workpiece, including Location (x, y, z) and Rotation (x, y, z).

Preferably, in the step of correcting the path of the workpiece, specifically for the robot processing path of the grinding unit equipment, in the path planning stage, the workpiece is scanned and recorded by using the visual three-dimensional imaging method, and the generated three-dimensional image information is transmitted to the digital The twin simulation platform performs analysis and processing, marks the pose information of the processed workpiece in the digital model of the grinding unit, compares the pose information of the processed workpiece with the pose information of the standard workpiece, and obtains the pose information of the processed workpiece and the standard workpiece The position and orientation information of the same processing point is the same as the position deviation value, and the path of the workpiece is corrected according to the deviation value, and the corrected path is obtained and fed back to the grinding unit equipment.

Preferably, in the virtual-real synchronization step of the device, the digital model, the control script and the communication interface are uniformly packaged, so that the standard interface communication between the model and the device can be used to build a virtual control network, and digital twin technology can be used to build a virtual-real synchronization device The physical simulation platform uses the communication interface to establish a synchronous control and sensing channel between the grinding unit equipment and the virtual digital model, so that the grinding unit equipment and the corresponding digital model can communicate and synchronize actions in real time.

Preferably, in the step of analyzing the processing conditions in real time with the digital twin, assigning physical attributes to the digital model of the grinding tool includes assigning quality, material or roughness attributes, and assigning physical attributes to the digital model of the workpiece includes assigning quality, material or roughness attributes. Roughness attribute, simulate and analyze the contact stress between the digital model of the grinding tool and the digital model of the processed workpiece, and judge whether continuing to grind with the current path will cause damage to the surface of the processed workpiece due to excessive feed. If not, continue processing. If the surface of the workpiece is damaged, the current grinding feed rate will be divided into multiple segments of grinding feed rate, and the method of multiple grinding instead of one-time grinding will be used, and the locally modified processing path data will be returned to the It is transmitted to the controller of the grinding unit equipment to realize "controlling the reality with the virtual" and avoid damage to the workpiece.

Preferably, in the step of verifying the path correction effect, at the same time interval, use the visual three-dimensional imaging method to scan the polishing device and the workpiece of the polishing unit equipment, obtain image data and transmit it to the simulation platform in real time, and convert the polishing unit equipment The image data and the image data of the digital twin are compared on the simulation platform. If the image data is completely consistent or highly similar after the comparison, it is considered that the real-time correction of the robot processing path is successful.

Beneficial effects of a technical solution of the present invention: This application uses visual three-dimensional imaging technology to obtain the actual situation of the standard workpiece pose and the processed workpiece pose and surface, and completes the data transmission without delay through the digital twin technology, and the actual grinding unit The working process of the equipment is completely synchronized. During the processing process, through the real-time data analysis and calculation of the digital model established by the digital twin technology, the processing path of the established grinding unit equipment is continuously corrected, the corrected processing path is generated, and the new processing path is synchronized. The processing path data is transmitted to the polishing unit equipment to realize real-time correction of the polishing path of the polishing unit equipment.

The invention solves the problem of processing defects caused by differences in pose and surface roughness between the workpiece to be processed and the standard workpiece in the path planning of the grinding unit equipment in the case of a predetermined processing path of the grinding unit equipment.

Description of drawings

Fig. 1 is a schematic diagram of the steps of an embodiment of the present invention;

Fig. 2 is a schematic diagram of path correction in an embodiment of the present invention;

Fig. 3 is a processing status analysis diagram one of an embodiment of the present invention;

Fig. 4 is a processing status analysis diagram 2 of an embodiment of the present invention;

Fig. 5 is a schematic diagram of path correction steps in an embodiment of the present invention.

Detailed ways

The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and through specific implementation methods.

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

In addition, the features defined as "first" and "second" may explicitly or implicitly include one or more of these features, which are used to describe the features differently, without order or importance.

In the description of the present invention, unless otherwise specified, "plurality" means two or more.

In the description of the present invention, it should be noted that unless otherwise specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection. Connected, or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention in specific situations.

Referring to Figures 1 to 5, a method for real-time correction of processing paths combined with vision and digital twins includes the following steps:

Digital modeling of the grinding unit: establish a digital model of the grinding unit according to the equipment of the grinding unit;

Record the pose information of the standard workpiece: scan and record the standard workpiece, analyze and process the recorded image information, and mark the pose information of the standard workpiece in the digital model of the grinding unit;

Correct the path of the workpiece: scan and record the processed workpiece, analyze and process the recorded image information, mark the pose information of the processed workpiece in the digital model of the grinding unit, and compare the pose information of the processed workpiece with the pose information of the standard workpiece Compare to get the position deviation value of the same processing point, and correct the path of the workpiece according to the deviation value;

Virtual and real synchronization of equipment: use digital twin technology to build a physical simulation platform for virtual and real synchronization of equipment;

The digital twin analyzes the processing status in real time: assign physical attributes to the digital model of the grinding unit, the digital model of the grinding tool, assign physical attributes to the digital model of the workpiece, and the contact stress between the digital model of the grinding tool and the digital model of the workpiece Make simulation and analysis, feedback and predict whether to continue grinding with the current contact stress;

Verify the path correction effect: compare the action of the grinding unit equipment with the digital model action of the digital twin to judge whether the path is corrected successfully in real time.

This application uses visual three-dimensional imaging technology to obtain the standard workpiece pose and the actual situation of the processed workpiece pose and surface, and completes data transmission through digital twin technology without delay, and is completely synchronized with the working process of the actual grinding unit equipment. In the process, through the real-time data analysis and calculation of the digital model established by the digital twin technology, the processing path of the established grinding unit equipment is continuously corrected, the corrected processing path is generated, and the new processing path data is transmitted to the grinding unit equipment synchronously. Real-time correction of the polishing path of the polishing unit equipment.

The invention solves the problem of processing defects caused by differences in pose and surface roughness between the workpiece to be processed and the standard workpiece in the path planning of the grinding unit equipment in the case of a predetermined processing path of the grinding unit equipment.

Explanation of terms in this application: Digital twin technology: is to make full use of data such as physical models, sensor updates, and operating history to integrate multi-disciplinary, multi-physical quantities, multi-scale, and multi-probability simulation processes, and complete the mapping in the virtual space to reflect The corresponding whole life cycle process of physical equipment is also called "digital mirror image", "digital twin" or "digital mapping".

Virtual and real synchronization: use PLC as a bridge to establish a communication channel between 3D simulation, equipment model, physical PLC, and configuration software to realize the interconnection of data and information, and realize real-time equipment real-time The real-time synchronization of data, configuration monitoring data and 3D virtual simulation data realizes the interaction and synchronization between virtual workshop, real workshop, on-site monitoring data and MES system execution data.

In the present application, in the step of digital modeling of the grinding unit, specifically, a digital model with a ratio of 1:1 to the grinding unit equipment is established based on the grinding unit equipment, and the grinding unit equipment is generated by a visual three-dimensional imaging method.

Establish the digital model of the grinding unit equipment, including the establishment of the model of the mechanical arm and the model of the moving parts of the robot. The model of the robot adopts a general three-dimensional CAD model with a clear and specific product structure; the model of the moving parts of the robot can be independently expressed and identified. Classify and model the moving parts and stationary parts, and complete the 3D modeling of the corresponding grinding unit equipment. Establish an accurate digital model of the actual grinding unit equipment to lay the foundation for the digital twin technology. According to the actual layout of the grinding unit equipment, all digital models are imported into the DTS digital twin simulation platform. The digital model of the grinding unit is consistent with the real thing on site, including the consistency of the digital model of the grinding unit and the consistency of the spatial relative position between the equipment.

The visual three-dimensional imaging method mentioned in the present invention is based on the fixed imaging eye-to-hand mode, and belongs to the active triangulation method in scanning 3D imaging.

Preferably, in the recording of the position and posture information of the standard workpiece, specifically for the robot processing path of the grinding unit equipment, in the path planning stage, the standard workpiece is scanned and recorded using the visual three-dimensional imaging method, and the generated three-dimensional image information is transmitted to the digital The twin simulation platform performs analysis and processing, and marks the pose information of the standard workpiece in the digital model of the grinding unit.

The visual 3D imaging method scans and records the standard workpiece, scans the standard workpiece through the active triangulation method in scanning 3D imaging, and generates a point cloud file on the surface of the standard workpiece. The point cloud is the 3D point set drawn on the x, y and z coordinates , the denser the point cloud, the higher the accuracy of the model, and the obtained point cloud file is transferred to the digital twin simulation platform for processing.

Since the point cloud file cannot be used directly, it is necessary to interpolate the points in the point cloud file to form a closed surface, so that the generated model surface is the same as the standard workpiece shape, and then fill the closed surface, and finally convert it into a general 3D file format, such as obj format, etc.

Preferably, the pose information includes six-axis angles of the standard workpiece, including Location (x, y, z) and Rotation (x, y, z).

For the recording of the pose information of a standard workpiece, the coordinates of the points in the point cloud file are actually recorded. The pose information comparison mentioned in this application is to compare the coordinates of the points in the two point cloud files. .

Specifically, in the step of modifying the path of the workpiece, specifically, for the robot processing path of the grinding unit equipment, in the path planning stage, the workpiece is scanned and recorded using the visual three-dimensional imaging method, and the generated three-dimensional image information is transmitted to the digital The twin simulation platform performs analysis and processing, marks the pose information of the processed workpiece in the digital model of the grinding unit, compares the pose information of the processed workpiece with the pose information of the standard workpiece, and obtains the pose information of the processed workpiece and the standard workpiece The position and orientation information of the same processing point is the same as the position deviation value, and the path of the workpiece is corrected according to the deviation value, and the corrected path is obtained and fed back to the grinding unit equipment.

Preferably, in the virtual-real synchronization step of the device, the digital model, the control script and the communication interface are uniformly packaged, so that the standard interface communication between the model and the device can be used to build a virtual control network, and digital twin technology can be used to build a virtual-real synchronization device The physical simulation platform uses the communication interface to establish a synchronous control and sensing channel between the grinding unit equipment and the virtual digital model, so that the grinding unit equipment and the corresponding digital model can communicate and synchronize actions in real time.

Specifically, in the step of analyzing the processing conditions in real time with the digital twin, assigning physical attributes to the digital model of the grinding tool includes assigning quality, material or roughness attributes, and assigning physical attributes to the digital model of the processed workpiece includes assigning quality, material or roughness attributes. Roughness attribute, simulate and analyze the contact stress between the digital model of the grinding tool and the digital model of the processed workpiece, and judge whether continuing to grind with the current path will cause damage to the surface of the processed workpiece due to excessive feed. If not, continue processing. If the surface of the workpiece is damaged, the current grinding feed rate will be divided into multiple segments of grinding feed rate, and the method of multiple grinding instead of one-time grinding will be used, and the locally modified processing path data will be returned to the It is transmitted to the controller of the grinding unit equipment to realize "controlling the reality with the virtual" and avoid damage to the workpiece.

In correcting the path of the workpiece, only the problem of grinding error caused by the pose deviation between the processed workpiece and the standard workpiece is solved. By establishing virtual-real synchronization between the digital twin and the grinding unit equipment, real-time analysis of the processing status steps, in the During the machining process, the digital twin model and the contact stress between the grinding tool and the workpiece can be used to judge whether there are protrusions or excessive burrs on the surface of the workpiece. For the processing surface with some protrusions or excessive burrs, if the fixed motion path is followed, the surface of the workpiece may be damaged due to excessive processing feed. The synchronization between the digital twin model and the grinding unit equipment relies on data transmission , the lower the data transmission delay, the better the virtual-real synchronization effect. In the most ideal situation, the data transmission delay time is 0, which can realize the virtual and real synchronization in the true sense, and the complete consistency of virtual and real actions, so that the digital twin model can simulate the actual processing situation in multiple directions.

The DTS digital twin simulation platform has a built-in finite element simulation module Simulation, which simulates the contact stress between the grinding tool and the digital model of the workpiece. The digital model of the grinding tool and the digital model of the workpiece have material, quality, roughness, etc. Physical properties, the motion path and speed of the digital twin and the grinding unit equipment are exactly the same. Therefore, the stress analysis of the digital twin can be regarded as the real-time stress state of the grinding unit equipment, and the digital model of the grinding tool is used as the force object to grind the workpiece The digital model of the workpiece is used as a force-bearing object, and the simulation module integrated with the digital twin simulation platform is used to grid the digital model of the workpiece, and then analyze the force of each grid unit when the grinding tool is in contact with the workpiece, and obtain the The stress state of each position, when the protrusion or the root of the burr on the workpiece reaches the critical stress of fracture, the grinding feed should be changed, the grinding path should be modified in real time, and the grinding part should be changed from one-time grinding to repeated grinding In this way, the locally modified processing path is sent back to the robot controller.

For example: Suppose there are two bumps on a workpiece that need to be ground, called bump 1 and bump 2, the first case: when the fixture grinds bump 1, it can be found that the stress radiates from the contact point (the farther away from the contact point , the smaller the stress). At this time, it is assumed that the stress at the root of the protrusion is 1Mpa, and the critical fracture stress of this material under the current structure is 0.8Mpa, then continued processing may cause fracture, and it is necessary to change the one-time grinding to multiple grinding (reduce the process) dosage);

The second case: When grinding the second bump, the same analysis as above will be carried out. At this time, the stress at the root of the second bump is 1Mpa, and the critical fracture stress is 1.2Mpa, so continuing processing will not damage the workpiece.

Preferably, in the step of verifying the path correction effect, at the same time interval, use the visual three-dimensional imaging method to scan the polishing device and the workpiece of the polishing unit equipment, obtain image data and transmit it to the simulation platform in real time, and convert the polishing unit equipment The image data and the image data of the digital twin are compared on the simulation platform. If the image data is completely consistent or highly similar after the comparison, it is considered that the real-time correction of the robot processing path is successful.

In the step of recording the pose information of the standard workpiece, in order to eliminate the pose deviation between the processed workpiece and the standard workpiece, the established processing path of the robot was corrected for the first time. In the step of analyzing the processing status of the digital twin in real time, the process of analyzing the processing status of the workpiece There may also be multiple local adjustments to the processing path. That is, during the processing process, the robot processing path in the digital twin is corrected, and the action presented in the digital twin is the final "solution". Then, as long as the equipment action of the grinding unit equipment is combined with the digital model action of the digital twin For comparison, if the actions of "virtual" and "real" are exactly the same, it proves that "controlling reality with virtual" is successful, and the path of the robot is successfully corrected. After the adjustment of the processing path plan is completed in the digital twin, the new path information is sent back to the robot controller in real time to achieve the purpose of real-time correction of the robot's processing path. The effect of real-time correction will be presented through the visual 3D imaging technology.

The above describes the technical principles of the present invention in conjunction with specific embodiments. These descriptions are only for explaining the principles of the present invention, and cannot be construed as limiting the protection scope of the present invention in any way. Based on the explanations herein, those skilled in the art can think of other specific implementation modes of the present invention without creative efforts, and these modes will all fall within the protection scope of the present invention.

Claims (6)

1. A processing path real-time correction method combining vision and digital twins, characterized in that, comprising the following steps: Digital modeling of grinding unit: according to the grinding unit equipment, establish a digital model with a ratio of 1:1 to the grinding unit equipment, and the grinding unit equipment is generated by visual three-dimensional imaging method; Record the pose information of the standard workpiece: scan and record the standard workpiece, analyze and process the recorded image information, and mark the pose information of the standard workpiece in the digital model of the grinding unit; Correct the path of the workpiece: scan and record the processed workpiece, analyze and process the recorded image information, mark the pose information of the processed workpiece in the digital model of the grinding unit, and compare the pose information of the processed workpiece with the pose information of the standard workpiece Compare to get the position deviation value of the same processing point, and correct the path of the workpiece according to the deviation value; Virtual and real synchronization of equipment: use digital twin technology to build a physical simulation platform for virtual and real synchronization of equipment; The digital twin analyzes the processing status in real time: assign physical attributes to the digital model of the grinding unit, the digital model of the grinding tool, assign physical attributes to the digital model of the workpiece, and the contact stress between the digital model of the grinding tool and the digital model of the workpiece Make simulation and analysis, feedback and predict whether to continue grinding with the current contact stress; In the step of real-time analysis of the processing status of the digital twin, assigning physical attributes to the digital model of the grinding tool includes assigning quality, material or roughness attributes, and assigning physical attributes to the digital model of the processed workpiece includes assigning quality, material or roughness attributes. Simulate and analyze the contact stress between the digital model of the grinding tool and the digital model of the workpiece to judge whether continuing to grind with the current path will cause damage to the surface of the workpiece due to excessive feed, if not, continue For processing, if it will cause damage to the surface of the processed workpiece, the current grinding feed rate will be divided into multiple segments of grinding feed rate, and the method of multiple grinding instead of one-time grinding will be used, and the partially modified processing path data will be sent back to the grinding unit The controller of the equipment realizes "controlling the reality with the virtual" and avoids damage to the workpiece; Verify the path correction effect: compare the action of the grinding unit equipment with the digital model action of the digital twin to judge whether the path is corrected successfully in real time. 2. The real-time correction method of a processing path combined with vision and digital twin according to claim 1, characterized in that, in the recording of the standard workpiece pose information, specifically for the robot processing path of the grinding unit equipment, in In the path planning stage, the standard workpiece is scanned and recorded using the visual 3D imaging method, and the generated 3D image information is transmitted to the digital twin simulation platform for analysis and processing, and the pose information of the standard workpiece in the digital model of the grinding unit is marked. 3. A method for real-time correction of processing paths combined with vision and digital twins according to claim 2, wherein the pose information includes six-axis angles of standard workpieces, including Location (x, y, z) and Rotation(x, y, z). 4. The real-time correction method of processing path combined with vision and digital twin according to claim 1, characterized in that, in the step of correcting the path of the workpiece, specifically for the robot processing path of the grinding unit equipment, in In the path planning stage, the workpiece is scanned and recorded using the visual 3D imaging method, the generated 3D image information is transmitted to the digital twin simulation platform for analysis and processing, the position and orientation information of the processed workpiece in the digital model of the grinding unit is marked, and the position and orientation of the processed workpiece are The pose information is compared with the pose information of the standard workpiece to obtain the position deviation value of the processing point where the pose information of the processed workpiece is the same as that of the standard workpiece, and the path of the workpiece is corrected according to the deviation value to obtain the corrected path Feedback to the grinding unit equipment. 5. The real-time correction method of processing path combined with vision and digital twin according to claim 1, characterized in that, in the virtual-real synchronization step of the device, the digital model, control script and communication interface are uniformly packaged, so that the model Use standard interface communication with equipment to build a virtual control network, use digital twin technology to build a virtual-real synchronization equipment physical simulation platform, and use communication interfaces to establish synchronous control and sensing channels between the grinding unit equipment and the virtual digital model , so that the grinding unit equipment and the corresponding digital model can communicate in real time and synchronize their actions. 6. The real-time correction method of a processing path combined with vision and digital twin according to claim 1, characterized in that, in the step of verifying the effect of path correction, the visual three-dimensional imaging method is used to scan and polish at the same time interval The image data of the grinding equipment and the workpiece of the unit equipment is obtained and transmitted to the simulation platform in real time, and the image data of the grinding unit equipment and the image data of the digital twin are compared on the simulation platform. If the image data is completely consistent or similar after comparison If it is very high, it is considered that the real-time correction of the robot processing path is successful.

Images