CN110763137A - Method and device for determining product posture - Google Patents
Method and device for determining product posture Download PDFInfo
- Publication number
- CN110763137A CN110763137A CN201911072376.2A CN201911072376A CN110763137A CN 110763137 A CN110763137 A CN 110763137A CN 201911072376 A CN201911072376 A CN 201911072376A CN 110763137 A CN110763137 A CN 110763137A
- Authority
- CN
- China
- Prior art keywords
- target product
- product
- line laser
- image
- light stripe
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a method and a device for determining the posture of a product. Wherein, the method comprises the following steps: controlling a line laser to project line laser to a target product; controlling an image acquisition device to acquire a light stripe image sequence on the product surface of a target product under the projection scene of the on-line laser; extracting three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information; obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determining the current posture of the target product based on the parallax image; the attitude of the predetermined actuator is adjusted based on the current attitude of the target product. The invention solves the technical problem of lower reliability of a mode for determining the posture of a product in the related technology.
Description
Technical Field
The invention relates to the technical field of product posture processing, in particular to a method and a device for determining a product posture.
Background
For the product placed on the dynamic production line at will, in order to realize intelligent flexible production, for example, automatic assembly, automatic screwing, automatic labeling, automatic detection and the like, the premise is that the placing posture of the product on the dynamic production line can be obtained in real time, then an executing mechanism, for example, a robot, an X/Y/Z platform, a module, a tool clamp and the like, is controlled, the posture of the mechanism is adjusted in real time according to the placing posture of the product, the relative posture between the mechanism and the product is kept uniform, and finally corresponding action is executed according to a designed relative path. At present, the common method is a contact method of additionally arranging a guide mechanism on a dynamic production line, and the product is guided into a fixed posture and then is subjected to subsequent intelligent flexible production, and the defects are that: need readjustment guiding mechanism when the product switches, there is great error in the gesture after the direction, not only reduces the takt of assembly line, scrapes colored product outward appearance face easily moreover, leads to quality abnormal occurence of failure to the product that has higher outward appearance face quality requirement can't be suitable for, for example: an air conditioning product.
Aiming at the problem that the reliability of a mode for determining the posture of a product in the related technology is low, an effective solution is not provided at present.
Disclosure of Invention
The embodiment of the invention provides a method and a device for determining the posture of a product, which at least solve the technical problem of low reliability of a mode for determining the posture of the product in the related technology.
According to an aspect of an embodiment of the present invention, there is provided a method for determining a product posture, including: controlling a line laser to project line laser to a target product; controlling an image acquisition device to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the line laser; extracting three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information; and obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determining the current posture of the target product based on the parallax image.
Optionally, controlling the line laser to project the line laser onto the target product comprises: acquiring an induction signal of a photoelectric sensor; when the induction signal represents that the target product enters a projection plane of the line laser, sending a starting signal to the line laser; and controlling the line laser to project line laser to the target product based on the starting signal.
Optionally, the line laser is disposed right above a production line for transporting the target product, and perpendicular to a projection plane of the line laser and a moving direction of the production line, and the photoelectric sensor is disposed at a predetermined position due to a holder for fixing the line laser.
Optionally, the image acquisition device is disposed right above the production line, and an optical axis of the image acquisition device and the line laser form a predetermined included angle.
Optionally, before controlling the image capturing device to acquire the sequence of light stripe images on the product surface of the target product in the projection scene of the line laser, the method for determining the product pose further includes: calibrating a projection plane of the line laser, image acquisition equipment, a reference end face and the movement direction of a production line for transmitting the target product, wherein the reference end face is perpendicular to the end face of a laser beam projected by the line laser; and establishing a relational mapping model between a Cartesian coordinate system and a three-dimensional point cloud coordinate system in which the three-dimensional point cloud data is located by taking the motion direction of the production line as the positive direction of the X axis.
Optionally, controlling the image acquisition device to acquire a sequence of light stripe images on the product surface of the target product under the projection scene of the line laser includes: acquiring a plurality of feedback signals of an encoder, wherein the encoder is arranged on a transmission motor of a production line for transmitting the target product; controlling the image acquisition equipment to acquire a plurality of light stripe images of the target product under the projection of the on-line laser under the triggering of each feedback signal in the plurality of feedback signals; determining the sequence of light stripe images based on the plurality of light stripe images.
Optionally, extracting three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information includes: extracting a light stripe central line of each light stripe image in the light stripe sequence; and sequentially splicing the light stripe center lines of each light stripe image along the motion direction of a production line for transmitting the target object to obtain three-dimensional point cloud data of the product surface of the target product on the production line.
Optionally, obtaining the parallax image of the product surface of the target product according to the three-dimensional point cloud data includes: determining a height characteristic of the target product and determining a segmentation threshold value of the three-dimensional point cloud data; and extracting the parallax image from the three-dimensional point cloud data based on the segmentation threshold.
Optionally, determining the current pose of the target product based on the parallax image comprises: acquiring appearance characteristic information of the target product; processing the appearance characteristic information and the parallax image according to a visual algorithm to obtain the placing posture of the target product on a production line for transmitting the target product; converting the placing posture into a product posture in a Cartesian coordinate system based on the relational mapping model; and determining the converted product posture as the current posture of the target product.
Optionally, after obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data and determining the current posture of the target product based on the parallax image, the method for determining the posture of the product further includes: transmitting the current posture of the target product to a preset execution mechanism, wherein the preset execution mechanism is used for executing preset operation on the target product; adjusting the current attitude of the predetermined actuator based on the current attitude of the target product; and executing a preset operation on the target product by using the adjusted preset executing mechanism.
According to another aspect of the embodiments of the present invention, there is also provided an apparatus for determining a product posture, including: the first control unit is used for controlling the line laser to project line laser to a target product; the second control unit is used for controlling the image acquisition equipment to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the line laser; the extraction unit is used for extracting the three-dimensional information of each light stripe image in the light stripe image sequence and obtaining the three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information; the determining unit is used for obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data and determining the current posture of the target product based on the parallax image.
Optionally, the first control unit comprises: the first acquisition module is used for acquiring an induction signal of the photoelectric sensor; the sending module is used for sending a starting signal to the line laser when the induction signal represents that the target product enters the projection plane of the line laser; and the first control module is used for controlling the line laser to project line laser to the target product based on the starting signal.
Optionally, the line laser is disposed right above a production line for transporting the target product, and perpendicular to a projection plane of the line laser and a moving direction of the production line, and the photoelectric sensor is disposed at a predetermined position due to a holder for fixing the line laser.
Optionally, the image acquisition device is disposed right above the production line, and an optical axis of the image acquisition device and the line laser form a predetermined included angle.
Optionally, the device for determining the product posture further comprises: the calibration unit is used for calibrating a projection plane of the line laser, the image acquisition equipment, a reference end face and the movement direction of a production line for transmitting the target product before controlling the image acquisition equipment to acquire a light stripe image sequence on the product surface of the target product in the projection scene of the line laser, wherein the reference end face is perpendicular to the end face of a laser beam projected by the line laser; and the establishing unit is used for establishing a relational mapping model between a Cartesian coordinate system and a three-dimensional point cloud coordinate system in which the three-dimensional point cloud data is positioned by taking the motion direction of the production line as the positive direction of the X axis.
Optionally, the second control unit comprises: the second acquisition module is used for acquiring a plurality of feedback signals of an encoder, wherein the encoder is arranged on a transmission motor of a production line for transmitting the target product; the second control module is used for controlling the image acquisition equipment to acquire a plurality of light stripe images of the target product under the projection of the on-line laser under the triggering of each feedback signal in the plurality of feedback signals; a first determination module to determine the sequence of light stripe images based on the plurality of light stripe images.
Optionally, the extraction unit includes: the first extraction module is used for extracting the light stripe central line of each light stripe image in the light stripe sequence; and the splicing module is used for sequentially splicing the light stripe center lines of each light stripe image along the motion direction of a production line for transmitting the target object to obtain three-dimensional point cloud data of the product surface of the target product on the production line.
Optionally, the determining unit includes: the second determination module is used for determining the height characteristic of the target product and determining a segmentation threshold value of the three-dimensional point cloud data; and the second extraction module is used for extracting the parallax image from the three-dimensional point cloud data based on the segmentation threshold.
Optionally, the determining unit includes: the third acquisition module is used for acquiring the appearance characteristic information of the target product; the processing module is used for processing the appearance characteristic information and the parallax image according to a visual algorithm to obtain the placing posture of the target product on a production line for transmitting the target product; the conversion module is used for converting the placing posture into a product posture under a Cartesian coordinate system based on the relational mapping model; and the third determining module is used for determining the converted product posture as the current posture of the target product.
Optionally, the device for determining the product posture further comprises: the transmission unit is used for transmitting the current posture of the target product to a preset execution mechanism after obtaining the parallax image of the product surface of the target product according to the three-dimensional point cloud data and determining the current posture of the target product based on the parallax image, wherein the preset execution mechanism is used for executing preset operation on the target product; the adjusting unit is used for adjusting the current posture of the preset actuating mechanism based on the current posture of the target product; and the execution unit is used for executing the preset operation on the target product by utilizing the adjusted preset execution mechanism.
According to another aspect of the embodiment of the present invention, there is also provided a product posture determination system, which uses the product posture determination method described in any one of the above.
According to another aspect of the embodiments of the present invention, there is also provided a storage medium including a stored program, wherein the program executes the method for determining the posture of the product according to any one of the above.
According to another aspect of the embodiment of the present invention, there is also provided a processor, configured to execute a program, where the program executes the method for determining the product posture described in any one of the above.
In the embodiment of the invention, a control line laser is adopted to project line laser to a target product; controlling an image acquisition device to acquire a light stripe image sequence on the product surface of a target product under the projection scene of the on-line laser; extracting three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information; the method for determining the product posture provided by the embodiment of the invention achieves the purpose of acquiring the current posture of the target product in real time; when the target product is switched, the target product does not need to be guided by a guide mechanism, so that the product posture deviation caused by the processing of the guide mechanism is avoided, the technical effect of improving the accuracy of positioning the current posture of the target product is achieved, and the technical problem of low reliability of a mode for determining the posture of the product in the related technology is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow diagram of a method of determining a product pose according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a product pose determination device according to an embodiment of the invention;
FIG. 3 is a front view of a product pose determination device according to an embodiment of the present invention;
FIG. 4 is a top view of a product pose determination device according to an embodiment of the present invention;
FIG. 5 is a side view of a product pose determination device according to an embodiment of the present invention;
FIG. 6 is an enlarged partial schematic view of a product pose determination device according to an embodiment of the present invention;
FIG. 7 is an enlarged partial schematic view of an alternative product pose determination device according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of the principles of laser triangulation according to an embodiment of the invention;
FIG. 9 is a schematic illustration of a parallax image of a target product according to an embodiment of the invention;
FIG. 10 is a schematic diagram of a current pose of a target product according to an embodiment of the invention;
fig. 11 is a schematic diagram of a product pose determination apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present invention, there is provided a method embodiment of a method for determining a pose of a product, it being noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than presented herein.
Fig. 1 is a flowchart of a method for determining a product posture according to an embodiment of the present invention, as shown in fig. 1, the method for determining a product posture includes the steps of:
and step S102, controlling the line laser to project line laser to the target product.
Optionally, in the embodiment of the present invention, the type of the target product is not specifically limited, and may be a washing machine, an air conditioner, a humidifier, a water heater, or the like, or may be other products.
And step S104, controlling the image acquisition equipment to acquire the light stripe image sequence on the product surface of the target product in the projection scene of the on-line laser.
Optionally, the image capturing device may be a camera, or may be other devices.
In an alternative embodiment, in the case that the line laser projects laser light to the target product, the camera may be controlled to collect optical stripe images of the target product under the projection of the laser light, and the plurality of optical stripe images collected by the camera during the movement of the target product in the production line may be determined as the above optical stripe image sequence.
And S106, extracting the three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining the three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information.
Optionally, the three-dimensional point cloud data may form a three-dimensional point cloud model of the target product, and then the outer contour feature of the target product may be obtained based on the three-dimensional point cloud model.
And S108, obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determining the current posture of the target product based on the parallax image.
And step S110, adjusting the posture of a preset actuating mechanism based on the current posture of the target product.
As can be seen from the above, in the embodiment of the present invention, the line laser may be controlled to project line laser to the target product, and then the image acquisition device is controlled to obtain the light stripe image sequence on the product surface of the target product in the projection scene of the line laser; the method comprises the steps of extracting three-dimensional information of each light stripe image in a light stripe image sequence, obtaining three-dimensional point cloud data of the product surface of a target product based on the three-dimensional information, obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determining the current posture of the target product based on the parallax image.
It is easy to notice that the line laser can be controlled to project line laser to the target product, and the image acquisition device is controlled to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the line laser, extract three-dimensional information of each light stripe image in the light stripe image sequence, and obtain three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information; obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determining the current posture of the target product based on the parallax image, thereby achieving the purpose of obtaining the current posture of the target product in real time; and when the target product is switched, the target product does not need to be guided by adopting a guide mechanism, so that the product posture deviation caused by the processing of the guide mechanism is avoided, and the technical effect of improving the accuracy of positioning the current posture of the target product is achieved.
Therefore, the method for determining the product posture provided by the embodiment of the invention solves the technical problem that the mode for determining the posture of the product in the related technology has low reliability.
Optionally, according to the above embodiment of the present invention, in step S102, controlling the line laser to project the line laser to the target product may include: acquiring an induction signal of a photoelectric sensor; when the induction signal indicates that the target product enters a projection plane of the line laser, a starting signal is sent to the line laser; the control line laser projects line laser to the target product based on the starting signal.
In an alternative embodiment, the line laser is arranged directly above the production line transporting the target product and perpendicular to the projection plane of the line laser and the direction of movement of the production line, and the photoelectric sensor is arranged at a predetermined position due to the holder holding the line laser.
It should be noted that the support may be a gantry support.
In addition, the image acquisition equipment is arranged right above the production line, and the optical axis of the image acquisition equipment and the line laser form a preset included angle.
Fig. 2 is a schematic diagram of a product posture determining apparatus according to an embodiment of the present invention, in which the support is a gantry support and the image capturing device is a camera in fig. 2, for example, two gantry supports are fixed on a platform on which a product is placed, a line laser is disposed directly above one of the two gantry supports, and a camera is disposed directly above the other of the two gantry supports, as shown in fig. 2. The sensor is arranged on one side of a gantry support fixed with the linear laser.
Namely, two gantry supports are erected on the dynamic production line and are respectively used for fixing a line laser and a camera, the line laser is fixed right above the dynamic production line, and a laser projection plane of the line laser is vertical to the motion direction of the dynamic production line; the camera is fixed right above the dynamic production line, and a fixed included angle is formed between an optical axis and a line laser projection plane, so that the laser triangulation principle model is met.
In addition, a pair of photoelectric sensors are arranged on a gantry bracket of the fixed line laser and used for sensing whether a product (such as an air conditioner) is on a line laser projection plane, and meanwhile, a feedback encoder is arranged on a dynamic assembly line transmission motor and used for acquiring the instantaneous movement speed of the dynamic assembly line, because the movement of the dynamic assembly line is not uniform in most cases.
Specifically, fig. 3 is a front view of the determining apparatus of the product posture according to the embodiment of the present invention, fig. 4 is a top view of the determining apparatus of the product posture according to the embodiment of the present invention, and fig. 5 is a side view of the determining apparatus of the product posture according to the embodiment of the present invention.
In the embodiment of the present invention, details of the product posture determining apparatus are also shown in an enlarged manner, fig. 6 is a partially enlarged schematic view of the product posture determining apparatus according to the embodiment of the present invention, fig. 7 is a partially enlarged schematic view of an optional product posture determining apparatus according to the embodiment of the present invention, and as can be seen from fig. 6 and 7, the laser beam end surface of the laser is perpendicular to the running direction of the production line.
In the following, the laser triangulation principle is explained in detail, and fig. 8 is a schematic diagram of the laser triangulation principle according to the embodiment of the invention, wherein fig. 8 is a vertical distance between a laser beam end face a and a reference plane R, and d isThe distance between the end surface a and a focal point O ' of the camera, D0 is the distance of OO ', f is the focal length of the camera, and can be obtained by calibrating a camera calibration method based on a 2D plane target, Δ is the distance between a measured object and a reference plane, θ is the included angle between a laser beam and OO ', and δ is the offset distance of an imaging point on an imaging plane, then: d ═ d0/(δ-1fsinθ+cosθ)。
According to the above embodiment of the present invention, before controlling the image capturing device to obtain the sequence of light stripe images on the product surface of the target product in the projection scene of the on-line laser, the method for determining the product pose may further include: calibrating the projection plane of the line laser, the image acquisition equipment, the reference end face and the movement direction of a production line for transmitting a target product, wherein the reference end face is vertical to the end face of a laser beam projected by the line laser; and establishing a relational mapping model between a Cartesian coordinate system and a three-dimensional point cloud coordinate system in which the three-dimensional point cloud data is located by taking the motion direction of the production line as the positive direction of the X axis.
In an alternative embodiment, the controlling the image capturing device to acquire the sequence of light stripe images on the product surface of the target product in the projection scene of the line laser may include: acquiring a plurality of feedback signals of an encoder, wherein the encoder is arranged on a transmission motor of a production line for transmitting a target product; controlling an image acquisition device to acquire a plurality of light stripe images of a target product under the projection of on-line laser under the trigger of each feedback signal in a plurality of feedback signals; a sequence of light stripe images is determined based on a plurality of light stripe images.
Optionally, the extracting three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information includes: extracting the light stripe central line of each light stripe image in the light stripe sequence; and sequentially splicing the light stripe center lines of each light stripe image along the motion direction of a production line for transmitting the target object to obtain three-dimensional point cloud data of the product surface of the target product on the production line.
According to the above embodiment of the present invention, in step S108, obtaining the parallax image of the product surface of the target product according to the three-dimensional point cloud data may include: determining the height characteristic of a target product and determining a segmentation threshold value of three-dimensional point cloud data; and extracting a parallax image from the three-dimensional point cloud data based on the segmentation threshold.
According to the above embodiment of the present invention, in step S108, determining the current posture of the target product based on the parallax image includes: acquiring appearance characteristic information of a target product; processing the appearance characteristic information and the parallax image according to a visual algorithm to obtain the placing posture of the target product on a production line for transmitting the target product; converting the placing posture into a product posture under a Cartesian coordinate system based on a relational mapping model; and determining the converted product posture as the current posture of the target product.
In addition, after obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data and determining the current posture of the target product based on the parallax image, the method for determining the posture of the product further comprises the following steps: transmitting the current posture of the target product to a preset execution mechanism, wherein the preset execution mechanism is used for executing preset operation on the target product; adjusting the current posture of a preset actuating mechanism based on the current posture of the target product; and executing the preset operation on the target product by using the adjusted preset executing mechanism.
In this embodiment, when the target product has a posture offset with respect to the predetermined execution architecture, the target product does not need to be directly adjusted, and the posture of the predetermined execution mechanism can be adjusted based on the current posture of the target product.
It is easy to note that, when determining that the target product has a posture offset relative to the predetermined actuator, the current posture of the target product does not need to be adjusted, so as to prevent the product from being damaged due to a contact type adjustment mode when the target product is subjected to posture adjustment by a device such as a manipulator, but after determining the current posture of the target product, the posture of the predetermined actuator is adjusted based on the determined current posture of the target product, so that the postures of the target product and the predetermined actuator are relatively uniform.
In addition, when the current posture of the target product is determined, a control line laser is adopted to project line laser to the target product, and then the image acquisition equipment is controlled to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the line laser; the three-dimensional information of each light stripe image in the light stripe image sequence is extracted, the three-dimensional point cloud data of the product surface of the target product is obtained based on the three-dimensional information, the parallax image of the product surface of the target product is obtained according to the three-dimensional point cloud data, and the current posture of the target product is determined based on the parallax image.
Through the method for determining the product attitude provided in the above embodiment of the present invention, first, system calibration is performed, which includes: the method comprises the following steps of datum plane calibration, camera calibration, laser projection plane calibration and movement direction calibration, wherein a certain fixed point on a dynamic production line is used as an origin of a Cartesian coordinate system, the movement direction of the dynamic production line is used as an X-axis square, and a mapping relation mathematical model between the Cartesian coordinate system and a three-dimensional point cloud coordinate system of the system is established.
Then, image acquisition is carried out, and if the product (air conditioner) is on the line laser projection plane, the photoelectric sensor outputs high level, and if the product (air conditioner) is not on the line laser projection plane, the photoelectric sensor outputs low level. When a product (air conditioner) flows through the line laser projection plane, the photoelectric sensor outputs a high level, the system monitors a real-time signal fed back by the feedback encoder in the process, the primary camera is triggered by a pulse increment fed back by the fixed feedback encoder (representing that a dynamic production line moves for a fixed distance), and a light stripe image projected onto the outer surface of the product (air conditioner) by one frame of line laser is obtained, so that a dynamic image sequence consisting of N images is obtained in the process that the product (air conditioner) flows through the line laser projection plane, each image in the sequence comprises a sequential section contour line of the outer surface of the product (air conditioner), and information is hidden in the light stripe in the image.
Then, data processing is performed, a visual algorithm is used for extracting a light stripe center line in each image in the dynamic image sequence, the light stripe center lines are sequentially spliced along the motion direction of the dynamic production line according to the image sequence to obtain outer surface three-dimensional point cloud data of a product (air conditioner) on the dynamic production line, a proper segmentation threshold value is set according to a height coordinate value of the three-dimensional point cloud data, and a parallax image of the product (air conditioner) on the dynamic production line can be simply extracted, wherein fig. 9 is a schematic diagram of the parallax image of a target product according to an embodiment of the invention.
And finally, calculating the placing posture of the product (air conditioner) by using a visual algorithm according to the acquired parallax image and the appearance characteristics of the product (air conditioner) (fig. 10 is a schematic diagram of the current posture of the target product according to the embodiment of the invention), converting the placing posture into the product posture in a Cartesian coordinate system according to the mapping relation mathematical model established in the first step, and feeding back the product posture to an execution mechanism in real time to automatically adjust the product posture into a uniform relative posture relative to the product (air conditioner), thereby providing possibility for subsequent intelligent flexible production.
In addition, in the embodiment of the invention, the relative positions of the line laser and the camera can be replaced, the camera is fixed right above the dynamic production line, the optical axis is vertical to the motion direction of the dynamic production line, the line laser is fixed right above the dynamic production line, and the line laser projection plane and the optical axis of the camera form a fixed included angle, so that the laser three-dimensional measurement principle model is conformed to.
Example 2
According to another aspect of the embodiments of the present invention, there is also provided an embodiment of an apparatus for performing the method for determining a product orientation in embodiment 1 above, and fig. 11 is a schematic diagram of an apparatus for determining a product orientation according to an embodiment of the present invention, as shown in fig. 11, the apparatus for determining a product orientation includes: a first control unit 1101, a second control unit 1103, an extraction unit 1105, and a determination unit 1107. The following describes the product attitude determination device in detail.
A first control unit 1101 for controlling the line laser to project line laser light to the target product.
The second control unit 1103 is configured to control the image capturing device to obtain a sequence of light stripe images on the product surface of the target product in the projection scene of the line laser.
The extracting unit 1105 is configured to extract three-dimensional information of each light stripe image in the light stripe image sequence, and obtain three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information.
The determining unit 1107 is configured to obtain a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determine the current posture of the target product based on the parallax image.
It should be noted here that the first control unit 1101, the second control unit 1103, the extraction unit 1105 and the determination unit 1107 correspond to steps S102 to S108 in embodiment 1, and the above units are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the contents disclosed in embodiment 1. It should be noted that the modules described above as part of an apparatus may be implemented in a computer system such as a set of computer-executable instructions.
From the above, in the above embodiments of the present application, the first control unit may be used to control the line laser to project the line laser to the target product; a second control unit is used for controlling the image acquisition equipment to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the on-line laser; extracting the three-dimensional information of each light stripe image in the light stripe image sequence by using an extraction unit, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information; and obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data by using the determining unit, and determining the current posture of the target product based on the parallax image. By the product posture determining device provided by the embodiment of the invention, the aim of acquiring the current posture of the target product in real time is fulfilled; when the target product is switched, the target product does not need to be guided by a guide mechanism, so that the product posture deviation caused by the processing of the guide mechanism is avoided, the technical effect of improving the accuracy of positioning the current posture of the target product is achieved, and the technical problem of low reliability of a mode for determining the posture of the product in the related technology is solved.
Optionally, the first control unit comprises: the first acquisition module is used for acquiring an induction signal of the photoelectric sensor; the transmitting module is used for transmitting a starting signal to the line laser when the induction signal indicates that the target product enters the projection plane of the line laser; and the first control module is used for controlling the line laser to project line laser to the target product based on the starting signal.
Optionally, the line laser is disposed right above a production line transporting the target product, and perpendicular to a projection plane of the line laser and a moving direction of the production line, and the photoelectric sensor is disposed at a predetermined position due to a holder fixing the line laser.
Optionally, the image acquisition device is arranged right above the production line, and an optical axis of the image acquisition device forms a predetermined included angle with the line laser.
Optionally, the device for determining the product posture further comprises: the calibration unit is used for calibrating a projection plane of the line laser, the image acquisition equipment, a reference end face and the movement direction of a production line for transmitting the target product before the light stripe image sequence on the product surface of the target product under the condition of controlling the image acquisition equipment to acquire the projection scene of the line laser, wherein the reference end face is vertical to the end face of a laser beam projected by the line laser; and the establishing unit is used for establishing a relational mapping model between a Cartesian coordinate system and a three-dimensional point cloud coordinate system in which the three-dimensional point cloud data is positioned by taking the motion direction of the production line as the positive direction of the X axis.
Optionally, the second control unit comprises: the second acquisition module is used for acquiring a plurality of feedback signals of the encoder, wherein the encoder is arranged on a transmission motor of a production line for transmitting the target product; the second control module is used for controlling the image acquisition equipment to acquire a plurality of light stripe images of the target product under the projection of the on-line laser under the trigger of each feedback signal in the plurality of feedback signals; a first determination module to determine a sequence of light stripe images based on a plurality of light stripe images.
Optionally, the extraction unit comprises: the first extraction module is used for extracting the light stripe central line of each light stripe image in the light stripe sequence; and the splicing module is used for sequentially splicing the light stripe center lines of each light stripe image along the motion direction of the assembly line for transmitting the target object to obtain the three-dimensional point cloud data of the target product on the product surface of the assembly line.
Optionally, the determining unit includes: the second determination module is used for determining the height characteristic of the target product and determining the segmentation threshold of the three-dimensional point cloud data; and the second extraction module is used for extracting the parallax image from the three-dimensional point cloud data based on the segmentation threshold.
Optionally, the determining unit includes: the third acquisition module is used for acquiring the appearance characteristic information of the target product; the processing module is used for processing the appearance characteristic information and the parallax image according to a visual algorithm to obtain the placing posture of the target product on a production line for transmitting the target product; the conversion module is used for converting the placing posture into a product posture under a Cartesian coordinate system based on the relational mapping model; and the third determining module is used for determining the converted product posture as the current posture of the target product.
The product attitude determination device further includes: the transmission unit is used for transmitting the current posture of the target product to a preset execution mechanism after obtaining the parallax image of the product surface of the target product according to the three-dimensional point cloud data and determining the current posture of the target product based on the parallax image, wherein the preset execution mechanism is used for executing preset operation on the target product; the adjusting unit is used for adjusting the current posture of the preset actuating mechanism based on the current posture of the target product; and the execution unit is used for executing the preset operation on the target product by utilizing the adjusted preset execution mechanism.
Example 3
According to another aspect of the embodiment of the invention, a system for determining the product posture is further provided, and the system for determining the product posture uses the method for determining the product posture.
Example 4
According to another aspect of the embodiments of the present invention, there is also provided a storage medium including a stored program, wherein the program performs the method of determining the product posture of any one of the above.
Example 5
According to another aspect of the embodiments of the present invention, there is provided a processor, configured to execute a program, where the program executes a method for determining a product posture of any one of the above.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for determining the pose of a product, comprising:
controlling a line laser to project line laser to a target product;
controlling an image acquisition device to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the line laser;
extracting three-dimensional information of each light stripe image in the light stripe image sequence, and obtaining three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information;
and obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data, and determining the current posture of the target product based on the parallax image.
2. The method of claim 1, wherein controlling a line laser to project line laser light onto the target product comprises:
acquiring an induction signal of a photoelectric sensor;
when the induction signal represents that the target product enters a projection plane of the line laser, sending a starting signal to the line laser;
and controlling the line laser to project line laser to the target product based on the starting signal.
3. The method of claim 2, wherein the line laser is disposed directly above a line transporting the target product and perpendicular to a projection plane of the line laser and a moving direction of the line, and the photosensor is disposed at a predetermined position due to a holder fixing the line laser.
4. The method of claim 3, wherein the image capture device is positioned directly above the pipeline and an optical axis of the image capture device forms a predetermined angle with the line laser.
5. The method of claim 1, further comprising, prior to controlling an image capture device to capture a sequence of light streak images on the product surface of the target product under the projection scene of the line laser,:
calibrating a projection plane of the line laser, image acquisition equipment, a reference end face and the movement direction of a production line for transmitting the target product, wherein the reference end face is perpendicular to the end face of a laser beam projected by the line laser;
and establishing a relational mapping model between a Cartesian coordinate system and a three-dimensional point cloud coordinate system in which the three-dimensional point cloud data is located by taking the motion direction of the production line as the positive direction of the X axis.
6. The method of claim 1, wherein controlling an image capture device to acquire a sequence of light streak images on a product surface of the target product under the projection scene of the line laser comprises:
acquiring a plurality of feedback signals of an encoder, wherein the encoder is arranged on a transmission motor of a production line for transmitting the target product;
controlling the image acquisition equipment to acquire a plurality of light stripe images of the target product under the projection of the on-line laser under the triggering of each feedback signal in the plurality of feedback signals;
determining the sequence of light stripe images based on the plurality of light stripe images.
7. The method of claim 1, wherein extracting three-dimensional information for each light stripe image in the sequence of light stripe images and deriving three-dimensional point cloud data for the product surface of the target product based on the three-dimensional information comprises:
extracting a light stripe central line of each light stripe image in the light stripe sequence;
and sequentially splicing the light stripe center lines of each light stripe image along the motion direction of a production line for transmitting the target object to obtain three-dimensional point cloud data of the product surface of the target product on the production line.
8. The method of claim 1, wherein obtaining the parallax image of the product surface of the target product from the three-dimensional point cloud data comprises:
determining a height characteristic of the target product and determining a segmentation threshold value of the three-dimensional point cloud data;
and extracting the parallax image from the three-dimensional point cloud data based on the segmentation threshold.
9. The method of any of claims 5 to 8, wherein determining the current pose of the target product based on the parallax image comprises:
acquiring appearance characteristic information of the target product;
processing the appearance characteristic information and the parallax image according to a visual algorithm to obtain the placing posture of the target product on a production line for transmitting the target product;
converting the placing posture into a product posture in a Cartesian coordinate system based on the relational mapping model;
and determining the converted product posture as the current posture of the target product.
10. An apparatus for determining the attitude of a product, comprising:
the first control unit is used for controlling the line laser to project line laser to a target product;
the second control unit is used for controlling the image acquisition equipment to acquire a light stripe image sequence on the product surface of the target product under the projection scene of the line laser;
the extraction unit is used for extracting the three-dimensional information of each light stripe image in the light stripe image sequence and obtaining the three-dimensional point cloud data of the product surface of the target product based on the three-dimensional information;
the determining unit is used for obtaining a parallax image of the product surface of the target product according to the three-dimensional point cloud data and determining the current posture of the target product based on the parallax image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911072376.2A CN110763137A (en) | 2019-11-05 | 2019-11-05 | Method and device for determining product posture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911072376.2A CN110763137A (en) | 2019-11-05 | 2019-11-05 | Method and device for determining product posture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110763137A true CN110763137A (en) | 2020-02-07 |
Family
ID=69336174
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911072376.2A Pending CN110763137A (en) | 2019-11-05 | 2019-11-05 | Method and device for determining product posture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110763137A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111618833A (en) * | 2020-04-27 | 2020-09-04 | Oppo（重庆）智能科技有限公司 | Method and device for assembling components, storage medium, and electronic device |
| CN111623706A (en) * | 2020-05-07 | 2020-09-04 | 哈尔滨工程大学 | Caragana microphylla stubble leveling machine information acquisition method |
| CN112964178A (en) * | 2021-02-07 | 2021-06-15 | 佛山科学技术学院 | Amorphous strip transmission position measuring method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070171393A1 (en) * | 2006-01-20 | 2007-07-26 | Samsung Electronics Co., Ltd. | Laser straight ruler and method for measuring distance and projecting line using the straight ruler |
| WO2007088789A1 (en) * | 2006-02-01 | 2007-08-09 | Tokyo Institute Of Technology | Surface shape measuring method and device using the same |
| JP2007248241A (en) * | 2006-03-15 | 2007-09-27 | Omron Corp | Inspection method and device of surface state |
| CN105277120A (en) * | 2014-06-25 | 2016-01-27 | 财团法人工业技术研究院 | Object measuring system and method |
| CN106780725A (en) * | 2016-12-23 | 2017-05-31 | 西安交通大学 | A kind of binocular three-dimensional reconstructing method and system |
| CN108444449A (en) * | 2018-02-02 | 2018-08-24 | 中国科学院西安光学精密机械研究所 | Method for measuring target space attitude with parallel line characteristics |
-
2019
- 2019-11-05 CN CN201911072376.2A patent/CN110763137A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070171393A1 (en) * | 2006-01-20 | 2007-07-26 | Samsung Electronics Co., Ltd. | Laser straight ruler and method for measuring distance and projecting line using the straight ruler |
| WO2007088789A1 (en) * | 2006-02-01 | 2007-08-09 | Tokyo Institute Of Technology | Surface shape measuring method and device using the same |
| JP2007248241A (en) * | 2006-03-15 | 2007-09-27 | Omron Corp | Inspection method and device of surface state |
| CN105277120A (en) * | 2014-06-25 | 2016-01-27 | 财团法人工业技术研究院 | Object measuring system and method |
| CN106780725A (en) * | 2016-12-23 | 2017-05-31 | 西安交通大学 | A kind of binocular three-dimensional reconstructing method and system |
| CN108444449A (en) * | 2018-02-02 | 2018-08-24 | 中国科学院西安光学精密机械研究所 | Method for measuring target space attitude with parallel line characteristics |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111618833A (en) * | 2020-04-27 | 2020-09-04 | Oppo（重庆）智能科技有限公司 | Method and device for assembling components, storage medium, and electronic device |
| CN111623706A (en) * | 2020-05-07 | 2020-09-04 | 哈尔滨工程大学 | Caragana microphylla stubble leveling machine information acquisition method |
| CN111623706B (en) * | 2020-05-07 | 2021-11-09 | 哈尔滨工程大学 | Caragana microphylla stubble leveling machine information acquisition method |
| CN112964178A (en) * | 2021-02-07 | 2021-06-15 | 佛山科学技术学院 | Amorphous strip transmission position measuring method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9279661B2 (en) | Information processing apparatus and information processing method | |
| CN110728715B (en) | A method for self-adaptive adjustment of the camera angle of an intelligent inspection robot | |
| CN108982546B (en) | Intelligent robot gluing quality detection system and method | |
| US20200096317A1 (en) | Three-dimensional measurement apparatus, processing method, and non-transitory computer-readable storage medium | |
| CN106625676B (en) | Three-dimensional visual accurate guiding and positioning method for automatic feeding in intelligent automobile manufacturing | |
| TWI408486B (en) | Camera with dynamic calibration and method thereof | |
| CN105674880B (en) | Contact net geometric parameter measurement method and system based on binocular principle | |
| US20160086343A1 (en) | Contour line measurement apparatus and robot system | |
| JP6324025B2 (en) | Information processing apparatus and information processing method | |
| CN103673876B (en) | For the swing type gage outfit robot on-line measurement system of body in white size detection | |
| CN110763137A (en) | Method and device for determining product posture | |
| KR101972432B1 (en) | A laser-vision sensor and calibration method thereof | |
| CN110017769A (en) | Part detection method and system based on industrial robot | |
| CN113014816A (en) | Method and device for determining trigger point of flying shooting | |
| JP2017142613A (en) | Information processing device, information processing system, information processing method and information processing program | |
| CN107633537B (en) | Camera calibration method based on projection | |
| CN117893610A (en) | Aeronautical assembly robot posture measurement system based on zoom monocular vision | |
| JP7414850B2 (en) | robot system | |
| CN112150547A (en) | Method and device for determining pose of vehicle body and look-around vision odometer system | |
| JP2019077026A (en) | Control device, robot system, and control device operating method and program | |
| CN102243064A (en) | Determination system and method for position and posture of video camera in virtual studio | |
| JP7028814B2 (en) | External shape recognition device, external shape recognition system and external shape recognition method | |
| CN114147727B (en) | Method, device and system for correcting pose of robot | |
| Rodríguez-Araújo et al. | ROS-based 3D on-line monitoring of LMD robotized cells | |
| JP6285765B2 (en) | Information processing apparatus and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200207 |