CN107598928B - Camera and robot control system based on semantic model and its automatic adaptation method - Google Patents

Abstract

The present invention relates to cameras and robot control system and its automatic adaptation method based on semantic model, semantization model service end module, semantization model client end module are established in camera controller, robot controller respectively, by the semantic modeling of semantization model service end module and the parsing with semantization model client end module, the automatic adaptation of industrial camera and robot control system is realized.The present invention can not need robot control and stop working when camera function upgrades and changes, and again to the function and order of controller programming modification camera, increase the flexibility of robot system work, saved the working time, plug and play.

Description

Camera and robot control system based on semantic model and its automatic adaptation method

technical field

The invention relates to a semantic model-based industrial camera and robot control system and an automatic adaptation method thereof, belonging to the field of robot control.

Background technique

In the context of the era of Industry 4.0, with the deepening of China's production 2025 strategy, the robot industry market is showing an explosive growth momentum, which is inseparable from the machine vision system, which is equivalent to the extension of human vision on the machine, making people Machine collaboration becomes a reality. If the robot industry wants to achieve true intelligence and automation, it must have the ability to accurately observe things, so as to be able to judge things well. As the core of the machine vision system, the importance of industrial cameras is self-evident.

In the traditional machine vision solution, the robot controller needs to perform fixed programming work on the communication configuration and functions of the camera. When there are a large number of random combinations of camera functions, extremely tedious robot programming work is required; when the camera functions are upgraded and changed, the robot controller needs to be Stop work and reprogram the function and configuration firmly. If you replace a different camera, you need to perform specific programming for this camera according to the camera manufacturer's functions and configurations, and the software between different cameras is not reusable. The traditional solution affects efficiency, is not flexible, and does not conform to the concept of intelligent manufacturing.

"Semantic" means that machines can study and collect information and understand machine language with less human intervention. It is currently widely used in the Internet field, and is widely used in web and HTML transmission. In the field of industrial control, there are different data formats and communication modes of various devices, which will result in the need for engineers to know different data and format meanings for data collection in a system, which wastes a lot of resources. Semantics solves this problem, allowing devices to be described in a unified manner, speaking the same language, and enabling better interconnection.

Aiming at the demand for intelligence in the field of robot control, based on semantic modeling, the adaptive method of industrial cameras and robot control systems is studied. Semantic description information, automatic adaptation between servers and clients, breaking through the cumbersome traditional visual fixed programming, realizing Software reuse, plug and play, and interconnection are of great significance to the development of China's intelligent manufacturing.

SUMMARY OF THE INVENTION

Aiming at the large number of communication configuration processes and the tedious reprogramming work during camera replacement or function update in the existing vision scheme, the present invention provides a semantic model-based industrial camera and robot control system and an automatic adaptation method thereof. Based on the semantic model module, through the semantic description, it realizes automatic adaptation, plug and play, synchronous update, and improves work efficiency.

The technical solution adopted by the present invention to achieve the above object is: an automatic adaptation method of a camera and a robot control system based on a semantic model, respectively establishing a semantic model server module and a semantic model client in the camera controller and the robot controller. The end module realizes the automatic adaptation of the industrial camera and the robot control system through the semantic modeling of the semantic model server module and the parsing with the semantic model client module, including the following steps:

The semantic model server module performs semantic description according to camera parameters and camera control commands, writes it into the elements and sub-elements of the address space, and then forms an XML file with the parameter organization structure and service list;

The semantic model client module accesses the semantic model server module in the camera controller and obtains the XML file, parses and obtains the camera parameters and camera control commands; sends the required camera parameters and the semantic commands corresponding to the camera control commands to camera controller;

The semantic model server module obtains the semantic command, the camera controller executes the operation corresponding to the semantic command, and feeds back the robot information; the robot controller performs control according to the robot information.

The camera parameters include camera shooting function, shooting parameter adjustment, and camera parameter information.

The parameter organization structure includes the corresponding relationship between elements and sub-elements; the service list includes the contents of the elements and sub-elements.

The semantic description is performed according to camera parameters and control commands, and the elements and sub-elements written into the address space include the following steps:

The camera shooting function, shooting parameter adjustment, and camera parameter information are written into the address space as elements respectively; the classification of each element is regarded as the sub-element under the element; the action corresponding to a sub-element, that is, the camera control command, is regarded as the sub-element under the sub-element. next-level child element.

The camera controller performs the corresponding command operation and feeds back the robot information, including the following steps:

The camera controller executes the shooting action according to the semantic command, and obtains the position information corresponding to the shooting result.

After feeding back the robot information, the robot controller calculates the robot motion trajectory according to the position information, and controls the robot to perform the grabbing action according to the motion trajectory.

For the camera and robot control system based on semantic model, a semantic model server module and a semantic model client module are established in the camera controller and robot controller respectively, including:

The semantic model server module is used to describe semantically according to camera parameters and camera control commands, write them into the elements and sub-elements of the address space, and then form an XML file with the parameter organization structure and service list; The camera controller executes the operations corresponding to the semantic commands and feeds back the robot information to the robot controller;

The semantic model client module is used to access the semantic model server module in the camera controller and obtain the XML file, and parse to obtain the camera parameters and camera control commands; the required camera parameters and camera control commands correspond to the semantics Commands are sent to the semantic model server module of the camera controller.

The present invention has the following advantages and beneficial effects:

1. The automatic adaptation of the present invention is not aimed at a specific type of robot controller and industrial camera, and the adaptation method is applicable to all robot controllers and industrial cameras equipped with the semantic model module of the present invention.

2. The feature of automatic adaptation of the present invention makes it possible to upgrade and change the camera function without the need for the robot controller to interrupt the work, instead of manually modifying the camera function and communication commands with the teach pendant, increasing the flexibility of the robot system. Save working time, plug and play.

3. The semantic model module of the present invention uses a service-oriented technology and does not depend on a specific hardware platform and operating system, and the software has reusability and wide application range.

Description of drawings

Fig. 1 is the embodiment block diagram of the hardware environment composition of the application of the present invention;

Fig. 2 is the controller semantic model module diagram of the present invention;

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

Referring to Figure 1, an industrial camera and robot control system based on a semantic model mainly involves a robot controller, a camera controller, and a robot body.

The robot controller is an X86 architecture platform running the QNX real-time operating system, and its hardware part includes a motion controller and a servo driver, and supports EtherCAT bus communication. The software part includes semantic model server module, control algorithm module, sensor module, motion control module; control algorithm module, including basic algorithm library, functional algorithm library and application algorithm library, used for algorithm application of robot control; sensor module, including Sensor hardware driver and data transformation model for robot-sensor interaction. Motion control module, including dynamics module and kinematics module, is used to parse commands and control robot operation.

The camera controller includes a semantic model client module, a communication configuration module, and a visual function module;

Referring to FIG. 2 , the core points of the automatic adaptation of the present patent are the client module and the server module of the semantic model. Semantic model server module is a service-oriented module based on semantics. It defines a model that integrates address space and information, and bases all parameters, operation methods, history, events and other information related to a specific object on the basis of semantic modeling. Semantically describe objects, establish mutual relationships, and form XML files of parameter organization structure and service list, which can represent complex data structures and processes, establish network communication servers, and complete data exchange based on TCP binary protocol. Said semantic modeling is based on data modeling, adding a semantic description layer, and describing in the form of "data-semantics-relationship". Semantic model client module is a service-oriented module based on semantics. It establishes a client for network communication, accesses the server module through ID, and obtains the parameter organization structure and service list based on semantic attribute information and XML file parsing.

Taking the robot controller and the industrial camera system as examples, the specific process includes: establishing a semantic model server module on the industrial camera controller side, and converting the camera parameter information such as: shooting function, adjustment of shooting parameters, basic information of the camera and other parameters Semantic modeling is used to describe semantically, and write it into the elements of the address space. The elements include hierarchical relationships such as element layer and sub-element layer. Relationships are established between elements to form an XML file of parameter organization structure and service list. The robot controller establishes a semantic model client module, and the client initiates a handshake with the server to establish a connection. Obtain the parameter organization structure and service list XML, parse the information of elements and sub-elements in the address space based on the semantic model, and finally obtain all the data of the camera's information model, so as to achieve the purpose of automatic adaptation. The robot controller sends the corresponding control command to the camera controller according to the adaptation result, the camera controller executes the sent command, and feeds back the position information to the robot controller. According to the position information, the robot controller combines the control algorithm module, The motion control module controls the robot body to execute the grab command.

The automatic adaptation method example of the present invention will be described below, and the specific steps are as follows:

Step 1: The camera controller establishes a semantic model server module, based on semantic modeling, adjusts the camera shooting functions (such as shooting shape, color recognition, storage function) and shooting parameters (such as exposure time, focus adjustment, shooting mode) ), the basic information of the camera (such as manufacturer, date of manufacture, maximum resolution) and other parameters are semantically described, and written into the elements and sub-elements of the address space; for example, functions and control commands: For example, the camera can shoot different shapes and Different colors, semantically describe colors and shapes at the element layer, describe shapes including circles, squares, etc., and colors such as red, yellow, etc., at their respective sub-element layers, and describe control commands such as shooting yellow circles Commands such as shape and red square are semantically described in the next sub-element layer. Then the parameter organization structure and service list are formed into an XML file.

Step 2: The robot controller sets the semantic model client module, initiates a handshake with the server, establishes communication, obtains the parameter organization structure and service list XML, parses the information of elements and sub-elements in the address space based on the semantic model, and finally obtains the camera's information. All data of the information model corresponds to the example of step 1. For example, after access, the control knows that the camera can shoot different shapes and colors, and various control commands, so that the controller and the camera can achieve the purpose of automatic adaptation.

Step 3: According to the functional requirements of the robot controller, such as shooting a yellow circle, the semantic model client module sends the semantic command of shooting the yellow circle obtained by parsing the semantics to the camera controller. After the camera controller obtains the command, the visual The function module starts to execute algorithms such as search and matching (there is a template such as a yellow circle in the camera controller, which is constantly searched. When the picture is exactly a yellow circle, it means that the match is reached, and the position information of the yellow circle will be calculated and given to the robot. controller), when the yellow circle is photographed, its position information is sent to the robot controller.

Step 4: The robot control calls the control algorithm module to perform calculation according to the position information, obtains the robot execution trajectory, and the motion control module controls the robot arm to perform the grasping task.

To sum up, the method of the present invention proposes a semantic model-based system and an automatic adaptation method for the needs of automatic adaptation of industrial cameras and robot control systems. The method provides convenience for the robot controller to flexibly control industrial cameras, plug and play, does not depend on specific models of controllers and cameras, its software is reusable, saves work time, is conducive to the expansion of multi-camera systems, and can achieve interconnection Interoperable self-organization decision-making.

Claims (7)

1. the automatic adaptation method of camera and robot control system based on semantic model, it is characterised in that: in camera control Semantization model service end module is established in device and establishes semantization model client end module in robot controller, passes through language The semantic modeling of justiceization model service end module and parsing with semantization model client end module, realize industrial camera with The automatic adaptation of robot control system, comprising the following steps:

Semantization model service end module carries out semantization description, writing address space according to camera parameter and camera control commands Element, in daughter element, parameter institutional framework and service list are then formed into XML file；

Semantization model client end module accesses the semantization model service end module in camera controller and obtains XML file, Parsing obtains camera parameter and camera control commands；Semantization corresponding to required camera parameter and camera control commands is ordered Order is sent to camera controller；

Semantization model service end module obtains semantization order, and camera controller executes operation corresponding to semantization order, And feed back robot information；Robot controller is controlled according to robot information.

2. the automatic adaptation method of the camera and robot control system according to claim 1 based on semantic model, institute Stating camera parameter includes camera shooting function, acquisition parameters adjusting, camera parameter information.

3. the automatic adaptation method of the camera and robot control system according to claim 1 based on semantic model, institute State the corresponding relationship that parameter institutional framework includes element, daughter element；Service list includes the content of element, daughter element.

4. the automatic adaptation method of the camera and robot control system according to claim 1 based on semantic model, institute It states and semantization description is carried out according to camera parameter and control command, include following step in the element in writing address space, daughter element It is rapid:

Using camera shooting function, acquisition parameters are adjusted, camera parameter information is as element writing address space；By each element Classification respectively as the daughter element under affiliated element；Movement, that is, camera control commands of certain corresponding daughter element are as the daughter element Under next stage daughter element.

5. the automatic adaptation method of the camera and robot control system according to claim 1 based on semantic model, institute State camera controller and carry out corresponding command operation, and feed back robot information the following steps are included:

Camera controller executes shooting action according to semantization order, and obtains the corresponding location information of shooting result.

6. the automatic adaptation method of the camera and robot control system according to claim 1 based on semantic model, instead After presenting robot information, robot controller resolves to obtain robot motion track according to location information, according to motion profile control Robot processed executes grasping movement.

7. camera and robot control system based on semantic model, which is characterized in that establish semantization in camera controller Model service end module and semantization model client end module is established in robot controller, comprising:

Semantization model service end module, for carrying out semantization description, write-in ground according to camera parameter and camera control commands In the element in location space, daughter element, parameter institutional framework and service list are then formed into XML file；Semantization order is obtained, So that camera controller is executed operation corresponding to semantization order and feeds back robot information to robot controller；

Semantization model client end module, for accessing the semantization model service end module in camera controller and obtaining XML File, parsing obtain camera parameter and camera control commands；By language corresponding to required camera parameter and camera control commands Justiceization orders the semantization model service end module for being sent to camera controller.