CN115883678B - Data processing system, method, device, equipment and medium - Google Patents
Data processing system, method, device, equipment and medium Download PDFInfo
- Publication number
- CN115883678B CN115883678B CN202310214847.9A CN202310214847A CN115883678B CN 115883678 B CN115883678 B CN 115883678B CN 202310214847 A CN202310214847 A CN 202310214847A CN 115883678 B CN115883678 B CN 115883678B
- Authority
- CN
- China
- Prior art keywords
- data
- application
- robot
- format
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 37
- 238000000034 method Methods 0.000 title claims abstract description 14
- 238000004891 communication Methods 0.000 claims abstract description 61
- 238000013500 data storage Methods 0.000 claims abstract description 35
- 238000003860 storage Methods 0.000 claims abstract description 18
- 238000013475 authorization Methods 0.000 claims description 30
- 238000013523 data management Methods 0.000 claims description 30
- 238000004140 cleaning Methods 0.000 claims description 22
- 238000003672 processing method Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 abstract description 3
- 238000007726 management method Methods 0.000 description 8
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 241001178520 Stomatepia mongo Species 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000009776 industrial production Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003319 supportive effect Effects 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Manipulator (AREA)
- Stored Programmes (AREA)
Abstract
The application discloses a data processing system, a method, a device, equipment and a medium, which are applied to the technical field of communication and specifically comprise the following steps: when the data receiving system receives robot operation data in a first data format sent by the robot based on a first communication protocol, the robot operation data are cleaned into a unified data format and then stored into a target database through the data storage system; when the data application system receives application demand data of a second data format sent by the data application client based on a second communication protocol, target robot operation data matched with the application demand data are obtained from a target database through the data storage system, and the target robot operation data are converted into the second data format and then sent to the data application client based on the second communication protocol, so that a standard flow and a unified mechanism of inflow, processing, storage and outflow of the robot operation data are established, and further the problems of data redundancy, poor compatibility, difficult use and the like are effectively solved.
Description
Technical Field
The present disclosure relates to the field of communications technologies, and in particular, to a data processing system, a method, an apparatus, a device, and a medium.
Background
At present, intelligent automation has penetrated into aspects of daily life and industrial production, robots replace manpower to engage in heavy work in many occasions, such as industrial robots like live working robots and household robots like sweeping robots, thereby bringing great convenience to aspects of daily life, industrial production and the like, and improving quality and efficiency of daily life and industrial production.
In the prior art, the robot generally sends the robot operation data to a data processing system in the form of message data for storage by various data application systems in the operation process, however, various message data formats are different along with the generation of a large amount of message data, so that the message data are difficult to use.
Disclosure of Invention
The application provides a data processing system, a method, a device, equipment and a medium, and specifically, the technical scheme provided by the application is as follows:
in one aspect, the present application provides a data processing system including a data application system, a data storage system, and a data receiving system; the data storage system is respectively in communication connection with the data receiving system and the data application system;
The data receiving system is used for receiving the robot operation data in the first data format sent by the robot based on the first communication protocol, cleaning the robot operation data in the first data format based on the data cleaning rule to obtain the robot operation data in the unified data format, and then sending the robot operation data in the unified data format to the data storage system;
the data storage system is used for receiving the robot operation data in the unified data format sent by the data receiving system and writing the robot operation data in the unified data format into the target database; receiving application demand data sent by a data application system, acquiring target robot job data in a unified data format matched with the application demand data from the robot job data in the unified data format stored in a target database, and returning the target robot job data in the unified data format to the data application system;
the data application system is used for receiving the application demand data in the second data format sent by the data application client based on the second communication protocol and sending the application demand data to the data storage system; and receiving target robot job data in a unified data format returned by the data storage system based on the application demand data, converting the target robot job data in the unified data format into target robot job data in a second data format, and then transmitting the target robot job data in the second data format to the data application client based on the second communication protocol.
In another aspect, the present application provides a data processing method, including:
when robot operation data in a first data format sent by a robot based on a first communication protocol is received, cleaning the robot operation data in the first data format based on a data cleaning rule to obtain robot operation data in a unified data format, and writing the robot operation data in the unified data format into a target database;
when receiving application demand data in a second data format sent by a data application client based on a second communication protocol, acquiring target robot job data in a unified data format matched with the application demand data from the robot job data in the unified data format stored in a target database;
and after the target robot job data in the unified data format are converted into the target robot job data in the second data format, the target robot job data in the second data format are sent to the data application client based on the second communication protocol.
In another aspect, the present application further provides a data processing apparatus, including:
the data receiving unit is used for cleaning the robot operation data in the first data format based on the data cleaning rule to obtain the robot operation data in the unified data format when receiving the robot operation data in the first data format sent by the robot based on the first communication protocol, and writing the robot operation data in the unified data format into the target database;
The data application unit is used for acquiring target robot operation data in a unified data format matched with the application demand data from the robot operation data in the unified data format stored in the target database when receiving the application demand data in the second data format sent by the data application client based on the second communication protocol;
the data return unit is used for transmitting the target robot job data in the second data format to the data application client based on the second communication protocol after converting the target robot job data in the unified data format into the target robot job data in the second data format.
In another aspect, the application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the data processing method described above when executing the computer program.
In another aspect, the present application also provides a computer readable storage medium storing computer instructions that when executed by a processor implement the above-described data processing method.
The beneficial effects of this application are as follows:
according to the method and the device, when the robot job data in the first data format sent by the robot based on the first communication protocol are received, the robot job data in the first data format are cleaned to the robot job data in the unified data format based on the data cleaning rule and then stored in the target database, when the application demand data in the second data format sent by the data application client based on the second communication protocol are received, the target robot job data in the unified data format matched with the application demand data are obtained from the target database and then converted to the target robot job data in the second data format, and the target robot job data are returned to the data application client, so that the standard flow and unified mechanism of inflow, processing, storage and outflow of the robot job data are established, and the problems of data redundancy, poor compatibility, difficulty in use and the like are effectively solved.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a schematic diagram of a data processing system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of another architecture of a data processing system in an embodiment of the present application;
FIG. 3 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart illustrating another overview of the data processing method according to the embodiment of the present application;
FIG. 5 is a schematic flow chart of another general method for data processing in the embodiment of the present application;
FIG. 6 is a schematic functional structure of a data processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic diagram of a hardware structure of an electronic device in an embodiment of the application.
Detailed Description
In order to make the objects, technical solutions and advantageous effects of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments, but not all embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
In order to facilitate a better understanding of the present application, technical terms related to the present application will be briefly described below.
The data center is used for collecting, calculating, storing and processing a large amount of data through a data technology, and forming standard data for storage after unifying standards and specifications, so that the data center is used as a big data production layer to provide high-efficiency data application and other services for a data application system.
Data cleansing refers to the discovery and correction of identifiable errors in data, including but not limited to checking for data consistency, processing invalid and missing values, and the like.
Message queues refer to queue-based messaging techniques, supportive software systems that provide synchronous or asynchronous, reliable message transmission to data application systems and data receiving systems in a network environment.
It should be noted that references to "first," "second," etc. in this application are for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that such terms are interchangeable under appropriate circumstances such that the embodiments described herein are capable of operation in other sequences than those illustrated or otherwise described herein. Furthermore, references to "and/or" in this application describe association relationships of associated objects, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
After technical terms related to the present application are introduced, a detailed description is given next to the technical solution provided in the present application.
The embodiment of the present application provides a data processing system, referring to fig. 1, where a data processing system 100 provided in the embodiment of the present application includes at least a data application system 101, a data storage system 102, and a data receiving system 103; the data storage system 102 is respectively in communication connection with the data receiving system 103 and the data application system 101;
The data receiving system 103 is configured to receive the robot job data in the first data format sent by the robot based on the first communication protocol, clean the robot job data in the first data format based on the data cleaning rule to obtain the robot job data in the unified data format, and then send the robot job data in the unified data format to the data storage system 102; wherein the first communication protocol is any one of TCP (Transmission Control Protocol ) and MQ (Message Queue);
a data storage system 102, configured to receive the robot job data in the unified data format sent by the data receiving system 103, and write the robot job data in the unified data format into a target database; receiving application demand data sent by the data application system 101, acquiring target robot job data in a unified data format matched with the application demand data from the robot job data in the unified data format stored in the target database, and returning the target robot job data in the unified data format to the data application system 101;
the data application system 101 is configured to receive application requirement data in a second data format sent by the data application client based on a second communication protocol, and send the application requirement data to the data storage system 102; and receiving target robot job data in a unified data format returned by the data storage system 102 based on the application demand data, converting the target robot job data in the unified data format into target robot job data in a second data format, and then transmitting the target robot job data in the second data format to the data application client based on the second communication protocol; wherein the second communication protocol is any one of TCP and MQ.
In one possible implementation, the data processing system 100 provided in the embodiments of the present application further includes a data management system 104; the data management system 104 is respectively in communication connection with the robot and the data application client;
the data management system 104 is configured to configure data pushing rights and data pushing keys for the robots and data application rights and data application keys for the data application clients.
In one possible implementation, the data management system 104 is communicatively coupled to the data receiving system 103;
the data management system 104 is configured to receive a push service authorization request sent by a robot, perform legal authentication on the robot based on a device identifier of the robot, generate a first push key and a second push key for the robot and a push permission identifier representing that the robot has data push permission when the legal authentication of the robot is determined to pass, send the first push key and the push permission identifier to the robot, and send the second push key to the data receiving system 103;
and the data receiving system 103 is configured to receive and store the second push key of the robot sent by the data management system 104, and when receiving the encrypted robot job data that is sent by the robot based on the first communication protocol and carries the push permission identifier and is encrypted by the first push key, determine that the robot has the data push permission based on the push permission identifier, decrypt the encrypted robot job data based on the second push key of the robot to obtain the robot job data in the first data format, clean the robot job data in the first data format based on the data cleaning rule to obtain the robot job data in the unified data format, and send the robot job data in the unified data format to the data storage system 102.
In one possible implementation, the data management system 104 is communicatively coupled to the data application system 101;
the data management system 104 is configured to receive an application service authorization request sent by the data application client, perform legal authentication on the data application client based on a client identifier of the data application client, generate a first application key and a second application key for the data application client and characterize an application authority identifier with data application authority when the legal authentication of the data application client is determined to pass, send the first application key and the application authority identifier to the data application client, and send the second application key to the data application system 101;
the data application system 101 is configured to receive and store a second application key of the data application client sent by the data management system 104, and when receiving encrypted application requirement data that is sent by the data application client based on a second communication protocol and carries an application permission identifier and is encrypted by the first application key, determine that the data application client has data application permission based on the application permission identifier, decrypt the encrypted application requirement data based on the second application key of the data application client to obtain application requirement data, and send the application requirement data to the data storage system 102; and receiving target robot job data returned by the data storage system 102 based on the application demand data, converting the target robot job data into a second data format, and then sending the second data format to the data application client based on a second communication protocol.
In this embodiment, referring to fig. 2, the data processing system 100 may be a data center, where the data center receives and transmits robot job data through MQ (for example kafka, rabbitmq, mqtt, etc.) and/or TCP, performs standardization processing on the received robot job data through an abnormal data cleaning service, and then flows into a target database such as a mongo database to store the received robot job data, and provides data application for a data application client through services such as data management, service authorization, gateway, etc. Specifically, the data center station is composed of four major parts, namely a data receiving system, a data storage system, a data management system and a data application system, wherein:
the data receiving system manages the whole butt joint process of robot data pushing, the robot operation data are butt-jointed through a standard MQ and/or TCP, the robot operation data are cleaned through a standard data cleaning rule, the pushed robot operation data are required to strictly conform to the data standards such as the MQ and/or TCP, the robot needs to acquire data pushing permission through service authorization, the robot operation data can be pushed only by acquiring the data pushing permission, and the robot operation data are further cleaned and filtered.
The data storage system stores the robot operation data cleaned by the data receiving system in a target database such as a mongo database, and stores the business data such as authority data and key data configured by the data management system in the target database such as a mysql database.
The data management system provides gateway service, call management service, data source management service, authorization service and the like; the gateway service provides a data query interface to the outside, and the data query needs permission authentication through the authorization service; the call management service calls log records, personnel authority configuration and the like aiming at the data query interface, and the data type authority configuration and the like can be used; the data source management service performs configuration management on each data source; the authorization service performs authorization control on data pushing, data application and the like, including interface calling, message queue use authorization and the like.
The data application system is responsible for the outflow and external use of robot operation data, and can use TCP interface service and MQ monitoring service to carry out application support of corresponding data through authority service authorization.
Based on the above data processing system 100 provided in the embodiment of the present application, the embodiment of the present application provides a data processing method, and referring to fig. 3, an overview flow of the data processing method provided in the embodiment of the present application is as follows:
Step 301: when robot operation data in a first data format sent by a robot based on a first communication protocol is received, cleaning the robot operation data in the first data format based on a data cleaning rule to obtain the robot operation data in a unified data format, and writing the robot operation data in the unified data format into a target database.
In practical application, the first communication protocol is any one of TCP and MQ (for example kafka, rabbitmq, mqtt, etc.), that is, data transmission can be performed between the robot and the data receiving system 103 through any one of TCP and MQ, in the implementation, before the robot pushes the operation data to the data receiving system 103, the robot can acquire the data pushing authority and the data pushing key through the data management system 104, specifically, the robot can send a push service authorization request to the data management system 104, when the data management system 104 receives the push service authorization request sent by the robot, based on the device identifier of the robot, legal authentication is performed on the robot, and when it is determined that legal authentication of the robot passes, the first pushing key and the second pushing key and the push authority identifier characterizing that the data pushing authority is provided are generated for the robot, the first pushing key and the push authority identifier are sent to the robot, and the second pushing key is sent to the data receiving system 103 to be saved, so far, the configuration of the data pushing authority and the data pushing key of the robot is completed, that the first pushing key and the second pushing key can be a symmetric key or a token or a key can be a symmetric key or a right token or the like can be obtained.
Accordingly, when the data receiving system 103 receives the encrypted robot operation data which is sent by the robot based on the first communication protocol and carries the pushing permission identification and is encrypted by the first pushing key, the permission authentication can be performed on the robot based on the pushing permission identification, after the data pushing permission of the robot is determined, the encrypted robot operation data is decrypted based on the second pushing key of the robot, so as to obtain the robot operation data in the first data format, and further, the robot operation data in the first data format is cleaned based on the data cleaning rule, so as to obtain the robot operation data in the unified data format, and then the robot operation data in the unified data format is sent to the data storage system 102, so that the robot operation data in the unified data format is written into the target database such as the mongo database through the data storage system 102.
Step 302: when receiving application demand data in a second data format sent by the data application client based on the second communication protocol, applying target robot job data in a unified data format matched with the application demand data from the robot job data in the unified data format stored in the target database.
In practical applications, the second communication protocol is any one of TCP and MQ (e.g. kafka, rabbitmq, mqtt, etc.), that is, data transmission can be performed between the data application client and the data application system 101 through any one of TCP and MQ, and in a specific implementation, the data application client can obtain the data application authority and the data application key through the data management system 104 before querying the data application system 101 for the robot job data. Specifically, the data application client may send an application service authorization request to the data management system 104, when the data management system 104 receives the application service authorization request sent by the data application client, based on the client identifier of the data application client, perform legal authentication on the data application client, and when determining that the legal authentication of the data application client passes, generate a first application key and a second application key for the data application client and characterize an application permission identifier with data application permission, send the first application key and the application permission identifier to the data application client, and send the second application key to the data application system 101 for storage, so far, the configuration of the data application permission and the data application key of the data application client is completed.
Correspondingly, when the data application system 101 receives the encrypted application demand data which is sent by the data application client based on the second communication protocol and carries the application permission identification and is encrypted by the first application key, permission authentication can be performed on the data application client based on the application permission identification, after the data application client is determined to have the data application permission, the encrypted application demand data is decrypted based on the second application key of the data application client to obtain the application demand data, and the application demand data is sent to the data storage system 102, so that the data storage system 102 obtains the target robot job data in the unified data format which is matched with the application demand data from the robot job data in the unified data format stored in the target database, and returns the target robot job data to the data application system 101.
Step 303: and after the target robot job data in the unified data format are converted into the target robot job data in the second data format, the target robot job data in the second data format are sent to the data application client based on the second communication protocol.
In practical application, after receiving the target robot job data returned by the data storage system 102 based on the application requirement data, the data storage system 102 may convert the target robot job data into a second data format and then send the second data format to the data application client based on the second communication protocol.
In this embodiment of the present application, the pushing of the robot operation data needs to obtain the authorization first, after the authorization passes through the data pushing authority, the message data including the robot operation data may be pushed by the encapsulated data pushing component, the robot operation data is stored in the target database such as the mongo database after being cleaned, the application of the robot operation data also needs to pass through the authorization, after the authorization passes through the data application authority, the robot operation data may be obtained by the gateway service, or the data pushing may be applied, and after the application passes, the robot operation data may be pushed to the designated queue. Specifically, referring to fig. 4, the robot job data pushing process includes: when the robot pushes the robot operation data for the first time, the request of push service authorization needs to be submitted, the data management system 104 configures data push permission, data push key and the like for the robot after the legal verification of the robot is passed, the robot needs to log in the message component management platform to acquire the data push permission, the data push key and the like, so that the message component pushes the robot operation data encrypted by the data push key, the data receiving system 103 verifies whether the robot has the data push permission, decrypts and cleans the data after the verification is passed, and the cleaned robot operation data is stored in the target database through the data storage system 102. Accordingly, referring to fig. 5, the robot job data application process includes: when the data application client first applies the robot job data, an application service authorization request needs to be submitted, after the data application client is validated and passed, the data management system 104 configures data application authority, a data application key and the like for the data application client, the data application client needs to log in a message component management platform to acquire the data application authority, the data application key and the like, thereby sending the application demand data encrypted by the data application key through a dispatching management component, the data application system 101 checks whether the data application client has the data application authority, after the data application client passes the verification, the data application client performs decryption and data query, and returns the queried target robot job data matched with the application demand data to the data application client, and the data application client calls or monitors the acquired target robot job data through a gateway interface. Therefore, a standard flow and a unified mechanism for inflow, processing, storage and outflow of the robot operation data are formed, and the problems of data redundancy, poor compatibility, difficult use and the like are effectively solved.
Based on the foregoing embodiments, the embodiments of the present application provide a data processing apparatus, and referring to fig. 6, the data processing apparatus 600 provided in the embodiments of the present application at least includes:
a data receiving unit 601, configured to, when receiving the robot job data in the first data format sent by the robot based on the first communication protocol, clean the robot job data in the first data format based on the data cleaning rule to obtain robot job data in a unified data format, and write the robot job data in the unified data format into the target database;
the data application unit 602 is configured to, when receiving application requirement data in a second data format sent by the data application client based on the second communication protocol, acquire target robot job data in a unified data format that matches the application requirement data from the robot job data in the unified data format stored in the target database;
the data returning unit 603 is configured to convert the target robot job data in the unified data format into target robot job data in the second data format, and then send the target robot job data in the second data format to the data application client based on the second communication protocol.
In one possible implementation manner, the data processing apparatus 600 provided in the embodiment of the present application further includes:
the first authorization unit 604 is configured to perform legal authentication on the robot based on the device identifier of the robot when receiving a push service authorization request sent by the robot; when the legal authentication of the robot is confirmed to pass, a first push key, a second push key and a push permission identifier with data push permission are generated for the robot, and the first push key and the push permission identifier are sent to the robot;
accordingly, when robot job data in a first data format sent by the robot based on the first communication protocol is received, the data receiving unit 601 is specifically configured to:
when encrypted robot operation data which is sent by the robot based on a first communication protocol and carries the pushing authority identification and is encrypted by the first pushing key is received, the encrypted robot operation data is decrypted based on a second pushing key of the robot after the fact that the robot has the data pushing authority is determined based on the pushing authority identification, and the robot operation data in a first data format is obtained.
In one possible implementation manner, the data processing apparatus 600 provided in the embodiment of the present application further includes:
A second authorization unit 605, configured to, when receiving an application service authorization request sent by a data application client, perform legal authentication on the data application client based on a client identifier of the data application client; when the legal authentication of the data application client passes, generating a first application key and a second application key for the data application client and representing an application authority identifier with data application authority, and sending the first application key and the application authority identifier to the data application client;
accordingly, when application requirement data in a second data format sent by the data application client based on the second communication protocol is received, the data application unit 602 is specifically configured to:
when receiving encrypted application demand data which is sent by the data application client based on the second communication protocol and carries the application permission identification and is encrypted by the first application key, after determining that the data application client has the data application permission based on the application permission identification, decrypting the encrypted application demand data based on the second application key of the data application client to obtain application demand data in a second data format.
It should be noted that, the principle of the data processing apparatus 600 provided in the embodiment of the present application to solve the technical problem is similar to that of the data processing method provided in the embodiment of the present application, so that the implementation of the data processing apparatus 600 provided in the embodiment of the present application may refer to the implementation of the data processing method provided in the embodiment of the present application, and the repetition is omitted.
Having described the data processing system, method and apparatus provided by the embodiments of the present application, a brief description of an electronic device provided by the embodiments of the present application follows.
Referring to fig. 7, an electronic device 700 provided in the embodiment of the present application at least includes a processor 701, a memory 702, and a computer program stored in the memory 702 and capable of running on the processor 701, where the processor 701 implements the data processing method provided in the embodiment of the present application when executing the computer program.
The electronic device 700 provided by the embodiments of the present application may also include a bus 703 that connects the different components, including the processor 701 and the memory 702. Bus 703 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and so forth.
The Memory 702 may include readable media in the form of volatile Memory, such as RAM (Random Access Memory ) 7021 and/or cache Memory 7022, and may further include ROM (Read Only Memory) 7023. The memory 702 may also include a program tool 7025 having a set (at least one) of program modules 7024, the program modules 7024 including, but not limited to: an operating subsystem, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The electronic device 700 may also communicate with one or more external devices 704 (e.g., keyboard, remote control, etc.), with one or more devices that enable a user to interact with the electronic device 700 (e.g., cell phone, computer, etc.), and/or with any device that enables the electronic device 700 to communicate with one or more other electronic devices 700 (e.g., router, modem, etc.). Such communication may occur through an I/O (Input/Output) interface 705. Also, the electronic device 700 may communicate with one or more networks (e.g., LAN (Local Area Network, local area network), WAN (Wide Area Network ) and/or public network, such as the Internet) through the network adapter 706. As shown in fig. 7, the network adapter 706 communicates with other modules of the electronic device 700 via the bus 703. It should be appreciated that although not shown in fig. 7, other hardware and/or software modules may be used in connection with electronic device 700, including, but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (Redundant Arrays of Independent Disks, disk array) subsystems, tape drives, data backup storage subsystems, and the like.
It should be noted that, the electronic device 700 shown in fig. 7 is only an example, and should not impose any limitation on the functions and application scope of the embodiments of the present application.
In addition, the embodiment of the application also provides a computer readable storage medium, and the computer readable storage medium stores computer instructions which are executed by a processor to implement the data processing method provided by the embodiment of the application. Specifically, the computer instructions may be built into or installed in the electronic device, so that the electronic device may implement the data processing method provided in the embodiments of the present application by executing the built-in or installed computer instructions.
In addition, the data processing method provided in the embodiment of the present application may also be implemented as a program product, where the program product includes a program code, and when the program code runs on a processor, the data processing method provided in the embodiment of the present application is implemented.
The program product provided by the embodiments of the present application may employ any combination of one or more readable media, where the readable media may be a readable signal medium or a readable storage medium, and the readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof, and more specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, RAM, ROM, EPROM (Erasable Programmable Read Only Memory, erasable programmable read-Only Memory), an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The program product provided by the embodiment of the application can adopt a CD-ROM and comprises program codes, and can also run on the electronic device. However, the program product provided by the embodiments of the present application is not limited thereto, and in the embodiments of the present application, the readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more of the elements described above may be embodied in one element in accordance with embodiments of the present application. Conversely, the features and functions of one unit described above may be further divided into a plurality of units to be embodied.
Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required to or suggested that these operations must be performed in this particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to encompass such modifications and variations.
Claims (9)
1. A data processing system comprising a data application system, a data storage system, and a data receiving system; the data storage system is respectively in communication connection with the data receiving system and the data application system;
the data receiving system is used for receiving the robot operation data in the first data format sent by the robot based on the first communication protocol, cleaning the robot operation data in the first data format based on the data cleaning rule to obtain the robot operation data in the unified data format, and then sending the robot operation data in the unified data format to the data storage system;
The data storage system is used for receiving the robot operation data in the unified data format sent by the data receiving system and writing the robot operation data in the unified data format into a target database; receiving application demand data sent by the data application system, acquiring target robot job data in a unified data format matched with the application demand data from the robot job data in the unified data format stored in the target database, and returning the target robot job data in the unified data format to the data application system;
the data application system is used for receiving application demand data in a second data format sent by the data application client based on a second communication protocol and sending the application demand data to the data storage system; receiving target robot job data in the unified data format returned by the data storage system based on the application demand data, converting the target robot job data in the unified data format into target robot job data in the second data format, and then sending the target robot job data in the second data format to the data application client based on the second communication protocol;
Wherein the data processing system further comprises a data management system; the data management system is respectively in communication connection with the robot and the data application client;
the data management system is used for configuring data pushing authority and data pushing key for the robot and configuring data application authority and data application key for the data application client.
2. The data processing system of claim 1, wherein the data management system is communicatively coupled to the data receiving system;
the data management system is used for receiving a push service authorization request sent by the robot, carrying out legal authentication on the robot based on the equipment identifier of the robot, generating a first push key and a second push key and a push permission identifier representing a data push permission for the robot when the legal authentication of the robot is confirmed to be passed, sending the first push key and the push permission identifier to the robot, and sending the second push key to the data receiving system;
the data receiving system is configured to receive and store the second push key of the robot sent by the data management system, and when receiving encrypted robot job data that is sent by the robot based on the first communication protocol and carries the push permission identifier and is encrypted by the first push key, determine that the robot has data push permission based on the push permission identifier, decrypt the encrypted robot job data based on the second push key of the robot to obtain robot job data in a first data format, and clean the first robot job data in a first data format based on a data cleaning rule to obtain robot job data in a unified data format, and then send the robot job data in the unified data format to the data storage system.
3. The data processing system of claim 1, wherein the data management system is communicatively coupled to the data application system;
the data management system is used for receiving an application service authorization request sent by the data application client, carrying out legal authentication on the data application client based on a client identifier of the data application client, generating a first application key and a second application key and an application authority identifier representing data application authority for the data application client when the legal authentication of the data application client is confirmed to pass, sending the first application key and the application authority identifier to the data application client, and sending the second application key to the data application system;
the data application system is configured to receive and store the second application key of the data application client sent by the data management system, and decrypt the encrypted application requirement data based on the second application key of the data application client after determining that the data application client has a data application right based on the application right identifier when receiving the encrypted application requirement data which is sent by the data application client based on the second communication protocol and carries the application right identifier and is encrypted by the first application key, and send the application requirement data to the data storage system; and receiving the target robot job data returned by the data storage system based on the application demand data, converting the target robot job data into the second data format, and then sending the second data format to the data application client based on the second communication protocol.
4. A method of data processing, comprising:
when robot operation data in a first data format sent by a robot based on a first communication protocol is received, cleaning the robot operation data in the first data format based on a data cleaning rule to obtain robot operation data in a unified data format, and writing the robot operation data in the unified data format into a target database;
when receiving application demand data in a second data format sent by a data application client based on a second communication protocol, acquiring target robot operation data in a unified data format matched with the application demand data from the robot operation data in the unified data format stored in the target database;
after the target robot job data in the unified data format are converted into the target robot job data in the second data format, the target robot job data in the second data format are sent to the data application client based on the second communication protocol;
wherein, the data processing method further comprises:
and configuring data pushing authority and data pushing key for the robot and configuring data application authority and data application key for the data application client.
5. The data processing method of claim 4, further comprising:
when receiving a push service authorization request sent by the robot, performing legal authentication on the robot based on the equipment identifier of the robot;
when the legal authentication of the robot is confirmed to pass, generating a first push key and a second push key for the robot and representing a push permission identifier with data push permission, and sending the first push key and the push permission identifier to the robot;
correspondingly, the receiving the robot job data in the first data format sent by the robot based on the first communication protocol includes:
when the encrypted robot operation data which is sent by the robot based on the first communication protocol and carries the pushing authority identification and is encrypted by the first pushing key is received, the encrypted robot operation data is decrypted based on the second pushing key of the robot after the fact that the robot has the data pushing authority is determined based on the pushing authority identification, so that the robot operation data in the first data format is obtained.
6. The data processing method of claim 4, further comprising:
When receiving an application service authorization request sent by the data application client, performing legal authentication on the data application client based on the client identifier of the data application client;
when the legal authentication of the data application client is confirmed to pass, a first application key and a second application key and an application permission identifier with data application permission are generated for the data application client, and the first application key and the application permission identifier are sent to the data application client;
accordingly, receiving the application requirement data in the second data format sent by the data application client based on the second communication protocol includes:
and when receiving the encrypted application demand data which is sent by the data application client based on the second communication protocol and carries the application permission identification and is encrypted by the first application key, decrypting the encrypted application demand data based on the second application key of the data application client after determining that the data application client has the data application permission based on the application permission identification, so as to obtain the application demand data in the second data format.
7. A data processing apparatus, comprising:
the data receiving unit is used for cleaning the robot operation data in the first data format based on a data cleaning rule to obtain the robot operation data in the unified data format when receiving the robot operation data in the first data format sent by the robot based on the first communication protocol, and writing the robot operation data in the unified data format into the target database;
the data application unit is used for acquiring target robot operation data in a unified data format matched with the application demand data from the robot operation data in the unified data format stored in the target database when receiving the application demand data in the second data format sent by the data application client based on the second communication protocol;
the data return unit is used for transmitting the target robot job data in the second data format to the data application client based on the second communication protocol after converting the target robot job data in the unified data format into the target robot job data in the second data format;
wherein the data processing apparatus further comprises:
The first authorization unit is used for configuring data pushing authority and data pushing keys for the robot;
and the second authorization unit is used for configuring data application authority and data application keys for the data application client.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the data processing method according to any of claims 4-6 when executing the computer program.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which when executed by a processor, implement the data processing method according to any of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310214847.9A CN115883678B (en) | 2023-03-08 | 2023-03-08 | Data processing system, method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310214847.9A CN115883678B (en) | 2023-03-08 | 2023-03-08 | Data processing system, method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115883678A CN115883678A (en) | 2023-03-31 |
| CN115883678B true CN115883678B (en) | 2023-06-16 |
Family
ID=85762039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310214847.9A Active CN115883678B (en) | 2023-03-08 | 2023-03-08 | Data processing system, method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115883678B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2211309A1 (en) * | 2001-01-12 | 2010-07-28 | Nippon Telegraph and Telephone Corporation | Authentication token and authentication system |
| CN110730066A (en) * | 2019-08-30 | 2020-01-24 | 深圳壹账通智能科技有限公司 | Network encryption and decryption method and device for data, storage medium and electronic device |
| CN112597250A (en) * | 2020-12-29 | 2021-04-02 | 广西交控智维科技发展有限公司 | Track traffic data relay station implementation method based on DataX data synchronization |
| CN112995022A (en) * | 2021-02-23 | 2021-06-18 | 中工互联(北京)智能科技有限公司 | Industrial Internet of things gateway |
| CN113162942A (en) * | 2021-04-27 | 2021-07-23 | 深圳市伊起来科技有限公司 | Method for forwarding self-adaptive protocol |
| CN113992646A (en) * | 2021-12-29 | 2022-01-28 | 天津市津科拓达科技有限责任公司 | Internet of things equipment protocol integration method and system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107656704A (en) * | 2017-09-28 | 2018-02-02 | 郑州云海信息技术有限公司 | Multi-protocol data shares storage method, device, equipment and computer-readable storage medium |
| CN109547474A (en) * | 2018-12-21 | 2019-03-29 | 深圳市创梦天地科技有限公司 | A kind of data transmission method and device |
| CN111212097A (en) * | 2020-04-21 | 2020-05-29 | 炬星科技（深圳）有限公司 | Method, device and storage medium for robot to access server |
| CN113542301B (en) * | 2021-07-30 | 2023-06-02 | 深圳追一科技有限公司 | Interaction method and device, electronic equipment and storage medium |
| CN114036216A (en) * | 2021-10-29 | 2022-02-11 | 中国联合网络通信集团有限公司 | Data processing method, device and storage medium |
| CN114679446B (en) * | 2022-03-28 | 2024-06-11 | 阿里云计算有限公司 | Data processing method, system and data processing equipment |
| CN115237987A (en) * | 2022-06-30 | 2022-10-25 | 国汽大有时空科技(安庆)有限公司 | Vehicle multi-source heterogeneous data access and processing method, module and system |
-
2023
- 2023-03-08 CN CN202310214847.9A patent/CN115883678B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2211309A1 (en) * | 2001-01-12 | 2010-07-28 | Nippon Telegraph and Telephone Corporation | Authentication token and authentication system |
| CN110730066A (en) * | 2019-08-30 | 2020-01-24 | 深圳壹账通智能科技有限公司 | Network encryption and decryption method and device for data, storage medium and electronic device |
| CN112597250A (en) * | 2020-12-29 | 2021-04-02 | 广西交控智维科技发展有限公司 | Track traffic data relay station implementation method based on DataX data synchronization |
| CN112995022A (en) * | 2021-02-23 | 2021-06-18 | 中工互联(北京)智能科技有限公司 | Industrial Internet of things gateway |
| CN113162942A (en) * | 2021-04-27 | 2021-07-23 | 深圳市伊起来科技有限公司 | Method for forwarding self-adaptive protocol |
| CN113992646A (en) * | 2021-12-29 | 2022-01-28 | 天津市津科拓达科技有限责任公司 | Internet of things equipment protocol integration method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115883678A (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8805596B2 (en) | Remote wind turbine reset system and method | |
| CN102097859A (en) | Power grid dispatching intelligentized and instruction informationized working platform | |
| CN111083153A (en) | Service access method, device and equipment between medical interfaces and readable storage medium | |
| TW515953B (en) | Method and apparatus for proprietary data collection and distribution | |
| KR101453671B1 (en) | Integrated interface system for monitoring and controling electric power system | |
| CN115967175B (en) | Edge data acquisition control device and method for energy storage power station | |
| CN115883678B (en) | Data processing system, method, device, equipment and medium | |
| CN117527873B (en) | Task processing method and system based on industrial Internet | |
| CN117743183A (en) | Business process testing method and device, electronic equipment and storage medium | |
| CN118175214A (en) | Data protocol converter | |
| CN117910787A (en) | Camunda workflow engine-based multi-tenant workflow system | |
| CN1973282A (en) | System and method for managing a change to a cluster configuration | |
| CN110290119B (en) | JSON message encryption method, decryption method and device | |
| KR100684445B1 (en) | Interworking system and method for interworking between heterogeneous databases | |
| JP2011096250A (en) | Method, system and computer-readable medium for comparing utility and warranty of services | |
| CN113179319A (en) | System for integrating monitoring data of nuclear power station key equipment | |
| CN116192853B (en) | Multi-cluster management method | |
| CN112948860B (en) | Data processing method, related node and medium | |
| CN111031132A (en) | Industrial Internet system architecture based on standard communication protocol and implementation method thereof | |
| CN221614989U (en) | Networking protection and monitoring system | |
| CN114683909B (en) | Method, system, equipment and readable storage medium for processing power change | |
| JP7661786B2 (en) | Control device, logging method and program | |
| CN116074080B (en) | Data processing method and platform of satellite ground station management system | |
| CN114860814A (en) | Credit investigation processing system, method, credit investigation system, device and medium | |
| KR100573060B1 (en) | How to match line information between dedicated line management systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |