CN115022045B - Data processing method and system based on edge cloud - Google Patents
Data processing method and system based on edge cloud Download PDFInfo
- Publication number
- CN115022045B CN115022045B CN202210624353.3A CN202210624353A CN115022045B CN 115022045 B CN115022045 B CN 115022045B CN 202210624353 A CN202210624353 A CN 202210624353A CN 115022045 B CN115022045 B CN 115022045B
- Authority
- CN
- China
- Prior art keywords
- data
- operation data
- job
- edge cloud
- rule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title abstract description 8
- 230000005540 biological transmission Effects 0.000 claims abstract description 87
- 238000012795 verification Methods 0.000 claims abstract description 71
- 238000009499 grossing Methods 0.000 claims abstract description 45
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 23
- 230000001105 regulatory effect Effects 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims description 18
- 238000004458 analytical method Methods 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000001276 controlling effect Effects 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000008030 elimination Effects 0.000 claims description 4
- 238000003379 elimination reaction Methods 0.000 claims description 4
- 230000008033 biological extinction Effects 0.000 claims description 3
- 238000011161 development Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000003111 delayed effect Effects 0.000 description 3
- 208000032370 Secondary transmission Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 229910002056 binary alloy Inorganic materials 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0209—Architectural arrangements, e.g. perimeter networks or demilitarized zones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention relates to the technical field of data processing, in particular to a data processing method and system based on edge cloud. According to the invention, the conventional information comparison method is replaced by the validity verification of the operation data, so that the data security of the edge cloud is ensured on one hand, and the efficiency of the validity verification of the data is improved on the other hand. In addition, the data validity verification based on the exponential smoothing algorithm can effectively avoid the situation that the vulnerability propagation in the data transmission is unfavorable for the practical development of lawless persons. And the illegal operation data is recorded and stored in a transmission source and mode, so that the photo, the characters and the numbers can be queried at any time, the illegal data is eliminated, the secondary propagation of the illegal data is prevented, and the security of data propagation is effectively enhanced. On the basis, redundancy of the operation data is eliminated, the operation data is regulated and controlled, and a proper port is allocated for the operation data, so that the data legal verification efficiency can be further improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data processing method and system based on edge cloud.
Background
The edge cloud computing, abbreviated as edge cloud, is a cloud computing platform based on the core of cloud computing technology and the capability of edge computing, is built on an edge infrastructure, forms an elastic cloud platform with comprehensive capabilities of computing, networking, storage, and the like of edge positions, and forms an end-to-end technical architecture of 'cloud edge end three-body coordination' with a central cloud and an Internet of things terminal.
In the prior art, a patent with publication number CN113572821a discloses a method and a system for collaborative processing of edge cloud node tasks, comprising: constructing an edge cloud node pool comprising a mobile edge cloud and a network edge cloud, and acquiring time delay topology of the mobile edge cloud and the network edge cloud and edge cloud node resources; receiving terminal service; according to the attribute of the terminal service and the service processing time delay, node resources and service processing benefits of the edge cloud nodes, the optimal edge cloud nodes or the optimal edge cloud node groups in the edge cloud node pool are selected, so that the terminal service is processed, the node resources of the mobile edge cloud and the network edge cloud are integrated, the optimal edge cloud nodes or the optimal edge cloud node groups are selected based on the service processing time delay, the node resources, the service processing benefits and the like of the edge cloud nodes, and the requirements of user service requirements and cost reduction and efficiency improvement of operators are met while the edge cloud resources are effectively utilized.
With the continuous development of cloud technology, the existing edge cloud is basically perfect, cloud computing can be well cooperatively processed, response time delay is reduced, and cloud pressure is relieved. However, the increasing number of edge cloud applications and the shortcomings of edge cloud are also apparent, and particularly, the potential safety hazard exists in the process of executing data and transmitting data by the edge cloud. Since most of data received by the edge cloud is from the outside, the legitimacy of the data and the legitimacy of data transmission cannot be guaranteed, and therefore the edge cloud needs to perform legitimacy verification on the data before calculating the data so as to confirm the type, the source and the authority of the data. The conventional validity verification generally adopts an information comparison method to compare non-uniform format data such as photos, characters, data and the like one by one and judge whether the data is legal, and the comparison method not only occupies large system resources and has low efficiency, but also has the problems of data transmission loss and the like. The conventional edge cloud cannot rapidly verify the legality of massive data with non-uniform formats, so that the data processing efficiency is low, and the problem to be solved in the technical field of data processing is solved. At present, a data processing method and system capable of improving the legal verification efficiency of edge cloud data based on edge cloud are needed.
Disclosure of Invention
The invention aims to overcome at least one defect of the prior art, and provides a data processing method and system based on edge cloud, which are used for solving the problem of low legal verification efficiency of edge cloud data.
The technical scheme adopted by the invention is as follows:
a data processing method based on edge cloud, comprising:
receiving job data transmitted by a central cloud;
verifying the validity of the operation data;
if the operation data do not pass the validity verification, the operation data transmitted by the center cloud are received again;
if the operation data passes the validity verification, analyzing the operation data, and calculating the data quantity and the expected transmission time of the operation data;
removing redundancy of the operation data according to the analysis result of the operation data;
compressing and decomposing the operation data with redundancy removed, and regulating and controlling the decomposed operation data to a corresponding port according to the data quantity of the operation data and the predicted transmission time;
and transmitting the job data at the port to a terminal server.
As a further aspect of the present invention, the performing validity verification on the job data includes:
performing data validity verification on the operation data based on an exponential smoothing algorithm;
if the job data does not pass the data validity verification, judging that the job data does not pass the data validity verification, deleting the job data, and storing a transmission record of the job data;
if the operation data passes the data validity verification, the operation data is subjected to transmission validity verification;
if the job data does not pass the transmission validity verification, judging that the job data does not pass the validity verification, and returning the job data to a transmitting place;
and if the operation data passes the transmission validity verification, judging that the operation data passes the validity verification.
As a further aspect of the present invention, the performing data validity verification on the job data based on an exponential smoothing algorithm includes:
establishing a detection model based on an exponential smoothing algorithm, and setting rule data; the rule data is data conforming to a legal rule;
calculating the job data and the rule data by using the detection model;
and comparing the calculation results of the job data and the rule data, thereby verifying the data validity of the job data.
As a further aspect of the present invention, the formula of the exponential smoothing algorithm is:
Y t+1 ＝αX t +α(1-α)X t-1 +α(1-α) 2 X t-2 +…+α(1-α) n X t-n
wherein Y is t+1 For the predicted value of the operation/rule data, alpha is a weight coefficient, n is a smooth index, t is the observation period number of the operation/rule data, and X is the observation value of the operation/rule data.
As a further aspect of the present invention, after transmitting the job data located at a port to a terminal server, the method further includes:
judging whether the terminal server receives the operation data;
if the terminal server does not receive the operation data within the expected transmission time, retransmitting the operation data to the terminal server;
and if the terminal server receives the job data within the expected transmission time, ending the job data transmission.
An edge cloud-based data processing system, comprising:
the edge cloud input module is used for receiving the operation data transmitted by the center cloud;
the edge cloud monitoring module is used for verifying the validity of the operation data;
the edge cloud analysis module is used for analyzing the operation data passing the validity verification and calculating the data quantity and the expected transmission time of the operation data;
the edge cloud computing module is used for eliminating redundancy of the operation data according to the analysis result of the operation data;
the edge cloud regulation and control module is used for compressing and decomposing the operation data with redundant elimination, and regulating and controlling the decomposed operation data to corresponding ports according to the data quantity and the expected transmission time of the operation data;
and the edge cloud output module is used for transmitting the operation data positioned at the port to the terminal server.
As a further aspect of the present invention, the edge cloud monitoring module includes:
the data legal unit is used for verifying the data validity of the operation data based on an exponential smoothing algorithm;
an extinction recording unit configured to delete the job data that has not passed the data validity verification, and to save a transmission record of the job data;
the transmission legal unit is used for carrying out transmission legal verification on the operation data passing through the data legal verification;
and the data return unit is used for returning the job data which does not pass the verification of the transmission validity to a sending place.
As a further aspect of the present invention, the data legal unit includes:
an initialization mechanism for establishing a detection model based on an exponential smoothing algorithm and setting rule data; the rule data is data conforming to a legal rule;
a detection model mechanism for calculating the job data and the rule data using the detection model;
and the data comparison mechanism is used for comparing the calculation results of the job data and the rule data so as to verify the data validity of the job data.
As a further aspect of the present invention, the formula of the exponential smoothing algorithm is:
Y t+1 ＝αX t +α(1-α)X t-1 +α(1-α) 2 X t-2 +…+α(1-α) n X t-n
wherein Y is t+1 For the predicted value of the operation/rule data, alpha is a weight coefficient, n is a smooth index, t is the observation period number of the operation/rule data, and X is the observation value of the operation/rule data.
As a further aspect of the present invention, the method further comprises:
and the edge cloud delay module is used for retransmitting the job data which is not received by the terminal server in the expected transmission time.
Compared with the prior art, the invention has the beneficial effects that: according to the method, the conventional information comparison method is replaced by the validity verification of the operation data, so that the data security of the edge cloud is guaranteed on one hand, and the efficiency of the validity verification of the data is improved on the other hand. In addition, the data validity verification based on the exponential smoothing algorithm can effectively avoid the situation that the vulnerability propagation in the data transmission is unfavorable for the practical development of lawless persons. And the illegal operation data is recorded and stored in a transmission source and mode, so that the photo, the characters and the numbers can be queried at any time, the illegal data is eliminated, the secondary propagation of the illegal data is prevented, and the security of data propagation is effectively enhanced. On the basis, redundancy of the operation data is eliminated, the operation data is regulated and controlled, and a proper port is allocated for the operation data, so that the data legal verification efficiency can be further improved.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a schematic view of an edge cloud of the present invention;
FIG. 3 is a schematic diagram of the system of the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the invention. For better illustration of the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the actual product dimensions; it will be appreciated by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
Examples
As shown in fig. 1, this embodiment provides a data processing method based on edge cloud, including:
s10, receiving operation data transmitted by a central cloud;
as shown in fig. 2, in this embodiment, the center cloud is communicatively connected to the edge cloud, and the edge cloud is communicatively connected to the terminal server. The center cloud transmits the operation data to the edge cloud, and the edge cloud processes the operation data and provides the operation data to the terminal server. The job data is a computer instruction, and the edge cloud is used for executing the job data to provide forwarding, storage and computing services for users. The center cloud comprises an internet data center and the edge cloud comprises a domain controller. The domain controller comprises a domain name resolver and a domain name server, and the Internet data center is respectively in communication connection with the domain name resolver and the domain name server. The terminal server is respectively in communication connection with the domain name resolver and the domain name server.
The edge cloud also includes DNS and dynamic DNS. The DNS is respectively in communication connection with the domain name resolver and the domain name server, and analyzes the domain name resolver and the domain name server. The dynamic DNS is in communication connection with the domain name resolver and the domain name server, and analyzes the domain name resolver and the domain name server.
The operation data sent by the user passes through the Internet data center, and at the moment, the center cloud performs preliminary cloud computing on the operation data. After the center cloud performs simple calculation on the operation data, the operation data passes through the domain controller, at the moment, the domain name resolver and the domain name server in the domain controller analyze the access IP address by utilizing the DNS and the dynamic DNS, so that further processing of the transmission data by the terminal server is realized, when the operation data is primarily processed by the Internet data center, the city backbone network is adopted for domestic data, the wide area network is adopted for data transmission for foreign data, and reasonable interval division is performed on the data transmission interval, so that the effect that the data transmission rate of the edge cloud is accelerated in the operation data processing process is realized.
S20, verifying the validity of the operation data;
the validity verification includes: data validity verification and transmission validity verification. Judging whether the data is legal or not, and judging the legitimacy of the data and the legitimacy of the data transmission process in batches. Data types include, but are not limited to, photographs, text, and numbers. The existing judging mode is to compare the photo, the text and the number with the data stored in the edge cloud so as to distinguish illegal data. If the edge cloud uses the existing judging mode, comparing and identifying the non-uniform format data such as photos, characters, numbers and the like one by one, the operation of identifying can occupy a large amount of computing resources, and the edge cloud server cannot process massive comparison data in time. To this end, the present invention employs an exponential smoothing algorithm to solve this problem.
As a further aspect of the present invention, the performing validity verification on the job data includes:
performing data validity verification on the operation data based on an exponential smoothing algorithm;
if the job data does not pass the data validity verification, judging that the job data does not pass the data validity verification, deleting the job data, and storing a transmission record of the job data;
if the operation data passes the data validity verification, the operation data is subjected to transmission validity verification;
if the job data does not pass the transmission validity verification, judging that the job data does not pass the validity verification, and returning the job data to a transmitting place;
and if the operation data passes the transmission validity verification, judging that the operation data passes the validity verification.
In the process of validity verification, for the situation that the operation data is legal but the operation data transmission process is illegal, the edge cloud returns the operation data to the user of data transmission, and the operation data is re-transmitted until the data transmission process is that the operation data is transmitted through a legal channel, the successful data transmission is displayed, and the situation that the vulnerability propagation in the data transmission is not beneficial to practical development by using an lawless person is effectively avoided. And recording and storing transmission sources and modes of the illegal operation data so as to inquire the photo, the characters and the numbers at any time, eliminate the illegal data, prevent the secondary propagation of the illegal data and further strengthen the security of data propagation.
As a further aspect of the present invention, the data validity verification is performed on the job data based on an exponential smoothing algorithm, including the steps of:
establishing a detection model based on an exponential smoothing algorithm; and setting rule data; the rule data is data conforming to a legal rule;
the exponential smoothing algorithm is a medium-short term time sequence data trend prediction algorithm which is compatible with the length of a full-term average and a moving average, and is characterized in that the past observations are given different weights, namely the weights of the more recent observations are larger than the weights of the long-term observations. According to different smoothing times, the exponential smoothing algorithm is divided into a primary exponential smoothing method, a secondary exponential smoothing method, a tertiary exponential smoothing method and the like. The basic ideas of the exponential smoothing algorithm are: the predicted value is a weighted sum of previous observations and different weights are given to different data, with new data given a larger weight and old data given a smaller weight.
The formula of the exponential smoothing algorithm is as follows:
Y t+1 ＝αX t +α(1-α)X t-1 +α(1-α) 2 X t-2 +…+α(1-α) n X t-n (1.1)
wherein Y is t+1 For the predicted value of the operation/rule data, alpha is a weight coefficient, n is a smooth index, t is the observation period number of the operation/rule data, and X is the observation value of the operation/rule data.
The exponential smoothing value in any period is the weighted average of the actual observed value in the present period and the exponential smoothing value in the previous period, the formula (1.1) is deduced, and the exponential smoothing value is smoothed again on the basis of primary exponential smoothing to obtain a secondary exponential smoothing formula:
in the formulae (1.2) and (1.3),for the first smoothed value of phase t, +.>Is the second smoothed value of the t-th stage, X t For the t-th-period job/rule observation, t=1, 2,3, …, n (n is the number of raw data).
The parameters of the above secondary exponential smoothing formula include:
calculating the job data and the rule data by using the detection model;
as a preferable mode of the present embodiment, job data of different formats, i.e., photographs, letters, numerals, etc., inputted into the detection model are converted into binary numbers to be sequentially developed. The expanded job data is then partitioned and arranged, and each segment is marked according to a time sequence. Setting the observation period number of the expanded operation data as t, and setting the observation value of each observation period of the operation data as X t 、X t-1 、X t-2 …、X t-n . Data outside the observed values are ignored, thereby reducing the amount of data that needs to be compared. And carrying out unequal right processing according with actual conditions on the data at different times, and dividing the data into different smoothing grades according to the smoothing times. Substituting the operation data observation value and the operation data observation period number into a detection model established based on an exponential smoothing algorithm, and calculatingTo obtain the corresponding secondary exponential smoothing parameter alpha t And b t 。
And similarly, converting the rule data into binary system, sequentially expanding, dividing and arranging the expanded rule data, and marking each segment according to the time sequence. Setting the observation period number of the expanded rule data to t (consistent with the observation period number of the job data), and setting the observation value of each observation period of the rule data to P t 、P t-1 、P t-2 …、P t-n . Data outside the observed values are ignored, thereby reducing the amount of data that needs to be compared. And carrying out unequal right processing according with actual conditions on the data at different times, and dividing the data into different smoothing grades according to the smoothing times. Substituting the rule data observation value and the rule data observation period number into a detection model established based on an exponential smoothing algorithm to calculate a corresponding secondary exponential smoothing parameter Qalpha t And Qb t 。
And comparing the calculation results of the job data and the rule data, thereby verifying the data validity of the job data.
Respectively the parameter alpha t Sum parameter qα t Parameter b t Sum parameter Qb t Comparison was performed. If the parameter alpha t Sum parameter qα t Consistent, and parameter b t Sum parameter Qb t And if the operation data of the part corresponding to the proving parameter is identical to the rule data and accords with the legal rule, judging that the part of operation data passes the data legal verification. If the parameter alpha t Sum parameter qα t Inconsistencies, or/and parameters b t Sum parameter Qb t And if the operation data of the corresponding part of the parameters are inconsistent with the rule data and do not accord with the legal rule, judging that the operation data of the part does not pass the data legal verification.
As a preferable scheme of the embodiment, the front part and the middle part or the middle part and the last part of the expanded operation data/rule data can be extracted for calculation comparison of the observation value extracted by the operation data/rule data, the length of the data is not required to be considered, and the observation period t of the operation data and the rule data is only required to be consistent. And comparing the operation data with a plurality of parameters of the rule data for a plurality of times, and judging that the whole operation data passes the verification of data validity as long as 80% of comparison results are consistent.
S30, if the operation data do not pass the validity verification, the operation data transmitted by the center cloud are received again;
s40, if the operation data passes the validity verification, analyzing the operation data, and calculating the data quantity and the expected transmission time of the operation data;
specifically, analysis of the job data obtains the type, the substantial content and the resource requirement of the job data. According to the type, the substantial content and the resource requirement of the data operation, the operation data are deployed to the corresponding virtual machine for execution, so that the execution efficiency is improved, the data quantity and the expected transmission time of the operation data are calculated, parameters are provided for subsequent port regulation and delay transmission, the phenomenon that the data transmission is invalid and the data transmission cannot be completed in time is effectively prevented. For example: and setting the expected transmission time to be 10S for the 20MB job data, and considering that the transmission fails if the expected transmission time exceeds the expected transmission time, and delaying the secondary transmission of the job data by the edge cloud after the transmission failure to prevent the data transmission failure.
S50, eliminating redundancy of the operation data according to the analysis result of the operation data;
specifically, the edge cloud analyzes the essential content from the job data, and the error and repeated parts of the essential content are removed, so that the time for data processing is effectively shortened. For example: and (3) analyzing and calculating a plurality of groups of repeated photos appearing in the transmission data to reserve the photos meeting the transmission requirement, and automatically rejecting the photos acquired by illegal ways in the transmission process.
S60, compressing and decomposing the operation data with redundant elimination, and regulating and controlling the decomposed operation data to corresponding ports according to the data quantity and the expected transmission time of the operation data;
specifically, the method and the device have the advantages that the decomposed operation data are reasonably regulated and controlled, the operation data are scheduled to idle/proper ports for transmission, the phenomenon of data congestion in the transmission process of the operation data can be reduced, and the circulation of the data is facilitated. For example: the operation data has 6 parts, three ports are reasonably distributed, and the output port starts to transmit the next part of data, so that the transmission efficiency is effectively improved. Also for example: and the decomposed operation data is transmitted by using the port with the idle edge cloud, so that the occupation of a transmission channel in the transmission process is reduced.
And S70, transmitting the job data positioned at the port to a terminal server.
Judging whether the terminal server receives the operation data;
if the terminal server does not receive the operation data within the expected transmission time, retransmitting the operation data to the terminal server;
and if the terminal server receives the job data within the expected transmission time, ending the job data transmission.
As a preferable scheme of the embodiment, in the process of transmitting the executed job data to the terminal server, the edge cloud monitors whether the terminal server receives the job data, so as to judge the validity of the data.
When the terminal server receives the job data, the edge cloud is effective to execute the data, and the job process is finished. The job data has been transmitted to the terminal server but has not been received beyond the expected transmission time, and the edge cloud sends an instruction to the output port that the job data has not been received. At this time, the output port starts to perform secondary transmission on the data stored by the edge cloud and transmitted in a delayed manner, so that the phenomenon that the data is lost in the transmission process is effectively prevented. In addition, the data transmitted in a delayed manner is automatically transmitted through the instruction, so that the problem that in the prior art, a data sender and a data receiver grasp the same data information is solved, and the data transmission in the edge cloud is facilitated. After the terminal server receives the operation data, the edge cloud receives a feedback instruction of the terminal server, eliminates the operation data which are transmitted in a delayed mode, and effectively prevents the edge cloud from generating data redundancy.
As shown in fig. 3, this embodiment provides a data processing system based on edge cloud, including:
the edge cloud input module is used for receiving the operation data transmitted by the center cloud;
the edge cloud monitoring module is used for verifying the validity of the operation data;
as a further aspect of the present invention, the edge cloud monitoring module includes:
the data legal unit is used for verifying the data validity of the operation data based on an exponential smoothing algorithm;
as a further aspect of the present invention, the data legal unit includes:
an initialization mechanism for establishing a detection model based on an exponential smoothing algorithm and setting rule data; the rule data is data conforming to a legal rule;
a detection model mechanism for calculating the job data and the rule data using the detection model;
and the data comparison mechanism is used for comparing the calculation results of the job data and the rule data so as to verify the data validity of the job data.
An extinction recording unit configured to delete the job data that has not passed the data validity verification, and to save a transmission record of the job data;
the transmission legal unit is used for carrying out transmission legal verification on the operation data passing through the data legal verification;
and the data return unit is used for returning the job data which does not pass the verification of the transmission validity to a sending place.
As a further aspect of the present invention, the formula of the exponential smoothing algorithm is:
Y t+1 ＝αX t +α(1-α)X t-1 +α(1-α) 2 X t-2 +…+α(1-α) n X t-n
wherein Y is t+1 For the predicted value of the operation/rule data, alpha is a weight coefficient, n is a smooth index, t is the observation period number of the operation/rule data, and X is the observation value of the operation/rule data.
The edge cloud analysis module is used for analyzing the operation data passing the validity verification and calculating the data quantity and the expected transmission time of the operation data;
the edge cloud computing module is used for eliminating redundancy of the operation data according to the analysis result of the operation data;
the edge cloud regulation and control module is used for compressing and decomposing the operation data with redundant elimination, and regulating and controlling the decomposed operation data to corresponding ports according to the data quantity and the expected transmission time of the operation data;
and the edge cloud output module is used for transmitting the operation data positioned at the port to the terminal server.
And the edge cloud delay module is used for retransmitting the job data which is not received by the terminal server in the expected transmission time.
It should be understood that the foregoing examples of the present invention are merely illustrative of the present invention and are not intended to limit the present invention to the specific embodiments thereof. Any modification, equivalent replacement, improvement, etc. that comes within the spirit and principle of the claims of the present invention should be included in the protection scope of the claims of the present invention.
Claims (7)
1. A method for processing data based on edge cloud, comprising:
receiving job data transmitted by a central cloud;
verifying the validity of the operation data;
if the operation data do not pass the validity verification, the operation data transmitted by the center cloud are received again;
if the operation data passes the validity verification, analyzing the operation data, and calculating the data quantity and the expected transmission time of the operation data;
removing redundancy of the operation data according to the analysis result of the operation data;
compressing and decomposing the operation data with redundancy removed, and regulating and controlling the decomposed operation data to a corresponding port according to the data quantity of the operation data and the predicted transmission time;
transmitting the job data at a port to a terminal server;
performing validity verification on the job data, including:
performing data validity verification on the operation data based on an exponential smoothing algorithm;
if the job data does not pass the data validity verification, judging that the job data does not pass the data validity verification, deleting the job data, and storing a transmission record of the job data;
if the operation data passes the data validity verification, the operation data is subjected to transmission validity verification;
if the job data does not pass the transmission validity verification, judging that the job data does not pass the validity verification, and returning the job data to a transmitting place;
if the operation data passes the transmission validity verification, judging that the operation data passes the validity verification;
and verifying the data validity of the operation data based on an exponential smoothing algorithm, wherein the method comprises the following steps:
establishing a detection model based on an exponential smoothing algorithm, and setting rule data; the rule data is data conforming to a legal rule;
calculating the job data and the rule data by using the detection model;
and comparing the calculation results of the job data and the rule data, thereby verifying the data validity of the job data.
2. The method for processing data based on edge cloud according to claim 1, wherein the formula of the exponential smoothing algorithm is:
Y t+1 ＝αX t +α(1-α)X t-1 +α(1-α) 2 X t-2 +…+α(1-α) n X t-n
wherein Y is t+1 Predicting for job/rule dataThe value alpha is a weight coefficient, n is a smooth index, t is the observation period number of the operation/rule data, and X is the observation value of the operation/rule data.
3. The method for processing data based on edge cloud according to claim 1, further comprising, after transmitting the job data located at a port to a terminal server:
judging whether the terminal server receives the operation data;
if the terminal server does not receive the operation data within the expected transmission time, retransmitting the operation data to the terminal server;
and if the terminal server receives the job data within the expected transmission time, ending the job data transmission.
4. An edge cloud-based data processing system, comprising:
the edge cloud input module is used for receiving the operation data transmitted by the center cloud;
the edge cloud monitoring module is used for verifying the validity of the operation data;
the edge cloud analysis module is used for analyzing the operation data passing the validity verification and calculating the data quantity and the expected transmission time of the operation data;
the edge cloud computing module is used for eliminating redundancy of the operation data according to the analysis result of the operation data;
the edge cloud regulation and control module is used for compressing and decomposing the operation data with redundant elimination, and regulating and controlling the decomposed operation data to corresponding ports according to the data quantity and the expected transmission time of the operation data;
the edge cloud output module is used for transmitting the operation data positioned at the port to the terminal server;
the edge cloud monitoring module comprises:
the data legal unit is used for verifying the data validity of the operation data based on an exponential smoothing algorithm;
an extinction recording unit configured to delete the job data that has not passed the data validity verification, and to save a transmission record of the job data;
the transmission legal unit is used for carrying out transmission legal verification on the operation data passing through the data legal verification;
and the data return unit is used for returning the job data which does not pass the verification of the transmission validity to a sending place.
5. The edge cloud-based data processing system of claim 4, wherein said data legal unit comprises:
an initialization mechanism for establishing a detection model based on an exponential smoothing algorithm and setting rule data; the rule data is data conforming to a legal rule;
a detection model mechanism for calculating the job data and the rule data using the detection model;
and the data comparison mechanism is used for comparing the calculation results of the job data and the rule data so as to verify the data validity of the job data.
6. The edge cloud based data processing system of claim 4, wherein said exponential smoothing algorithm is formulated as:
Y t+1 ＝αX t +α(1-α)X t-1 +α(1-α) 2 X t-2 +…+α(1-α) n X t-n
wherein Y is t+1 For the predicted value of the operation/rule data, alpha is a weight coefficient, n is a smooth index, t is the observation period number of the operation/rule data, and X is the observation value of the operation/rule data.
7. The edge cloud-based data processing system of claim 4, further comprising:
and the edge cloud delay module is used for retransmitting the job data which is not received by the terminal server in the expected transmission time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210624353.3A CN115022045B (en) | 2022-06-02 | 2022-06-02 | Data processing method and system based on edge cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210624353.3A CN115022045B (en) | 2022-06-02 | 2022-06-02 | Data processing method and system based on edge cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115022045A CN115022045A (en) | 2022-09-06 |
| CN115022045B true CN115022045B (en) | 2023-09-19 |
Family
ID=83072792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210624353.3A Active CN115022045B (en) | 2022-06-02 | 2022-06-02 | Data processing method and system based on edge cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115022045B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0411588D0 (en) * | 2004-05-24 | 2004-06-23 | Toshiba Res Europ Ltd | Data transmission method |
| CN109714370A (en) * | 2019-03-07 | 2019-05-03 | 四川长虹电器股份有限公司 | A kind of implementation method based on http protocol end Yunan County full communication |
| WO2020207264A1 (en) * | 2019-04-08 | 2020-10-15 | 阿里巴巴集团控股有限公司 | Network system, service provision and resource scheduling method, device, and storage medium |
| CN112187798A (en) * | 2020-09-28 | 2021-01-05 | 安徽大学 | Bidirectional access control method and system applied to cloud-side data sharing |
| CN112543187A (en) * | 2020-11-26 | 2021-03-23 | 齐鲁工业大学 | Industrial Internet of things safety data sharing method based on edge block chain |
| CN114238323A (en) * | 2021-12-11 | 2022-03-25 | 宜昌优智科技有限公司 | Internet of things data collection, cleaning, rating, transmission and storage method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9052939B2 (en) * | 2011-05-27 | 2015-06-09 | Red Hat, Inc. | Data compliance management associated with cloud migration events |
| US20200004503A1 (en) * | 2017-03-17 | 2020-01-02 | Mitsubishi Electric Corporation | Information processing device, information processing method, and computer readable medium |
-
2022
- 2022-06-02 CN CN202210624353.3A patent/CN115022045B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0411588D0 (en) * | 2004-05-24 | 2004-06-23 | Toshiba Res Europ Ltd | Data transmission method |
| CN109714370A (en) * | 2019-03-07 | 2019-05-03 | 四川长虹电器股份有限公司 | A kind of implementation method based on http protocol end Yunan County full communication |
| WO2020207264A1 (en) * | 2019-04-08 | 2020-10-15 | 阿里巴巴集团控股有限公司 | Network system, service provision and resource scheduling method, device, and storage medium |
| CN112187798A (en) * | 2020-09-28 | 2021-01-05 | 安徽大学 | Bidirectional access control method and system applied to cloud-side data sharing |
| CN112543187A (en) * | 2020-11-26 | 2021-03-23 | 齐鲁工业大学 | Industrial Internet of things safety data sharing method based on edge block chain |
| CN114238323A (en) * | 2021-12-11 | 2022-03-25 | 宜昌优智科技有限公司 | Internet of things data collection, cleaning, rating, transmission and storage method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115022045A (en) | 2022-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021244211A1 (en) | Blockchain message processing method and apparatus, computer and readable storage medium | |
| CN110138575B (en) | Network slice creating method, system, network device and storage medium | |
| CN110704518B (en) | Business data processing method and device based on Flink engine | |
| JP6587330B2 (en) | Random forest model training method, electronic apparatus, and storage medium | |
| US12219515B2 (en) | Data transmission method and apparatus, computer readable medium, and electronic device | |
| JPH10243018A (en) | Communication equipment, frequency band reservation method and terminal equipment | |
| CN112583715A (en) | Equipment node connection adjustment method and device | |
| CN108063653A (en) | A kind of delay control method, apparatus and system | |
| CN108809520B (en) | Edge computing environment-based code distributed computing method and device | |
| JP7356581B2 (en) | Information processing methods, devices, equipment and computer readable storage media | |
| Wang et al. | Determining delay bounds for a chain of virtual network functions using network calculus | |
| CN115022045B (en) | Data processing method and system based on edge cloud | |
| RU2407170C2 (en) | Method for interface adaptation of television internet protocol with stream data storage device | |
| CN110855424A (en) | Method and device for synthesizing asymmetric flow xDR in DPI field | |
| CN116367223B (en) | XR service optimization method and device based on reinforcement learning, electronic equipment and storage medium | |
| CN104168206B (en) | Adapter gateway load balancing control method, device and system | |
| CN112600906B (en) | Resource allocation method, device and electronic device for online scene | |
| CN112600753B (en) | Equipment node communication path selection method and device according to equipment access amount | |
| CN115168302A (en) | Business data export method and device and electronic equipment | |
| CN114554496A (en) | 5G network slice resource allocation method based on machine learning | |
| CN114727099A (en) | Video conference transmission quality optimization and evaluation method, device, equipment and medium | |
| CN102546593B (en) | Node cooperation method and system in peer-to-peer network streaming media system | |
| TWI735520B (en) | Method and device for adjusting the number of component logic threads | |
| CN115996404B (en) | Dynamic adjustment method and device for node link | |
| CN115150216B (en) | Flow forwarding system, method and control plane equipment of vBRAS |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20220906 Assignee: Shanghai Haoyun Changsheng Data Service Co.,Ltd. Assignor: LIANTONG (GUANGDONG) INDUSTRY INTERNET Co.,Ltd. Contract record no.: X2024980000732 Denomination of invention: A data processing method and system based on edge cloud Granted publication date: 20230919 License type: Common License Record date: 20240118 |