CN112738060B - Method and device for processing micro-service data, micro-service processing platform and medium - Google Patents
Method and device for processing micro-service data, micro-service processing platform and medium Download PDFInfo
- Publication number
- CN112738060B CN112738060B CN202011556562.6A CN202011556562A CN112738060B CN 112738060 B CN112738060 B CN 112738060B CN 202011556562 A CN202011556562 A CN 202011556562A CN 112738060 B CN112738060 B CN 112738060B
- Authority
- CN
- China
- Prior art keywords
- micro
- service
- data
- microservice
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 165
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000006978 adaptation Effects 0.000 claims abstract description 22
- 238000012544 monitoring process Methods 0.000 claims description 72
- 238000004458 analytical method Methods 0.000 claims description 44
- 238000004590 computer program Methods 0.000 claims description 14
- 230000010354 integration Effects 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000004140 cleaning Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000003672 processing method Methods 0.000 abstract description 4
- 238000012827 research and development Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1073—Registration or de-registration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Multimedia (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Stored Programmes (AREA)
Abstract
The application is applicable to the technical field of research and development management, and provides a micro-service data processing method, a device, a micro-service processing platform and a medium, wherein the method comprises the following steps: determining a target registration center containing registration information of the micro service and a target registration type of the target registration center; generating a target calling interface in a micro-service registration adaptation engine of a micro-service processing platform according to the target registration type; acquiring micro-service data of the micro-service according to the target calling interface and the target open interface; and writing the micro-service data into a database of the micro-service processing platform and performing centralized processing. By adopting the method, the interface adaptation is carried out with the registration centers of different registration types to obtain the micro-service data of each micro-service, and the effect of directly and effectively processing the micro-service data can be realized when the registration information of a plurality of micro-services is respectively deployed in the registration centers of different registration types.
Description
Technical Field
The application belongs to the technical field of big data, and particularly relates to a micro-service data processing method and device, a micro-service processing platform and a medium.
Background
In the prior art, when registration information of a plurality of micro services is respectively deployed in registration centers of different registration types, it is difficult to directly process the plurality of micro services through a single micro service processing platform due to the different registration types of the registration centers. At present, registration information of a large number of micro services is usually deployed in a registry of the same registration type. However, if a large number of micro services have corresponding registration centers, the registration information deployed by the micro services in the corresponding registration centers needs to be deleted. And then, the registration information is redeployed in the same registration type registration center. At this time, when the registration information is redeployed, the workload of the worker is increased. Therefore, at present, when registration information of a plurality of micro services are respectively deployed in registration centers of different registration types, it is difficult to directly and effectively process a plurality of micro service data through a single micro service processing platform.
Disclosure of Invention
The embodiment of the application provides a method and a device for processing micro-service data, a micro-service processing platform and a medium, which can solve the problem that when registration information of a plurality of micro-services is respectively deployed in registration centers with different registration types, the plurality of micro-service data are difficult to be directly and effectively processed through a single micro-service processing platform.
In a first aspect, an embodiment of the present application provides a method for processing microservice data, which is applied to a microservice processing platform, and the method includes:
determining a target registration center containing registration information of the micro service, and determining a target registration type of the target registration center;
generating a target calling interface adapted to a target open interface of the target registration center in a micro-service registration adaptation engine of the micro-service processing platform according to the target registration type;
acquiring micro-service data of the micro-service according to the target calling interface and the target open interface;
and writing the micro-service data into a database of the micro-service processing platform, and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data corresponding to a plurality of micro-services one by one.
In one embodiment, the target registry comprises a unified registry of the microservice processing platform and a registry that the microservice has already; before the determining the target registry containing the registration information of the microservice, the method further comprises the following steps:
if the micro service does not have a corresponding registration center, acquiring registration information of the micro service, and deploying the registration information in the unified registration center;
and if the micro service has a corresponding registration center, determining the registration center corresponding to the micro service as a target registration center of the micro service.
In one embodiment, before the determining the target registry containing the registration information of the microservice, the method further comprises:
if the micro service has a corresponding registration center, deleting the registration information deployed by the micro service in the corresponding registration center;
and acquiring the registration information of the micro service, and deploying the registration information in the unified registration center again.
In an embodiment, after the writing the micro service data into the database of the micro service processing platform and performing centralized processing on the micro service data, the method further includes:
determining a frame type of a microservice frame used by the microservice;
calling a tool kit matched with the frame type through a micro-service monitoring engine of the micro-service processing platform;
and when the micro service is operated, accessing a micro service framework of the micro service according to the tool kit so as to call a service component of the micro service.
In an embodiment, when the microservice is run, after accessing a microservice framework of the microservice according to the toolkit to call a service component of the microservice, the method further includes:
acquiring monitoring data of the micro-service during operation;
performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result;
storing the analysis results in a database.
In an embodiment, the acquiring the monitoring data of the microservice runtime includes:
storing the monitoring data of the micro service in operation into a message queue of the micro service;
and writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
In an embodiment, the performing real-time integration and analysis on the monitoring data within a preset time period to obtain an analysis result, and obtaining the analysis result includes:
sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain processed monitoring data;
integrating the processed monitoring data in a charting mode to respectively obtain table data in a table mode and graphic data in a graphic mode;
and analyzing by combining the table data and the graph data to obtain the analysis result.
In a second aspect, an embodiment of the present application provides an apparatus for processing microservice data, which is applied to a microservice processing platform, and the apparatus includes:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for determining a target registration center containing registration information of the micro service and determining a target registration type of the target registration center;
the generating module is used for generating a target calling interface which is adaptive to a target open interface of the target registration center in a micro service registration adaptive engine of the micro service processing platform according to the target registration type;
the second acquisition module is used for acquiring the microservice data of the microservice according to the target calling interface and the target open interface;
and the processing module is used for writing the micro-service data into a database of the micro-service processing platform and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data which correspond to a plurality of micro-services one by one.
In a third aspect, an embodiment of the present application provides a microservice processing platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, implements the method according to any one of the above first aspects.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method according to any one of the above first aspects.
In a fifth aspect, embodiments of the present application provide a computer program product, which, when run on a terminal device, causes the terminal device to perform the method of any one of the above aspects.
Compared with the prior art, the embodiment of the application has the advantages that: after determining the target registration center corresponding to the micro-service and the target registration type of the target registration center, the micro-service processing platform only needs to adapt to the target open interface of the existing target registration center. Therefore, the micro-service processing platform can be connected with the target registration center, and the registration information deployed by the micro-service in the target registration center can be acquired. And then, acquiring the micro service data of the corresponding micro service according to the registration information for processing. Based on the method, the micro-service processing platform can not only uniformly process the micro-service data of a plurality of micro-services corresponding to the registration centers of different registration types, but also effectively reduce the workload of workers and reduce the risk of micro-service data leakage when the registration information is redeployed in the micro-service processing platform.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a flowchart illustrating an implementation of a method for processing microservice data according to an embodiment of the present application;
fig. 2 is a flowchart of an implementation of a method for processing microservice data according to another embodiment of the present application;
fig. 3 is a flowchart illustrating an implementation of a method for processing microservice data according to still another embodiment of the present application;
fig. 4 is a flowchart illustrating an implementation of a method for processing microservice data according to another embodiment of the present application;
FIG. 5 is a flowchart illustrating an implementation of a method for processing microservice data according to another embodiment of the present application;
fig. 6 is a schematic diagram illustrating an implementation manner of S10CA of a method for processing microservice data according to an embodiment of the present application;
fig. 7 is a schematic diagram illustrating an implementation manner of an S10CB of a method for processing microservice data according to an embodiment of the present application;
fig. 8 is a block diagram illustrating a micro-service data processing apparatus according to an embodiment of the present application;
fig. 9 is a block diagram illustrating a micro-service processing platform according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing a relative importance or importance.
The method for processing the micro-service data provided by the embodiment of the application is applied to a micro-service processing platform. The micro-service processing platform comprises a unified registration center, a micro-service registration adaptation engine, a micro-service monitoring engine and a database. The unified registry can be used for deploying the registration information of each micro service, so that each newly online micro service can realize the centralized processing of the registration information of the micro services without deploying the respective registration information in a plurality of registries. The microservice registration adaptation engine is used for carrying out interface adaptation on the existing registration center of the microservice so that the microservice processing platform can obtain registration information of the microservice from the existing registration center through the interface and obtain microservice data of the microservice according to the registration information. The microservice monitoring engine can be used for calling a tool kit matched with the frame type to access the microservice frame after the frame type of the microservice frame in the microservice is determined so as to call microservice components in the microservice. The database may be used to store micro-service data of each micro-service, monitoring data generated during the operation of the micro-service, and an analysis result obtained by analyzing the monitoring data, which is not limited herein.
Referring to fig. 1, fig. 1 is a flowchart illustrating an implementation of a method for processing microservice data according to an embodiment of the present application, where the method is applied to a microservice processing platform, and the method specifically includes the following steps:
s101, determining a target registration center containing registration information of the micro service, and determining a target registration type of the target registration center.
In application, the micro service is an architectural scheme for constructing service application. The architecture scheme of the service application is different from the traditional monolithic scheme, and the service application can be split into a plurality of core service functions. Each service function may be referred to as a service and may be built and deployed separately, i.e., each service function may be considered a microservice. When each micro service is started, data information such as a network address of the micro service, a micro service component and the like needs to be registered in a registration center and stored by the registration center. Based on this, when invoking the microservice, the service consumer may query the network address of the microservice from the registry. And then, acquiring the micro-service component through the network address, connecting the micro-service component with a calling interface for accessing the micro-service, and calling the micro-service component.
In application, whether the micro service has the corresponding target registration center and the target registration type of the target registration center is determined, and workers can determine the micro service according to actual conditions and send the determination result to the micro service processing platform.
In application, the registration type of the above-mentioned registry is various, and specifically, the registration type includes, but is not limited to, zookeeper type, eureka type, and the like. It will be appreciated that the registry may be the same type of registration or different types of registration for different microservices. It can be understood that the registration types of the micro services corresponding to the registries are different, and it is difficult for the micro service processing platform of each micro service to be compatible with the registries of other registration types. Further, it is difficult to implement unified processing for micro-services corresponding to a plurality of different registration types.
S102, according to the target registration type, generating a target calling interface adapted to a target open interface of the target registration center in a micro-service registration adaptation engine of the micro-service processing platform.
In the application, it has been described above that when each micro service is started, data information such as a network address and a micro service component of the first micro service needs to be deployed in the registry. The data information such as network addresses, micro-service components and the like correspondingly deployed by all micro-services in the target registration center can be obtained after the micro-service registration adaptation engine is in adaptation connection with the target registration center. Based on this, in order to realize the adaptive connection between the micro-service registration adaptation engine and the registration center, an open interface of the target registration center needs to be acquired. And then, the micro-service registration adaptation engine adapts the target registration center according to the open interface to generate a target call interface which can carry out data transmission with the target registration center. The calling interface and the open interface can be API interfaces. It should be noted that the microservice registration adaptation engine is a component in the microservice processing platform, and is used for adapting open interfaces provided by registries of different registration types and generating a call interface corresponding to the registry of each registration type.
It will be appreciated that a component (registry) has an open API, i.e. data of its own (data within the registry) can be opened to the outside on behalf of the component. In this case, the open API includes the usage authority of the API, for example, the number of calls of the API, the call target, and the like.
S103, acquiring micro service data of the micro service according to the target calling interface and the target open interface.
And S104, writing the micro-service data into a database of the micro-service processing platform, and performing centralized processing on the micro-service data, wherein the database contains micro-service data corresponding to a plurality of micro-services one by one.
In application, the microservice data may be understood as data information such as a network address of a microservice, a microservice component, and the like. It should be noted that, when the target call interface of the micro-service registration adaptation engine is adapted to the target open interface of the target registration center, the micro-service processing platform may call the target call interface to connect with the target open interface. Further, the network address of the micro-service and the data information such as the micro-service component can be searched in the target registry.
In application, after the micro-service data is obtained, the micro-service data is written into a database of the micro-service processing platform for storage. Therefore, after the database contains the micro-service data which are in one-to-one correspondence with the plurality of micro-services, the centralized processing of the plurality of micro-service data can be realized. The centralized processing may be to set identification information of each micro service data to distinguish each micro service, and may perform processing such as adding, deleting, modifying, querying, and the like on registration information deployed by each micro service in the target registration center, which is not limited herein.
In this embodiment, after determining the target registry corresponding to the micro service and the target registration type of the target registry, the micro service processing platform only needs to adapt to a target open interface of the existing target registry. Therefore, the micro-service processing platform can be connected with the target registration center, and the registration information deployed by the micro-service in the target registration center can be acquired. And then, acquiring the micro service data of the corresponding micro service according to the registration information for processing. Based on the method, the micro-service processing platform can not only uniformly process the micro-service data of a plurality of micro-services corresponding to the registration centers of different registration types, but also effectively reduce the workload of workers and reduce the risk of micro-service data leakage when the registration information is redeployed in the micro-service processing platform.
Referring to fig. 2, in an embodiment, the target registry includes a unified registry of the microservice processing platform and a registry already existing for the microservice; before S101 determines the target registry containing the registration information of the microservice, the method further includes the following steps S101A-S101B, which are detailed as follows:
S101A, if the micro service does not have a corresponding registration center, acquiring registration information of the micro service, and deploying the registration information in the unified registration center.
In application, the unified registry may be a registry which is set in advance in the microservice processing platform for a worker, and may be used to deploy registration information for microservices which do not have a corresponding registry.
In applications, for newly brought online microservices, they typically do not have a corresponding registry. Based on the method, the registration information of the micro-service can be deployed in a unified registration center in the micro-service processing platform. At this time, there is only one unified registry, that is, there is only one registration type of the unified registry. At this time, the unified registry is the target registry, and the unified registry and the microservice registration adaptation engine are all components in the microservice processing platform. Namely, the microservice registration adaptation engine is considered to have a target calling interface adapted to the target open interface of the unified registry. Therefore, the registration information of the newly online micro-service is deployed in the unified registration center, the registration center types corresponding to the newly online micro-service can be unified, the registration centers with various registration types are prevented from being constructed in the micro-service processing platform, and the component resources required to be used in the micro-service processing platform are reduced.
S101B, if the micro service has a corresponding registration center, determining the registration center corresponding to the micro service as a target registration center of the micro service.
In the application, for the micro-service of the existing corresponding registration center, at this time, the existing corresponding registration center can be determined as the target registration center. Based on this, the way of S101 to S102 described above can be adopted, and the microservice registration adaptation engine is used to perform interface adaptation on the target open interface of the target registry.
In this embodiment, after determining that a corresponding registration center exists for the micro service, the existing registration center is subjected to interface adaptation by using the above S101 to S102 to obtain micro service data of the micro service, so that the registration information of the micro service does not need to be redeployed, thereby reducing the workload of workers; and after determining that the micro-service has no corresponding registration center, deploying the registration information of each micro-service by using the uniform registration center of the same registration type, so that the registration centers of various registration types can be prevented from being constructed in the micro-service processing platform, and the component resources required to be used in the micro-service processing platform are reduced.
Referring to fig. 3, in an embodiment, before S101 determines the target registry containing the registration information of the microservice, the following steps S101C-S101D are further included, which are detailed as follows:
S101C, if the micro service has a corresponding registration center, deleting the registration information deployed by the micro service in the corresponding registration center.
S101D, obtaining the registration information of the micro service, and deploying the registration information in the unified registration center again.
In application, after determining that the micro service has the corresponding registration center, the registration information deployed by the micro service in the corresponding registration center can be deleted. And then, the registration information of the microservice is deployed in the unified registration center again. It should be noted that, when the number of the micro services of the existing corresponding registry is small, the registration information may be redeployed by adopting the above manner, so as to reduce the number of the interfaces for the interface adaptation between the micro service registration adaptation engine and the external registry.
Referring to fig. 4, in an embodiment, after the step S104 writes the micro service data into the database of the micro service processing platform and performs centralized processing on the micro service data, the following steps S104A to S104C are further included, which are detailed as follows:
S104A, determining the frame type of the micro-service frame used by the micro-service.
In an application, the micro-service framework can be used for deploying a running environment required by the micro-service in running. Therefore, when the micro service is generated, a micro service framework of the micro service needs to be constructed in advance to deploy various service functions of the micro service in the micro service framework so that the micro service can operate normally. The types of the framework of the microservice framework used are different because the service functions are different between different microservices. The frame type includes, but is not limited to, a Service Mesh frame type, a Linkerd frame type, and the like.
It should be noted that, for determining the framework type of the microservice framework, the method for determining the target registration type of the target registry may be consistent. For example, the staff member may determine the micro service according to the actual situation of the micro service, and send the determination result to the micro service processing platform, which is not limited herein.
And S104B, calling a tool kit matched with the frame type through a micro-service monitoring engine of the micro-service processing platform.
And S104C, when the micro service is operated, accessing a micro service framework of the micro service according to the tool kit so as to call a service component of the micro service.
In an application, after acquiring microservice data of a microservice, in order to run the microservice (that is, to call a microservice component in the microservice data), a corresponding microservice framework needs to be deployed in a microservice processing platform. Based on the above, in order to avoid deploying the micro-service framework of each framework type in the micro-service processing platform, the micro-service processing platform can call the tool kit adapted to the framework type and the micro-service framework to connect through the micro-service monitoring engine. Furthermore, the micro-service processing platform can be remotely accessed to the micro-service framework, so that the micro-service components can be called. The microservice monitoring engine is a functional component in the microservice processing platform, is used for storing tool packages respectively matched with a plurality of frame types, and can call the tool packages to be matched with microservice frames of corresponding frame types.
In an application, the above-mentioned tool package is a Software Development Kit (SDK), and a set of Development tools for application Software is generally established for a specific Software package, a Software framework (microservice framework), an operating system, and the like. That is, the staff member can install a tool kit adapted to various frame types in the microservice monitoring engine in advance, so that the microservice processing platform can access the microservice frame.
In the prior art, micro services of different framework types generally have respective corresponding micro service processing platforms. In addition, because the frame types are different, a plurality of micro service processing platforms are not mutually adaptive. Based on the method, the SDK toolkit can be used for adapting the capability of the micro-services with multiple framework types and accessing the micro-service frameworks of the micro-services, so that the micro-services with different framework types can be called and processed.
Referring to fig. 5, in an embodiment, when the microservice is executed at S104C, after accessing the microservice framework of the microservice according to the toolkit to invoke the service component of the microservice, the following steps S104CA-S104CC are further included, which are detailed as follows:
and S104CA, acquiring the monitoring data during the micro service operation.
In application, after the micro-service processing platform is accessed to the micro-service framework through the SDK toolkit, a micro-service component of the micro-service can be called, and then the micro-service is operated. Based on this, when the micro-service is running, the micro-service processing platform can obtain the monitoring data when the micro-service is running. The monitoring data includes, but is not limited to, working time of the microservice, processed processing data, and the like.
And S104CB, performing real-time integration analysis on the monitoring data in a preset time period to obtain an analysis result.
And S104CC, storing the analysis result into a database.
In an application, the time period may be a period preset by a user. The micro-service processing platform can integrate the monitoring data in each time period and then analyze the monitoring data to obtain an analysis result. The analysis result may be the result of the micro-service in the time period, such as the working efficiency and the working state. By analyzing the running state of the micro-service within a certain time period, the staff can reasonably adjust and optimize the micro-service configuration of the micro-service better according to the analysis result of the micro-service. The microservice configuration includes, but is not limited to, the running environment of the microservice, and the functional components of the microservice.
In application, the database is used for storing analysis results, and can store the analysis results in a diagrammatized manner, so that the microservice processing platform can sort, plan and analyze the disordered analysis results (analyze again according to the analysis results of different time periods). Furthermore, the staff can adjust and optimize the configuration information of the micro-service in real time according to the existing analysis result in the database.
Referring to fig. 6, in an embodiment, in acquiring the monitoring data of the micro service runtime at S104CA, the following sub-steps S104CA1-S104CA2 are further included, which are detailed as follows:
s104CA1, storing the monitoring data of the micro service in operation to the message queue of the micro service.
S104CA2, writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
In an application, the message queue is a temporary storage area in the microservice, and is used for storing all data (monitoring data) of the microservice in operation. The file system is a local file system in the micro service processing platform, wherein the file system can persistently store the monitoring data. In the actual process, problems such as crash, restart and the like may occur when the micro service runs. Based on this, if the micro service crashes suddenly during operation, the monitoring data stored in the message queue may have a data loss condition during the process of recovering the operation of the micro service. Therefore, after storing data in a message queue local to the microservice, monitoring data in the message queue needs to be stored in the file system to prevent the monitoring data of the microservice from being lost.
In application, the thread refers to an execution flow in a process, and is used for executing a service in the microservice. Asynchronous is understood to mean a way of transmitting data. Specifically, when the monitoring data stored in the message queue is written into the file system by using the asynchronous thread, the monitoring data does not need to be synchronously processed between the micro-service processing platform and the micro-service. For example, the micro service stores the monitoring data to the message queue in real time, and the micro service processing platform may obtain the monitoring data in the time period from the message queue every preset time period and store the monitoring data to the file system. Therefore, the micro-service processing platform and the micro-service do not need to wait for the response of the other party, and the data transmission efficiency is improved. The file system is specifically an HDFS system (distributed file system). Because the data volume transmitted between the micro-service processing platform and the plurality of micro-services is very large, the HDFS system has the characteristic of high fault tolerance, and can provide high throughput to acquire the monitoring data in the plurality of micro-services.
Referring to fig. 7, in an embodiment, in the step S104CB, performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result, and the step S104CB1 to S104CB3 of obtaining the analysis result further include the following sub-steps, which are described in detail as follows:
and S104CB1, sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain the processed monitoring data.
In an application, the monitoring data is generally raw data of the micro service operation, and formats and dimensions of the raw data may be different. Based on this, data cleaning processing, format conversion processing, and the like need to be performed on the monitoring data. The data cleaning process may be to clean data with transmission errors in the monitoring data. In an application, the obtained monitoring data may be monitoring data in multiple formats, such as double, int, char, and the like. And the data format of each monitoring data can be unified for the convenience of subsequent analysis and processing. That is, the current data format of the monitoring data is converted into the data in the preset target format, which is not limited in this respect. At this time, the data in the target format is the processed monitoring data.
And S104CB2, integrating the processed monitoring data in a charting form to respectively obtain table data in a table form and graphic data in a graphic form.
And S104CB3, analyzing by combining the table data and the graphic data to obtain the analysis result.
In application, the data integration is performed on the processed monitoring data, and the data of the same type can be integrated. For example, the processed monitoring data includes data of the micro-service working efficiency and some type of data processed by the micro-service. Based on this, integrating the processed monitoring data can be understood as centralizing and analyzing all data belonging to the micro-service working efficiency.
In application, the analysis result obtained by analyzing the table data may be obtained by performing operations (difference operation, square error operation) and the like with each data value in the previous table data directly according to each data value in the table data, so as to obtain an operation numerical value. And then, determining a target numerical range containing the operation numerical value according to a plurality of preset numerical ranges and the analysis result corresponding to each numerical range, and obtaining the analysis result of the table data. The analysis performed on the image data may be performed to compare the image data with a plurality of preset patterns, and determine a target preset pattern having a highest similarity with the image data from the preset patterns. Based on the analysis result, the analysis result corresponding to the target preset graph can be determined from the analysis results corresponding to the preset graphs respectively.
In application, after the analysis result is obtained, the micro service processing platform monitoring data can also display table data in a table form and graphic data in a graphic form. It can be understood that the table data can only visually show the data values, and the relation between the data values cannot be clearly shown. Based on the method, the graphic data in the graphic form are displayed simultaneously, so that when a user sees the table data, the user can understand the internal rules of the table data more easily according to the graphic data, and the user can understand the monitoring data more comprehensively.
In application, the micro-service processing platform can also receive a control instruction issued by a worker, so that the micro-service processing platform can manage the working state of the micro-service according to the control instruction. Specifically, the control instruction includes, but is not limited to, an instruction for controlling the microserver to change the execution task, and an instruction for controlling the microserver to stop executing the task. The operating state of the control microservice may be that the control microservice is in a service state, an idle state, or the like. It is necessary to supplement that, when the working state of the micro-service changes, it can also save the latest working state of the micro-service in the database by the data transmission mode of the asynchronous thread.
Referring to fig. 8, fig. 8 is a block diagram illustrating a micro service data processing apparatus according to an embodiment of the present disclosure. The processing apparatus of microservice data in this embodiment includes modules for executing the steps in the embodiments corresponding to fig. 1 to fig. 7. Please refer to fig. 1 to 7 and fig. 1 to 7 for related descriptions. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 8, a device 800 for processing microservice data includes: a first obtaining module 810, a generating module 820, a second obtaining module 830, and a processing module 840, wherein:
the first obtaining module 810 is configured to determine a target registry including registration information of the microservice, and determine a target registration type of the target registry.
A generating module 820, configured to generate, in the micro service registration adaptation engine of the micro service processing platform, a target call interface adapted to the target open interface of the target registration center according to the target registration type.
A second obtaining module 830, configured to obtain the microservice data of the microservice according to the target calling interface and the target open interface.
The processing module 840 is configured to write the microservice data into a database of the microservice processing platform, and perform centralized processing on the microservice data, where the database includes microservice data corresponding to multiple microservices one to one.
In one embodiment, the target registry comprises a unified registry of the microservice processing platform and a registry that the microservice has already; the apparatus 800 for processing microservice data further comprises:
and the third acquisition module is used for acquiring the registration information of the micro service and deploying the registration information in the unified registration center if the micro service does not have the corresponding registration center.
And the first determining module is used for determining the registry corresponding to the micro service as a target registry of the micro service if the micro service has the corresponding registry.
In an embodiment, the apparatus 800 for processing microservice data further includes:
and the deleting module is used for deleting the registration information deployed by the micro service in the existing corresponding registration center if the micro service has the corresponding registration center.
And the deployment module is used for acquiring the registration information of the microservice and re-deploying the registration information in the unified registration center.
In an embodiment, the apparatus 800 for processing microservice data further includes:
and the second determining module is used for determining the frame type of the micro-service frame used by the micro-service.
And the first calling module is used for calling the tool kit matched with the frame type through a micro-service monitoring engine of the micro-service processing platform.
And the second calling module is used for accessing the micro-service framework of the micro-service according to the tool pack when the micro-service is operated so as to call the service component of the micro-service.
In an embodiment, the apparatus 800 for processing microservice data further includes:
and the third acquisition module is used for acquiring the monitoring data during the micro-service operation.
And the analysis module is used for performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result.
And the storage module is used for storing the analysis result into a database.
In an embodiment, the third obtaining module is further configured to:
storing the monitoring data of the micro service in operation into a message queue of the micro service;
and writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
In an embodiment, the analysis module is further configured to:
sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain processed monitoring data;
integrating the processed monitoring data in a charting mode to respectively obtain table data in a table mode and graphic data in a graphic mode;
and analyzing by combining the table data and the graph data to obtain the analysis result.
It should be understood that, in the structural block diagram of the data processing apparatus shown in fig. 8, each unit/module is used to execute each step in the embodiment corresponding to fig. 1 to 7, and each step in the embodiment corresponding to fig. 1 to 7 has been explained in detail in the above embodiment, and specific reference is made to the relevant description in the embodiment corresponding to fig. 1 to 7 and fig. 1 to 7, which is not repeated herein.
Fig. 9 is a block diagram of a microservice processing platform according to another embodiment of the present application. As shown in fig. 9, the microservice processing platform 900 of this embodiment includes: a processor 910, a memory 920 and a computer program 930, e.g. a program of a data processing method, stored in the memory 920 and executable on the processor 910. The processor 910 implements the steps in each embodiment of the data processing method described above, such as S101 to S104 shown in fig. 1, when executing the computer program 930. Alternatively, the processor 910, when executing the computer program 930, implements the functions of the modules in the embodiment corresponding to fig. 8, for example, the functions of the modules 810 to 840 shown in fig. 8, specifically referring to the relevant description in the embodiment corresponding to fig. 8.
Illustratively, the computer program 930 may be divided into one or more units, which are stored in the memory 920 and executed by the processor 910 to accomplish the present application. One or more of the elements may be a series of computer program instruction segments capable of performing certain functions, the instruction segments describing the execution of the computer program 930 in the terminal device 900.
The microservice processing platform may include, but is not limited to, a processor 910, a memory 920. Those skilled in the art will appreciate that fig. 9 is merely an example of a microservice processing platform 900 and does not constitute a limitation of the microservice processing platform 900 and may include more or less components than those shown, or some components may be combined, or different components may be included, for example, the microservice processing platform may also include input output devices, network access devices, buses, etc.
The processor 910 may be a central processing unit, but may also be other general purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method for processing microservice data is applied to a microservice processing platform, and the method comprises the following steps:
determining a target registration center containing registration information of the micro service, and determining a target registration type of the target registration center;
generating a target calling interface adapted to a target open interface of the target registration center in a micro-service registration adaptation engine of the micro-service processing platform according to the target registration type;
acquiring micro-service data of the micro-service according to the target calling interface and the target open interface;
and writing the micro-service data into a database of the micro-service processing platform, and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data corresponding to a plurality of micro-services one by one.
2. The method for processing microservice data of claim 1, wherein the target registry comprises one of a unified registry of the microservice processing platform and a registry to which the microservice has a correspondence;
before the determining the target registry containing the registration information of the microservice, the method further comprises the following steps:
if the micro service does not have a corresponding registration center, acquiring registration information of the micro service, and deploying the registration information in the unified registration center;
and if the micro service has a corresponding registration center, determining the registration center corresponding to the micro service as a target registration center of the micro service.
3. The method of claim 2, wherein prior to determining the target registry containing registration information for the microservice, further comprising:
if the micro service has a corresponding registration center, deleting the registration information deployed by the micro service in the corresponding registration center;
and acquiring the registration information of the micro service, and deploying the registration information in the unified registration center again.
4. The method for processing microservice data according to claim 1, further comprising, after said writing said microservice data into a database of said microservice processing platform and performing centralized processing on said microservice data:
determining a frame type of a microservice frame used by the microservice;
calling a tool pack adapted to the frame type through a micro-service monitoring engine of the micro-service processing platform;
and when the micro service is operated, accessing a micro service framework of the micro service according to the tool kit so as to call a service component of the micro service.
5. The method for processing microservice data according to claim 4, wherein when the microservice is run, after accessing a microservice framework of the microservice according to the toolkit to invoke a service component of the microservice, the method further comprises:
acquiring monitoring data of the micro-service during operation;
performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result;
and storing the analysis result in a database.
6. The method for processing microservice data according to claim 5, wherein the obtaining the monitoring data of the microservice runtime includes:
storing the monitoring data of the micro service in operation into a message queue of the micro service;
and writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
7. The method for processing microservice data according to claim 5, wherein the performing real-time integration analysis on the monitored data within a preset time period to obtain an analysis result comprises:
sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain processed monitoring data;
integrating the processed monitoring data in a charting form to respectively obtain table data in a table form and graphic data in a graphic form;
and analyzing by combining the table data and the graph data to obtain the analysis result.
8. An apparatus for processing microservice data, applied to a microservice processing platform, the apparatus comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for determining a target registration center containing registration information of the microservice and determining a target registration type of the target registration center;
the generating module is used for generating a target calling interface which is adaptive to a target open interface of the target registration center in a micro service registration adaptive engine of the micro service processing platform according to the target registration type;
the second acquisition module is used for acquiring the micro-service data of the micro-service according to the target calling interface and the target open interface;
and the processing module is used for writing the micro-service data into a database of the micro-service processing platform and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data which correspond to a plurality of micro-services one by one.
9. A microservice processing platform comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011556562.6A CN112738060B (en) | 2020-12-24 | 2020-12-24 | Method and device for processing micro-service data, micro-service processing platform and medium |
| PCT/CN2021/084251 WO2022134358A1 (en) | 2020-12-24 | 2021-03-31 | Microservice data processing method, apparatus, microservice processing platform, and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011556562.6A CN112738060B (en) | 2020-12-24 | 2020-12-24 | Method and device for processing micro-service data, micro-service processing platform and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112738060A CN112738060A (en) | 2021-04-30 |
| CN112738060B true CN112738060B (en) | 2022-11-18 |
Family
ID=75615666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011556562.6A Active CN112738060B (en) | 2020-12-24 | 2020-12-24 | Method and device for processing micro-service data, micro-service processing platform and medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112738060B (en) |
| WO (1) | WO2022134358A1 (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113656096A (en) * | 2021-08-10 | 2021-11-16 | 成都长城开发科技有限公司 | Microservice rapid integration system, method, device and storage medium |
| CN115442372A (en) * | 2022-09-16 | 2022-12-06 | 平安付科技服务有限公司 | Interface calling method and micro-service application system applying same |
| CN115269612B (en) * | 2022-09-26 | 2022-12-13 | 公安部信息通信中心 | Cross-platform multi-dimensional data fusion system based on micro-service |
| CN115934765B (en) * | 2022-10-11 | 2024-12-06 | 中电金信软件(上海)有限公司 | Data sharing method, device and electronic device between microservice systems |
| CN115604333B (en) * | 2022-10-12 | 2023-09-12 | 江苏赛融科技股份有限公司 | Distributed big data analysis service scheduling method and system based on dubbo |
| CN115914351A (en) * | 2022-11-30 | 2023-04-04 | 中国银行股份有限公司 | Data processing engine system and computer equipment |
| CN116155715A (en) * | 2023-02-15 | 2023-05-23 | 北京比特易湃信息技术有限公司 | Service treatment platform |
| CN116346773A (en) * | 2023-03-31 | 2023-06-27 | 河南合众伟奇云智科技有限公司 | Single registration method under multi-registration center environment |
| CN117609973B (en) * | 2024-01-23 | 2024-05-07 | 江苏博云科技股份有限公司 | Registration center nano-tube method, system, device and server |
| CN118714191B (en) * | 2024-08-28 | 2024-11-15 | 西安奇点能源股份有限公司 | Method for unified registration, discovery and calling of multi-type service |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10242073B2 (en) * | 2016-07-27 | 2019-03-26 | Sap Se | Analytics mediation for microservice architectures |
| CN109246251B (en) * | 2018-11-13 | 2021-01-22 | 杭州数梦工场科技有限公司 | Micro-service calling method, device, system, equipment and readable storage medium |
| CN109714319A (en) * | 2018-12-06 | 2019-05-03 | 深圳市中农网有限公司 | Management system, method, apparatus, computer equipment and the storage medium of micro services |
| CN109547570B (en) * | 2018-12-29 | 2021-07-06 | 深圳云天励飞技术有限公司 | Service registration method, device, registration center management equipment and storage medium |
| CN110262902B (en) * | 2019-05-16 | 2021-08-31 | 杭州朗和科技有限公司 | Information processing method and system, medium, and computing device |
| CN110392114B (en) * | 2019-07-26 | 2022-02-08 | 中国工商银行股份有限公司 | Service call processing system and method across multiple registration centers |
| CN110381163B (en) * | 2019-07-30 | 2022-09-20 | 普信恒业科技发展（北京）有限公司 | Method for forwarding service request by gateway node and gateway node |
| CN110990047B (en) * | 2019-11-06 | 2021-11-19 | 北京云思畅想科技有限公司 | Fusion method and device for multiple microservice architectures |
| CN111767095A (en) * | 2020-06-30 | 2020-10-13 | 平安国际智慧城市科技股份有限公司 | Microservice generation method, apparatus, terminal device and storage medium |
-
2020
- 2020-12-24 CN CN202011556562.6A patent/CN112738060B/en active Active
-
2021
- 2021-03-31 WO PCT/CN2021/084251 patent/WO2022134358A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| CN112738060A (en) | 2021-04-30 |
| WO2022134358A1 (en) | 2022-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112738060B (en) | Method and device for processing micro-service data, micro-service processing platform and medium | |
| CN109634728B (en) | Job scheduling method and device, terminal equipment and readable storage medium | |
| CN113971095A (en) | KUBERNETES application program interface in extended process | |
| EP3660665A1 (en) | Business processing method, apparatus, device and system using the same, and readable storage medium of the same | |
| CN112187933B (en) | Method and system for monitoring services in a multi-architecture cloud platform | |
| EP2972824B1 (en) | Computer system using in-service software upgrade | |
| CN113032125B (en) | Job scheduling method, job scheduling device, computer system and computer readable storage medium | |
| CN116795647A (en) | A database heterogeneous resource management and scheduling method, device, equipment and medium | |
| CN116302708A (en) | Data backup method, device, equipment and storage medium based on load balancing | |
| CN109889406A (en) | For managing method, apparatus, equipment and the storage medium of network connection | |
| CN102540953A (en) | Programmable logic controller and data processing method thereof | |
| CN115858499A (en) | Database partition processing method and device, computer equipment and storage medium | |
| CN113658351B (en) | Method and device for producing product, electronic equipment and storage medium | |
| CN114168297A (en) | Method, device, equipment and medium for scheduling collection tasks | |
| CN114327484A (en) | K8S integration and deployment method, system and storage medium supporting multi-architecture | |
| US9323509B2 (en) | Method and system for automated process distribution | |
| CN115484187B (en) | Method, equipment and storage medium for testing container network interface in container environment | |
| CN115357433A (en) | Database backup method, device, equipment and storage medium under container environment | |
| CN114610413A (en) | Method, device, equipment and storage medium for executing synchronous and asynchronous tasks based on Java | |
| CN113010363B (en) | Container monitoring method under swarm clusters and shared service cloud platform | |
| CN115344370A (en) | Task scheduling method, device, equipment and storage medium | |
| CN113031964A (en) | Management method, device, equipment and storage medium for big data application | |
| CN114064225A (en) | Self-adaptive scheduling method, device, computer storage medium and system | |
| CN113342540A (en) | Terminal user payment starting method, system, computer equipment and storage medium | |
| CN111461641A (en) | Approval flow processing method, system, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |