CN114035861B - Cluster configuration method, device, electronic device and computer readable medium - Google Patents
Cluster configuration method, device, electronic device and computer readable medium Download PDFInfo
- Publication number
- CN114035861B CN114035861B CN202111307620.6A CN202111307620A CN114035861B CN 114035861 B CN114035861 B CN 114035861B CN 202111307620 A CN202111307620 A CN 202111307620A CN 114035861 B CN114035861 B CN 114035861B
- Authority
- CN
- China
- Prior art keywords
- cluster
- configuration
- target
- result
- recommendation result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
- G06F9/4451—User profiles; Roaming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
本公开的实施例公开了集群配置方法、装置、电子设备和计算机可读介质。该方法的一具体实施方式包括：响应于确定对应于目标集群组件的配置任务满足预设触发条件，获取对应于目标集群组件的监控信息和对应于配置任务的集群配置信息；从集群配置信息中提取配置推荐算子；基于监控信息和配置推荐算子，生成集群配置推荐结果；响应于集群配置推荐结果和对应于配置推荐结果的目标集群的配置结果满足预设异常条件，将集群配置推荐结果发送至目标集群对应的各个集群节点，其中，目标集群组件部署在目标集群。该实施方式可以实现自动化地、动态地修改集群的资源配置。
The embodiments of the present disclosure disclose a cluster configuration method, device, electronic device, and computer-readable medium. A specific implementation of the method includes: in response to determining that a configuration task corresponding to a target cluster component meets a preset trigger condition, obtaining monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task; extracting a configuration recommendation operator from the cluster configuration information; generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator; in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result meeting a preset abnormal condition, sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster, wherein the target cluster component is deployed in the target cluster. This implementation can realize automatic and dynamic modification of the resource configuration of the cluster.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a cluster configuration method, apparatus, electronic device, and computer readable medium.
Background
With the development of big data and AI technology, a big data platform built on a cloud service IaaS (Infrastructure AS A SERVICE) layer is sequentially pushed out, and the big data platform can deploy a plurality of cluster components. At present, when resource allocation is performed on cluster components of a big data cluster, the method is generally adopted, namely, the allocation of each cluster node of the big data cluster is kept consistent, or the allocation is performed through operation and maintenance means or tools, for example, configuration information is completely issued into the big data cluster in an anstable mode, and the cluster components are restarted to take effect.
However, when the resource configuration is performed in the above manner, there are often technical problems that operation and maintenance personnel are required to manually operate, the configuration cannot be automatically modified, the configuration optimization depends on expert experience, and the configuration cannot be dynamically adjusted at any time.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose cluster configuration methods, apparatuses, electronic devices, and computer-readable media to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a cluster configuration method, which includes, in response to determining that a configuration task corresponding to a target cluster component meets a preset trigger condition, acquiring monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task, extracting a configuration recommendation operator from the cluster configuration information, generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator, and in response to the cluster configuration recommendation result and a configuration result of a target cluster corresponding to the configuration recommendation result meeting a preset exception condition, transmitting the cluster configuration recommendation result to each cluster node corresponding to the target cluster, wherein the target cluster component is deployed in the target cluster.
Optionally, the sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster includes sending the cluster configuration recommendation result to an associated display device, and sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to receiving a confirmation sending information record corresponding to the cluster configuration recommendation result.
Optionally, the method further comprises determining the configuration result as a target configuration result in response to receiving a cancel transmission information record corresponding to the cluster configuration recommendation result, or the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result do not meet the preset abnormal condition.
Optionally, the monitoring information is collected according to the cluster configuration information, and the generating of the cluster configuration recommendation result comprises the steps of inputting the monitoring information into the configuration recommendation operator to obtain the cluster configuration recommendation result.
Optionally, the acquiring the monitoring information corresponding to the target cluster component and the cluster configuration information corresponding to the configuration task in response to determining that the configuration task corresponding to the target cluster component meets the preset trigger condition includes acquiring the monitoring information corresponding to the target cluster component and the cluster configuration information corresponding to the configuration task in response to determining that the current time meets the preset period trigger condition included in the preset trigger condition.
Optionally, the sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster includes storing the cluster configuration recommendation result in a target database.
Optionally, the cluster configuration information includes configuration modification priority information, and the sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster includes sending the configuration modification priority information and the cluster configuration recommendation result to each cluster node.
In a second aspect, some embodiments of the present disclosure provide a cluster configuration apparatus, where the apparatus includes an obtaining unit configured to obtain, in response to determining that a configuration task corresponding to a target cluster component meets a preset trigger condition, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task, an extracting unit configured to extract a configuration recommendation operator from the cluster configuration information, a generating unit configured to generate a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator, and a transmitting unit configured to transmit the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result meeting a preset exception condition, where the target cluster component is deployed in the target cluster.
Optionally, the sending unit of the apparatus is further configured to send the cluster configuration recommendation result to an associated display device, and send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to receiving a confirmation sending information record corresponding to the cluster configuration recommendation result.
Optionally, the device further comprises a determining unit configured to determine the configuration result as a target configuration result in response to receiving a cancel transmission information record corresponding to the cluster configuration recommendation result, or the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result do not meet the preset abnormal condition.
Optionally, the monitoring information is collected according to the cluster configuration information. The generation unit is further configured to input the monitoring information into the configuration recommendation operator to obtain a cluster configuration recommendation result.
Optionally, the obtaining unit is further configured to obtain monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task in response to determining that the current time meets a preset period trigger condition included in the preset trigger condition.
Optionally, the sending unit comprises a storage unit configured to store the cluster configuration recommendation result to a target database.
Optionally, the cluster configuration information includes configuration modification priority information. The sending unit is further configured to send the configuration modification priority information and the cluster configuration recommendation result to the respective cluster nodes.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising one or more processors, and storage means having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the method described in any of the implementations of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.
The above embodiments of the present disclosure have the advantage that by the cluster configuration method of some embodiments of the present disclosure, an automatic and dynamic modification of the resource configuration of the clusters can be achieved. In particular, the inability to automatically and dynamically modify the resource configuration of a cluster is due to the manual operation required by the operation and maintenance personnel, the inability to automatically modify the configuration, and the inability to dynamically adjust the configuration at any time depending on expert experience. Based on this, the cluster configuration method of some embodiments of the present disclosure first obtains, in response to determining that a configuration task corresponding to a target cluster component meets a preset trigger condition, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task. Therefore, the monitoring information of the cluster component (namely, the target cluster component) corresponding to the configuration task which is triggered at present and the cluster configuration information of the configuration task which is triggered at present can be automatically obtained through the triggering condition corresponding to the configuration task. Then, a configuration recommendation operator is extracted from the cluster configuration information. And then, generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator. Therefore, the cluster configuration recommendation result can be automatically generated through the configuration recommendation operator and the monitoring information included in the cluster configuration information. And finally, responding to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result to meet a preset abnormal condition, and sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster. Wherein the target cluster component is deployed on the target cluster. Therefore, when the automatically generated cluster configuration recommendation result is different from the current configuration result of the target cluster, the cluster configuration recommendation result can be automatically pushed to each cluster node of the target cluster, so that each cluster node carries out configuration modification according to the cluster configuration recommendation result. And because the cluster configuration recommendation result can be automatically generated and pushed, the automatic and dynamic modification of the resource configuration of the cluster can be realized.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a schematic illustration of one application scenario of a cluster configuration method according to some embodiments of the disclosure;
FIG. 2 is a flow chart of some embodiments of a cluster configuration method according to the present disclosure;
FIG. 3 is a flow chart of further embodiments of a cluster configuration method according to the present disclosure;
FIG. 4 is a flow chart of yet other embodiments of cluster configuration methods according to the present disclosure;
FIG. 5 is a schematic structural diagram of some embodiments of a cluster configuration device according to the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 is a schematic diagram of one application scenario of a cluster configuration method according to some embodiments of the disclosure.
In the application scenario of fig. 1, first, in response to determining that the configuration task 102 corresponding to the target cluster component meets a preset trigger condition, the computing device 101 may obtain monitoring information 103 corresponding to the target cluster component and cluster configuration information 104 corresponding to the configuration task 102. The computing device 101 may then extract the configuration recommendation operator 105 from the cluster configuration information 104 described above. Thereafter, the computing device 101 may generate cluster configuration recommendations 106 based on the monitoring information 103 and the configuration recommendation operator 105. Finally, the computing device 101 may send the cluster configuration recommendation result 106 to each cluster node 108 corresponding to the target cluster in response to the cluster configuration recommendation result 106 and the configuration result 107 corresponding to the target cluster of the configuration recommendation result 106 meeting a preset abnormal condition. Wherein the target cluster component is deployed on the target cluster.
It should be noted that, the computing device 101 may be hardware, or may be software. When the computing device is hardware, the computing device may be implemented as a distributed cluster formed by a plurality of servers or terminal devices, or may be implemented as a single server or a single terminal device. When the computing device is embodied as software, it may be installed in the hardware devices listed above. It may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.
It should be understood that the number of computing devices in fig. 1 is merely illustrative. There may be any number of computing devices, as desired for an implementation.
With continued reference to fig. 2, a flow 200 of some embodiments of a cluster configuration method according to the present disclosure is shown. The cluster configuration method comprises the following steps:
In step 201, in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task are obtained.
In some embodiments, an execution body of the cluster configuration method (for example, the computing device 101 shown in fig. 1) may obtain, from a target database, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task through a wired connection manner or a wireless connection manner, in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition. The preset trigger condition may be "a trigger instruction for detecting the configuration task". The trigger instruction may be an instruction generated by a user manually triggering the configuration task. The configuration task may be a task for performing cluster configuration on the target cluster component. For example, the configuration task may be a timing task for performing cluster configuration on the target cluster component. The target cluster component may be a cluster component corresponding to a configuration task. The cluster component may be a big data component. For example, the target cluster component may be, but is not limited to, one of a hadoop component, a hbase component, and a ELASTICSEARH component. The target database may be a database storing cluster configuration information of each cluster component and monitoring information of each cluster component submitted by a user. The monitoring information can be real-time monitoring related information of the operation of the target cluster component, and can include but is not limited to basic information and component service information. The basic information may be related information of the device in operation, and may include, but not limited to, memory occupancy rate and CPU occupancy rate. The service information of the components can be related service information corresponding to the cluster components, and can include but not limited to query rate per second, reading speed, JVM maximum memory and JVM minimum memory. The cluster configuration information may be related information for configuring the target cluster component, which is included in the configuration task, and may include, but is not limited to, a cluster component name, a configuration parameter corresponding to the cluster component name, a configuration recommendation operator, and an execution period. The cluster configuration information may further include an identifier that characterizes whether the corresponding configuration task is retriable. The execution cycle may be a cycle that triggers a configuration task. The configuration parameters may include, but are not limited to, one of JVM memory, concurrency count, file chunk size, and number of card partitions. The configuration recommendation operator can be an operator which is self-configured by a user and used for generating a cluster configuration recommendation result by adding indexes and operators. For example, the configuration recommendation operator may be a formula configured by a user, where the index added by the user is an independent variable, and the configuration parameter is an independent variable. The configuration recommendation operator may be a code block of a method using an index added by a user as a parameter and using a value of the configuration parameter as a return value. By way of example, the cluster configuration information may be "cluster component name: HIVE, configuration parameter: HIVE. Heel size, configuration recommendation operator: HIVE. Heel. Max×1.2, execution period: 20 minutes".
It should be noted that the wireless connection may include, but is not limited to, 3G/4G connection, wiFi connection, bluetooth connection, wiMAX connection, zigbee connection, UWB (ultra wideband) connection, and other now known or later developed wireless connection.
In some optional implementations of some embodiments, the executing body may obtain the monitoring information corresponding to the target cluster component and the cluster configuration information corresponding to the configuration task in response to determining that the current time meets a preset period trigger condition included in the preset trigger condition. Wherein, the preset trigger condition includes a preset period trigger condition. The preset period trigger condition may be "the current time is the period trigger time of the configuration task". For example, the current time may be "2021/8/1:11:00". The period of the configuration task trigger may be 20 minutes. The last time the configuration task was triggered may be "2021/8/1 10:40". The current time "2021/8/1 11:00" is the periodic trigger time of the configuration task after "2021/8/1 10:40" described above. Therefore, the monitoring information and the cluster configuration information can be automatically and periodically obtained through the preset period triggering condition.
Step 202, extracting a configuration recommendation operator from cluster configuration information.
In some embodiments, the execution body may extract a configuration recommendation operator from the cluster configuration information. In practice, the execution body may extract, from the cluster configuration information, a field value with a field of "configuration recommendation operator" as a configuration recommendation operator.
And 203, generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator.
In some embodiments, the executing entity may generate a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator. In practice, first, the execution subject may extract, from the monitoring information, respective pieces of information corresponding to respective indices in the configuration recommendation operator as respective index value information. And then, respectively inputting the index value information into the configuration recommendation operators to obtain a cluster configuration recommendation result. For example, the configuration recommendation operator may be "hive. The index value information may be "hive. Map. Max:128G". The generated cluster configuration recommendation may be "hive. Heel size 153.6G" (153.6g=128 g×1.2).
Optionally, the monitoring information is collected according to the cluster configuration information. The monitoring information only includes information corresponding to each index in the configuration recommendation operator included in the cluster configuration information. The execution body can directly input the monitoring information into the configuration recommendation operator to obtain a cluster configuration recommendation result. Therefore, the utilization rate of the monitoring information can be improved, so that the monitoring information can be directly input into the configuration recommendation operator, and a cluster configuration recommendation result is obtained.
And step 204, in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result meeting the preset abnormal condition, sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster.
In some embodiments, the execution body may send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to the cluster configuration recommendation result and a configuration result of the target cluster corresponding to the configuration recommendation result meeting a preset exception condition. The target cluster may be a cluster of computers deploying the target cluster component. The configuration result may be a configuration parameter value of the target cluster corresponding to a configuration parameter included in the configuration recommendation result. For example, the configuration result may be "hive. Heel size:128G". The preset abnormal condition may be that the configuration recommendation result of the cluster is different from the configuration result. In practice, the execution body may send the cluster configuration recommendation result to each cluster node corresponding to the target cluster synchronously. It is understood that the execution body may be any node in the cluster nodes.
In some optional implementations of some embodiments, the execution body may further store the cluster configuration recommendation result to a target database. The target database may be a database storing the monitoring information and the cluster configuration information. For example, the target database may be a MYSQL database. Therefore, the sent cluster configuration recommendation result can be stored and backed up.
The above embodiments of the present disclosure have the advantage that by the cluster configuration method of some embodiments of the present disclosure, an automatic and dynamic modification of the resource configuration of the clusters can be achieved. In particular, the inability to automatically and dynamically modify the resource configuration of a cluster is due to the manual operation required by the operation and maintenance personnel, the inability to automatically modify the configuration, and the inability to dynamically adjust the configuration at any time depending on expert experience. Based on this, the cluster configuration method of some embodiments of the present disclosure first obtains, in response to determining that a configuration task corresponding to a target cluster component meets a preset trigger condition, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task. Therefore, the monitoring information of the cluster component (namely, the target cluster component) corresponding to the configuration task which is triggered at present and the cluster configuration information of the configuration task which is triggered at present can be automatically obtained through the triggering condition corresponding to the configuration task. Then, a configuration recommendation operator is extracted from the cluster configuration information. And then, generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator. Therefore, the cluster configuration recommendation result can be automatically generated through the configuration recommendation operator and the monitoring information included in the cluster configuration information. And finally, responding to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result to meet a preset abnormal condition, and sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster. Wherein the target cluster component is deployed on the target cluster. Therefore, when the automatically generated cluster configuration recommendation result is different from the current configuration result of the target cluster, the cluster configuration recommendation result can be automatically pushed to each cluster node of the target cluster, so that each cluster node carries out configuration modification according to the cluster configuration recommendation result. And because the cluster configuration recommendation result can be automatically generated and pushed, the automatic and dynamic modification of the resource configuration of the cluster can be realized.
With further reference to FIG. 3, a flow 300 of further embodiments of a cluster configuration method is shown. The cluster configuration method flow 300 includes the following steps:
in step 301, in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task are obtained.
Step 302, extracting a configuration recommendation operator from cluster configuration information.
Step 303, generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator.
In some embodiments, the specific implementation of steps 301 to 303 and the technical effects thereof may refer to steps 201 to 203 in those embodiments corresponding to fig. 2, which are not described herein.
And step 304, sending the cluster configuration recommendation result to the associated display equipment.
In some embodiments, an execution body of the cluster configuration method (e.g., the computing device 101 shown in fig. 1) may send the cluster configuration recommendation result to an associated display device in response to the cluster configuration recommendation result and a configuration result of a target cluster corresponding to the configuration recommendation result satisfying a preset exception condition, so that the display device displays the cluster configuration recommendation result.
And step 305, in response to receiving the confirmation sending information record corresponding to the cluster configuration recommendation result, sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster.
In some embodiments, the executing entity may send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to receiving a confirmation sending information record corresponding to the cluster configuration recommendation result. The confirmation sending information record may be an information record generated by the execution body and representing that the cluster configuration recommendation result has been confirmed to be sent after the user browses the displayed cluster configuration recommendation result through the display device and performs a confirmation sending operation on the displayed cluster configuration recommendation result. The acknowledgement sending operation may include, but is not limited to, clicking, sliding, hovering, dragging. Therefore, after the user confirms the automatically generated cluster configuration recommendation result, the cluster configuration recommendation result can be pushed.
Optionally, in response to receiving a cancel transmission information record corresponding to the cluster configuration recommendation result, or the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result do not satisfy the preset exception condition, the execution body may determine the configuration result as a target configuration result. The sending cancellation information record may be an information record generated by the execution body and representing that the cluster configuration recommendation result has been cancelled for sending after the user browses the displayed cluster configuration recommendation result through the display device and executes a sending cancellation operation on the displayed cluster configuration recommendation result. The cancel send operation may include, but is not limited to, click, swipe, hover, drag. The target configuration result may be a configuration result that characterizes the configuration result without modification. Therefore, when the user cancels the sending of the cluster configuration recommendation result or the cluster configuration recommendation result is the same as the configuration result, the original configuration of the target cluster can be still maintained.
As can be seen in fig. 3, the flow 300 of the cluster configuration method in some embodiments corresponding to fig. 3 embodies the steps of displaying the cluster configuration recommendation results and expanding the confirmation transmission information record, as compared to the description of some embodiments corresponding to fig. 2. Therefore, the solutions described in these embodiments can push the cluster configuration recommendation result after the user confirms the automatically generated cluster configuration recommendation result.
With further reference to fig. 4, a flow 400 of still further embodiments of a cluster configuration method is shown. The cluster configuration method flow 400 includes the following steps:
In step 401, in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition, monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task are obtained.
Step 402, extracting a configuration recommendation operator from cluster configuration information.
Step 403, generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator.
In some embodiments, the specific implementation of the steps 401 to 403 and the technical effects thereof may refer to the steps 201 to 203 in those embodiments corresponding to fig. 2, which are not described herein.
And step 404, transmitting configuration modification priority information and cluster configuration recommendation results to each cluster node.
In some embodiments, the execution body of the cluster configuration method (e.g., the computing device 101 shown in fig. 1) may send the configuration modification priority information and the cluster configuration recommendation result to the respective cluster nodes in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result satisfying a preset exception condition. Wherein the cluster configuration information includes the configuration modification priority information. The configuration modification priority information may be information indicating the degree of urgency of modifying the configuration or the degree of priority processing. For example, the configuration modification priority information may include, but is not limited to, one of high priority, medium priority, low priority. The high priority may indicate that if the current configuration parameter is abnormal, the cluster performance will be seriously affected, and the current configuration parameter needs to be modified immediately. The priority of modifying the configuration characterized by the medium priority is next to the high priority. The low priority characterizes a priority of modifying the configuration that is inferior to the medium priority. Therefore, the configuration modification priority information and the cluster configuration recommendation result can be simultaneously sent to each cluster node of the cluster, so that each node of the cluster sequentially carries out configuration modification according to the configuration modification priority information corresponding to the cluster configuration recommendation result.
As can be seen in fig. 4, the flow 400 of the cluster configuration method in some embodiments corresponding to fig. 4 embodies the step of expanding the configuration modification priority information as compared to the description of some embodiments corresponding to fig. 2. Therefore, the schemes described in the embodiments can simultaneously send the configuration modification priority information and the cluster configuration recommendation result to each cluster node of the cluster, so that each node of the cluster sequentially carries out configuration modification according to the configuration modification priority information corresponding to the cluster configuration recommendation result.
With further reference to fig. 5, as an implementation of the method shown in the foregoing figures, the present disclosure provides some embodiments of a cluster configuration apparatus, which correspond to those method embodiments shown in fig. 2, and which are particularly applicable in various electronic devices.
As shown in fig. 5, the cluster configuration device 500 of some embodiments includes an acquisition unit 501, an extraction unit 502, a generation unit 503, and a transmission unit 504. The obtaining unit 501 is configured to obtain monitoring information corresponding to a target cluster component and cluster configuration information corresponding to the configuration task in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition, the extracting unit 502 is configured to extract a configuration recommendation operator from the cluster configuration information, the generating unit 503 is configured to generate a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator, and the sending unit 504 is configured to send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result meeting a preset abnormal condition.
Optionally, the sending unit 504 of the cluster configuration apparatus 500 may be further configured to send the cluster configuration recommendation result to an associated display device, and send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to receiving the acknowledgement sending information record corresponding to the cluster configuration recommendation result.
Optionally, the cluster configuration device 500 may further include a determining unit (not shown in the figure) configured to determine the configuration result as the target configuration result in response to receiving the cancellation transmission information record corresponding to the cluster configuration recommendation result, or the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result do not satisfy the preset abnormal condition.
Optionally, the monitoring information is collected according to the cluster configuration information. The generating unit 503 may be further configured to input the monitoring information to the configuration recommendation operator to obtain a cluster configuration recommendation result.
Optionally, the obtaining unit 501 may be further configured to obtain the monitoring information corresponding to the target cluster component and the cluster configuration information corresponding to the configuration task in response to determining that the current time meets the preset period triggering condition included in the preset triggering condition.
Optionally, the sending unit 504 may include a storage unit (not shown in the figure) configured to store the cluster configuration recommendation result to the target database.
Optionally, the cluster configuration information includes configuration modification priority information. The sending unit 504 may be further configured to send the configuration modification priority information and the cluster configuration recommendation result to the respective cluster nodes.
It will be appreciated that the elements described in the apparatus 500 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 500 and the units contained therein, and are not described in detail herein.
Referring now to FIG. 6, a schematic diagram of an electronic device 600 (e.g., computing device 101 of FIG. 1) suitable for use in implementing some embodiments of the disclosure is shown. The electronic device shown in fig. 6 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, devices may be connected to I/O interface 605 including input devices 606, including for example, touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc., output devices 607, including for example, liquid Crystal Displays (LCDs), speakers, vibrators, etc., storage devices 608, including for example, magnetic tape, hard disk, etc., and communication devices 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 6 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 609, or from storage device 608, or from ROM 602. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing device 601.
It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be included in the electronic device or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to acquire monitoring information corresponding to a target cluster component and cluster configuration information corresponding to the configuration task in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition, extract a configuration recommendation operator from the cluster configuration information, generate a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator, and send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the configuration recommendation result meeting a preset abnormal condition, wherein the target cluster component is deployed in the target cluster.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, which may be described, for example, as a processor comprising an acquisition unit, an extraction unit, a generation unit and a transmission unit. The names of the units do not limit the unit itself in some cases, for example, the obtaining unit may also be described as "a unit that obtains monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task in response to determining that the configuration task corresponding to the target cluster component satisfies a preset trigger condition".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic that may be used include Field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems-on-a-chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (10)
1. A cluster configuration method, comprising:
Responding to the fact that a configuration task corresponding to a target cluster component meets a preset trigger condition, and acquiring monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task, wherein the target cluster component is a cluster component corresponding to the configuration task, and the cluster component is a big data component;
Extracting a configuration recommendation operator from the cluster configuration information, wherein the configuration recommendation operator is an operator which is self-configured by a user through adding indexes and operators and is used for generating a cluster configuration recommendation result;
Generating a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator, wherein the generating the cluster configuration recommendation result comprises the steps of extracting each piece of information corresponding to each index in the configuration recommendation operator from the monitoring information as each index value information;
And responding to the cluster configuration recommendation result and a configuration result of a target cluster corresponding to the cluster configuration recommendation result to meet a preset abnormal condition, and sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster, wherein the target cluster component is deployed in the target cluster, and the preset abnormal condition is that the cluster configuration recommendation result is different from the configuration result.
2. The method of claim 1, wherein the sending the cluster configuration recommendation to each cluster node corresponding to the target cluster comprises:
Sending the cluster configuration recommendation result to an associated display device;
And transmitting the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to receiving the confirmation transmission information record corresponding to the cluster configuration recommendation result.
3. The method of claim 2, wherein the method further comprises:
And determining the configuration result as a target configuration result in response to receiving a cancel sending information record corresponding to the cluster configuration recommendation result or the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the cluster configuration recommendation result do not meet the preset abnormal condition.
4. The method of claim 1, wherein the monitoring information is collected based on the cluster configuration information, and
The generating cluster configuration recommendation results includes:
And inputting the monitoring information into the configuration recommendation operator to obtain a cluster configuration recommendation result.
5. The method of claim 1, wherein the obtaining monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition comprises:
And responding to the preset period triggering condition included in the condition that the current time meets the preset triggering condition, and acquiring monitoring information corresponding to the target cluster component and cluster configuration information corresponding to the configuration task.
6. The method of claim 1, wherein the sending the cluster configuration recommendation to each cluster node corresponding to the target cluster comprises:
and storing the cluster configuration recommendation result to a target database.
7. The method of claim 1, wherein the cluster configuration information comprises configuration modification priority information, and
The sending the cluster configuration recommendation result to each cluster node corresponding to the target cluster includes:
And sending the configuration modification priority information and the cluster configuration recommendation result to each cluster node.
8. A cluster configuration device, comprising:
The acquisition unit is configured to acquire monitoring information corresponding to a target cluster component and cluster configuration information corresponding to a configuration task in response to determining that the configuration task corresponding to the target cluster component meets a preset trigger condition, wherein the target cluster component is a cluster component corresponding to the configuration task, and the cluster component is a big data component;
The extraction unit is configured to extract a configuration recommendation operator from the cluster configuration information, wherein the configuration recommendation operator is an operator which is self-configured by a user through adding indexes and operators and used for generating a cluster configuration recommendation result;
The generation unit is configured to generate a cluster configuration recommendation result based on the monitoring information and the configuration recommendation operator, wherein the generation unit is used for generating the cluster configuration recommendation result and comprises the steps of extracting each piece of information corresponding to each index in the configuration recommendation operator from the monitoring information as each index value information;
the sending unit is configured to send the cluster configuration recommendation result to each cluster node corresponding to the target cluster in response to the cluster configuration recommendation result and the configuration result of the target cluster corresponding to the cluster configuration recommendation result meeting a preset abnormal condition, wherein the target cluster component is deployed in the target cluster, and the preset abnormal condition is that the cluster configuration recommendation result is different from the configuration result.
9. An electronic device, comprising:
one or more processors;
A storage device having one or more programs stored thereon,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
10. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111307620.6A CN114035861B (en) | 2021-11-05 | 2021-11-05 | Cluster configuration method, device, electronic device and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111307620.6A CN114035861B (en) | 2021-11-05 | 2021-11-05 | Cluster configuration method, device, electronic device and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114035861A CN114035861A (en) | 2022-02-11 |
| CN114035861B true CN114035861B (en) | 2025-02-07 |
Family
ID=80136504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111307620.6A Active CN114035861B (en) | 2021-11-05 | 2021-11-05 | Cluster configuration method, device, electronic device and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114035861B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114896296B (en) * | 2022-02-24 | 2023-08-11 | 浙江华通云数据科技有限公司 | Cloud service resource allocation method and device, electronic equipment and computer readable medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262899A (en) * | 2019-06-20 | 2019-09-20 | 无锡华云数据技术服务有限公司 | Monitor component elastic telescopic method, apparatus and controlled terminal based on Kubernetes cluster |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8996568B2 (en) * | 2009-07-14 | 2015-03-31 | Qualcomm Incorporated | Methods and apparatus for efficiently processing multiple keyword queries on a distributed network |
| US11009836B2 (en) * | 2016-03-11 | 2021-05-18 | University Of Chicago | Apparatus and method for optimizing quantifiable behavior in configurable devices and systems |
| KR20180086791A (en) * | 2017-01-23 | 2018-08-01 | 한국전자통신연구원 | Cloud system for surpporting big data process and managing method thereof |
| CN108776696B (en) * | 2018-06-05 | 2023-04-18 | 平安科技（深圳）有限公司 | Node configuration method and device, storage medium and electronic equipment |
| CN112333010B (en) * | 2020-10-23 | 2022-06-21 | 苏州浪潮智能科技有限公司 | Method and device for supporting automatic configuration of high-availability environment |
| CN113032412B (en) * | 2021-03-03 | 2023-12-05 | 北京沃东天骏信息技术有限公司 | Data synchronization method, device, electronic equipment and computer readable medium |
-
2021
- 2021-11-05 CN CN202111307620.6A patent/CN114035861B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262899A (en) * | 2019-06-20 | 2019-09-20 | 无锡华云数据技术服务有限公司 | Monitor component elastic telescopic method, apparatus and controlled terminal based on Kubernetes cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114035861A (en) | 2022-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112363879B (en) | Cache server performance test method, device, equipment and medium | |
| CN110795446A (en) | List updating method and device, readable medium and electronic equipment | |
| CN112015816A (en) | Data synchronization method, device, medium and electronic equipment | |
| CN113760536A (en) | Data caching method and device, electronic equipment and computer readable medium | |
| CN112907942A (en) | Vehicle scheduling method, device, equipment and medium based on edge calculation | |
| CN113722055A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN114035861B (en) | Cluster configuration method, device, electronic device and computer readable medium | |
| CN113486968B (en) | Method, device, equipment and medium for monitoring life cycle of camera | |
| CN112464039B (en) | Tree-structured data display method and device, electronic equipment and medium | |
| CN111625692B (en) | Feature extraction method, device, electronic equipment and computer readable medium | |
| CN111898061B (en) | Method, apparatus, electronic device and computer readable medium for searching network | |
| CN111612434B (en) | Method, apparatus, electronic device and medium for generating processing flow | |
| CN112507676B (en) | Method and device for generating energy report, electronic equipment and computer readable medium | |
| CN115102992B (en) | Data publishing method and device, electronic equipment and computer readable medium | |
| CN110245120A (en) | The daily record data processing method of streaming computing system and streaming computing system | |
| CN114697206B (en) | Method, device, equipment and computer readable medium for managing nodes of Internet of things | |
| CN112818139A (en) | Edge calculation data management method, device and equipment applied to security monitoring | |
| CN112182002A (en) | Data disaster tolerance method and device, electronic equipment and computer readable medium | |
| CN114692898A (en) | MEC federated learning method, device and computer-readable storage medium | |
| CN113778850A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN112543228A (en) | Data transmission method and device, electronic equipment and computer readable medium | |
| CN112256714A (en) | Data synchronization method, apparatus, electronic device and computer readable medium | |
| CN112559087B (en) | Information generation method, apparatus, electronic device and computer readable medium | |
| CN114629821B (en) | Internet of things consumption data generation method, device, equipment and medium | |
| CN116820354B (en) | Data storage method, data storage device and data storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |