CN111770172A - Messaging middleware consumption proxy method, device, proxy server and storage medium - Google Patents

Abstract

The invention relates to the technical field of financial technology (Fintech), and discloses a message middleware consumption proxy method, a device, a proxy server and a storage medium; the method comprises the following steps: when detecting that the proxy server is started, accessing a configuration center corresponding to the proxy server to acquire configuration information; determining a consumption group according to the configuration information, and starting an agent cluster according to the consumption group; obtaining subscription information in the message middleware through the agent cluster, and storing the subscription information to a buffer pool; and sending the subscription information in the buffer pool to each consumption node corresponding to the consumption group according to the configuration information. The invention controls and starts the proxy cluster through the proxy server by presetting the proxy server, accesses the message middleware through the proxy cluster, reduces the operation and maintenance cost of the consumption node, and improves the safety and the stability of the message middleware.

Description

Messaging middleware consumption proxy method, device, proxy server and storage medium

technical field

The present invention relates to the technical field of financial technology (Fintech), and in particular, to a message middleware consumption proxy method, device, proxy server and storage medium.

Background technique

With the rapid development of Internet technology, especially Internet financial technology (Fintech), more and more technologies (big data, distributed, artificial intelligence, etc.) are applied in the financial field, causing the explosion of financial business-related business data and financial users growth.

There are more and more data interactions between financial business platforms and financial users. Currently, financial business platforms integrate distributed systems through message middleware (message middleware is based on data communication, and message middleware provides message delivery and message queuing models by providing , extending the communication between processes in a distributed environment) to interact with the consumer nodes corresponding to financial users. The current interaction protocol between the message middleware and the consumer nodes is complex, and the consumer nodes need to be developed according to the external interface provided by the message middleware. , the development cost of the code is high. When the protocol changes, the code of the consumer node needs to be changed to upgrade the consumer node, and the code maintenance cost is high. The upgrade operation of the consumer node is troublesome; and because the number of consumer nodes accessing the message middleware is uncontrollable, Too many accesses will affect the reliability and performance of the message middleware; in addition, when the message middleware interacts with the consumer node, there are some security risks when the consumer node directly accesses the message middleware.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a message middleware consumption proxy method, device, proxy server and storage medium, aiming to solve the current interaction between the consumption node and the message middleware, the consumption node code development and maintenance cost is high, and the excessive consumption nodes Access will affect the performance of the message middleware, and the message middleware has technical problems of potential security risks.

In order to achieve the above object, the present invention provides a message middleware consumption proxy method, and the message middleware consumption proxy method includes the following steps:

When detecting that the proxy server is started, access the configuration center corresponding to the proxy server to obtain configuration information;

Determine a consumption group according to the configuration information, and start an agent cluster according to the consumption group;

Obtain subscription information in the message middleware through the proxy cluster, and save the subscription information in a buffer pool;

Send the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information.

Optionally, before the step of accessing the configuration center corresponding to the proxy server to obtain configuration information when it is detected that the proxy server is started, the method includes:

Create a configuration center corresponding to the proxy server;

When receiving the proxy configuration request, output the information configuration page;

Acquire the configuration information input based on the information configuration page, and save the configuration information to the configuration center.

Optionally, the step of determining a consumption group according to the configuration information and starting a proxy cluster according to the consumption group includes:

Obtain the consumer node identifier in the configuration information and the subscription topic corresponding to the consumer node identifier, and form a consumer group with the consumer node identifiers with the same subscription topic;

Each consumer group is individually associated with a preset proxy node, the associated preset proxy nodes are formed into a proxy cluster, and the proxy cluster is started.

Optionally, the step of obtaining the subscription information in the message middleware through the proxy cluster and saving the subscription information to the buffer pool includes:

Obtain the subscription topic in the configuration information, and obtain the subscription information corresponding to the subscription topic in the message middleware through the proxy cluster;

Obtain the number of nodes of the consumer node in the consumer group;

When the number of the consuming nodes is at least two, the subscription information is saved to the buffer pool.

Optionally, after the step of obtaining the subscription information in the message middleware through the proxy cluster and saving the subscription information to the buffer pool, the method includes:

Detecting whether all the proxy nodes in the proxy cluster are successfully started;

When all the proxy nodes in the proxy cluster are not successfully started, determine the target proxy node that fails to start, start the hot device node corresponding to the target proxy node, and obtain the subscription information in the message middleware through the hot device node , and save the subscription information to the buffer pool;

When all the proxy nodes in the proxy cluster are successfully started, the steps of acquiring the subscription information in the message middleware through the proxy cluster and saving the subscription information in the buffer pool are performed.

Optionally, after the step of sending the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information, the method includes:

If the confirmation character information fed back by the consuming node is not received within a preset time period, execute the sending of the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information until the confirmation character information fed back by the consuming node is received.

Optionally, the message middleware consumption proxy method includes:

When detecting that a new message subscription request is added to the message queue corresponding to the consumer node, obtain the consumer node identifier corresponding to the message subscription request;

Access the configuration center, obtain the consumer group to which the consumer node identifier in the configuration information belongs, and start the proxy cluster according to the consumer group;

Obtain subscription information in the message middleware through the proxy cluster, and send the subscription information to the consuming node.

In addition, in order to achieve the above object, the present invention also provides a message middleware consumption proxy device, the message middleware consumption proxy device includes:

an access acquisition module, configured to access the configuration center corresponding to the proxy server to acquire configuration information when detecting that the proxy server is started;

a node startup module, configured to determine a consumption group according to the configuration information, and start an agent cluster according to the consumption group;

an acquiring and saving module, configured to acquire subscription information in the message middleware through the proxy cluster, and save the subscription information to a buffer pool;

an information sending module, configured to send the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information.

In addition, in order to achieve the above object, the present invention also provides a proxy server, the proxy server includes: a memory, a processor, and a message middleware consumption proxy program stored on the memory and running on the processor, When the message middleware consuming proxy program is executed by the processor, the steps of the above message middleware consuming proxy method are implemented.

In addition, in order to achieve the above object, the present invention also provides a computer storage medium, where a message middleware consumption agent program is stored on the computer storage medium, and the message middleware consumption agent program is executed by a processor to realize the above-mentioned Steps of message middleware consuming proxy method.

The present invention provides a message middleware consumption proxy method, device, proxy server and storage medium. In the embodiment of the present invention, when it is detected that the proxy server is started, the configuration center corresponding to the proxy server is accessed to obtain configuration information; a consumption group is determined according to the configuration information, and a proxy cluster is started according to the consumption group; The cluster acquires the subscription information in the message middleware, saves the subscription information in a buffer pool, and sends the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information. In the embodiment of the present invention, a proxy server is preset. When the proxy server is started, it accesses the configuration center to obtain configuration information. The proxy server starts the proxy cluster according to the configuration information, and accesses the message middleware through the proxy cluster. In this way, when the protocol of the message middleware is changed, it only needs to update Configuration information, the proxy server can start the corresponding proxy cluster according to the new configuration information to obtain the subscription information in the message middleware, without the need to upgrade each consumer node, reduce the operation and maintenance cost of the consumer node, and at the same time According to the configuration information, each proxy node in the proxy cluster can proxy multiple consumer nodes, so that there are fewer connections to the consumer nodes, which improves the stability of the message middleware and reduces the performance occupation of the message middleware; in addition, because the proxy The proxy nodes in the cluster are controllable, so that the message middleware can be prevented from being attacked and the security of the message middleware is improved.

Description of drawings

1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention;

FIG. 2 is a schematic flowchart of the configuration of a consumption agent in the first embodiment of the message middleware consumption agent method of the present invention;

3 is a schematic flowchart of a first embodiment of a message middleware consumption proxy method according to the present invention;

FIG. 4 is a schematic diagram of functional modules of an embodiment of a message middleware consumption proxy device according to the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The current message middleware access method uses the consumption of complex interaction protocols based on TCP (Transmission Control Protocol TCP is a connection-oriented, reliable, byte stream-based transport layer communication protocol). The node (also called the client) accesses the message middleware, and the steps for the consuming node to subscribe to the message include:

1. The message middleware (for example, MQServer1 and MQServer2) registers its own information with the registry (the registry is also called the Registry, and the registry contains subscription topics and service information. For example, the registry records T1 is maintained on MQServer1, and T2 is maintained on MQServer1. maintain routing information on MQServer2);

2. The consumer node subscribes to T1, and the consumer node queries the Registry to obtain the routing information of T1;

3. The consumer node determines that the message of the T1 subscription topic is maintained in MQServer1 according to the routing information, and the consumer node establishes a long link with MQServer1;

4. The consumer node uses the established long link to initiate a continuous long polling consumption request to MQServer1, and sends the message to the local for continuous consumption. At this time, the rhythm of consumption depends on whether a new message is delivered on T1. .

In the current consumption mode of message middleware based on TCP, the interaction protocol between the consumer node and the message middleware is complex, the message subscription logic in the consumer node is cumbersome, and the logic block of the consumer node needs to execute, including but not limited to maintaining the instance heartbeat state. , load balancing of sending and consumption, ACL (AccessControlLis, control access list) authentication, flow control, ACK (Acknowledgement, confirmation character) mechanism, Trace tracking operations, any major change in any of the logic blocks needs to be promoted The consumer node on the consumer side updates the code, so the operation and maintenance of the consumer node is complicated, the upgrade operation of the consumer node is complicated, and the interaction protocol between the consumer node and the message middleware is cumbersome, and the development and maintenance cost of the consumer node in different languages is too high , it is currently difficult to meet the demand for language diversity of consumer nodes.

In view of the problems existing in the above-mentioned consumption mode of message middleware based on TCP, an embodiment of the middleware consumption proxy method of the present invention is proposed; in the embodiment of the present invention, the method is based on the HTTP (HTTP, Hyper Text Transfer Protocol) protocol, which The method includes: when it is detected that a proxy server is started, accessing a configuration center corresponding to the proxy server to obtain configuration information; determining a consumption group according to the configuration information, and starting a proxy cluster according to the consumption group; obtaining through the proxy cluster The subscription information in the message middleware is stored in a buffer pool; the subscription information in the buffer pool is sent to each consuming node corresponding to the consuming group according to the configuration information.

The use of the HTTP-based message middleware consumption proxy method in the embodiment of the present invention effectively solves the problems existing in the TCP-based message middleware consumption mode, and has the following advantages, including:

1. By maintaining the heartbeat state, load balancing between sending and consumption, ACL authentication, flow control, ACK mechanism, trace tracking, etc. placed in the logic block of the proxy server, it does not need to be maintained by the consumer node, that is, it can be maintained by itself , there is no need to frequently promote the upgrade of consumer nodes, which reduces the difficulty of operation and maintenance. Migrate the logic block of the consumer node to the proxy server, then the proxy server can only retain the transmission and consumption carried by the HTTP protocol, and consume the ACK command. To develop a proxy node in a certain language only needs to implement a few simple HTTP commands. Reduce the workload of language diversification.

2. The consumer node establishes a connection with the proxy server, and the proxy server controls the proxy nodes in the proxy cluster to access the message middleware. The number of proxy nodes is limited, which greatly reduces the number of TCP long links on the message middleware and enhances the message middleware. The reliability of the server, and the security check can eliminate the security risk of directly exposing the message middleware server to the consumer node.

As shown in FIG. 1 , FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention.

The message middleware consumption proxy method according to the embodiment of the present invention is applied to a proxy server.

As shown in FIG. 1 , the proxy server may include: a processor 1001 , such as a CPU, a network interface 1004 , a user interface 1003 , a memory 1005 , and a communication bus 1002 . Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (eg, a WI-FI interface). The memory 1005 may be high-speed RAM memory, or may be non-volatile memory, such as disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the device structure shown in FIG. 1 does not constitute a limitation on the device, and may include more or less components than the one shown, or combine some components, or arrange different components.

As shown in FIG. 1 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a message middleware consumption agent program.

In the device shown in FIG. 1 , the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect the consumer node (client) and perform data communication with the consumer node; and the processor 1001 can be used to call the message middleware consuming proxy program stored in the memory 1005, and perform the operations in the following message middleware consuming proxy method.

Based on the above hardware structure, an embodiment of the message middleware consumption proxy method of the present invention is proposed.

When detecting that the proxy server is started, access the configuration center corresponding to the proxy server to obtain configuration information;

Determine a consumption group according to the configuration information, and start an agent cluster according to the consumption group;

Obtain subscription information in the message middleware through the proxy cluster, and save the subscription information in a buffer pool;

Send the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information.

The message middleware consumption proxy method in this embodiment is applied to a proxy server, and the proxy server is the hardware carrier of the message middleware consumption proxy computer program. The proxy server can also be set in the front end of the message middleware.

For ease of understanding, in this embodiment, the proxy server is set at the front end of the message middleware as an example for description.

Before the steps of the first embodiment of the message middleware consumption proxy method of the present invention, a consumption proxy framework is created in advance, and the message push operation is performed through the consumption proxy framework. Referring to FIG. 2, FIG. 2 is the first implementation of the message middleware consumption proxy method of the present invention. In the example, a schematic flowchart of the configuration of the consumption agent, in this embodiment, the steps of the configuration of the consumption agent include:

Step S01, creating a configuration center corresponding to the proxy server;

The proxy server creates the consumer proxy framework based on the public code information of the consumer proxy pre-written by the developer. The consumer proxy framework refers to the execution of the public code of the consumer proxy. The developer writes the function code in the consumer proxy framework, and the proxy server sets the configuration center synchronously. The center is a data table or database used to save configuration information. Configuration information refers to the subscription topic, subscription time, and consumer node identification of each consumer node configured by the user or developer. The configuration information calls the proxy cluster (the proxy cluster replaces the consumer node to obtain subscription messages from the message middleware. Each proxy node in the proxy cluster can replace one consumer node or multiple consumer nodes, and obtain each consumer node from the message middleware. subscribed subscription messages, then save the subscription messages to the cache, and then push the subscription messages in the cache to each consumer node), and perform the consumption proxy operation through the proxy cluster, specifically:

Step S02, when receiving the proxy configuration request, output the information configuration page;

The user triggers the proxy configuration request on the proxy server. The triggering method of the proxy configuration request is not specifically limited. When the proxy server receives the proxy configuration request, the proxy server outputs the information configuration page for the user to input the information of each consumer node on the information configuration page. Configuration information such as subscription topic, subscription time, and consumer node ID.

Step S03: Acquire configuration information input based on the information configuration page, and save the configuration information to a configuration center.

The proxy server obtains the configuration information input based on the information configuration page, wherein the configuration information includes a mapping relationship (the mapping relationship can be a one-to-one mapping, or a one-to-many mapping, preferably a one-to-many mapping, which can prevent the message middleware from connecting. The proxy configuration information and consumption configuration information of the incoming long links), the proxy configuration information refers to the proxy cluster information that has a communication connection with the proxy server, and the consumption configuration information refers to the subscription topic, subscription time and Consumer node identification, etc., the proxy server then saves the proxy configuration information and the consumption configuration information to the configuration center according to different dimensions.

It can be understood that the configuration information in this embodiment can be updated in real time, that is, the time when the proxy configuration request is triggered in this embodiment is not limited, for example, the proxy configuration request can be triggered during the process of executing the consumption proxy; The proxy configuration request can be triggered; the triggering method of the proxy configuration request is not specifically limited. The proxy configuration request can be automatically triggered by the proxy server. For example, when a new consumer node is detected to access the message middleware, the proxy server automatically triggers the proxy configuration request. To obtain the configuration information of the consuming node, and store the configuration information in association with the consuming node; or the proxy configuration request can be triggered manually, for example, when the user clicks the "Configure" virtual button on the display interface of the proxy server to trigger the proxy Configure the request.

In this embodiment, the proxy server is used to perform the consumption proxy operation of the message middleware, so that the consuming node does not need to directly connect with the data interface provided by the message middleware. When the message middleware is upgraded, only the configuration information in the configuration center needs to be updated, and the proxy server The consumption agent operation can be carried out according to the latest configuration information, and there is no need to upgrade each consumption node. The maintenance of the consumption node is relatively convenient, and the long link accessed by the message middleware comes from the proxy node controlled by the proxy server. A proxy node can proxy Multiple consumer nodes, so that the number of long links connected in the message middleware is small, and the phenomenon of data overload in the message middleware will not occur, which improves the stability and reliability of the message middleware. Each proxy node controlled by the proxy server is verified in advance, so that the security of the message middleware is better guaranteed.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of a first embodiment of a message middleware consumption proxy method according to the present invention, and the method includes:

In step S10, when it is detected that the proxy server starts, the configuration center corresponding to the proxy server is accessed to obtain configuration information.

The proxy server is provided with a listener. When the proxy server starts, the proxy server accesses the configuration center set by default, and obtains the configuration information in the configuration center, so that the proxy server starts each proxy node in the proxy cluster according to the configuration information to execute the consumption proxy operation. , that is, the proxy server determines whether the configuration information includes proxy configuration information and consumption configuration information with a mapping relationship; if the configuration information includes proxy configuration information and consumption configuration information with a mapping relationship, the proxy server according to the mapping relationship in the configuration information, Start each proxy node in the proxy cluster. In this embodiment, start each proxy node in the proxy cluster according to the mapping relationship. The proxy server can establish a mapping relationship according to information such as network addresses, so that the message proxy is more convenient; The proxy configuration information and consumption configuration information of the mapping relationship. The proxy server randomly starts each proxy node in the proxy cluster. In this embodiment, each proxy node in the proxy cluster is randomly activated. In this way, each proxy node is a hot device for each other. When a proxy node fails, it can be replaced by other proxy nodes to avoid the problem of proxy exceptions; it is understandable that the proxy server and the proxy nodes in the proxy cluster are distributed architectures, and the proxy cluster has the proxy to access the configuration center Nodes can act as proxy servers.

Step S20: Determine a consumption group according to the configuration information, and start a proxy cluster according to the consumption group.

The proxy server determines the consumption group according to the configuration information, that is, the configuration center contains pre-configured configuration information, and the proxy server uses the The consumer nodes corresponding to the node identifiers are divided into groups to form multiple consumer groups. For example, the proxy server uses the consumer nodes corresponding to the consumer node identifiers with the same subscription topic as a consumer group; or the proxy server uses the same subscription time. The consumer node corresponding to each consumer node identifier is used as a consumer group; or the proxy server determines the network address information of each consumer node according to the consumer node identifier, etc., and the proxy server divides the consumer node corresponding to each consumer node identifier according to the network address information. Consumption group; the proxy server associates each consumption group with the proxy node, and the proxy server uses the associated proxy node as the proxy cluster, and then controls to start the proxy cluster.

Step S20 in this embodiment specifically includes:

Step a1, acquiring the consumer node identifier in the configuration information and the subscription topic corresponding to the consumer node identifier, and forming a consumer group with the consumer node identifiers with the same subscription topic;

In step a2, each consumer group is individually associated with a preset proxy node, the associated preset proxy nodes are formed into a proxy cluster, and the proxy cluster is started.

The proxy server obtains the consumer node identifier in the configuration information, and the subscription topic corresponding to each consumer node identifier, and the proxy server forms a consumer group with the consumer node identifier with the same subscription topic, that is, in this embodiment, the proxy server subscribes the topic in the configuration information The same consumer nodes form a consumer group, and each consumer group is individually associated with a preset proxy node (preset proxy node refers to a client with a preset specific message subscription logic), and the associated preset proxy nodes are formed. Agent cluster, and start the agent cluster to perform message subscription operation.

It can be understood that, in this embodiment, the consumption group is determined by taking the subscription topic as an example to divide. In the actual application process, the proxy server can also divide the consumption group according to other dimensions, which makes the message subscription more flexible.

For example, in this embodiment, the proxy server is connected to the proxy cluster in communication, and the proxy node in the proxy cluster replaces the consumer node to obtain subscription information from the message middleware, and feeds back the subscription information to the consumer node. There are many long links. In this embodiment, each proxy node in the proxy cluster may correspond to a consumer group (the number of consumer nodes included in a consumer group is not limited. Including multiple consuming nodes), for example, there are actually 100 consuming nodes, but these 100 consuming nodes have certain homogeneity. The 30 consumer nodes N49 are the nodes of the business system S2, and the other 50 consumer nodes N50-N99 are the nodes of the business system S3; the proxy server only needs to construct three homogeneous consumer groups for S1, S2 and S3 respectively. , each consumer group is associated with a proxy node, that is, through 3 proxy nodes, the message distribution of 100 nodes can be completed.

However, it needs to face the problem of data distribution within the group, because on the proxy server there is a one-to-many relationship between the proxy node and the consumer node. For example, when the message arrives at S2, it needs to be distributed to N20-N49 according to certain rules, which is more suitable for consumer nodes. In many scenarios, steps S30 to S40 in this embodiment specifically describe the steps of consumption distribution.

Step S30: Obtain subscription information in the message middleware through the proxy cluster, and save the subscription information in a buffer pool.

The proxy server obtains the subscription information in the message middleware through the proxy cluster, that is, the proxy server obtains the subscription information of each consumer group through each proxy node in the proxy cluster. A consumer group includes one or more consumer nodes. At this time, the proxy node cannot directly perform message distribution operations. Therefore, the proxy server saves the acquired subscription information to the buffer pool of the proxy server.

Step S40, sending the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information.

The proxy server sends the subscription information in the buffer pool to each consumer node corresponding to the consumer group according to the configuration information, that is, the proxy server determines the consumer node identifier corresponding to the subscription information in the buffer pool, and the proxy server distributes the subscription information according to the consumer node identifier. to the consumer node.

It should be added that the proxy server in the first embodiment of the message middleware consumption proxy method of the present invention adopts the Push model, and the proxy server needs to maintain subscription information. This method is simple enough and does not introduce the risk of insufficient resources.

In the embodiment of the present invention, a proxy server is preset. When the proxy server is started, it accesses the configuration center to obtain configuration information. The proxy server starts the proxy cluster according to the configuration information, and accesses the message middleware through the proxy cluster. In this way, when the protocol of the message middleware is changed, it only needs to update Configuration information, the proxy server can start the corresponding proxy cluster according to the new configuration information to obtain the subscription information in the message middleware, without the need to upgrade each consumer node, reduce the operation and maintenance cost of the consumer node, and at the same time According to the configuration information, each proxy node in the proxy cluster can proxy multiple consumer nodes, so that there are fewer connections to the consumer nodes, which improves the stability of the message middleware and reduces the performance occupation of the message middleware; in addition, because the proxy The proxy nodes in the cluster are pre-authenticated, so the message middleware is unavoidably attacked, which improves the security of the message middleware.

After step S40 of the first embodiment, in order to prevent the consuming node from not receiving the subscription message, a determination step is set in this embodiment, which specifically includes:

If the confirmation character information fed back by the consuming node is not received within a preset time period, execute the sending of the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information until the confirmation character information fed back by the consuming node is received.

When the proxy server does not receive the confirmation character information fed back by the consuming node during the preset time period (the preset time period can be flexibly set according to the specific scenario), the proxy server executes sending the subscription information in the buffer pool to all users according to the configuration information. The steps of each consuming node corresponding to the consuming group are performed until the confirmation character information fed back by the consuming node is received.

That is, in this embodiment, when the proxy server pushes the message to the consumer node, the network is disconnected, the host is powered off, faulty, or the business party has a program failure when processing the message, which will cause the in-transit message delivery to fail. , the proxy server believes that the delivery has been successful, but the message has not actually reached the consumer node. The ACK mechanism is introduced. When the consumer node receives the message, it responds to the proxy server with ACK confirmation information. For the proxy server, the message that has been delivered but has not received the ACK response will be retried and sent to the same consumer group. The instance of , and then Push N times, the blocking synchronous ACK method or the non-blocking asynchronous ACK method can be used here. In order to improve the Push performance of the proxy server, the non-blocking asynchronous ACK method is adopted.

Further, based on the first embodiment of the message middleware consumption proxy method of the present invention, a second embodiment of the message middleware consumption proxy method of the present invention is proposed.

This embodiment is a refinement of step S30 in the first embodiment, and the difference between this embodiment and the above-mentioned embodiment is:

Obtain the subscription topic in the configuration information, and obtain the subscription information corresponding to the subscription topic in the message middleware through the proxy cluster;

Obtain the number of nodes of the consumer node in the consumer group;

When the number of the consuming nodes is at least two, the subscription information is saved to the buffer pool.

The proxy server obtains the subscription topic in the configuration information, and obtains the subscription information corresponding to the subscription topic in the message middleware through the proxy cluster; obtains the number of nodes of the consumer node in the consumer group; when the number of nodes of the consumer node is one, that is, the proxy server It is determined that the subscription message does not need to be cached, and the proxy server pushes the subscription information to the consuming node; when the number of consuming nodes is at least two, the proxy server determines that the subscription message needs to be distributed, and the proxy server saves the subscription information to the buffer pool. In this embodiment, when the number of consuming nodes in the consuming group is one, messages are sent directly without joining the buffer pool, which reduces the operations of consuming nodes and speeds up the efficiency of message delivery.

Further, based on the above embodiments of the message middleware consumption proxy method of the present invention, a third embodiment of the message middleware consumption proxy method of the present invention is proposed.

This embodiment is a step after step S20 in the first embodiment. The difference between this embodiment and the above-mentioned embodiment is:

Detecting whether all the proxy nodes in the proxy cluster are successfully started;

When all the proxy nodes in the proxy cluster are not successfully started, determine the target proxy node that fails to start, start the hot device node corresponding to the target proxy node, and obtain the subscription information in the message middleware through the hot device node , and save the subscription information to the buffer pool;

When all the proxy nodes in the proxy cluster are successfully started, the steps of acquiring the subscription information in the message middleware through the proxy cluster and saving the subscription information in the buffer pool are performed.

The proxy server detects whether all the proxy nodes in the proxy cluster have been successfully started; when all the proxy nodes in the proxy cluster are not successfully started, the proxy server determines the target proxy node that fails to start, and starts the hot device node corresponding to the target proxy node. The server also obtains the subscription information in the message middleware through the hot device node, and saves the subscription information in the buffer pool; when all the proxy nodes in the proxy cluster are successfully started, the proxy server executes the described obtaining message through the proxy cluster The subscription information in the middleware, the step of saving the subscription information to the buffer pool.

In this embodiment, the proxy server determines that each proxy node in the proxy cluster is each other's hot devices, and when one proxy node fails, it is replaced by another hot device node, that is, the subscription information maintained by different proxy nodes in the same proxy cluster is Unified and homogeneous, each agent node in the cluster is a hot device node for each other, so that the consumer agent has high reliability.

Further, based on the above embodiments of the message middleware consumption proxy method of the present invention, a fourth embodiment of the message middleware consumption proxy method of the present invention is proposed.

The difference between this embodiment and the above-mentioned embodiment is:

When detecting that a new message subscription request is added to the message queue corresponding to the consumer node, obtain the consumer node identifier corresponding to the message subscription request;

Access the configuration center, obtain the consumer group to which the consumer node identifier in the configuration information belongs, and start the proxy cluster according to the consumer group;

Obtain subscription information in the message middleware through the proxy cluster, and send the subscription information to the consuming node.

In this embodiment, when the proxy server detects a new message subscription request in the message queue corresponding to the consumer node, the proxy server obtains the consumer node identifier corresponding to the message subscription request; the proxy server accesses the configuration center to obtain the consumer node identifier to which the consumer node identifier in the configuration information belongs. group, the proxy server starts the proxy cluster according to the consumer group; the proxy server obtains the subscription information in the message middleware through the proxy cluster, and sends the subscription information to the consumer node.

The difference between this embodiment and the first embodiment is that:

In the first embodiment, the proxy server can adopt the Push model. When the proxy server starts, the configuration center corresponding to the proxy server is accessed to obtain configuration information; the consumption group is determined according to the configuration information, and the proxy cluster is started according to the consumption group; the message intermediate is obtained through the proxy cluster. The subscription information in the file is stored in the buffer pool, and the proxy server needs to maintain the subscription information. The push model is used for the proxy server to be concise enough and does not introduce the risk of insufficient resources.

In this embodiment, the Pull model is adopted, the consumer node sets up a message queue, and the proxy server obtains the identifier of the consumer node corresponding to the message subscription request when detecting a new message subscription request in the message queue corresponding to the consumer node; the proxy server accesses the configuration center to obtain the configuration In the information, the consumer node identifies the consumer group to which it belongs, starts the proxy cluster according to the consumer group to obtain the subscription information in the message middleware, and sends the subscription information to the consumer node, so that the proxy server only needs to undertake the state maintenance of the pull message (such as pull rhythm, local message cache, start and stop pull, etc.), and need to maintain the message queue in the consumer node, because each consumer node has a corresponding message queue, so the proxy server does not need to set a buffer pool, the proxy server No need for message maintenance information, just passively accept the request to get the message from the message queue corresponding to the consuming node and return the message, which can avoid the risk of ramming machine caused by insufficient resources.

It is understandable that when the message middleware is directly connected to the consuming node, the consuming node can directly query the message middleware to quickly obtain subscription messages, or the message middleware can automatically push the messages subscribed by the consuming node to the consuming node quickly. However, in the message middleware consumption proxy method in the embodiment of the present application, the message middleware and the consuming node are indirectly connected through a proxy server, which may cause message delay. Maintenance, the configuration center (ConfigCenter) is introduced to centrally manage, store and distribute subscription information, and the proxy server actively accesses the configuration center to obtain subscription information in a polling manner. If the polling period is T (T can be set according to the scene, for example, it is set to 1s ), that is, within 0<t<T, all proxy nodes in the proxy cluster corresponding to the proxy server can be consistent with the message middleware, and then sent to the proxied consumer node to achieve consistency.

4, the present invention also provides a message middleware consumption proxy device, the message middleware consumption proxy device includes:

The access acquisition module 10 is configured to access the configuration center corresponding to the proxy server to acquire configuration information when detecting that the proxy server is started;

a node startup module 20, configured to determine a consumption group according to the configuration information, and start an agent cluster according to the consumption group;

an acquiring and saving module 30, configured to acquire subscription information in the message middleware through the proxy cluster, and save the subscription information to a buffer pool;

The information sending module 40 is configured to send the subscription information in the buffer pool to each consuming node corresponding to the consuming group according to the configuration information.

In one embodiment, the message middleware consumption proxy device includes:

Create a setting module to create a configuration center corresponding to the proxy server;

The receiving and outputting module is used to output the information configuration page when receiving the proxy configuration request;

an obtaining and saving module, configured to obtain configuration information input based on the information configuration page, and save the configuration information to the configuration center;

In one embodiment, the node startup module 20 includes:

a group determination unit, configured to obtain the consumer node identifier in the configuration information and the subscription topic corresponding to the consumer node identifier, and form a consumer group with the consumer node identifiers with the same subscription topic;

The agent starting unit is used for associating each consumer group with a preset agent node individually, forming an agent cluster with the associated preset agent nodes, and starting the agent cluster.

In one embodiment, the obtaining and saving module 30 includes:

an information acquisition unit, configured to acquire a subscription topic in the configuration information, and acquire subscription information corresponding to the subscription topic in the message middleware through the proxy cluster;

a quantity obtaining unit, used to obtain the node quantity of the consuming nodes in the consuming group;

An information caching unit, configured to save the subscription information to a buffer pool when the number of the consuming nodes is at least two.

In one embodiment, the message middleware consumption proxy device includes:

A startup detection unit for detecting whether all the agent nodes in the agent cluster are successfully started;

A node startup unit, configured to determine a target proxy node that fails to start when all the proxy nodes in the proxy cluster are not successfully started, start a hot device node corresponding to the target proxy node, and obtain a message through the hot device node Subscription information in the middleware, and save the subscription information to the buffer pool;

The information caching unit is configured to execute the steps of acquiring subscription information in the message middleware through the proxy cluster and saving the subscription information to a buffer pool when all the proxy nodes in the proxy cluster are successfully started.

In one embodiment, the message middleware consumption proxy device includes:

A second sending module, configured to execute the sending of the subscription information in the buffer pool to the consumer according to the configuration information if the confirmation character information fed back by the consumer node is not received within a preset time period The steps of each consuming node corresponding to the group until the confirmation character information fed back by the consuming node is received.

In one embodiment, the message middleware consumption proxy device includes:

A detection and acquisition module, configured to acquire a consumer node identifier corresponding to the message subscription request when a new message subscription request is detected in the message queue corresponding to the consumer node;

an access startup module, configured to access a configuration center, obtain a consumer group to which the consumer node identifier in the configuration information belongs, and start an agent cluster according to the consumer group;

an information sending module, configured to obtain subscription information in the message middleware through the proxy cluster, and send the subscription information to the consuming node.

In the embodiment of the present invention, a proxy server is preset. When the proxy server is started, it accesses the configuration center to obtain configuration information. The proxy server starts the proxy cluster according to the configuration information, and accesses the message middleware through the proxy cluster. In this way, when the protocol of the message middleware is changed, it only needs to update Configuration information, the proxy server can start the corresponding proxy cluster according to the new configuration information to obtain the subscription information in the message middleware, without the need to upgrade each consumer node, reduce the operation and maintenance cost of the consumer node, and at the same time According to the configuration information, each proxy node in the proxy cluster can proxy multiple consumer nodes, so that there are fewer connections to the consumer nodes, which improves the stability of the message middleware and reduces the performance occupation of the message middleware; in addition, because the proxy The proxy nodes in the cluster are pre-authenticated, so the message middleware is unavoidably attacked, which improves the security of the message middleware.

For the methods executed by the above program modules, reference may be made to the various embodiments of the message middleware consumption proxy method of the present invention, which will not be repeated here.

The present invention also provides a computer storage medium.

The computer storage medium of the present invention stores a message middleware consumption agent program, and when the message middleware consumption agent program is executed by the processor, implements the steps of the above message middleware consumption agent method.

The method implemented when the message middleware consumption proxy program running on the processor is executed may refer to the various embodiments of the message middleware consumption proxy method of the present invention, which will not be repeated here.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on such understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. A message middleware consumption agent method, characterized in that the message middleware consumption agent method comprises:

when detecting that the proxy server is started, accessing a configuration center corresponding to the proxy server to acquire configuration information;

determining a consumption group according to the configuration information, and starting an agent cluster according to the consumption group;

obtaining subscription information in the message middleware through the agent cluster, and storing the subscription information to a buffer pool;

and sending the subscription information in the buffer pool to each consumption node corresponding to the consumption group according to the configuration information.

2. The message middleware consumption proxy method according to claim 1, wherein before the step of accessing the configuration center corresponding to the proxy server to obtain the configuration information when the proxy server is detected to be started, the method comprises:

establishing a configuration center corresponding to the proxy server;

when an agent configuration request is received, outputting an information configuration page;

and acquiring configuration information input based on the information configuration page, and storing the configuration information to the configuration center.

3. The message middleware consumption agent method of claim 1, wherein the step of determining a consumption group according to the configuration information, and starting an agent cluster according to the consumption group, comprises:

acquiring consumption node identifiers in the configuration information and subscription topics corresponding to the consumption node identifiers, and forming consumption groups by the consumption node identifiers with the same subscription topics;

and associating each consumption group with a preset agent node independently, forming the associated preset agent nodes into an agent cluster, and starting the agent cluster.

4. The message middleware consumption proxy method of claim 1, wherein the step of obtaining subscription information in message middleware through the proxy cluster, saving the subscription information to a buffer pool, comprises:

obtaining a subscription theme in the configuration information, and obtaining subscription information corresponding to the subscription theme in a message middleware through the agent cluster;

acquiring the number of nodes of the consumption nodes in the consumption group;

and when the number of the consumption nodes is at least two, storing the subscription information into a buffer pool.

5. The message middleware consumption proxy method of claim 1, wherein after the step of obtaining subscription information in message middleware by the proxy cluster, saving the subscription information to a buffer pool, the method comprises:

detecting whether all the proxy nodes in the proxy cluster are successfully started;

when all the proxy nodes in the proxy cluster are not successfully started, determining a target proxy node which is failed to start, starting a hot equipment node corresponding to the target proxy node, acquiring subscription information in message middleware through the hot equipment node, and storing the subscription information to a buffer pool;

and when all the proxy nodes in the proxy cluster are successfully started, executing the step of obtaining the subscription information in the message middleware through the proxy cluster and storing the subscription information into a buffer pool.

6. The message middleware consumption agent method according to claim 1, wherein after the step of sending the subscription information in the buffer pool to each consumption node corresponding to the consumption group according to the configuration information, the method comprises:

and if the confirmation character information fed back by the consumption node is not received within a preset time period, the step of sending the subscription information in the buffer pool to each consumption node corresponding to the consumption group according to the configuration information is executed until the confirmation character information fed back by the consumption node is received.

7. The message middleware consumption proxy method of any one of claims 1 to 6 wherein the message middleware consumption proxy method comprises:

when detecting that a new message subscription request is added in a message queue corresponding to a consumption node, acquiring a consumption node identifier corresponding to the message subscription request;

accessing a configuration center, acquiring a consumption group to which the consumption node identification belongs in configuration information, and starting an agent cluster according to the consumption group;

and obtaining subscription information in the message middleware through the agent cluster, and sending the subscription information to the consumption node.

8. A message-middleware consumption agent apparatus, characterized in that the message-middleware consumption agent apparatus comprises:

the access acquisition module is used for accessing a configuration center corresponding to the proxy server to acquire configuration information when the proxy server is detected to be started;

the node starting module is used for determining a consumption group according to the configuration information and starting the agent cluster according to the consumption group;

the acquisition and storage module is used for acquiring subscription information in the message middleware through the agent cluster and storing the subscription information to a buffer pool;

and the information sending module is used for sending the subscription information in the buffer pool to each consumption node corresponding to the consumption group according to the configuration information.

9. A proxy server, characterized in that the proxy server comprises: memory, a processor and a message middleware consumption agent stored on the memory and executable on the processor, the message middleware consumption agent when executed by the processor implementing the steps of the message middleware consumption agent method according to any one of claims 1 to 7.

10. A computer storage medium having stored thereon a message middleware consumption agent program which, when executed by a processor, implements the steps of the message middleware consumption agent method according to any one of claims 1 to 7.

Images