CN110022257B - Distributed messaging system - Google Patents
Distributed messaging system Download PDFInfo
- Publication number
- CN110022257B CN110022257B CN201810014768.2A CN201810014768A CN110022257B CN 110022257 B CN110022257 B CN 110022257B CN 201810014768 A CN201810014768 A CN 201810014768A CN 110022257 B CN110022257 B CN 110022257B
- Authority
- CN
- China
- Prior art keywords
- message
- zookeeper
- database
- configuration parameters
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to the field of distributed data technology, and provides a distributed message system, comprising: the database is used for storing the configuration parameters of the message; the message application servers are used for generating messages according to message requests and configuration parameters of the requesting party and sending the messages to the requesting party; the configuration server is used for issuing a database updating notice to all the message application servers when the database is updated so that the message application servers read the updated configuration parameters from the database and write the configuration parameters into the local cache of the message application servers; wherein the message application server reads the corresponding configuration parameters from the local cache when generating the message. The high-availability cache system supporting high concurrency is designed, the configuration server dynamically issues database updating notifications to all message application servers and triggers the servers to automatically reload the databases to the local cache, so that various problems caused by application restart are avoided, and the system availability is improved.
Description
Technical Field
The present disclosure relates to the field of distributed data technologies, and in particular, to a distributed message system.
Background
Before the advent of the message pushing technology, messages pushed by a message server to a user can be seen only by waiting for active pulling of the user, and in order to improve the user activity, the user stickiness and the user retention rate, the message server, particularly the message server of mobile equipment, needs to acquire the active right of message display instead of passively waiting for the user to pull the messages. The message pushing mechanism comes along. The operator actively pushes messages to the user mobile equipment through a product or a third-party tool, the user can see the message notification pushed by the message server in the screen locking state and the notification bar of the mobile equipment, and the user clicks the notification bar message to call the APP and jump to a corresponding page. WeChat messages QQ and the like which are seen in screen locking at ordinary times belong to a mobile terminal message pushing line and column.
Message centers currently access as many as one hundred or more messages. Whereas prior message-centric system architectures did not configure the necessary parameters for each message in a configuration file or database but rather coupled in code. This results in that not only the service end needs to add parameter check related to the new message for each new message accessed, but also the client end needs to distinguish the messages according to the type of the message returned by the service end, check the necessary parameters, and add corresponding service logic. Therefore, each time a new message is accessed, not only the code of the server needs to be modified, but also the edition period of the server and the edition period of the client need to be consistent.
In order to solve the problem, most of the implementation schemes in the prior art directly configure the configuration parameters in the database, but each time the database is read, network IO and disk IO are involved, and the system performance is greatly affected at high concurrency. Especially after modifying the database configuration, if the application server is manually restarted to make the application server read the updated database configuration, the server is in a temporary downtime state during restarting, so that the system availability is seriously affected.
Therefore, there is a need for a distributed messaging system that supports high concurrency and high availability.
The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
An object of the present disclosure is to provide a distributed messaging system, thereby overcoming, at least to some extent, one or more of the problems due to the limitations and disadvantages of the related art.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to an embodiment of the present disclosure, a distributed messaging system is disclosed, comprising:
the database is used for storing the configuration parameters of the message;
the message application servers are used for generating messages according to message requests and configuration parameters of the requesting party and sending the messages to the requesting party;
the configuration server is used for issuing a database updating notice to all the message application servers when the database is updated so that the message application servers read the updated configuration parameters from the database and write the configuration parameters into the local cache of the message application servers;
wherein the message application server reads the corresponding configuration parameters from the local cache when generating the message.
According to an example embodiment of the present disclosure, when a message application program of a message application server starts, the message application server reads configuration parameters from a database and writes the configuration parameters into a local cache of the message application server.
According to an example embodiment of the present disclosure, messages are divided into different message categories, and messages of the same category adopt the same configuration parameters.
According to an example embodiment of the present disclosure, a message template corresponding to a message category is established, and a message application server generates a message using the message template corresponding to the message category to which a message request belongs.
According to an example embodiment of the present disclosure, the message template includes at least one of a message center display template, a notification bar display template, a landing page display template, an android lock screen template, an ios force touch template, or an ios small graph template.
According to an example embodiment of the present disclosure, the configuration server is a zookeeper server, and each message application server also serves as a zookeeper client.
According to an example embodiment of the present disclosure, the Zookeeper client is registered as a temporary sequential node of the Zookeeper.
According to an example implementation manner of the disclosure, one of the zookeeper clients serves as a main service for monitoring whether the zookeeper server side is alive or not, the other zookeeper clients serve as slave services for monitoring whether the main service is alive or not, when the main service is down, a plurality of slave services compete again and elect one as a new main service to continue monitoring whether the zookeeper server side is alive or not.
According to an example embodiment of the present disclosure, wherein the re-competing and electing one of the several slave services as a new master service comprises: and taking the smallest sequence number of the temporary sequence nodes in the Zookeeper clients as the slave services as a new master service.
According to an example embodiment of the present disclosure, wherein a requestor sends a message request and receives a message through a messaging client.
According to an example embodiment of the present disclosure, wherein the database is a mysql database.
According to some example embodiments of the present disclosure, by designing a high-availability cache system supporting high concurrency, dynamically issuing a database update notification to all message application servers and triggering the servers to automatically reload the database configuration to the local cache through the configuration server, many problems caused by application restart are avoided, and the system availability is improved.
According to some example embodiments of the present disclosure, all messages are categorized by templates, and the necessary and optional parameters of each template are changed from hard-coded to configuration in the database. The necessary and optional parameters can be obtained by accessing the new message server every time as long as the template to which the new message server belongs is queried according to the message, and the function of verifying the parameters is executed by a uniform tool class; the client can complete data verification only by knowing the template corresponding to the message.
According to some example embodiments of the disclosure, through a zookeeper server survival monitoring mechanism, a distributed lock is realized by using characteristics of temporary sequence nodes of zookeeper to support master-slave hot switching in a distributed environment, and a problem of monitoring zookeeper server survival service single point failure is avoided.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
Fig. 1 shows a block diagram of a distributed messaging system according to an example embodiment of the present disclosure.
Fig. 2 illustrates a block diagram of a distributed messaging system employing a Zookeeper architecture, according to an example embodiment of the present disclosure.
Fig. 3 illustrates a flowchart of the operation of a distributed messaging system to update configuration parameters according to an example embodiment of the present disclosure.
Fig. 4 shows an architecture diagram of a Zookeeper client in a distributed messaging system according to an example embodiment of the present disclosure.
Detailed description of the exemplary embodiments
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more example embodiments. In the following description, numerous specific details are provided to give a thorough understanding of example embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other systems, components, steps, and the like. In other instances, well-known structures, systems, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
An object of the present disclosure is to provide a distributed message system, including: the database is used for storing the configuration parameters of the message; the message application servers are used for generating messages according to message requests and configuration parameters of the requesting party and sending the messages to the requesting party; the configuration server is used for issuing a database updating notice to all the message application servers when the database is updated so that the message application servers read the updated configuration parameters from the database and write the configuration parameters into the local cache of the message application servers; wherein the message application server reads the corresponding configuration parameters from the local cache when generating the message. The high-availability cache system supporting high concurrency is designed, the configuration server dynamically issues database updating notifications to all message application servers and triggers the servers to automatically reload the databases to the local cache, so that various problems caused by application restart are avoided, and the availability of the system is improved. Meanwhile, all messages are classified according to the templates, and the necessary and optional parameters of each template are changed from hard coding to configuration in the database. The necessary and optional parameters can be obtained by accessing the new message server every time as long as the template to which the new message server belongs is queried according to the message, and the function of verifying the parameters is executed by a uniform tool class; the client can complete data verification only by knowing the template corresponding to the message. In addition, through a zookeeper server survival monitoring mechanism, the distributed lock is realized by using the characteristic of a temporary sequence node of the zookeeper to support master-slave hot switching in a distributed environment, and the problem of monitoring the zookeeper server survival service single point fault is avoided.
The distributed messaging system of the present disclosure is described in detail below with reference to fig. 1-4, wherein fig. 1 illustrates a block diagram of a distributed messaging system in accordance with an example embodiment of the present disclosure; FIG. 2 illustrates a block diagram of a distributed messaging system employing a Zookeeper architecture, according to an example embodiment of the present disclosure; FIG. 3 illustrates an operational flow diagram of a distributed messaging system updating configuration parameters, according to an example embodiment of the present disclosure; fig. 4 shows an architecture diagram of a Zookeeper client in a distributed messaging system according to an example embodiment of the present disclosure.
A distributed messaging system according to an exemplary embodiment of the present disclosure is first described in detail with reference to fig. 1-3, wherein fig. 1 shows a block diagram of a distributed messaging system according to an exemplary embodiment of the present disclosure; FIG. 2 illustrates a block diagram of a distributed messaging system employing a Zookeeper architecture, according to an example embodiment of the present disclosure; fig. 3 illustrates a flowchart of the operation of a distributed messaging system to update configuration parameters according to an example embodiment of the present disclosure.
As shown in fig. 1, a distributed messaging system, comprising: a database 1 for storing configuration parameters of messages; a plurality of message application servers 21, 22.. 2n, forming a message application server cluster 2, for generating a message according to a message request and configuration parameters of a requester and sending the message to the requester, wherein the configuration parameters include necessary parameters and optional parameters; the configuration server3 is used for issuing a database update notification to all the message application servers when the database is updated, so that the message application servers read the updated configuration parameters from the database and write the configuration parameters into the local cache of the message application servers; wherein the message application server reads the corresponding configuration parameters from the local cache when generating the message. The method adopts a special server as a configuration server and is provided with a hot backup server capable of hot switching.
In the prior art, an implementation scheme of a distributed message system is generally to directly configure data in a database, where each time a database is read, network IO and disk IO are involved, and especially when the system is highly concurrent, system performance is greatly affected. In contrast, the distributed message system of the present disclosure only reads data in the database when the service of the message application server is started and the database is updated, and only reads data from the local cache in other cases, thereby greatly reducing the number of network IO and disk IO, and improving the system throughput. Meanwhile, a large distributed message system usually has hundreds (for example, 300) message application servers, and if all message application servers need to be restarted each time the database modifies the message configuration parameters, the message request is lost, and even the service is unavailable. The distributed message system of the present disclosure can read the data in the updated database without restarting the server when the database is updated. The distributed message system disclosed by the invention dynamically issues the database updating notification to all the message application servers through the configuration server and triggers the message application servers to automatically reload the configuration parameters of the database to the local cache, so that various problems caused by application restart are avoided, and the system availability is improved.
Further, to further ensure strong consistency of data in the distributed environment, the present disclosure combines a zookeeper architecture on the basis of the distributed message system as shown in fig. 1, where zookeeper is a highly available framework for distributed data management and system coordination, and based on the implementation of paxos algorithm, the framework ensures strong consistency of data in the distributed environment. A distributed message system using the Zookeeper architecture is described below with reference to fig. 2.
As shown in fig. 2, the distributed message system adopting the Zookeeper architecture includes: a database 1 for storing configuration parameters of messages; a plurality of message application servers 21, 22.. 2n, forming a message application server cluster 2, for generating a message according to a message request and configuration parameters of a requester and sending the message to the requester, wherein the configuration parameters include necessary parameters and optional parameters, and each message application server also serves as a zookeeper client; the configuration server3 serving as a zookeeper server is used for issuing a database update notification to all zookeeper clients when a database is updated so that the message application server reads updated configuration parameters from the database and writes the updated configuration parameters into a local cache of the message application server; wherein the message application server reads the corresponding configuration parameters from the local cache when generating the message. Due to the adoption of the zookeeper framework, the strong consistency of data in the distributed environment is further ensured while various problems caused by application restart are avoided.
How the distributed message system of the present disclosure performs the configuration parameter update is described below with reference to fig. 3, and fig. 3 shows a flow chart of the operation of the distributed message system to update the configuration parameters according to an example embodiment of the present disclosure.
As shown in fig. 3, the distributed message system performs the update of the configuration parameters as follows: at S31, start; at S32, when the service of the message application server is started, then at S33, the message application server reads the configuration parameters of the message from the database and at S34, the configuration parameters are maintained in a set of local cache systems realized by ConcurrentHashMap; or in S32', when the zookeeper configuration center running on the configuration server3, that is, the zookeeper server, receives information such as database update/degradation switch/system configuration from the database, the zookeeper configuration center issues a database update notification to each message application server (as a zookeeper client), and then in S33, the message application server reads configuration parameters of the information from the database and in S34, maintains the configuration parameters in a set of local cache system implemented by using ConcurrentHashMap; finally, in S35, the update ends. The key point of the method is that a segmented lock technology is used, data is stored in a segment, then each segment of data is provided with a lock, when one thread occupies the lock to access one segment of data, the data of other segments can be accessed by other threads, and real concurrent access can be realized.
According to an example embodiment of the present disclosure, wherein the requestor sends a message request and receives a message through the messaging client 4.
According to an example embodiment of the present disclosure, wherein the database is a mysql database.
According to an example embodiment of the present disclosure, messages are divided into different message categories, and messages of the same category adopt the same configuration parameters.
According to an example embodiment of the present disclosure, a message template corresponding to a message category is established, and a message application server generates a message using the message template corresponding to the message category to which a message request belongs. The application server of the new message is accessed every time, the necessary and optional parameters can be obtained only by inquiring the template to which the application server belongs according to the message, and the function of checking the parameters is executed by a uniform tool class; the message client can complete data verification only by knowing the template corresponding to the message. After the message template is adopted, the system can check the parameters of the messages by taking the template as the dimension instead of taking a single message as the dimension check parameter.
According to an example embodiment of the present disclosure, the message template includes at least one of a message center display template, a notification bar display template, a landing page display template, an android lock screen template, an ios Force Touch template (where ios is an Apple operating system; force Touch is a Touch sensing technology used by Apple computer for Apple Watch, brand new MacBook, and brand new MacBook Pro. Each template is about 3-5 types, and 30 display effects are supported.
In addition, the distributed message system can provide the function of automatically generating the access protocol only by inputting the template type for the accessed third party, thereby greatly saving the communication cost of new message access and the development test and joint debugging cost. More importantly, the access of the new message is not dependent on the message client sending version any more, as long as the newly accessed message belongs to the existing template, the message can be accessed at any time, and the old message client can also receive the newly accessed message.
In view of the strong dependence of the distributed message system adopting the Zookeeper architecture on Zookeeper, the distributed message system adopting the Zookeeper architecture must pay close attention to/monitor the survival of the Zookeeper server, and when the Zookeeper server is monitored to be down, an application responsible person of the distributed message system must be timely notified to take measures to recover the Zookeeper server as soon as possible, such as restarting a server serving as the Zookeeper server or hot-switching other hot backup servers as new Zookeeper servers. Therefore, in the following example embodiments, in order to improve system availability, the present disclosure utilizes the characteristic of Zookeeper's temporary sequential nodes, and adds a Zookeeper server survival monitoring mechanism supporting master-slave hot handover to a distributed message system adopting a Zookeeper architecture to uninterruptedly monitor whether a Zookeeper server is alive or not.
According to an example embodiment of the disclosure, a plurality of zookeeper clients are provided, each message application server is also a zookeeper client, and not only is a database update notification sent by a zookeeper server received, but also the survival of the zookeeper server can be monitored. Each message application server (as a zookeeper client) selects a master service for monitoring whether the zookeeper server side is alive through a temporary sequential node registered as zookeeper, and the other zookeeper clients monitor whether the master service is alive as slave services. When the main service monitoring whether the zookeeper server side is alive is down, another zookeeper client side which is not selected as the slave service of the main service is reselected to be used as a new main service to continue monitoring the survival of the zookeeper server side. The zookeeper server survival monitoring mechanism utilizes the characteristic of a temporary sequence node of the zookeeper to realize a distributed lock to support master-slave hot switching in a distributed environment, avoid single point failure of monitoring zookeeper server survival service, and improve system HA (high availability).
Before making a detailed description, the concepts of temporary nodes, sequential nodes, HA (high availability) and single point of failure are introduced as follows:
and (4) temporary nodes: its lifecycle is tied to the zookeeper client session, and if the client session fails, this node is automatically deleted by the zookeeper.
A sequence node: so to say, if three zookeeper clients are shared, each creating 1 sequential node under the/monitor/directory (assumed to be this directory), the zookeeper cluster will create nodes in the order in which the clients created the nodes, which are/monitor/01,/monitor/02,/monitor/03, respectively.
HA (High Availability) is highly available. One node in the cluster fails and its tasks can be passed to the other nodes. Single point failures can be effectively prevented. High availability clusters aim to provide highly reliable services, i.e. 7 x 24 hours uninterrupted service to the outside using the cluster's fault tolerance.
Single point of failure: a single point failure can propagate to the entire system or network, resulting in a breakdown of the entire system or network. This is to be avoided when designing IT infrastructures, which are generally avoided by means of both master and slave. The present disclosure uses a master-slave approach to avoid single points of failure.
This will be described in detail with reference to fig. 4.
Fig. 4 shows an architecture diagram of a Zookeeper client in a distributed messaging system according to an example embodiment of the present disclosure. As shown in fig. 4, the message application server registers its own information as a temporary sequence node of zookeeper when it is started, and there are no three message application servers as zookeeper clients, which are respectively set as server1 (message application server 21), server2 (message application server 22), and server3 (message application server 23), and the temporary sequence nodes registered in zookeeper are respectively/monitor/01,/monitor/02,/monitor/03. At this time, the smallest node is the/monitor/01 node created by server1, that is, server1 can be regarded as master/master service to monitor whether zookeeper server is alive, and server2 and server3 are slave/slave services to monitor whether master service is alive, because the life cycle of the temporary node is bound to zookeeper client (i.e. message application server here) session, that is, if zookeeper client session fails, this node will be automatically deleted by zookeeper, that is, assuming server1 message application server is down, zookeeper will delete the temporary node/monitor/01 registered by the message application server, and/monitor/02 becomes the smallest node, that is, server2 becomes master/master service. Similarly, if server2 is down, server3 becomes master/primary service.
The principle of implementing a distributed lock with temporary sequential nodes is as follows:
a zookeeper client calls the create () method to create a temporary sequential child node with a parent/monitor.
The zookeeper client calls the getChildren ("/monitor") method to get all child nodes that have been created.
After the zookeeper client acquires all the child node paths, if the node created in the step 1 is found to be the node with the smallest sequence number in all the nodes, the zookeeper client is considered to obtain the lock, and the zookeeper client becomes the main service for monitoring the survival of the zookeeper server.
4. If the node is found not to be the smallest of all the child nodes in the step 3, which indicates that the node does not acquire the lock, monitoring whether the node with the smallest size survives and waits until the notification of the change of the smallest child node is monitored, then acquiring the child nodes, and judging whether to acquire the lock.
The process of releasing the lock is relatively simple, and the child node created by the child node is deleted.
Assuming that the downtime probability of a single message application server in a message application server cluster is 0.25, when a master-slave switching mechanism of a monitoring service is not introduced, the probability of unavailability of the service for monitoring the survival of a zookeeper server is 0.25 no matter how many message application servers are in the cluster, after the master-slave hot switching mechanism realized by using the characteristic of zookeeper temporary sequence nodes is added in the system, assuming that n message application servers are deployed in the cluster, the probability of unavailability of the monitoring service is as follows: (0.25) n . It can be seen that when the number n of message application servers in the cluster reaches more than 8 (i.e. when there are more than 8 message application servers as zookeeper clients), the probability of monitoring service being unavailable is less than 0.0000001%, and when n is greater, the probability of monitoring service being unavailable is smaller, and the limit is 0.
From the foregoing detailed description, those skilled in the art will readily appreciate that a system according to embodiments of the present disclosure may have one or more of the following advantages.
According to some example embodiments of the present disclosure, by designing a high-availability cache system supporting high concurrency, dynamically issuing a database update notification to all message application servers and triggering the servers to automatically reload the database configuration to the local cache through the configuration server, many problems caused by application restart are avoided, and the system availability is improved.
According to some example embodiments of the present disclosure, all messages are categorized by templates, and the necessary and optional parameters of each template are changed from hard-coded to configuration in the database. The necessary and optional parameters can be obtained by accessing the new message server every time as long as the template to which the new message server belongs is queried according to the message, and the function of verifying the parameters is executed by a uniform tool class; the client can complete data verification only by knowing the template corresponding to the message.
According to some example embodiments of the disclosure, through a zookeeper server survival monitoring mechanism, a distributed lock is realized by using characteristics of temporary sequence nodes of zookeeper to support master-slave hot switching in a distributed environment, and a problem of monitoring zookeeper server survival service single point failure is avoided.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (7)
1. A distributed messaging system employing a Zookeeper architecture, comprising:
the database is used for storing the configuration parameters of the message; the configuration parameters comprise necessary parameters and optional parameters, and when a message application program of the message application server is started, the message application server reads the configuration parameters from a database and writes the configuration parameters into a local cache of the message application server;
the message application servers are used for generating messages according to message requests and configuration parameters of the requesting party and sending the messages to the requesting party;
the configuration server is used for issuing a database updating notice to all the message application servers when the database is updated so that the message application servers read the updated configuration parameters from the database and write the configuration parameters into the local cache of the message application servers;
when the message application server generates a message, reading corresponding configuration parameters from a local cache;
the requesting party sends a message request and receives messages through the message client, the messages are divided into different message types, and the messages of the same type adopt the same configuration parameters; and establishing a message template corresponding to the message type, wherein the message application server generates a message by adopting the message template corresponding to the message type to which the message request belongs.
2. The system of claim 1, wherein the message template comprises at least one of a message center display template, a notification bar display template, a floor page display template, an android lock screen template, an ios force touch template, or an ios teletext template.
3. The system of claim 1, wherein the configuration server is a zookeeper server, and each message application server also serves as a zookeeper client.
4. The system of claim 3, wherein the Zookeeper client is registered as a temporal sequential node of the Zookeeper.
5. The system according to claim 4, wherein one of the zookeeper clients is used as a main service for monitoring whether the zookeeper server is alive or not, the other zookeeper clients are used as slave services for monitoring whether the main service is alive or not, and when the main service is down, a plurality of slave services compete again and elect one as a new main service to continue monitoring whether the zookeeper server is alive or not.
6. The system of claim 5, wherein the re-competing and electing one of the plurality of slave services as the new master service comprises: and taking the smallest sequence number of temporary sequence nodes in the Zookeeper clients as the slave services as a new master service.
7. The system of claim 1, wherein the database is a mysql database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810014768.2A CN110022257B (en) | 2018-01-08 | 2018-01-08 | Distributed messaging system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810014768.2A CN110022257B (en) | 2018-01-08 | 2018-01-08 | Distributed messaging system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110022257A CN110022257A (en) | 2019-07-16 |
| CN110022257B true CN110022257B (en) | 2023-04-07 |
Family
ID=67187395
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810014768.2A Active CN110022257B (en) | 2018-01-08 | 2018-01-08 | Distributed messaging system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110022257B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463514A (en) * | 2019-09-06 | 2021-03-09 | 北京京东尚科信息技术有限公司 | Monitoring method and device for distributed cache cluster |
| CN113364822B (en) * | 2020-03-05 | 2024-12-03 | 北京沃东天骏信息技术有限公司 | A method and device for updating cache data |
| CN112084065B (en) * | 2020-08-24 | 2024-02-20 | 贵州易鲸捷信息技术有限公司 | A rolling restart method based on EsgynDB database |
| CN112035164B (en) * | 2020-09-03 | 2024-10-11 | 中国银行股份有限公司 | Method for updating node configuration of distributed system, application node and control node |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2590581A1 (en) * | 2007-05-31 | 2008-11-30 | Strategic Connections | Lead distribution and tracking with integrated corporate data usage and reporting capabilities with message templating |
| CN105100200A (en) * | 2015-06-02 | 2015-11-25 | 北京京东尚科信息技术有限公司 | Method and system for distributed deployment, unified configuration and automatic adaptation |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE602006017089D1 (en) * | 2005-09-29 | 2010-11-04 | Teamon Systems Inc | E-MAIL SERVER FOR PROCESSING A THRESHOLD NUMBER OF E-MAIL JOBS FOR A GIVEN USER AND METHOD THEREOF |
| CN102739720B (en) * | 2011-04-14 | 2015-01-28 | 中兴通讯股份有限公司 | Distributed cache server system and application method thereof, cache clients and cache server terminals |
| CN102255752B (en) * | 2011-06-30 | 2014-08-06 | 北京新媒传信科技有限公司 | Configuration management system and method of server cluster |
| CN103500173B (en) * | 2013-09-03 | 2017-07-28 | 北京泰乐德信息技术有限公司 | A kind of querying method of track traffic Monitoring Data |
| US10027728B2 (en) * | 2015-03-06 | 2018-07-17 | Ebay Inc. | Systems and methods of streaming data |
| CN106878053A (en) * | 2016-12-22 | 2017-06-20 | 努比亚技术有限公司 | A kind of config update devices and methods therefor based on ZOOKEEPER |
-
2018
- 2018-01-08 CN CN201810014768.2A patent/CN110022257B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2590581A1 (en) * | 2007-05-31 | 2008-11-30 | Strategic Connections | Lead distribution and tracking with integrated corporate data usage and reporting capabilities with message templating |
| CN105100200A (en) * | 2015-06-02 | 2015-11-25 | 北京京东尚科信息技术有限公司 | Method and system for distributed deployment, unified configuration and automatic adaptation |
Non-Patent Citations (1)
| Title |
|---|
| 一个可重用的应用服务器框架的设计与实现;王成耀;《计算机工程与设计》;20021028(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110022257A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110022257B (en) | Distributed messaging system | |
| US9680692B2 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN111865632B (en) | Switching method of distributed data storage cluster and switching instruction sending method and device | |
| US20220413937A1 (en) | Node management method, device and apparatus, storage medium, and system | |
| CN114900449B (en) | Resource information management method, system and device | |
| CN103488526A (en) | System and method for locking business resource in distributed system | |
| WO2022007908A1 (en) | Method for service collaboration between network element devices, and network element device | |
| CN108874531B (en) | Method, device and system for fusing service and electronic equipment | |
| CN107508700B (en) | Disaster recovery method, device, equipment and storage medium | |
| CN108418859B (en) | Method and device for writing data | |
| CN111752892B (en) | Distributed file system and implementation method, management system, equipment and medium thereof | |
| CN111355605A (en) | Virtual machine fault recovery method and server of cloud platform | |
| CN116185697B (en) | Container cluster management method, device, system, electronic equipment and storage medium | |
| CN116668269A (en) | Arbitration method, device and system for dual-activity data center | |
| CN111309515A (en) | Disaster recovery control method, device and system | |
| CN116347467B (en) | UDR user data management method and system in 5G network | |
| WO2024146184A1 (en) | Cloud database management method and related system | |
| JP2015114952A (en) | Network system, monitoring control unit, and software verification method | |
| CN118214721A (en) | Fusing current limiting method and system based on Sentinel | |
| JP7646874B2 (en) | Disaster recovery method, device, network element device, and storage medium for user data | |
| CN114584462B (en) | A method and device for processing network services | |
| CN116346834A (en) | Session synchronization method, device, computing equipment and computer storage medium | |
| CN109753292B (en) | Method and device for deploying multiple applications in multiple single instance database service | |
| CN111722988A (en) | Method and device for failover of data space node | |
| CN113407369B (en) | Intelligent platform management system supporting main and standby system management and implementation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |