[go: up one dir, main page]

CN116389514A - Communication processing method and device for distributed data storage - Google Patents

Communication processing method and device for distributed data storage Download PDF

Info

Publication number
CN116389514A
CN116389514A CN202310357743.3A CN202310357743A CN116389514A CN 116389514 A CN116389514 A CN 116389514A CN 202310357743 A CN202310357743 A CN 202310357743A CN 116389514 A CN116389514 A CN 116389514A
Authority
CN
China
Prior art keywords
node
node information
information table
nodes
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310357743.3A
Other languages
Chinese (zh)
Inventor
李一鸣
贾永强
张宏海
杨永凯
崔斌豪
智强
张凯
王英华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Travelsky Technology Co Ltd
Original Assignee
China Travelsky Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Travelsky Technology Co Ltd filed Critical China Travelsky Technology Co Ltd
Priority to CN202310357743.3A priority Critical patent/CN116389514A/en
Publication of CN116389514A publication Critical patent/CN116389514A/en
Priority to PCT/CN2023/125350 priority patent/WO2024207708A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/02Details
    • H04L12/16Arrangements for providing special services to substations
    • H04L12/18Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/04Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/164Adaptation or special uses of UDP protocol

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer And Data Communications (AREA)
  • Multi Processors (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a communication processing method and device for data distributed storage, which are used for generating a starting node according to a storage instruction after receiving the data distributed storage instruction and transmitting node information of the starting node to at least one propagation node. And then constructing a node information table according to the node information of the starting node and the node information of the transmitting node, and transmitting the node information table in sequence in all nodes in the cluster so that the node information table obtains the node information of all the nodes in the cluster. And finally, retransmitting the node information table to all nodes in the cluster. By the method, all nodes in the distributed storage cluster store node information of each node, when data is required to be stored in a distributed mode, the nodes can finish data transmission and storage only through the stored node information, network mass sending or participation of intermediate nodes is not required any more through a broadcasting mode, and the data transmission process is more efficient and safer.

Description

Communication processing method and device for distributed data storage
Technical Field
The present disclosure relates to the field of distributed storage technologies, and in particular, to a communication processing method and apparatus for data distributed storage.
Background
With the gradual diversification of the functions of the aviation business, the data and types required to be stored by the aviation business also tend to be complex. In the current air ticket query service, a huge amount of message data with a great variety are often required, so that a distributed storage technology is generally adopted to store the message data in the prior art.
In the process of distributed storage of data in the prior art, data are generally stored in parts by adopting self-developed group nodes, so that the communication quality among the group nodes plays a crucial role in the accuracy of data distributed storage. The existing group nodes generally adopt a TCP distributed protocol to complete the communication among the nodes, although the TCP distributed protocol can effectively guarantee the reliability of information, under the condition of relatively tension of network resources, as the communication based on the TCP distributed protocol needs an intermediate node to maintain the communication among all the nodes, the existence of the intermediate node leads the communication among the nodes to need to be frequently interacted through the intermediate node, and the use pressure of the network resources is relatively high.
In another process of communication between nodes by using a UDP distributed protocol, the nodes can complete communication with all nodes in the local area network directly through a broadcast mode under the protocol, but perform data group transmission in the network in the broadcast mode, which is easy to cause data loss in the transmission process, and has low reliability of data transmission.
Therefore, how to solve the problem that in the prior art, it is difficult to accurately maintain the distributed storage of data under the condition of relatively tense network resources becomes a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
Based on the above problems, in order to solve the problem that it is difficult to accurately maintain data distributed storage under the condition of shortage of network resources, the application provides a communication processing method and device for data distributed storage.
In a first aspect, the application discloses a communication processing method for distributed storage of data, which is applied to a distributed storage cluster and includes:
after receiving a data distribution storage instruction, generating a starting node according to the data distribution storage instruction; the distributed storage cluster comprises a plurality of propagation nodes;
transmitting node information of the start node to at least one propagation node; the node information is used for communication between nodes;
constructing a node information table according to the node information of the starting node and the node information of the propagation node; the propagation node is a node which receives the node information of the starting node;
sequentially spreading the node information table in all nodes in the cluster so that the node information table obtains node information of all nodes in the cluster;
and retransmitting the node information table to all nodes in the cluster.
Optionally, after retransmitting the node information table to all nodes in the cluster, the method further includes:
when the update times of the node information table reach a preset threshold value, performing failure node analysis on the cluster according to the current node information table and the historical node information table;
and if the failure node exists in the cluster, performing offline processing on the failure node.
Optionally, when the update times of the node information table reach a preset threshold, performing failure node analysis on the cluster according to the current node information table and the historical node information table, including:
and comparing the current node information table with the historical node information table, and if the node information in the historical node information table is not displayed in the current node information table, determining the node corresponding to the node information which is not displayed as a failure node.
Optionally, the data distribution storage instruction contains service data to be distributed stored; before the generating the starting node, the method further comprises:
carrying out security verification on the service data through a preset analysis algorithm;
and if the service data passes the security check, performing format conversion on the service data to obtain identifiable service data.
Optionally, before retransmitting the node information table to all nodes in the cluster, the method further includes:
and carrying out encryption processing on the node information table.
In a second aspect, the present application discloses a communication processing apparatus for distributed storage of data, including:
the generating module is used for generating a starting node according to the data distribution storage instruction after receiving the data distribution storage instruction; the distributed storage cluster comprises a plurality of propagation nodes;
a sending module, configured to send node information of the start node to at least one propagation node; the node information is used for communication between nodes;
the construction module is used for constructing a node information table according to the node information of the starting node and the node information of the propagation node; the propagation node is a node which receives the node information of the starting node;
the first sending module is used for sequentially transmitting the node information table in all nodes in the cluster so that the node information table obtains the node information of all the nodes in the cluster;
and the second sending module is used for resending the node information table to all nodes in the cluster.
Optionally, the communication processing device further includes: a failure node analysis module; the failure node analysis module is used for:
when the update times of the node information table reach a preset threshold value, performing failure node analysis on the cluster according to the current node information table and the historical node information table;
and if the failure node exists in the cluster, performing offline processing on the failure node.
Optionally, the failure node analysis module is specifically configured to:
and comparing the current node information table with the historical node information table, and if the node information in the historical node information table is not displayed in the current node information table, determining the node corresponding to the node information which is not displayed as a failure node.
Optionally, the communication processing device further includes: a security verification module; the safety verification module is used for:
carrying out security verification on the service data through a preset analysis algorithm;
and if the service data passes the security check, performing format conversion on the service data to obtain identifiable service data.
Optionally, the communication processing device further includes: and the encryption module is used for carrying out encryption processing on the node information table.
Compared with the prior art, the application has the following beneficial effects: the application provides a communication processing method and device for data distributed storage, which are used for generating a starting node according to a storage instruction after receiving the data distributed storage instruction and transmitting node information of the starting node to at least one propagation node. And then constructing a node information table according to the node information of the starting node and the node information of the transmitting node, and transmitting the node information table in sequence in all nodes in the cluster so that the node information table obtains the node information of all the nodes in the cluster. And finally, retransmitting the node information table to all nodes in the cluster. By the method, all nodes in the distributed storage cluster store node information of each node, when data is required to be stored in a distributed mode, the nodes can finish data transmission and storage only through the stored node information, network mass sending or participation of intermediate nodes is not required any more through a broadcasting mode, and the data transmission process is more efficient and safer.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive faculty for a person skilled in the art.
Fig. 1 is a schematic flow chart of a communication processing method for distributed storage of data according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a communication processing device for distributed storage of data according to an embodiment of the present application.
Detailed Description
As described above, in the process of distributed storage of data in the prior art, data is generally stored in portions by using self-developed group nodes, so that the communication quality between the group nodes plays a crucial role in the accuracy of data distributed storage. The existing group nodes generally adopt a TCP distributed protocol to complete the communication among the nodes, although the TCP distributed protocol can effectively guarantee the reliability of information, under the condition of relatively tension of network resources, as the communication based on the TCP distributed protocol needs an intermediate node to maintain the communication among all the nodes, the existence of the intermediate node leads the communication among the nodes to need to be frequently interacted through the intermediate node, and the use pressure of the network resources is relatively high.
In another process of communication between nodes by using a UDP distributed protocol, the nodes can complete communication with all nodes in the local area network directly through a broadcast mode under the protocol, but perform data group transmission in the network in the broadcast mode, which is easy to cause data loss in the transmission process, and has low reliability of data transmission.
The inventor provides a communication processing method and a communication processing device for data distributed storage, which are used for generating a starting node according to a storage instruction after receiving the data distributed storage instruction and transmitting node information of the starting node to at least one propagation node. And then constructing a node information table according to the node information of the starting node and the node information of the transmitting node, and transmitting the node information table in sequence in all nodes in the cluster so that the node information table obtains the node information of all the nodes in the cluster. And finally, retransmitting the node information table to all nodes in the cluster. By the method, all nodes in the distributed storage cluster store node information of each node, when data is required to be stored in a distributed mode, the nodes can finish data transmission and storage only through the stored node information, network mass sending or participation of intermediate nodes is not required any more through a broadcasting mode, and the data transmission process is more efficient and safer.
In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Method embodiment
In this embodiment, a UDP distributed protocol is used as a communication protocol for the nodes in the storage cluster of the present application. Referring to fig. 1, the flow chart of a communication processing method for distributed storage of data provided in the present application includes the following specific steps:
s101: after receiving a data distribution storage instruction, generating a starting node according to the data distribution storage instruction; the distributed storage cluster comprises a plurality of propagation nodes.
After receiving the data distribution storage instruction, a starting node is correspondingly generated according to the storage instruction, so that the node is used for completing the storage of partial data. In this distributed storage cluster, there already exist a plurality of propagation nodes, each of which can perform distributed storage of data. When one propagation node needs the data stored in the other propagation node, the two propagation nodes can complete data transmission through the node information of the other party, so that the data is distributed and stored.
S102: transmitting node information of the start node to at least one propagation node; the node information is used for communication between nodes.
After the start node is generated, the start node is controlled to send own node information to other propagation nodes in the storage cluster so as to inform that a new node is added in the cluster. The node information of the start node contains address information such as an IP address and an identification code of the node. After receiving the node information of the starting node, other propagation nodes can directly communicate with the starting node through the node information so as to complete data transmission.
In the process of transmitting the node information of the start node to the propagation nodes in the cluster, the transmission can be performed by a broadcast mode of the UDP protocol, so that the propagation nodes in the cluster can receive the node information of the start node. Because the broadcasting mode of the UDP protocol may cause packet loss of the nodes, thereby causing data loss, in the process of sending the node information, broadcast communication can be carried out for a plurality of times, thereby ensuring that at least one propagation node in the cluster receives the node information of the starting node.
S103: constructing a node information table according to the node information of the starting node and the node information of the propagation node; the propagation node is a node which receives the node information of the starting node.
When at least one propagation node in the storage cluster receives the node information of the starting node, the received node information of the starting node is stored, the node information of the starting node and the node information of the starting node are subjected to aggregation processing, and a node information table is constructed, wherein the node information table comprises the node information of the starting node and the node information of the at least one propagation node.
S104: and sequentially transmitting the node information tables in all nodes in the cluster so that the node information tables obtain the node information of all the nodes in the cluster.
At least one propagation node in the cluster receives the node information of the starting node, after the construction of the node information table is completed, the propagation node is controlled to propagate the node information table to other propagation nodes in the cluster, and after the other propagation nodes in the cluster receive the node information table, the self node information is stored in the node information table, and meanwhile, the node information table is sent to the next propagation node. The method and the system complete the sequential propagation of all propagation nodes in the cluster, thereby completing the collection of node information of all nodes in the cluster, and meanwhile, the node information table is finally retransmitted to the starting node, thereby collecting the node information of all nodes in the cluster.
S105: and retransmitting the node information table to all nodes in the cluster.
After collecting the node information of all the nodes in the cluster, the node information table storing the node information of all the nodes is sent to all the nodes in the cluster, so that all the nodes in the cluster contain the node information of different nodes. When communication is needed to complete distributed storage of data, the propagation node and the starting node can complete data communication only through a node information table stored by the propagation node and the starting node, communication consumption of any node is usually only 1-2 times of data transmission and data reception, consumption of network resources in a data distributed storage process is minimized as much as possible, unnecessary waste of network resources is reduced, and simultaneous processing of more data services can be supported.
Specifically, the transmission of the node information table in the cluster can be completed through the heartbeat signal between the nodes, and the propagation mode of the node information table is not particularly limited in this embodiment.
As an alternative embodiment, after step S105, further includes:
when the update times of the node information table reach a preset threshold value, performing failure node analysis on the cluster according to the current node information table and the historical node information table;
and if the failure node exists in the cluster, performing offline processing on the failure node.
And each time a new starting node is added, the node information table stores the node information of the new starting node into the node information table, so that the updating of the node information table is completed. In the actual application process, along with updating of the node information table and time lengthening, some nodes may fail, so that normal operation of the work is affected. Therefore, after the update times of the node information table reach a preset threshold, failure node analysis is performed on the storage cluster by combining the current latest node information table and the historical node information table.
Specifically, in the process of performing failure node analysis on the storage cluster, comparing the node information table of the current node with the node information table of the history, if the node information in the node information table of the history does not exist in the current node information table, indicating that the node in the cluster exists and cannot store the node information of the node in the node information table, determining the node corresponding to the node information which is not displayed as the failure node, and performing the downlink processing on the failure node.
As another optional implementation manner, the data distribution storage instruction contains service data needing to be stored in a distributed manner, and before generating the start node, the method further includes:
carrying out security verification on the service data through a preset analysis algorithm;
and if the service data passes the security check, performing format conversion on the service data to obtain identifiable service data.
After receiving the data distribution storage instruction, the service data in the storage instruction can be subjected to safety verification through a preset analysis algorithm, so that the integrity and safety of the service data are detected. If the service data passes the security check, format conversion is carried out on the service data to generate service data which can be identified by the propagation nodes in the storage cluster. Specifically, in the process of converting the format of the service data, the format of the service data may be converted by a preset parsing logic algorithm, and the format conversion mode of the specific service data is not specifically limited in this embodiment.
As another alternative embodiment, before step S105, the method further includes:
and carrying out encryption processing on the node information table.
Before the node information table is sent to all nodes in the cluster, special protocol processing and encryption processing can be carried out on the node information table according to the current network environment and a specific communication protocol, so that the node information table is prevented from being received and analyzed by other illegal servers, and the safety of data sending is ensured.
The embodiment provides a communication processing method and device for data distributed storage, which are used for generating a starting node according to a storage instruction after receiving the data distributed storage instruction and sending node information of the starting node to at least one propagation node. And then constructing a node information table according to the node information of the starting node and the node information of the transmitting node, and transmitting the node information table in sequence in all nodes in the cluster so that the node information table obtains the node information of all the nodes in the cluster. And finally, retransmitting the node information table to all nodes in the cluster. By the method, all nodes in the distributed storage cluster store node information of each node, when data is required to be stored in a distributed mode, the nodes can finish data transmission and storage only through the stored node information, network mass sending or participation of intermediate nodes is not required any more through a broadcasting mode, and the data transmission process is more efficient and safer.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the following, a description will be given of a communication processing apparatus for data distributed storage according to an embodiment of the present application, and a communication processing apparatus for data distributed storage described below and a communication processing method for data distributed storage described above may be referred to correspondingly with each other.
Device embodiment
Referring to fig. 2, the schematic structural diagram of a communication processing device for distributed storage of data provided in an embodiment of the present application specifically includes:
the generating module 100 is configured to generate a start node according to the data distribution storage instruction after receiving the data distribution storage instruction; the distributed storage cluster comprises a plurality of propagation nodes;
a sending module 200, configured to send node information of the start node to at least one propagation node; the node information is used for communication between nodes;
a construction module 300, configured to construct a node information table according to the node information of the start node and the node information of the propagation node; the propagation node is a node which receives the node information of the starting node;
a first sending module 400, configured to sequentially propagate the node information table in all nodes in the cluster, so that the node information table obtains node information of all nodes in the cluster;
and a second sending module 500, configured to resend the node information table to all nodes in the cluster.
Optionally, the communication processing device further includes: a failure node analysis module; the failure node analysis module is used for:
when the update times of the node information table reach a preset threshold value, performing failure node analysis on the cluster according to the current node information table and the historical node information table;
and if the failure node exists in the cluster, performing offline processing on the failure node.
Optionally, the failure node analysis module is specifically configured to:
and comparing the current node information table with the historical node information table, and if the node information in the historical node information table is not displayed in the current node information table, determining the node corresponding to the node information which is not displayed as a failure node.
Optionally, the communication processing device further includes: a security verification module; the safety verification module is used for:
carrying out security verification on the service data through a preset analysis algorithm;
and if the service data passes the security check, performing format conversion on the service data to obtain identifiable service data.
Optionally, the communication processing device further includes: and the encryption module is used for carrying out encryption processing on the node information table.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment is mainly described in a different point from other embodiments. In particular, for the method and apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and references to the details of the method embodiments are only provided. The method and apparatus embodiments described above are merely illustrative, in which elements illustrated as separate elements may or may not be physically separate, and elements presented as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing is merely one specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A communication processing method for data distributed storage is applied to a distributed storage cluster, and is characterized by comprising the following steps:
after receiving a data distribution storage instruction, generating a starting node according to the data distribution storage instruction; the distributed storage cluster comprises a plurality of propagation nodes;
transmitting node information of the start node to at least one propagation node; the node information is used for communication between nodes;
constructing a node information table according to the node information of the starting node and the node information of the propagation node; the propagation node is a node which receives the node information of the starting node;
sequentially spreading the node information table in all nodes in the cluster so that the node information table obtains node information of all nodes in the cluster;
and retransmitting the node information table to all nodes in the cluster.
2. The method of claim 1, wherein after retransmitting the node information table to all nodes in the cluster, further comprising:
when the update times of the node information table reach a preset threshold value, performing failure node analysis on the cluster according to the current node information table and the historical node information table;
and if the failure node exists in the cluster, performing offline processing on the failure node.
3. The method according to claim 2, wherein when the update times of the node information table reach a preset threshold, performing failure node analysis on the cluster according to the current node information table and the historical node information table, includes:
and comparing the current node information table with the historical node information table, and if the node information in the historical node information table is not displayed in the current node information table, determining the node corresponding to the node information which is not displayed as a failure node.
4. The method according to claim 1, wherein the data distribution storage instruction contains service data to be distributed stored; before the generating the starting node, the method further comprises:
carrying out security verification on the service data through a preset analysis algorithm;
and if the service data passes the security check, performing format conversion on the service data to obtain identifiable service data.
5. The method of claim 1, wherein before retransmitting the node information table to all nodes in the cluster, further comprising:
and carrying out encryption processing on the node information table.
6. A communication processing apparatus for distributed storage of data, comprising:
the generating module is used for generating a starting node according to the data distribution storage instruction after receiving the data distribution storage instruction; the distributed storage cluster comprises a plurality of propagation nodes;
a sending module, configured to send node information of the start node to at least one propagation node; the node information is used for communication between nodes;
the construction module is used for constructing a node information table according to the node information of the starting node and the node information of the propagation node; the propagation node is a node which receives the node information of the starting node;
the first sending module is used for sequentially transmitting the node information table in all nodes in the cluster so that the node information table obtains the node information of all the nodes in the cluster;
and the second sending module is used for resending the node information table to all nodes in the cluster.
7. The apparatus as recited in claim 6, further comprising: a failure node analysis module; the failure node analysis module is used for:
when the update times of the node information table reach a preset threshold value, performing failure node analysis on the cluster according to the current node information table and the historical node information table;
and if the failure node exists in the cluster, performing offline processing on the failure node.
8. The apparatus of claim 7, wherein the failure node analysis module is specifically configured to:
and comparing the current node information table with the historical node information table, and if the node information in the historical node information table is not displayed in the current node information table, determining the node corresponding to the node information which is not displayed as a failure node.
9. The apparatus as recited in claim 6, further comprising: a security verification module; the safety verification module is used for:
carrying out security verification on the service data through a preset analysis algorithm;
and if the service data passes the security check, performing format conversion on the service data to obtain identifiable service data.
10. The apparatus as recited in claim 6, further comprising: and the encryption module is used for carrying out encryption processing on the node information table.
CN202310357743.3A 2023-04-04 2023-04-04 Communication processing method and device for distributed data storage Pending CN116389514A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202310357743.3A CN116389514A (en) 2023-04-04 2023-04-04 Communication processing method and device for distributed data storage
PCT/CN2023/125350 WO2024207708A1 (en) 2023-04-04 2023-10-19 Communication processing method and device for distributed data storage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310357743.3A CN116389514A (en) 2023-04-04 2023-04-04 Communication processing method and device for distributed data storage

Publications (1)

Publication Number Publication Date
CN116389514A true CN116389514A (en) 2023-07-04

Family

ID=86972757

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310357743.3A Pending CN116389514A (en) 2023-04-04 2023-04-04 Communication processing method and device for distributed data storage

Country Status (2)

Country Link
CN (1) CN116389514A (en)
WO (1) WO2024207708A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024207708A1 (en) * 2023-04-04 2024-10-10 中国民航信息网络股份有限公司 Communication processing method and device for distributed data storage

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013054425A1 (en) * 2011-10-13 2013-04-18 富士通株式会社 Node device and communication method
WO2013116979A1 (en) * 2012-02-06 2013-08-15 华为技术有限公司 Method and system for transmitting node state information in distributed cluster system
CN111090495A (en) * 2019-12-02 2020-05-01 中兴通讯股份有限公司 Node management method, device, equipment, storage medium and system
CN113572820B (en) * 2021-07-01 2024-02-02 西安震有信通科技有限公司 Cluster construction method and device, terminal equipment and storage medium
CN116389514A (en) * 2023-04-04 2023-07-04 中国民航信息网络股份有限公司 Communication processing method and device for distributed data storage

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024207708A1 (en) * 2023-04-04 2024-10-10 中国民航信息网络股份有限公司 Communication processing method and device for distributed data storage

Also Published As

Publication number Publication date
WO2024207708A1 (en) 2024-10-10

Similar Documents

Publication Publication Date Title
EP2810180B1 (en) Multi-path data transfer using network coding
CN110069354A (en) The full link trace method of micro services and micro services framework
EP2661028B1 (en) Duplicate Packet Suppression
US20190280914A1 (en) Scalability, fault tolerance and fault management for twamp with a large number of test sessions
WO2018171641A1 (en) Method and device for transmitting and receiving network management information, transmitting apparatus and receiving apparatus
CN110505123B (en) Packet loss rate calculation method, server and computer-readable storage medium
KR101714227B1 (en) vehicle data communication method and apparatus using the same
CN105634977A (en) Method and device for discovering a path maximum transmission unit (PMTU)
CN116389514A (en) Communication processing method and device for distributed data storage
CN115002016B (en) Message processing system, method, device and storage medium
EP3672189B1 (en) Data transmission method, device and system
KR20180122816A (en) Operation method of communication node in network supporting long-range and low-power transmission
EP3171565B1 (en) Methods, devices and system for netconf hello packets interaction
CN115486041B (en) Data message format for communication across different networks
CN113890897A (en) Message processing method and related device
US9300438B2 (en) Method and system for transmitting large object
CN109525496B (en) Method and device for updating link state information
Phung et al. Enhancing rest http with random linear network coding in dynamic edge computing environments
CN106850153A (en) Data repeating method and system
CN117082054A (en) Data transmission method, device, system and medium
CN109792444B (en) Play-out buffering in a live content distribution system
CN114157716B (en) Block chain-based data processing method and device and electronic equipment
CN111385069A (en) Data transmission method and computer equipment
US20160234655A1 (en) Compression with Multicast Dictionary
CN103166847A (en) Method and device for ensuring graceful restart

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination