CN107948248A - Distributed storage method, control server and computer-readable recording medium - Google Patents
Distributed storage method, control server and computer-readable recording medium Download PDFInfo
- Publication number
- CN107948248A CN107948248A CN201711060240.0A CN201711060240A CN107948248A CN 107948248 A CN107948248 A CN 107948248A CN 201711060240 A CN201711060240 A CN 201711060240A CN 107948248 A CN107948248 A CN 107948248A
- Authority
- CN
- China
- Prior art keywords
- storage
- memory node
- node
- memory
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of distributed storage method, the method comprising the steps of：The communication that control server is established with one or more memory nodes is connected, and a shared storage allocation table is created in the control server；Every preset time, the instantaneous operating conditions information of each memory node is obtained, and the instantaneous operating conditions information of each memory node is write into the shared storage allocation table；When receiving data storage request, according to the instantaneous operating conditions information of each memory node recorded in the shared storage allocation table, choose particular memory node and preserve the corresponding data of the data storage request.The present invention can improve the efficiency and reliability of distributed storage.
Description
Technical field
The present invention relates to computer information technology field, more particularly to a kind of distributed storage method, control server and
Computer-readable recording medium.
Background technology
At present, the storage for Web object uses traditional centralised storage server mostly, in centralised storage
Under the memory module of server, storage server easily becomes the bottleneck of system, once server fail, may cause whole
The paralysis of a system, and can not realize Mass storage.Distributed storage is the effective of current solution mass data storage
Approach, still, existing distributed storage scheme cannot meet the own characteristic and application-specific scene of Web object well.
Therefore, it is badly in need of the distributed storage scheme for meeting particular requirement for Web object design.Therefore distribution of the prior art is deposited
Method for storing design is not reasonable, and there is an urgent need for improve.
The content of the invention
In view of this, the present invention proposes a kind of distributed storage method, control server and computer-readable recording medium,
By setting shared storage allocation table and distributed storage architecture, the storage efficiency of distributed storage is improved.
First, to achieve the above object, the present invention proposes a kind of control server, and the control server includes memory
And processor, the distributed memory system that can be run on the processor is stored with the memory, and the distribution is deposited
Storage system realizes following steps when being performed by the processor：
The communication that control server is established with one or more memory nodes is connected, and one is created in the control server
Shared storage allocation table；
Every preset time, the instantaneous operating conditions information of each memory node is obtained, and by the reality of each memory node
When work state information write the shared storage allocation table；And
When receiving data storage request, according to the reality of each memory node recorded in the shared storage allocation table
When work state information, choose particular memory node preserve the corresponding data of the data storage request.
Preferably, each memory node includes a master server and one from server；
Data, memory node array and the data that the shared storage allocation table includes memory node storage are dispensed on
The position of memory node, wherein, the memory node array is used for the status information of record storage node；And
Whether the storage data quantity of the status information of the memory node including memory node, node state, node live,
Upper a node and next node, total capacity and load factor.
Preferably, the specific memory node of selection, which preserves the corresponding data of the data storage request, includes：
According to the remaining memory capacity and load factor size of each memory node, chosen from all memory nodes remaining
Memory capacity meets the particular memory node of the data storage request and load factor minimum, wherein, each memory node remains
The total capacity that remaining memory capacity is equal to each memory node subtracts storage data quantity；And
The corresponding data of the data storage request are preserved into the master server storage unit of the particular memory node, and
Master server of the corresponding data of the data storage request in the particular memory node is recorded in the shared storage allocation table
Memory unit address.
Preferably, it is additionally operable to realize following steps when the distributed memory system is performed by the processor：
After the corresponding data of the data storage request are preserved to the master server of the particular memory node, control this specific
Memory node turn-on data synchronized process, by the slave clothes of the corresponding data duplication of the data storage request to the particular memory node
In device storage unit of being engaged in, and the replicate data is recorded in the shared storage allocation table in the particular memory node from service
Device memory unit address.
Preferably, it is additionally operable to realize following steps when the distributed memory system is performed by the processor：
When the master server of a memory node is stopped, the storage section is updated in the shared storage allocation table
The master server status information of point, and it is added to the shared storage using the backup server of the memory node as master server
Allocation table；And
When the master server of memory node and when being all stopped from server, by the memory node from described shared
Deleted in storage allocation table, and a upper node for the memory node is connected to next node.
In addition, to achieve the above object, the present invention also provides a kind of distributed storage method, this method is applied to control and takes
Business device, the described method includes：
The communication that control server is established with one or more memory nodes is connected, and one is created in the control server
Shared storage allocation table；
Every preset time, the instantaneous operating conditions information of each memory node is obtained, and by the reality of each memory node
When work state information write the shared storage allocation table；And
When receiving data storage request, according to the reality of each memory node recorded in the shared storage allocation table
When work state information, choose particular memory node preserve the corresponding data of the data storage request.
Preferably, each memory node includes a master server and one from server；
Data, memory node array and the data that the shared storage allocation table includes memory node storage are dispensed on
The position of memory node, wherein, the memory node array is used for the status information of record storage node；And
Whether the storage data quantity of the status information of the memory node including memory node, node state, node live,
Upper a node and next node, total capacity and load factor.
Preferably, the specific memory node of selection, which preserves the corresponding data of the data storage request, includes：
According to the remaining memory capacity and load factor size of each memory node, chosen from all memory nodes remaining
Memory capacity meets the particular memory node of the data storage request and load factor minimum, wherein, each memory node remains
The total capacity that remaining memory capacity is equal to each memory node subtracts storage data quantity；And
The corresponding data of the data storage request are preserved into the master server storage unit of the particular memory node, and
Master server of the corresponding data of the data storage request in the particular memory node is recorded in the shared storage allocation table
Memory unit address.
Preferably, this method further includes：
After the corresponding data of the data storage request are preserved to the master server of the particular memory node, control this specific
Memory node turn-on data synchronized process, by the slave clothes of the corresponding data duplication of the data storage request to the particular memory node
In device storage unit of being engaged in, and the replicate data is recorded in the shared storage allocation table in the particular memory node from service
Device memory unit address；
When the master server of a memory node is stopped, the storage section is updated in the shared storage allocation table
The master server status information of point, and it is added to the shared storage using the backup server of the memory node as master server
Allocation table；And
When the master server of memory node and when being all stopped from server, by the memory node from described shared
Deleted in storage allocation table, and a upper node for the memory node is connected to next node.
Further, to achieve the above object, the present invention also provides a kind of computer-readable recording medium, the computer
Readable storage medium storing program for executing is stored with distributed memory system, and the distributed memory system can be performed by least one processor, with
The step of making at least one processor perform distributed storage method described above.
Compared to the prior art, control server proposed by the invention, distributed storage method and computer-readable deposit
Storage media, by setting shared storage allocation table and the distributed storage architecture (storage service of control server and memory node
Device separates), there is provided a kind of distributed storage scheme based on shared storage allocation table, improves the storage effect of distributed storage
Rate, reliability and fault-tolerance, better than traditional centralized data storage scheme.
Brief description of the drawings
Fig. 1 is the system architecture schematic diagram between present invention control server and memory node；
Fig. 2 is the schematic diagram of present invention control one optional hardware structure of server；
Fig. 3 is the program module schematic diagram of one embodiment of distributed memory system in present invention control server；
Fig. 4 is the implementation process diagram of one embodiment of distributed storage method of the present invention.
Reference numeral：
| Control server | 2 |
| Shared storage allocation table | 24 |
| Memory node | 4、5、6 |
| Master server | 41、51、61 |
| From server | 42、52、62 |
| Memory | 21 |
| Processor | 22 |
| Network interface | 23 |
| Distributed memory system | 20 |
| Creation module | 201 |
| Writing module | 202 |
| Memory module | 203 |
| Process step | S41-S43 |
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, it is right
The present invention is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the present invention, not
For limiting the present invention.Based on the embodiments of the present invention, those of ordinary skill in the art are not before creative work is made
All other embodiments obtained are put, belong to the scope of protection of the invention.
It should be noted that the description for being related to " first ", " second " etc. in the present invention is only used for description purpose, and cannot
It is interpreted as indicating or implies its relative importance or imply the quantity of the technical characteristic indicated by indicating.Thus, define " the
One ", at least one this feature can be expressed or be implicitly included to the feature of " second ".In addition, the skill between each embodiment
Art scheme can be combined with each other, but must can be implemented as basis with those of ordinary skill in the art, when technical solution
It will be understood that the combination of this technical solution is not present with reference to there is conflicting or can not realize when, also not in application claims
Protection domain within.
Explanation is needed further exist for, herein, term " comprising ", "comprising" or its any other variant are intended to contain
Lid non-exclusive inclusion, so that process, method, article or device including a series of elements not only will including those
Element, but also including other elements that are not explicitly listed, or further include as this process, method, article or device
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Also there are other identical element in process, method, article or device including the key element.
As shown in fig.1, it is the system architecture schematic diagram between present invention control server and memory node.The present embodiment
In, control server 2 is established with one or more memory nodes (such as memory node 4-6) and communicates connection, in the control server 2
Establishment has a shared storage allocation table 24.Further, memory node 4 includes master server 41 and from server 42, stores
Node 5 includes master server 51 and from server 52, and memory node 6 includes master server 61 and from server 62.Below by way of
The description of Fig. 2 to Fig. 4 further illustrates technical scheme.
First, the present invention proposes a kind of control server 2.
As shown in fig.2, it is the schematic diagram of present invention control 2 one optional hardware structure of server.In the present embodiment, institute
Stating control server 2 may include, but be not limited to, and connection memory 21, processor 22, network can be in communication with each other by system bus
Interface 23.It is pointed out that Fig. 2 illustrate only the control server 2 with component 21-23, it should be understood that not
It is required that implement all components shown, the more or less component of the implementation that can be substituted.
Wherein, the control server 2 can be rack-mount server, blade server, tower server or cabinet
The computing devices such as formula server, which can be independent server or multiple servers are formed
Server cluster.
The memory 21 includes at least a type of readable storage medium storing program for executing, the readable storage medium storing program for executing include flash memory,
Hard disk, multimedia card, card-type memory (for example, SD or DX memories etc.), random access storage device (RAM), static random are visited
Ask memory (SRAM), read-only storage (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only deposit
Reservoir (PROM), magnetic storage, disk, CD etc..In certain embodiments, the memory 21 can be the control clothes
The internal storage unit of business device 2, such as the hard disk or memory of the control server 2.In further embodiments, the memory
21 can also be the External memory equipment of the control server 2, such as the plug-in type hard disk being equipped with the control server 2,
Intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash
Card) etc..Certainly, the memory 21 can also both include the internal storage unit of the control server 2 or including outside it
Portion's storage device.In the present embodiment, the memory 21 is installed on the operating system of the control server 2 commonly used in storage
With types of applications software, such as program code of the distributed memory system 20 etc..In addition, the memory 21 can also be used
In temporarily storing the Various types of data that has exported or will export.
The processor 22 can be in certain embodiments central processing unit (Central Processing Unit,
CPU), controller, microcontroller, microprocessor or other data processing chips.The processor 22 is commonly used in the control control
The overall operation of control server 2, for example, perform with it is described control server 2 carry out data interaction or communicate it is relevant control and
Processing etc..In the present embodiment, the processor 22 is used to run the program code stored in the memory 21 or processing number
According to, such as run described distributed memory system 20 etc..
The network interface 23 may include radio network interface or wired network interface, which is commonly used in
Establish and communicate to connect between the control server 2 and other electronic equipments.For example, the network interface 23 is used to pass through network
Control server 2 and the external data platform (such as master server 41 or from server 42) are connected, serviced in described control
Data transmission channel and communication connection are established between device 2 and external data platform.The network can be intranet
(Intranet), internet (Internet), global system for mobile communications (Global System of Mobile
Communication, GSM), wideband code division multiple access (Wideband Code Division Multiple Access,
WCDMA), the wirelessly or non-wirelessly network such as 4G networks, 5G networks, bluetooth (Bluetooth), Wi-Fi.
So far, oneself is through describing the application environment of each embodiment of the present invention and the hardware configuration and work(of relevant device in detail
Energy.In the following, above application environment and relevant device will be based on, each embodiment of the present invention is proposed.
As shown in fig.3, it is the program module of 20 1 embodiment of distributed memory system in present invention control server 2
Figure.In the present embodiment, the distributed memory system 20 can be divided into one or more program modules, it is one or
The multiple program modules of person are stored in the memory 21, and (are described in the present embodiment by one or more processors
Manage device 22) it is performed, to complete the present invention.For example, in figure 3, the distributed memory system 20 can be divided into wound
Model block 201, writing module 202 and memory module 203.Program module alleged by the present invention is to refer to complete specific work(
The series of computation machine programmed instruction section of energy, takes than program more suitable for describing the distributed memory system 20 in the control
The implementation procedure being engaged in device 2.The function of putting up with each program module 201-203 below is described in detail.
The creation module 201, for establishing control server 2 and one or more memory node (such as memory node 4-
6) communication connection, creates a shared storage allocation table 24 in the control server 2.In the present embodiment, each storage
Node is arranged to a storage server group, and each storage server group, from server, is used including a master server and one
In data synchronization and backup.For example, memory node 4 includes master server 41 and includes main service from server 42, memory node 5
Device 51 and from server 52, memory node 6 include master server 61 and from servers 62.
Preferably, in the present embodiment, the shared storage allocation table 24 is stored in the memory 21 of control server 2,
Each memory node shares a storage allocation table in whole system, and safeguards the shared storage distribution by control server 2
The state of table 24.Further, the shared storage allocation table 24 includes, but not limited to the data of memory node storage, storage
Node array and data are dispensed on position (memory unit address of such as master server or the depositing from server of memory node
Storage unit address).Wherein, the memory node array is used for the data such as the status information of record storage node.
Further, in the present embodiment, the status information of the memory node includes, but are not limited to following information：Deposit
Whether the storage data quantity (size) of storage node, node state (opening and closing), node live (isAvlie), a upper section
Point and next node, total capacity (total) and load factor (storage data quantity/total capacity=size/total) etc..Its
In, in the present embodiment, the node state includes the state of master server and the state from server.
For example, the data structure of the shared storage allocation table 24 can be defined as form：
Said write module 202, for every preset time (such as 3 seconds), obtaining the real-time working of each memory node
Status information, and the instantaneous operating conditions information of each memory node is write into the shared storage allocation table 24, i.e., in real time more
The nodal information stored in the new shared storage allocation table 24.Wherein, the instantaneous operating conditions packet of each memory node
Include, but be not limited to, whether storage data quantity (size), node state, node live, a upper node and next node, total
Capacity and load factor etc..
Preferably, in the present embodiment, control and communicated between server 2 and memory node using heartbeat mechanism.Tool
For body, the heartbeat mechanism used in the present embodiment is one function keepAlive () of addition, function in memory node
Send a request message (such as http request) every preset time (such as 3 seconds) includes to control server 2, the request message
The instantaneous operating conditions information (information such as such as status, isAlive, size) of memory node.Control server 2 receives storage section
After the instantaneous operating conditions information put, the nodal information stored in storage allocation table 24 will be shared described in real-time update, and return
One response message (such as ACK responses) is to memory node.
The memory module 203, for when receiving data storage request (such as Web object storage request), according to institute
The instantaneous operating conditions information of each memory node recorded in shared storage allocation table 24 is stated, particular memory node is chosen and preserves
The corresponding data of the data storage request.
Preferably, in the present embodiment, the specific memory node of selection preserves the corresponding number of the data storage request
According to including：
According to the remaining memory capacity and load factor size of each memory node, chosen from all memory nodes remaining
Memory capacity meets the particular memory node of the data storage request and load factor minimum, and the data storage request is corresponding
Data are preserved into the master server storage unit of the particular memory node, and record should in the shared storage allocation table 24
Memory unit address of the corresponding data of data storage request in the master server of the particular memory node.Wherein, each storage
The total capacity (total) that the remaining memory capacity of node is equal to each memory node subtracts storage data quantity (size).Due to
Load balancing operation is carried out according to the storage data quantity (size) of each memory node in the present embodiment, so as to ensure that data energy
It is evenly distributed over and is stored on each memory node.
Further, after the corresponding data of the data storage request are preserved to the master server of the particular memory node,
The particular memory node turn-on data synchronized process is controlled, by the corresponding data duplication of the data storage request to the particular memory
In the slave server storage unit of node, and the replicate data is recorded in the particular memory in the shared storage allocation table 24
The slave server storage unit address of node, realizes that data are synchronous and back up.That is, same part data, specific deposited at this
There are two copies, when master server is stopped, corresponding data can also be being found from server on storage node.
Further, in other embodiments, the distributed memory system 20 is additionally operable to：
When the master server of a memory node is stopped, the storage is updated in the shared storage allocation table 24
The master server status information (being such as updated to close) of node, and the backup server (such as from server) of the memory node is made
It is added to the shared storage allocation table 24 for master server, to ensure the availability of system；
When the master server of memory node and when being all stopped from server, by the memory node from described shared
Deleted in storage allocation table 24, and a upper node for the memory node is connected to next node.
By above procedure module 201-203, distributed memory system 20 proposed by the invention, by setting shared deposit
Storage allocation table and distributed storage architecture (storage server of control server and memory node separates), there is provided one kind is based on
The distributed storage scheme of shared storage allocation table, improves the storage efficiency, reliability and fault-tolerance of distributed storage, more
Better than traditional centralized data storage scheme.
In addition, the present invention also proposes a kind of distributed storage method.
As shown in fig.4, it is the implementation process diagram of one embodiment of distributed storage method of the present invention.In the present embodiment
In, according to different demands, the execution sequence of the step in flow chart shown in Fig. 4 can change, and some steps can be omitted.
Step S41, the communication for establishing control server 2 with one or more memory nodes (such as memory node 4-6) are connected,
A shared storage allocation table 24 is created in the control server 2.In the present embodiment, each memory node is arranged to one
Storage server group, each storage server group includes a master server and one from server, synchronous and standby for data
Part.For example, memory node 4 includes master server 41 and from server 42, memory node 5 includes master server 51 and from server
52, memory node 6 includes master server 61 and from server 62.
Preferably, in the present embodiment, the shared storage allocation table 24 is stored in the memory 21 of control server 2,
Each memory node shares a storage allocation table in whole system, and safeguards the shared storage distribution by control server 2
The state of table 24.Further, the shared storage allocation table 24 includes, but not limited to the data of memory node storage, storage
Node array and data are dispensed on position (memory unit address of such as master server or the depositing from server of memory node
Storage unit address).Wherein, the memory node array is used for the data such as the status information of record storage node.
Further, in the present embodiment, the status information of the memory node includes, but are not limited to following information：Deposit
Whether the storage data quantity (size) of storage node, node state (opening and closing), node live (isAvlie), a upper section
Point and next node, total capacity (total) and load factor (storage data quantity/total capacity=size/total) etc..Its
In, in the present embodiment, the node state includes the state of master server and the state from server.
For example, the data structure of the shared storage allocation table 24 can be defined as form：
Step S42, every preset time (such as 3 seconds), obtains the instantaneous operating conditions information of each memory node, and incites somebody to action
The instantaneous operating conditions information of each memory node writes the shared storage allocation table 24, i.e., storage is shared described in real-time update
The nodal information stored in allocation table 24.Wherein, the instantaneous operating conditions information of each memory node includes, but not limited to, and deposits
Whether storage data volume (size), node state, node live, a upper node and next node, total capacity and load factor
Deng.
Preferably, in the present embodiment, control and communicated between server 2 and memory node using heartbeat mechanism.Tool
For body, the heartbeat mechanism used in the present embodiment is one function keepAlive () of addition, function in memory node
Send a request message (such as http request) every preset time (such as 3 seconds) includes to control server 2, the request message
The instantaneous operating conditions information (information such as such as status, isAlive, size) of memory node.Control server 2 receives storage section
After the instantaneous operating conditions information put, the nodal information stored in storage allocation table 24 will be shared described in real-time update, and return
One response message (such as ACK responses) is to memory node.
Step S43, when receiving data storage request (such as Web object storage request), according to the shared storage point
Instantaneous operating conditions information with each memory node recorded in table 24, selection particular memory node preserves data storage please
Seek corresponding data.
Preferably, in the present embodiment, the specific memory node of selection preserves the corresponding number of the data storage request
According to including：
According to the remaining memory capacity and load factor size of each memory node, chosen from all memory nodes remaining
Memory capacity meets the particular memory node of the data storage request and load factor minimum, and the data storage request is corresponding
Data are preserved into the master server storage unit of the particular memory node, and record should in the shared storage allocation table 24
Memory unit address of the corresponding data of data storage request in the master server of the particular memory node.Wherein, each storage
The total capacity (total) that the remaining memory capacity of node is equal to each memory node subtracts storage data quantity (size).Due to
Load balancing operation is carried out according to the storage data quantity (size) of each memory node in the present embodiment, so as to ensure that data energy
It is evenly distributed over and is stored on each memory node.
Further, after the corresponding data of the data storage request are preserved to the master server of the particular memory node,
The particular memory node turn-on data synchronized process is controlled, by the corresponding data duplication of the data storage request to the particular memory
In the slave server storage unit of node, and the replicate data is recorded in the particular memory in the shared storage allocation table 24
The slave server storage unit address of node, realizes that data are synchronous and back up.That is, same part data, specific deposited at this
There are two copies, when master server is stopped, corresponding data can also be being found from server on storage node.
Further, in other embodiments, the distributed storage method further includes following steps：
When the master server of a memory node is stopped, the storage is updated in the shared storage allocation table 24
The master server status information (being such as updated to close) of node, and the backup server (such as from server) of the memory node is made
It is added to the shared storage allocation table 24 for master server, to ensure the availability of system；
When the master server of memory node and when being all stopped from server, by the memory node from described shared
Deleted in storage allocation table 24, and a upper node for the memory node is connected to next node.
Pass through above-mentioned steps S41-S43 and other correlation steps, distributed storage method proposed by the invention, by setting
Put shared storage allocation table and distributed storage architecture (storage server of control server and memory node separates), there is provided
A kind of distributed storage scheme based on shared storage allocation table, improves the storage efficiency, reliability and appearance of distributed storage
Mistake, better than traditional centralized data storage scheme.
Further, to achieve the above object, the present invention also provides a kind of computer-readable recording medium (such as ROM/RAM,
Magnetic disc, CD), the computer-readable recording medium storage is distributed formula storage system 20, the distributed memory system 20
It can be performed by least one processor 22, so that at least one processor 22 performs distributed storage method as described above
The step of.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to realized by hardware, but very much
In the case of the former be more preferably embodiment.Based on such understanding, technical scheme is substantially in other words to existing
The part that technology contributes can be embodied in the form of software product, which is stored in a storage
In medium (such as ROM/RAM, magnetic disc, CD), including some instructions are used so that a station terminal equipment (can be mobile phone, calculate
Machine, server, air conditioner, or network equipment etc.) perform method described in each embodiment of the present invention.
Above by reference to the preferred embodiment of the present invention has been illustrated, not thereby limit to the interest field of the present invention.On
State that sequence number of the embodiment of the present invention is for illustration only, do not represent the quality of embodiment.Patrolled in addition, though showing in flow charts
Order is collected, but in some cases, can be with the steps shown or described are performed in an order that is different from the one herein.
Those skilled in the art do not depart from the scope of the present invention and essence, can have a variety of flexible programs to realize the present invention,
It can be used for another embodiment for example as the feature of one embodiment and obtain another embodiment.It is every to utilize description of the invention
And the equivalent structure or equivalent flow shift that accompanying drawing content is made, other related technical areas are directly or indirectly used in,
It is included within the scope of the present invention.
Claims (10)
1. one kind control server, it is characterised in that the control server includes memory and processor, on the memory
The distributed memory system that can be run on the processor is stored with, the distributed memory system is performed by the processor
Shi Shixian following steps：
The communication that control server is established with one or more memory nodes is connected, and one is created in the control server and is shared
Storage allocation table；
Every preset time, the instantaneous operating conditions information of each memory node is obtained, and by the real-time work of each memory node
Make status information and write the shared storage allocation table；And
When receiving data storage request, according to the real-time work of each memory node recorded in the shared storage allocation table
Make status information, choose particular memory node and preserve the corresponding data of the data storage request.
2. server is controlled as claimed in claim 1, it is characterised in that each memory node includes a master server and one
It is a from server；
Data, memory node array and the data that the shared storage allocation table includes memory node storage are dispensed on storage
The position of node, wherein, the memory node array is used for the status information of record storage node；And
Whether the storage data quantity of the status information of the memory node including memory node, node state, node live, upper one
A node and next node, total capacity and load factor.
3. control server as claimed in claim 2, it is characterised in that the specific memory node of selection preserves the data
Storage asks corresponding data to include：
According to the remaining memory capacity and load factor size of each memory node, remaining storage is chosen from all memory nodes
Capacity meets the particular memory node of the data storage request and load factor minimum, wherein, the residue of each memory node is deposited
The total capacity that storage capacity is equal to each memory node subtracts storage data quantity；And
The corresponding data of the data storage request are preserved into the master server storage unit of the particular memory node, and in institute
State depositing for the corresponding data of the data storage request are recorded in shared storage allocation table in the particular memory node master server
Storage unit address.
4. control server as claimed in claim 3, it is characterised in that the distributed memory system is held by the processor
It is additionally operable to realize following steps during row：
After the corresponding data of the data storage request are preserved to the master server of the particular memory node, the particular memory is controlled
Node turn-on data synchronized process, by the slave server of the corresponding data duplication of the data storage request to the particular memory node
In storage unit, and record the replicate data in the shared storage allocation table and deposited in the slave server of the particular memory node
Storage unit address.
5. control server as claimed in claim 2, it is characterised in that the distributed memory system is held by the processor
It is additionally operable to realize following steps during row：
When the master server of a memory node is stopped, the memory node is updated in the shared storage allocation table
Master server status information, and the backup server of the memory node is added to the shared storage as master server and is distributed
Table；And
When the master server of memory node and when being all stopped from server, by the memory node from the shared storage
Deleted in allocation table, and a upper node for the memory node is connected to next node.
A kind of 6. distributed storage method, applied to control server, it is characterised in that the described method includes：
The communication that control server is established with one or more memory nodes is connected, and one is created in the control server and is shared
Storage allocation table；
Every preset time, the instantaneous operating conditions information of each memory node is obtained, and by the real-time work of each memory node
Make status information and write the shared storage allocation table；And
When receiving data storage request, according to the real-time work of each memory node recorded in the shared storage allocation table
Make status information, choose particular memory node and preserve the corresponding data of the data storage request.
7. distributed storage method as claimed in claim 6, it is characterised in that each memory node includes a master server
With one from server；
Data, memory node array and the data that the shared storage allocation table includes memory node storage are dispensed on storage
The position of node, wherein, the memory node array is used for the status information of record storage node；And
Whether the storage data quantity of the status information of the memory node including memory node, node state, node live, upper one
A node and next node, total capacity and load factor.
8. distributed storage method as claimed in claim 7, it is characterised in that the specific memory node of selection preserves should
The corresponding data of data storage request include：
According to the remaining memory capacity and load factor size of each memory node, remaining storage is chosen from all memory nodes
Capacity meets the particular memory node of the data storage request and load factor minimum, wherein, the residue of each memory node is deposited
The total capacity that storage capacity is equal to each memory node subtracts storage data quantity；And
The corresponding data of the data storage request are preserved into the master server storage unit of the particular memory node, and in institute
State depositing for the corresponding data of the data storage request are recorded in shared storage allocation table in the particular memory node master server
Storage unit address.
9. distributed storage method as claimed in claim 8, it is characterised in that this method further includes：
After the corresponding data of the data storage request are preserved to the master server of the particular memory node, the particular memory is controlled
Node turn-on data synchronized process, by the slave server of the corresponding data duplication of the data storage request to the particular memory node
In storage unit, and record the replicate data in the shared storage allocation table and deposited in the slave server of the particular memory node
Storage unit address；
When the master server of a memory node is stopped, the memory node is updated in the shared storage allocation table
Master server status information, and the backup server of the memory node is added to the shared storage as master server and is distributed
Table；And
When the master server of memory node and when being all stopped from server, by the memory node from the shared storage
Deleted in allocation table, and a upper node for the memory node is connected to next node.
10. a kind of computer-readable recording medium, the computer-readable recording medium storage is distributed formula storage system, described
Distributed memory system can be performed by least one processor, so that at least one processor performs such as claim 6-9
Any one of distributed storage method the step of.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711060240.0A CN107948248A (en) | 2017-11-01 | 2017-11-01 | Distributed storage method, control server and computer-readable recording medium |
| PCT/CN2018/076172 WO2019085334A1 (en) | 2017-11-01 | 2018-02-10 | Distributed storage method, control server and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711060240.0A CN107948248A (en) | 2017-11-01 | 2017-11-01 | Distributed storage method, control server and computer-readable recording medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107948248A true CN107948248A (en) | 2018-04-20 |
Family
ID=61934099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711060240.0A Pending CN107948248A (en) | 2017-11-01 | 2017-11-01 | Distributed storage method, control server and computer-readable recording medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107948248A (en) |
| WO (1) | WO2019085334A1 (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109542352A (en) * | 2018-11-22 | 2019-03-29 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN109739435A (en) * | 2018-12-03 | 2019-05-10 | 中科恒运股份有限公司 | File storage and update method and device |
| CN109960587A (en) * | 2019-02-27 | 2019-07-02 | 厦门市世纪网通网络服务有限公司 | The storage resource distribution method and device of super fusion cloud computing system |
| CN110286852A (en) * | 2019-05-20 | 2019-09-27 | 平安科技（深圳）有限公司 | Dual control framework distributed memory system, method for reading data, device and storage medium |
| CN110535969A (en) * | 2019-09-16 | 2019-12-03 | 腾讯科技（深圳）有限公司 | Date storage method, device, storage medium and equipment based on block chain network |
| CN110838938A (en) * | 2019-10-11 | 2020-02-25 | 成都飞机工业（集团）有限责任公司 | DNC data storage server scheduling method based on industrial control network |
| CN110968259A (en) * | 2018-09-30 | 2020-04-07 | 武汉斗鱼网络科技有限公司 | Step-by-step object storage system, object storage method and storage medium |
| CN111124316A (en) * | 2019-12-30 | 2020-05-08 | 青岛海尔科技有限公司 | Method and device for sharing storage space and computer readable storage medium |
| CN111193804A (en) * | 2020-01-02 | 2020-05-22 | 深圳市网心科技有限公司 | Distributed storage method and device, network node and storage medium |
| CN111190549A (en) * | 2019-12-30 | 2020-05-22 | 浪潮电子信息产业股份有限公司 | Method, device, equipment and medium for acquiring available capacity of shared volume |
| CN112637327A (en) * | 2020-12-21 | 2021-04-09 | 北京奇艺世纪科技有限公司 | Data processing method, device and system |
| WO2021249165A1 (en) * | 2020-06-12 | 2021-12-16 | 华为技术有限公司 | Ethernet storage system, and information notifying method therefor and related apparatus thereof |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114115718B (en) * | 2021-08-31 | 2024-03-29 | 济南浪潮数据技术有限公司 | Distributed block storage system service quality control method, device, equipment and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143215A (en) * | 2011-01-20 | 2011-08-03 | 中国人民解放军理工大学 | Network-based PB level cloud storage system and processing method thereof |
| CN102202087A (en) * | 2011-04-25 | 2011-09-28 | 中兴通讯股份有限公司 | Method for identifying storage equipment and system thereof |
| CN103019614A (en) * | 2011-09-23 | 2013-04-03 | 阿里巴巴集团控股有限公司 | Distributed storage system management device and method |
| CN103124299A (en) * | 2013-03-21 | 2013-05-29 | 杭州电子科技大学 | A Distributed Block Level Storage System in Heterogeneous Environment |
| CN103229487A (en) * | 2012-12-27 | 2013-07-31 | 华为技术有限公司 | Partition balance method, device and server in distributed storage system |
| CN104378423A (en) * | 2014-11-13 | 2015-02-25 | 普华基础软件股份有限公司 | Metadata cluster distribution storage system and storage data reading and writing method |
| CN104967691A (en) * | 2015-07-08 | 2015-10-07 | 浪潮(北京)电子信息产业有限公司 | Distributed storage control method and system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540775B (en) * | 2009-04-30 | 2012-05-23 | 华为技术有限公司 | Method and device for distributing contents and network system for distributing contents |
| US9727266B2 (en) * | 2009-12-29 | 2017-08-08 | International Business Machines Corporation | Selecting storage units in a dispersed storage network |
| CN103797770B (en) * | 2012-12-31 | 2015-12-02 | 华为技术有限公司 | A kind of method and system of shared storage resources |
| TW201431355A (en) * | 2013-01-25 | 2014-08-01 | Elta Technology Co Ltd | Expandable multimedia storage system, multimedia distribution device, and relevant computer program product |
| CN104219318B (en) * | 2014-09-15 | 2018-02-13 | 北京联创信安科技股份有限公司 | A kind of distributed file storage system and method |
| US9826038B2 (en) * | 2015-01-30 | 2017-11-21 | International Business Machines Corporation | Selecting a data storage resource of a dispersed storage network |
| US20170192688A1 (en) * | 2015-12-30 | 2017-07-06 | International Business Machines Corporation | Lazy deletion of vaults in packed slice storage (pss) and zone slice storage (zss) |
-
2017
- 2017-11-01 CN CN201711060240.0A patent/CN107948248A/en active Pending
-
2018
- 2018-02-10 WO PCT/CN2018/076172 patent/WO2019085334A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143215A (en) * | 2011-01-20 | 2011-08-03 | 中国人民解放军理工大学 | Network-based PB level cloud storage system and processing method thereof |
| CN102202087A (en) * | 2011-04-25 | 2011-09-28 | 中兴通讯股份有限公司 | Method for identifying storage equipment and system thereof |
| CN103019614A (en) * | 2011-09-23 | 2013-04-03 | 阿里巴巴集团控股有限公司 | Distributed storage system management device and method |
| CN103229487A (en) * | 2012-12-27 | 2013-07-31 | 华为技术有限公司 | Partition balance method, device and server in distributed storage system |
| CN103124299A (en) * | 2013-03-21 | 2013-05-29 | 杭州电子科技大学 | A Distributed Block Level Storage System in Heterogeneous Environment |
| CN104378423A (en) * | 2014-11-13 | 2015-02-25 | 普华基础软件股份有限公司 | Metadata cluster distribution storage system and storage data reading and writing method |
| CN104967691A (en) * | 2015-07-08 | 2015-10-07 | 浪潮(北京)电子信息产业有限公司 | Distributed storage control method and system |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110968259A (en) * | 2018-09-30 | 2020-04-07 | 武汉斗鱼网络科技有限公司 | Step-by-step object storage system, object storage method and storage medium |
| CN109542352A (en) * | 2018-11-22 | 2019-03-29 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN109542352B (en) * | 2018-11-22 | 2020-05-08 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN109739435A (en) * | 2018-12-03 | 2019-05-10 | 中科恒运股份有限公司 | File storage and update method and device |
| CN109739435B (en) * | 2018-12-03 | 2023-10-13 | 中科恒运股份有限公司 | File storage and updating method and device |
| CN109960587A (en) * | 2019-02-27 | 2019-07-02 | 厦门市世纪网通网络服务有限公司 | The storage resource distribution method and device of super fusion cloud computing system |
| CN110286852A (en) * | 2019-05-20 | 2019-09-27 | 平安科技（深圳）有限公司 | Dual control framework distributed memory system, method for reading data, device and storage medium |
| CN110535969B (en) * | 2019-09-16 | 2021-06-15 | 腾讯科技（深圳）有限公司 | Data storage method, device, storage medium and equipment based on block chain network |
| CN110535969A (en) * | 2019-09-16 | 2019-12-03 | 腾讯科技（深圳）有限公司 | Date storage method, device, storage medium and equipment based on block chain network |
| CN111049902A (en) * | 2019-09-16 | 2020-04-21 | 腾讯科技（深圳）有限公司 | Data storage method, device, storage medium and equipment based on block chain network |
| CN111049902B (en) * | 2019-09-16 | 2021-08-13 | 腾讯科技（深圳）有限公司 | Data storage method, device, storage medium and equipment based on block chain network |
| CN110838938B (en) * | 2019-10-11 | 2021-09-07 | 成都飞机工业（集团）有限责任公司 | DNC data storage server scheduling method based on industrial control network |
| CN110838938A (en) * | 2019-10-11 | 2020-02-25 | 成都飞机工业（集团）有限责任公司 | DNC data storage server scheduling method based on industrial control network |
| CN111190549A (en) * | 2019-12-30 | 2020-05-22 | 浪潮电子信息产业股份有限公司 | Method, device, equipment and medium for acquiring available capacity of shared volume |
| CN111124316A (en) * | 2019-12-30 | 2020-05-08 | 青岛海尔科技有限公司 | Method and device for sharing storage space and computer readable storage medium |
| CN111124316B (en) * | 2019-12-30 | 2023-12-19 | 青岛海尔科技有限公司 | Storage space sharing method and device and computer readable storage medium |
| CN111193804A (en) * | 2020-01-02 | 2020-05-22 | 深圳市网心科技有限公司 | Distributed storage method and device, network node and storage medium |
| CN111193804B (en) * | 2020-01-02 | 2022-09-09 | 深圳市网心科技有限公司 | Distributed storage method and device, network node and storage medium |
| WO2021249165A1 (en) * | 2020-06-12 | 2021-12-16 | 华为技术有限公司 | Ethernet storage system, and information notifying method therefor and related apparatus thereof |
| CN112637327A (en) * | 2020-12-21 | 2021-04-09 | 北京奇艺世纪科技有限公司 | Data processing method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019085334A1 (en) | 2019-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107948248A (en) | Distributed storage method, control server and computer-readable recording medium | |
| CN107766132B (en) | Multi-task scheduling method, application server and computer readable storage medium | |
| CN103095687B (en) | metadata processing method and device | |
| CN110166409A (en) | Equipment cut-in method, related platform and computer storage medium | |
| CN109376197B (en) | Data synchronization method, server and computer storage medium | |
| CN104683161B (en) | Network management and device based on SaaS | |
| CN110213326B (en) | Method, system, equipment and storage medium for capacity expansion of metadata node cluster | |
| CN107689982A (en) | Multi-data source method of data synchronization, application server and computer-readable recording medium | |
| CN112328170B (en) | Cloud hard disk capacity expansion method and device, computer equipment and storage medium | |
| CN107682426A (en) | interface proxy method and application server | |
| CN109739690A (en) | Backup method and Related product | |
| CN108052281A (en) | Business Information storage method, application server and computer storage media | |
| CN104168326A (en) | Server load balancing method and system | |
| CN105471662A (en) | Cloud server and virtual network strategy centralized control system and method | |
| CN109634735A (en) | A kind of method and device for dispatching Pod | |
| CN107770207A (en) | The method of data syn-chronization between multi-terminal equipment | |
| CN107526616A (en) | A kind of main frame carries out the method and device of data write-in and reading to BMC | |
| CN1330124C (en) | Method and apparatus for virtualizing network resources | |
| CN110515728B (en) | Server scheduling method and device, electronic equipment and machine-readable storage medium | |
| CN108427635A (en) | Quickly method, server and the computer readable storage medium of test web page | |
| CN114020405B (en) | Virtual machine scheduling method, device, equipment and medium | |
| CN110933192A (en) | Local area network IP address allocation method, device, system and computer equipment | |
| CN112804276B (en) | Virtual broadband remote access server, control method thereof and communication system | |
| CN112131010A (en) | Server layout method and device, computer equipment and storage medium | |
| CN106161068B (en) | Recovery prompting and distributing method for network resources and controller |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180420 |
|
| RJ01 | Rejection of invention patent application after publication |