CN110945548B - Computer-implemented system and method for managing large distributed storage pools in a blockchain network - Google Patents

Abstract

A computer-implemented method implemented in a blockchain network is described. A validator node receives data about a newly mined block including a plurality of transactions and sends a delete request to a distributed storage pool to delete the plurality of transactions from the distributed storage pool. A node storing the distributed storage pool stores a plurality of transactions, the plurality of transactions forming at least a portion of a distributed storage pool of transactions awaiting mining into a block of a blockchain. The computer-implemented method further includes receiving a delete request from a validator node of the blockchain network, the delete request identifying one or more transactions that have been included in the newly mined block, the delete request indicating that the one or more transactions should be deleted from the distributed storage pool.

Description

Computer-implemented systems and methods for managing large distributed storage pools in blockchain networks

Technical Field

The present application generally relates to computer-implemented methods and systems suitable for implementation in nodes of a blockchain network. A modified blockchain node structure, network architecture, and protocol for processing a large number of transactions and large transaction blocks are described.

Background Art

In this document, the term "blockchain" is used to include all forms of electronic, computer-based distributed ledgers, including but not limited to blockchain and transaction chain technology, permissioned and unpermitted ledgers, shared ledgers and their variants. It should be noted that other blockchain implementations and protocols also fall within the scope of the present invention.

A blockchain is a consensus-based electronic ledger implemented as a decentralized, distributed computer-based system composed of blocks, which consist of transactions and other information. Each transaction is a data structure that encodes the transfer of control of digital assets between participants in the blockchain system and includes at least one input and at least one output. Each block contains a hash of the previous block, so that the blocks are linked together to create a permanent, unchangeable record of all transactions written to the blockchain since its inception. Transactions contain small programs, called scripts, embedded in the inputs and outputs of the transaction, which specify how and by whom the output of the transaction can be accessed. These scripts are written in a stack-based scripting language.

In order for a transaction to be written to the blockchain, it must be "validated". Some network nodes perform work to ensure that each transaction is valid, and invalid transactions are rejected by the network. For example, a software client installed on the node may reference an unspent transaction output (Unspent Transaction2/46

Verification work is performed on transactions of Unlimited Token Outputs (UTXO). Verification can be performed by executing its locking and unlocking scripts. If the execution of the locking and unlocking scripts evaluates to true (TRUE), and if certain specific conditions are met, the transaction is valid and can be written to the blockchain. Therefore, in order for a transaction to be written to the blockchain, the transaction must: i) be verified by the node that receives the transaction - if the transaction is verified, the node relays the transaction to other nodes in the network; ii) be added to a new block; iii) be mined, that is, added to the public ledger of past transactions. A transaction is considered confirmed when a sufficient number of blocks are added to the blockchain to make the transaction effectively irreversible.

Digital entrepreneurs have begun exploring the use of cryptographically secure systems and data that can be stored on blockchains to implement new systems. It would be extremely beneficial if blockchains could be used to automate tasks and processes. Such solutions would be able to leverage the advantages of blockchains (e.g., permanent tamper-proof records of events, distributed processing, etc.) while being more versatile in the application of said solutions.

One area of research is the use of blockchain to implement "smart contracts". These smart contracts are computer programs designed to automatically execute the terms of a machine-readable contract or agreement. Unlike traditional contracts written in natural language, a smart contract is a machine-executable program that includes rules that process inputs to produce results, and the smart contract can perform actions based on these results.

Summary of the invention

The future of blockchain technology depends, at least in part, on the proposal of a new architecture that can increase the number of issued transactions per second. One requirement of this new architecture is to remove the current restrictions on the block size limit.

In this case, one technical problem is how to manage large blocks of data and how to manage the storage of very large blockchains. Another technical problem is how to manage the pool of transactions waiting to be mined into blocks and incorporated into the blockchain. The local storage pool cannot provide sufficient storage capacity, so a new model of distributed memory pool (DMP) needs to be designed. Once a transaction is included

In blockchain, it is hoped that transactions can be removed from such a distributed storage pool. However, another technical problem is that

The problem is how to manage the removal of transactions from the distributed storage pool in a secure way without causing data inconsistency problems in the distributed storage pool. Another technical problem is how to mitigate the problem of data inconsistency while improving security against attacks that attempt to destroy data stored in the distributed storage pool.

The purpose of certain embodiments of the present invention is to solve these technical problems by providing the technical solutions described herein. In particular, the present application describes a communication and data management protocol between verification nodes and distributed storage pool nodes in a blockchain network, which allows pruning of distributed storage pools while mitigating data inconsistency issues within the distributed storage pools. Therefore, the present invention provides a technical contribution by increasing the capacity of a blockchain network while maintaining its distributed nature.

A computer-implemented method for a node of a blockchain network is described, the computer-implemented method comprising:

storing a plurality of transactions forming at least a portion of a distributed storage pool of transactions awaiting mining into a block of a blockchain; and

A deletion request is received from a verification node of the blockchain network, the deletion request identifying one or more transactions that have been included in a newly mined block, the deletion request indicating that the one or more transactions should be deleted from the distributed storage pool.

The foregoing method allows a verification node to communicate with a distributed storage pool to remove transactions that have been incorporated into the blockchain and can therefore be removed from the distributed storage pool. Sending/receiving an explicit delete request indicating that one or more transactions should be deleted from the distributed storage pool is different from notification of only mined transactions. In the configuration of the prior art, mined transactions are not removed from the storage pool until enough blocks have been mined on top to confirm the transaction. Therefore, the storage pool contains a large number of transactions that are unlikely to be needed for a considerable period of time after being mined into the blockchain. In addition, if the network nodes automatically delete transactions after a period of time without following a specific deletion request protocol, then in some cases this will lead to data inconsistencies. The present invention recognizes and solves these problems.

The method may also include marking the transaction as removed from the distributed storage pool when a first threshold number of deletion requests for the transaction are received. The method may also include physically removing the transaction from the distributed storage pool when a second threshold number of deletion requests for the transaction are received. The second threshold is greater than the first threshold. In addition, the first and second threshold numbers of deletion requests need to come from a threshold number of different verification nodes in the blockchain network. These features require a consensus level for marking transactions as deleted in the distributed storage pool, and a further consensus level is required before physically removing transactions from the distributed storage pool. This alleviates the problem of data inconsistency and also improves security against attacks that attempt to destroy data stored in the distributed storage pool. A transaction effectively includes three states with respect to the distributed storage pool: available; removed; and physically removed. If necessary, a transaction marked as removed can be restored to be available again in the distributed storage pool. Depending on the configuration of the system, physically removed transactions may not be recoverable.

A transaction may be physically removed from the distributed storage pool only after a threshold time has elapsed since the deletion request was received, during which no further data requests for the transaction were received. For example, the threshold time may correspond to at least 1, 2, 3, 4, 5, 6 or more blocks incorporated into the blockchain after the transaction was incorporated into the blockchain requested to be deleted from the distributed storage pool. Moreover, the transaction may be physically removed from the distributed storage pool in a decreasing chronological order since the deletion request was received, during which no further data requests for the transaction were received. The computer-implemented method also includes marking the transaction as removed from the distributed storage pool when the transaction depends on a previously removed transaction. These features further help mitigate data inconsistency issues and also improve security against attacks that attempt to corrupt data stored in the distributed storage pool.

The deletion request may be stored in a new deletion request database associated with the distributed storage pool. The deletion request received from a validator that has not verified the transaction may be discarded. The transaction is discarded only when the check verifier option is enabled. The number of data requests for transactions that have been marked as removed from the distributed storage pool may also be monitored. Using such a system, after receiving a threshold number of data requests for the transaction marked as removed from the distributed storage pool, the transaction may be marked as a candidate for restoration to the distributed storage pool. When a restoration request for the transaction is received or when a threshold number of restoration requests for the transaction is received, the transaction may be unmarked as removed from the distributed storage pool. The threshold number of restoration requests may require a threshold number of different verification nodes from the blockchain network. In response to a query for a transaction that has been removed from the distributed storage pool, a message indicating that the transaction has been removed from the distributed storage pool is sent. Moreover, such a message may indicate that the transaction has been removed from the distributed storage pool and the number of deletion messages received that have been removed. Again, these features further help mitigate the problem of data inconsistency and also improve security against attacks that attempt to destroy data stored in the distributed storage pool.

The above features of the distributed storage pool management protocol are described from the perspective of the distributed storage pool storage nodes in the blockchain network. From the perspective of the verification nodes in the blockchain network, a computer-implemented method is provided, the method comprising:

receiving data regarding a newly mined block including a plurality of transactions; and

A deletion request is sent to the distributed storage pool to delete the plurality of transactions from the distributed storage pool.

The deletion request should preferably contain a random number and/or a timestamp to avoid spreading fraudulent copies that could affect the consensus protocol. The deletion request is preferably signed using the private key of the verification node.

The node may store a record of transactions it has verified, and only send delete requests for transactions it has previously verified. When the check verifier option is enabled, delete requests may be sent for transactions that the node has previously verified. Also, restore requests may be sent to make the transaction available again in the distributed storage pool, for example, after a blockchain reorganization to maintain data consistency within the system.

The deletion request may be activated by one or more of the following:

Receive mined blocks;

said blockchain reorganization; and

Double spending or other forms of transaction conflicts.

6/46

Transaction pruning from the distributed storage pool may be activated by one or more of the following:

Manual pruning;

the expiry of the transaction; and

The transaction in the distributed storage pool has reached the storage limit.

Embodiments of the present invention may be provided in a variety of forms. For example, a computer-readable storage medium including computer-executable instructions may be provided, which, when executed, cause one or more processors to perform the methods described herein. An electronic device may also be provided, the electronic device comprising: an interface device; one or more processors connected to the interface device; a memory connected to the one or more processors, the memory storing computer-executable instructions, which, when executed, cause the one or more processors to perform the methods described herein. In addition, a node of a blockchain network may also be provided, the node being used to perform the methods described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will become apparent and elucidated with reference to the embodiments described herein. Embodiments of the invention will now be described by way of example only and with reference to the accompanying drawings, in which:

Figure 1 shows the overall structure of the block;

FIG2 shows the steps from a user submitting a transaction to the transaction being completed on the blockchain in the form of an operation diagram;

FIG3 is a graph showing an example of the total size of transactions awaiting confirmation in a storage pool;

Figure 4 shows multiple nodes linked to an internal decentralized storage facility.

FIG5 illustrates a configuration in which each node is part of a distributed storage pool and a distributed storage facility;

Figure 6 shows a network configuration where validators are members of storage pools.

Together they form a decentralized distributed network;

Figure 7 shows the functionality of the new node;

FIG8 illustrates a new Merkle tree structure that constitutes a modification to the current protocol.

Figure 9 shows the workflow for creating a Bloom filter;

FIG10 shows a workflow illustrating how transactions are encoded in Invertible Bloom Filters (IBFs) and Invertible Bloom Lookup Tables (IBLTs);

FIG. 11 shows a storage pool cluster including a validator node v and a storage pool node s.

Figure 12 shows that the validator node _vn (0≤n＜N) receives updates about the newly mined block.

——After validating the block, the validator sends a delete (DELETE) request for the published transaction to the distributed storage pool (DMP), and each storage pool node _sm (0≤m＜M) tracks the storage request on the _SDB table and the removal request on the _RDB table; and

FIG13 shows a state diagram of a transaction id _i - depending on the type (i.e., STORE, REMOVE, REVERT) and number (e.g., N,

N*, N**), a transaction can change the state to Available, Removed, or Physically removed.

DETAILED DESCRIPTION

Types of blockchain network nodes

A blockchain network can be described as a peer-to-peer open membership network that anyone can join without an invitation or consent from other members. Distributed electronic devices that run an instance of the blockchain protocol (under which the blockchain network runs) can participate in the blockchain network. Such distributed electronic devices can be called nodes.

The electronic devices that run the blockchain protocol and form the nodes of the blockchain network can be various 8/46

Types include, for example, computers (such as desktop computers, laptops, tablet computers, servers, computer farms), mobile devices (such as smartphones), wearable computers (such as smart watches), or other electronic devices.

The nodes of the blockchain network are connected to each other using suitable communication technologies, which may include wired and wireless communication technologies. In many cases, the blockchain network is implemented at least in part on the Internet, and some nodes may be located in geographically dispersed locations.

Currently, nodes maintain a global ledger of all transactions on the blockchain, with all transactions grouped into multiple blocks, where each block contains a hash of the previous block on the blockchain. The global ledger is a distributed ledger, and each node can store a full copy or a partial copy of the global ledger. Transactions of nodes that affect the global ledger are verified by other nodes, thereby maintaining the validity of the global ledger. One of ordinary skill in the art will understand the details of implementing and operating a blockchain network.

Each transaction typically has one or more inputs and one or more outputs. The scripts embedded in the inputs and outputs specify how and who can access the output of the transaction. The output of a transaction can be an address to which the value is transferred as a result of the transaction. The value is then associated with the output address as an unspent transaction output (UTXO). Subsequent transactions can use the address as an input to spend or disperse the value.

Nodes can be of different types or categories depending on their functions. Four basic functions associated with nodes have been proposed: wallet, mining, full blockchain maintenance, and network routing. These functions may vary. Nodes may have multiple functions. For example, a "full node" provides all four functions. Lightweight nodes, such as those implemented in digital wallets, may only have wallet and network routing functions. Instead of storing the full blockchain, digital wallets may track block headers, which are used as indexes when querying blocks. Nodes communicate with each other using connection-oriented protocols such as the Transmission Control Protocol (TCP/IP).

Additional types or categories of nodes can be provided: Merchant Node (

M nodes are designed to focus on the rapid propagation of transactions. M nodes can store 9/46

It may not store the full blockchain and does not perform mining functions. In this sense, M-nodes are similar to lightweight nodes or wallets. However, M-nodes include additional features to enable fast propagation of transactions.

The operation of the M-node is focused on the rapid verification and propagation of unconfirmed transactions, especially to other M-nodes, from which unconfirmed transactions are quickly pushed out to other nodes in the blockchain network. To facilitate this function, M-nodes are allowed to make a greater number of incoming connections, especially outgoing connections, which might otherwise be allowed to be used to manage nodes under the protocol.

M-nodes can be collectively referred to as merchant networks (M-nets) (or M-networks). The term "merchant" can be interpreted as "dedicated". M-nodes can be integrated into blockchain networks. Each M-node is a dedicated node on a blockchain network that meets specific hardware and performance capabilities to ensure that it can perform the functions of an M-node. In other words, an M-network can be considered a subnet within a blockchain network and distributed across the blockchain network. M-nodes can be set up to perform one or more dedicated functions or services.

In order for the M network to operate reliably and provide services at a certain level of security, the M nodes need to maintain a good overview of the entire M network, so an effective routing protocol needs to be established. Whenever an M node receives a start transaction, the M node needs to broadcast the transaction to several other M nodes and other nodes. In the context of the M network, this is equivalent to finding a solution to the Multiple Traveling Salesman Problem (MTSP). There are many solutions to the Multiple Traveling Salesman Problem, any of which can be applied to the M network. Each M node runs routing optimization in some form of state-of-the-art.

In some embodiments, the M network is implemented as a decentralized IP multicast type network. That is, in order to enable incoming transactions to quickly spread to the blockchain network, multicast can be used to ensure that the transaction is quickly broadcast throughout the M network, so that all M nodes focus on forwarding transactions to other nodes in the blockchain network.

The multicast network architecture allows the possibility to distribute data to a group of destination nodes simultaneously, without duplicating the data for each node interested in receiving the information. If a node wants to receive a multicast transmission, it joins the multicast group (registration phase), after which said node will be able to receive all data sent through the multicast group. IP multicast can be extended to a larger group of receivers without the need for a priori knowledge of how many receivers there are, and by requiring the source to send a packet only once, the network infrastructure can be efficiently utilized. Due to the nature of multicast networks, it is impractical to use connection-oriented protocols such as TCP due to the simultaneous communication with a large number of other nodes. Accordingly, connectionless protocols are used.

Some blockchain networks use TCP for node-to-node communication. Data packets sent using TCP have associated sequence numbers, which are used for sorting. In addition to this, the TCP protocol involves a three-way handshake process both when establishing a connection and when terminating a connection. Data packets sent over TCP have associated sequence numbers and a three-way handshake protocol. When establishing a connection, 128-136 bytes are transmitted, while closing a connection costs 160 bytes. Therefore, the handshake cost in packet transmission is as high as 296 bytes. In addition, when a node receives a new transaction, the node notifies other nodes through an inventory (INV) message containing the transaction hash. The node receiving the INV message checks whether the hash of the transaction has been seen before; if not, the node will request the transaction by sending a GetData message. The time required for a transaction to be transmitted from node A to node B is T1=verification+TCP(inv+detdata+tx), where TCP() represents the time overhead introduced by the TCP handshake procedure.

M-nodes can be used to communicate with other nodes authorized by existing protocols using TCP. However, M-nodes can use connectionless protocols such as User Datagram Protocol (UDP) to communicate from M-node to M-node in multicast situations, or even more appropriately from M-node to multiple M-nodes. Unlike TCP, UDP does not involve a handshake protocol, so M-nodes are able to propagate transactions faster. This can also prevent malicious nodes from bundling other nodes by sending repeated INV messages instead of sending actual transactions.

The lightweight nature of UDP is associated with certain tradeoffs. There is less error checking and no error recovery. In some implementations, these limitations of UDP can be overcome at the application level by making error recovery, sequencing, and retransmission functions of the application layer. Putting error checking at the application level eliminates the overhead of the network.

In one example scenario, a regular node on a blockchain network generates a transaction (e.g., a merchant-based payment) that it wishes to process through the M network. The regular node can send the transaction to an M node, which then broadcasts the transaction to other M nodes using multicast, or can send the transaction directly to multiple M nodes if the regular node knows the IP multicast address of the M node. In some examples, all M nodes of an M network are members of a single multicast address, so all transactions sent to that address are received by all M nodes. However, in some cases, there may be multiple multicast addresses associated with an M network, and the receiving M node can evaluate from the routing information whether the transaction needs to be further broadcast to other multicast addresses to propagate the transaction to the entire M network.

Multicast helps ensure rapid initial propagation of new transactions to all M nodes; however, multicast solutions do not necessarily address the scalability issues of blockchain networks caused by increased transaction throughput. Each node in the network typically maintains a storage pool (Mempool) containing unconfirmed transactions it has seen that have not yet been merged into the blockchain by the network nodes that have completed the proof of work. A significant increase in the number of transactions used in payment processing can increase the number of transactions stored in each storage pool. Therefore, although nodes in the M network are able to receive new transactions almost simultaneously, the nodes may have storage capacity limitations for large and rapidly changing storage pools.

To solve this problem, M nodes can use a shared storage pool implemented through a distributed hash table (DHT) as an alternative to using multicast.

Assuming an average transaction (TX) size of 500 bytes and a transaction rate of approximately 104 transactions/second, the M network can receive approximately 400GB of daily incoming data. All of this data needs to be stored in a storage pool of unconfirmed transactions for varying periods of time. Therefore, the M network requires a lot of storage space and the ability to store data quickly. In order not to place too many demands on each individual M node, the M nodes implement a shared storage pool that relies on DHT. Instead of having each M node save all incoming transactions in its own storage pool, each M node only stores a certain portion of the total, as well as the hash and associated key values for the rest.

DHTs are a class of decentralized distributed systems that allow for the partitioning of membership of key sets between nodes, enabling messages to be sent only to the owner of a given key in an efficient and optimized manner. Each node of the network can be seen as a cell of an array of hash tables. DHTs are designed to manage a large number of nodes, allowing new nodes to join the network, old nodes to leave the network or crash, without compromising the integrity of the shared data. DHTs ensure decentralization (there is no central authority and no central coordination), scalability (the system has efficient behavior with millions of nodes), and fault tolerance (the system is reliable and is able to manage nodes that join and leave the network or crash). Each node in the network can be in contact with only a few other nodes, so the network is not overloaded when changes or new data appear.

This concept can also be applied to the UTXO database, which contains the set of all unspent outputs on the blockchain. The UTXO database can be built using a DHT to share the contents among a group of nodes.

Many possible DHT structures and protocols can be used to implement a shared storage pool for an M network. One example is Pastry ^TM , but there are many others. Pastry ^TM is a protocol designed to maintain an overlay network that can store and transmit information over a distributed system. Each node in a Pastry ^TM network is assigned a 128-bit identifier that indicates the node's position in a circular node identifier (ID) space (ranging from 0 to 2 ¹²⁸ -1). When a node joins the network, the ID is randomly assigned. Each node maintains a routing table, a neighborhood set, and a leaf set.

One factor to consider when determining the dimensionality of a stable DHT is the number of replicas required to ensure robustness and reliability of the entire network. As mentioned earlier, nodes can join and leave the network, which should not affect the availability of data. If the node storing transaction A leaves the network, it is necessary to find transaction A in another part of the network. In existing blockchain networks, the number of networks or the number of blockchain replicas is equal to the number of full nodes in the network (5000 replicas on average), but this affects scalability.

In an M-network configuration, the storage pool is not fully replicated on each M-node, but is implemented via a DHT. To provide reliability, the DHT can be implemented with some overlap; that is, each transaction data item is replicated in multiple M-nodes, although not in every M-node. For example, the DHT can be implemented to specify a minimum number of 2 replicas, while assuming complete independence between the two nodes. This would potentially result in two nodes being down at any given time.

Therefore, a process for storing a new transaction in a distributed storage pool may include the following steps, wherein the distributed storage pool is implemented using a DHT. The process includes: a node sends a transaction to an M node. The M node hashes a transaction or a transaction ID according to the implementation to obtain a key value. The key value indicates the M node or multiple M nodes (in the case of replicated data) where the transaction is stored. The M node then stores the transaction in the distributed storage pool, which may include: routing the transaction to the correct M node where the transaction is to be stored based on the key value and the assigned ID of the M node in the M network. Depending on the DHT protocol involved, the M node may receive a confirmation. When the M node receives a new transaction from a regular node, the M node may perform certain verification operations to verify the authenticity of the transaction.

The transaction can be hashed to generate a key for the transaction. The key can indicate where the transaction should be stored in the DHT, which can be on a node other than the current M node. The M node evaluates whether the transaction is already in the running DHT. Based on the key space partitioning between the M nodes that make up the M network, each M node has a portion of stored transactions. In some configurations, the key space is divided between the participating M nodes. The division may involve overlap to cause replication for network resilience. In some embodiments (for example, using Pastry ^TM ), each M node is assigned a unique key or ID number, and transactions can be stored in the M node or multiple M nodes (where replication is required) based on the proximity of the transaction key value. The M node can have a hash or key value of the stored portion of the transaction and the rest locally. Therefore, the M node is able to evaluate whether a new transaction is in the DHT based on the local data in operation.

If the transaction is not in the DHT, then in operation, the M-node stores the transaction in the DHT according to its key value. In general, this can take the form of a put(k,tx) operation, where k is the key value and tx is the transaction. The applicable DHT routing protocol ensures that the transaction is sent to and stored at the appropriate M-node. Based on the selected implementation, the DHT can operate according to various protocols for distributed hash tables. Using the DHT to store transactions in the M-network avoids the use of inventory/get data (INV/GETDATA) messages in the M-network to route transactions to each M-node.

In the operation of this example, the M node can send the transaction to a regular node in the blockchain network according to the normal transaction forwarding protocol of the blockchain network. For example, the communication to the ordinary node can use TCP for node-to-node connection.

In one configuration, the M node includes a processor, a network interface, and a memory. The M node can be implemented using any suitable computing hardware that has network connectivity and sufficient processing and storage resources to perform the functions described herein. The M node may include processor executable instructions to implement the functions described herein. In some cases, the processor executable instructions may be referred to as a blockchain merchant node application, but it is understood that based on the hardware and operating system, the instructions may be implemented in one or more modules, applications, scripts, or other programming structures. The processor may include a multi-core processor and/or multiple processors.

The memory stores data based in part on its DHT key value (i.e., M node ID), including an allocated portion of a DHT-based storage pool. In this example embodiment, the memory also stores a routing table, a neighborhood set, and a leaf set. The routing table contains a list of specific routing destinations within the M network, and when a node receives a data packet, it will refer to the routing table to understand where to send the data. The routing table may also contain information about the distance between each destination and the M node. The neighborhood set (e.g., based on a proximity metric (ping delay)) contains information about M nodes that are close. The leaf set contains M nodes that are close in number. If the key value (node ID) of the M node is close in number, then the node is close in number. The memory also includes an M node reputation table, which will be further described below.

In order to provide scalability, in addition to implementing storage pools using DHT, the M-network also allows nodes to join the M-network. The new node needs to have the address of at least one M-node that is already part of the M-network so that the new node can direct its joining request to one of the M-nodes. The M-node can perform certain verification actions, which may involve querying the new node. For example, the M-network can have a set of minimum standards associated with joining the M-network that is assigned to the M-node. For example, the standards may include minimum processing resources available, minimum free storage available, or connectivity requirements.

Assuming the M-node completes any verification operations performed to validate the new node, the M-node forwards a join request (Joinrequest()) to the DHT in accordance with any DHT protocol that controls the operation of the DHT. The DHT communicates with the new node to provide the new node with the routing table, key value (node ID), and any other data to enable the new node to act as a new M-node on the M-network.

It should be understood that the ease with which nodes can join the M network makes it easy for malicious nodes to join the network. In order to identify and isolate potential malicious nodes, a configuration provides that the M node stores an M node reputation table for tracking and updating node behavior rankings. When a new node joins the network, the node can be added to the M node reputation table, as shown in the node ID field. In some embodiments, the table may also include the joining time. The table also includes the score or rating of the M node.

Scores can be adjusted up or down based on certain behavioral indicators. For example, if an M-node fails to forward transactions, remains silent for a period of time, floods the M-network with traffic determined to be non-transactional, or otherwise engages in negative behavior, then the ranking of the M-node may be lowered or decremented. If a node's score falls below a preset minimum, then the node may be excluded from the M-network.

The M-node reputation table maintained on a particular M-node may be limited to tracking the scores of its neighbors, rather than the entire M-network. Thus, when a new M-node joins the network at time t, the M-node reputation tables of its neighbors do not contain any information about the new node, but starting at time t, the neighbors begin to build the reputation of the new node, storing the information in the node registry. For example, if the new node is a silent node, meaning that the node does not transmit information received over the network, then all neighbors begin to record this behavior in their respective M-node reputation tables (e.g., assigning a negative value to the ID of the new node). After a certain time t+n, if the M-node reputation tables of all nodes that know the new node contain negative values, then the node may decide to isolate the new node and ban the new node from the network.

Transactions in the M network's distributed storage pool may have to wait a considerable amount of time before being confirmed, i.e. before being incorporated into a block that is added to the blockchain and confirmed. A block is considered "confirmed" when a sufficient number of subsequent blocks are added to the blockchain above it, such that it is computationally infeasible to reverse growth in the chain and remove blocks to change to a different branch or fork.

Due to the size and flexibility of the storage pool and the volume of transactions, a given transaction may remain unconfirmed longer than in some blockchain implementations. In traditional implementations, once a transaction is incorporated into a block, it is removed from the storage pool. This means that if the block ends up being orphaned, all transactions in the block are retransmitted across the network. In the case of fast transaction networks, this may not be practical or may result in long delays in the confirmation of certain transactions.

Thus, in some embodiments, the storage pool may track the number of confirmations of the block into which the transaction has been incorporated, i.e., the number of blocks added to the blockchain after the block into which the transaction has been incorporated. The transaction will only be removed from the storage pool after a predetermined number of confirmations have occurred. The predetermined number may be 4, 5, 6, 7, or any suitable number for a given embodiment. The storage pool data entry may be structured to include a transaction ID field, a transaction field, and a number of confirmations (NoC) field. In another embodiment, the storage pool data entry may simply record the number of blocks without tracking the NoC. Based on the current number of blocks in the blockchain, the storage pool data entry may assess how many confirmations have occurred from the number of blocks.

Once the necessary number of confirmations has occurred, the transaction can be safely removed from the storage pool. This way, there is no loss of transactions in the event of orphaned blocks, and after the necessary number of confirmations, the transaction is permanently deleted.

The solution described below in this specification utilizes a modified type of fast verification node as described above. A new full node configuration is described, which is actually an M-node verification architecture with large-scale storage capabilities and a modified operating protocol. M-nodes and storage nodes together form the core of the new full node. The new node structure is described in detail, including the necessary technical requirements and technical solutions, and a sustainable incentive model is provided.

Block size and storage requirements

The current block size is 1Mb. Currently, a block consists of a field containing a so-called magic number (always the same value), a value indicating the actual size of the block, a so-called block header, the number of transactions contained in the block, and a list of actual transactions. The list of actual transactions always starts with the coinbase transaction. In Figure 1, the overall structure of a block is shown.

The block header contains the following:

1. Version number (4 bytes)

2. Hash of the previous block header (32 bytes)

3. Merkle root hash (32 bytes)

4. Time (4 bytes)

5. Target threshold (encoded as nBits–4 bytes)

6. A non-repeating random number that is used only once (4 bytes)

Currently, blocks contain about 2,000 transactions and are mined approximately every 10 minutes (the 10-minute block time is set as a compromise between first confirmation time and time wasted on chain splits). This provides a transaction rate of about 3.5 transactions per second, with a theoretical maximum of 7 transactions per second. In comparison, VISA operates at a rate of about 10,000 transactions per second, and can reach more than 50,000 transactions per second.

Clearly, in order to build a competitive payment system, some circumvention of the current constraints is necessary. Since a 10 minute block time has been established, changes in block size and, therefore, changes in the blockchain itself must be considered. In this specification, a scalable solution is described that is capable of handling, for example, approximately 50,000 transactions per second.

Increasing the current block size, or even removing the limit altogether, is a much discussed and sometimes controversial topic. There appear to be strong arguments on both sides, as there are clear benefits and trade-offs to both keeping the current size and increasing it.

Assuming a transaction rate of r, we can calculate the necessary block size. Let's assume that the block time is (on average) 10 minutes. Therefore, let's assume that the number of transactions per block is T(r). We get

T(r)＝r·6·10 ² block ^-1

If s _Tx is the average transaction size (in bytes), the block size B(r,s _Tx ) can be expressed as

B(r,s _Tx )＝s _Tx ·T(r)＝s _Tx ·r·6·10 ²

So, considering 50,000 transactions/second and s _Tx = 500 bytes, a quick back calculation shows:

This in turn led to Storage requirements. Clearly, for blocks of this size, we need a slightly different approach to block propagation and storage. Table 1 below shows the relationship between transaction rate, average transaction size, block size, and the amount of storage space required per month and per year.

Table 1: Relationship between transaction rate, average transaction size, block size, and the amount of storage space required per month and per year

FIG2 shows a schematic diagram of the operation from the user submitting a transaction to the transaction being completed in the blockchain.

A system is provided in which special verification nodes (a shared storage pool between special verification nodes is maintained through a distributed hash table DHT) receive transactions, verify transactions, and distribute transactions in the storage pool. The verification node provides services to the network nodes, namely, a list of valid transaction hashes. The network nodes assemble pre-blocks (block skeletons) (BlockSkeletons) based on these hashes and try to solve hash puzzles. When a solution to the puzzle is found, the winning network node sends the block skeleton back to the verification node. The verification node verifies the block and ensures block storage. Initially, it is possible and feasible for the verification node to store the block itself. When the block size eventually exceeds a certain size threshold, the verification node will: a) expand its own storage capacity; or b) outsource storage to a dedicated storage node. These two architectures will be discussed below in this specification.

New full node

In order With the block size of , it seems no longer feasible to rely on PC-type nodes to provide the storage capacity for hosting the full image of the blockchain. Instead, The challenge then becomes creating a system that can accommodate new blocks while maintaining the distributed, decentralized, and trustless nature of the network.

Two types of full node architectures are envisioned, as well as combinations of the two types:

1. A validator node with an associated petabyte storage rack

2. A verification node with an associated storage pool based on an internally decentralized distributed peer-to-peer (P2P) single-node network

Combination of 3.1 and 2

The proposed solution seeks to address the problem of keeping a distributed and decentralized record of the blockchain by introducing nodes similar to today’s so-called full nodes, but with the ability to scale as the size of blocks and the number of transactions increase.

The differences are not limited to purely architectural and hardware related issues. The new nodes proposed in this article will be dedicated nodes, compared to the home computer-based full nodes running at the time of writing. Said new nodes require a significant investment, and therefore, the incentives will be very different. In a scalable paradigm, both M-nodes (validation nodes) and new full nodes (combination of validation and storage nodes) will expect to be compensated for their services.

On the other hand, we have decentralized distributed storage solutions that are mostly composed of a single node. Storj (Wilkinson et al., 2016), Sia (NebulousLabs), and MaidSafe (maidsafe) are good examples.

As mentioned above, supernodes consisting of petabyte (Pb) racks and peer-to-peer (P2P) storage systems are also conceivable.

Obviously, it is important to compensate all full nodes, as the network will rely on their (winning) blocks to eventually make it into the public blockchain.

Nodes will be organized into pools, which act as supernodes. In order to maintain the distributed nature of the blockchain, there must be a certain number of such supernodes (≥100). Supernodes are connected but do not overlap.

Technical requirements

As mentioned above, there are two completely different architectures to consider when discussing new full nodes (see Table 2).

New full nodes need to maintain two types of storage:

1) A distributed hash table (DHT) memory/storage similar to random access memory (RAM) of the storage pool.

2) Persistent tape/disk storage of blockchain.

As mentioned above, for a transaction rate of 50,000 transactions/second, blocks are expected to be This means that the annual storage requirement is 365×24×6×15 Gb=7.9·10 ⁵ Gb=0.8 Pb/yr (see Table 1).

Table 2 shows the comparison between current full nodes and future full nodes:

At the same time, racks/clusters need to maintain a storage pool. This allows for fast recovery of chunks. The necessary size of the storage pool is more difficult to estimate. Currently, chunk sizes are around 1 megabyte. About 4 transactions per second The total size of transactions waiting in the storage pool ranges from 2 megabytes to about 70 megabytes. FIG3 shows an example graph of the total size of transactions waiting for confirmation in the storage pool.

As mentioned above, we envision two fundamentally different structures capable of storing large amounts of data, as well as a combination of the two structures. These two structures are illustrated in Figures 4 and 5. Figure 4 shows a configuration that includes multiple nodes with access to an internal decentralized storage facility. Figure 5 shows a configuration in which each node is part of a distributed storage pool and a distributed storage facility. The architecture described in Figure 4 seems suitable for larger entities that own and maintain multiple verification nodes, all of which have access to the entity's own storage facilities. In contrast, the architecture described in Figure 5 is fully decentralized. This solution is suitable for a single node (such as a home PC with sufficient storage capacity) that wants to join a shared distributed storage pool. There are already underlying storage technologies for this purpose (e.g., Storj, Sia, MaidSafe).

Figure 6 shows a network configuration where validators are members of storage pools. Together, these pools form a decentralized distributed network.

Full Node Operation

In the large block scenario, we will face another situation, and not just due to space requirements. The storage pool should be able to hold the equivalent of one block, which is about 15 gigabytes (~15Gb), and preferably the equivalent of the next block to be mined. This will also be combined with the overhead that needs to be considered.

1) The storage pool needs to be synchronized with other validating nodes. This involves exchanging reversible bloom filter lookup tables (Michael T. Goodrich, 2011)

2) Need to double check IBLTs and retrieve lost transactions (Tx)

3) Transactions that require additional retrieval for verification

4) Assemble blocks based on block skeletons received from network nodes or other full nodes. The new full node keeps the latest storage pool. It does this by exchanging IBLTs and other validations with network nodes and new full nodes.

The network node sends a block skeleton (Tuple) containing the following:

1. A non-repeating random number that is used only once, n

2.IBLT

3. Coinbase transactions

Based on this, the new full node sorts the transactions accordingly (according to a specific set of rules) and assembles the newly mined block. The new full node continues to store the block in its own storage and propagates the skeleton to other new full nodes. The protocol will be described in more detail later in this specification.

excitation

An important feature of a particular configuration is that incentives are built into the system to incentivize the provision of new node structures and services. Incentives are needed because storing a blockchain requires a lot of costs. Figure 7 shows the functionality of a new full node.

[Transaction] Validation nodes will be compensated for transaction validation via a fee system.

The incentive lies in the 100 block confirmation time T ₁₀₀ .

1) The network node needs to wait for a time t of about T ₁₀₀ .

2) The verification node needs to wait for a time t of approximately T ₁₀₀ before receiving the fee for verifying the transactions in the block.

3) A new full node needs to wait for a time t of approximately T ₁₀₀ before receiving the block assembly fee and storage fees depending on the size.

Therefore, a confirmation time of 100 blocks will provide the necessary incentive for network nodes to propagate the skeleton block (including the payment) to a series of new full nodes, and incentivize new full nodes to propagate the skeleton block to other new full nodes.

It should also be noted that network nodes are free to choose the list of transactions they wish to include in a block. Therefore, we envision a market consisting of validating nodes that compete by compiling lists of verified transactions from which network nodes can select and purchase through commitment transactions.

Revisiting the Excavation

The network nodes collect transactions (Txs) from a storage pool (or, as envisioned in this article, from dedicated validation nodes), organize the transactions into blocks, and try to find a solution to a hash puzzle (a non-repeating random number used only once). The block header contains the hash of the previous block on the blockchain, the root of the Merkle tree of transactions, and a non-repeating random number used only once contained by the network node. The solution of the puzzle consists of calculating a double SHA256 hash of a non-repeating random number used only once (chosen iteratively) concatenated with the previous block hash and the Merkle root, and checking whether it is less than the so-called difficulty target. If it is less than the difficulty target, the puzzle is solved; if it is greater than the difficulty target, the iteration of non-repeating random numbers used only once continues. This remains unchanged in the new paradigm. What poses a challenge is the greatly enlarged block size and the distribution of mined blocks throughout the network. For gigabyte-sized blocks, broadcasting the entire block through the network is not necessarily feasible.

Instead, we propose a solution consisting of the following steps:

1. The network node receives a list of verified transactions from the validating/M-node and/or the new full node.

2. The network nodes themselves may or may not operate their own storage pool of transaction hash values that follows a specific ordering convention.

3. Network nodes solve the hash puzzle by determining a non-repeating random number n that is used only once.

4. Next, the hash tree (Merkle tree, hereinafter referred to as HT) is calculated and the root of the Merkle tree is stored (see the next section).

5. The transaction list is used to create an IBLT. IBLT can be used to calculate the content difference between two collections (such as storage pools) and to coordinate the two collections.

6. The tuple {n; IBLT; CoinBase Tx; HT root} is broadcasted to the validation/M nodes.

7. The new full node is the blockchain’s storage pool and storage operation DHT.

8. The pool reassembles the block based on the tuple {n; IBLT; CoinBase Tx; HT root} and records the block on the blockchain by a) storing the block itself or b) by storing it on a dedicated storage node.

Avoid contention between network nodes

Network nodes are able to select a list of verified transactions from a market consisting of multiple validating nodes. Unless otherwise stated, it is reasonable to assume that network nodes will select the list that maximizes their potential revenue. A careful reader may point out that this can lead to network nodes predominantly selecting the same list from the same node. This, in turn, can lead to a situation where some network nodes compete with each other, trying to mine the same block. This favors the network nodes with the most hashing power.

We propose to add an extra field to the block header. Said field contains a random number chosen by each network node. This ensures that each network node starts from a different starting point, thus preventing solving a block from being attributed solely to hashing power. This in turn will simulate the current situation where network nodes tend to mine similar but independently chosen and slightly different blocks.

protocol

Here, we describe the protocols necessary to operate a new full node.

In order for the proposed system to work, the storage pools of the involved nodes (validators, new full nodes, ...) should follow a convention for ordering transactions. Here, we propose to use the canonical order proposed by Gavin Andresen. In said ordering, the ordering is related to the list of transactions in the block, but here, we propose an idea: all validators and new full nodes use the same convention for their storage pools.

The convention can be summarized as follows:

1) Sort transactions in ascending order relative to the previous transaction hash.

2) Add the first transaction from the sorted list that does not depend on subsequent transactions.

As mentioned before, blocks contain what is called a Merkle root hash. It is produced by hashing all transactions (including coinbase transactions) and then hashing the concatenated hashes until the Merkle root hash is reached. Obviously, if the network nodes are not producing coinbase transactions, the validating nodes can calculate the entire Merkle tree and thus the Merkle root and the corresponding hash.

Here, we propose a procedure to compute the Merkle tree in the following way:

The validator node calculates a Little Merkle Root. The procedure is the same as when calculating a standard Merkle root, with a few exceptions:

1) Exclude coinbase transactions.

2) Includes so-called commitment transactions.

3) The network node creates a coinbase transaction and concatenates it with the small Merkle root hash to produce the Merkle root hash.

The new Merkle tree structure is shown in Figure 8. Note that the structure constitutes a modification to the current protocol.

Modify block header

As mentioned above, we propose to add an extra field to the block header containing a random number chosen by network nodes. Therefore, the solution to the hash puzzle changes as follows:

SHA256[SHA256[Prev.Block Hash//Merkle Root//nonce]]

SHA256[SHA256[RND||Prev.Block Hash//Merkle Root//nonce]]

Therefore, we propose to enhance the block header of a newly mined block with an additional field containing a random number. The block header will contain the following:

1. Version number (4 bytes)

2. Hash of the previous block header (32 bytes)

3. Merkle Root Hash (32 bytes)

4. Time (4 bytes)

5. Target threshold (encoded as nBits–4 bytes)

6. A non-repeating random number that is used only once (4 bytes)

7. Random number (4 bytes)

Verification → Network Node

Validation Node:

o Upon request, a validating node (which may or may not be a new full node) prepares a list of verified transactions to be mined.

οThe validator node creates a commitment transaction.

ο Compute the so-called Little Merkle Root (see previous subsection), which includes the Commitment Transaction.

οThe validator node prepares two IBLTs:

1) for all transactions in the block (IBLT1); and

2) All corresponding transaction identifiers (TxID) in the block (IBLT2)

οThe validator node sends to the network node:

1) Little Merkel root.

2)IBLT1.

3) IBLT2 (optional - only if the network node runs with its own TxID-/storage pool).

4) Previous block hash.

5) The above hash checksum.

Network Nodes:

After receiving the data from the validator node, the network node proceeds to create a coinbase transaction. In addition, the coinbase transaction contains an output field with a secret that matches the secret in the commitment transaction.

ο The network node uses the small Merkle root received from the validator node and combines it with the coinbase transaction to create the Merkle root hash.

ο The network nodes now have all the information they need to start solving the hash puzzle.

οExcavate as described above.

Network node → new full node

Network Nodes:

After mining a block, the network node sends the following to the new full node list:

ο A non-repeating random number that is used only once (the solution to the puzzle), n.

οCoinbase transactions.

οBlock header.

ο Merkle root.

οLittle Merkel root.

οIBLT1.

οIBLT2 (optional).

ο Hash checksum.

New full node:

The o node checks whether the received data is consistent by calculating a checksum (hash).

οNodes use IBLT1 to ensure that transactions in a block are present in the storage pool.

o Using IBLT1 to query the storage pool, the node assembles the chunk. It then stores the chunk (see the section on new chunk nodes and storage).

οData received from network nodes will be broadcast to other new full nodes.

Customized transaction list

We envision a situation where the market for verified transactions will adapt to the needs of network nodes. Network nodes tend to choose listings that maximize their potential, and validating M-nodes will follow these trends.

In some cases, a network node may wish to customize its blocks by merging transactions from two or more lists. A set reconciliation can be performed between two sets by calculating the difference between two IBLTs. The network node then sends the IBLT containing the difference back to one of the providing nodes, in this way retrieving the information needed to make a list containing all transactions from both lists.

Obviously, if a network node wants to compile its own list based on several lists, this will introduce additional challenges. This article briefly describes the key points.

If network nodes want to merge lists from various validators, it is unclear how to merge the Merkle roots. Here, we make the following suggestions:

Constructing a Big Little Merkle Root of multiple individual Little Merkle Roots; and

Combine big and small Merkle roots with coinbase transactions.

The additional fee is not proportional to the additional amount of transactions added to the list/block. Since it is reasonable to assume that the various storage pools overlap a lot, merging lists is equivalent to adding a few (relatively speaking) transactions from another list. However, in order to merge lists, network nodes must "buy" the full list from each validating node (via commitment transactions). Whether this is a profitable approach for network nodes remains to be seen.

Merging lists from multiple validating nodes requires a commitment between the network node and each providing validation node. It is conceivable that the network node could abuse the system. Currently, there is no rule/protocol that forces all committed transactions to end up in a block. One possibility is that the validating node could check each block and veto those that contain its transactions.

Summary of node structure and operation

With a significant increase in transaction volume, it may not be feasible to focus on mining. The solution described in this specification relegates various tasks to corresponding specialized nodes, and the network nodes themselves become more specialized. Compiling a list of verified transactions, reconstructing blocks based on block skeletons, and storage are all functions that require a lot of resources. We have described these issues in detail in this specification.

The elements described in this manual include:

o A new type of node structure, referred to in this article as a new full node or super node, may or may not be an extension of the validating M node.

o Nodes operate according to a protocol that efficiently allows the broadcast of gigabyte-sized blocks from validating nodes to network nodes and from network nodes to new full nodes.

o Two overall storage structures for storing the blockchain, which may or may not be part of the proposed new full nodes.

o Incentive model that allows for a market for pre-block lists of verified transactions to be assembled and stored in blocks after mining.

oThe new Merkle tree structure eliminates the need for network nodes to maintain their own storage pools.

o Adding an extra field in the block header with a random number chosen by network nodes to avoid mining becoming a competition based purely on hashing power.

According to one implementation, a computer-implemented method for a node of a blockchain network is provided, the computer-implemented method comprising:

receiving mined data from the blockchain network corresponding to a plurality of verified transactions;

Assembling blocks based on the mined data; and

The assembled blocks are sent to a storage entity for storage on the blockchain.

This approach enables nodes to construct large blocks to be stored on a storage entity without requiring network nodes to construct and store large blocks and transmit such blocks across the blockchain network. In addition, the described architecture allows the use of large storage entities dedicated to storing large and growing blockchains.

The computer-implemented method may further include:

Receiving the transaction from the blockchain network;

verifying the transaction received from the blockchain network;

Maintaining, together with other nodes in the blockchain network, a distributed, decentralized storage of the verified transactions; and

Data corresponding to the verified transactions are distributed to the blockchain network for mining, the data including a list of the verified transactions. Each list can provide a complete list of verified transactions to be mined into a block.

This computer-implemented method effectively eliminates the requirement for network nodes to perform verification functions while retaining a distributed, decentralized storage of verified transactions with other nodes in the blockchain network. In addition, the method enables transaction verification nodes to provide services to network nodes by preparing and distributing data corresponding to verified transactions to the blockchain network for mining. For example, the method can prepare and distribute a list of verified transactions.

The step of maintaining the distributed, decentralized storage of the verified transactions with other transaction verification nodes in the blockchain network may include synchronizing the transaction verification nodes on the blockchain network to maintain the latest list of the verified transactions in a decentralized and distributed manner. For example, the verification nodes may be synchronized by exchanging reversible bloom filter lookup tables. The verified transactions are sorted in a defined order so that a common sorting system is used on the transaction verification nodes in the blockchain network to maintain the distributed, decentralized storage of the verified transactions. For example, a canonical sorting system may be used to maintain the distributed, decentralized storage of the verified transactions. It has been found that this is a particularly effective method of ensuring that transaction data on the network is maintained in a consistent manner while maintaining a decentralized, distributed storage.

The step of distributing data corresponding to the verified transactions to the blockchain network for mining may include preparing data corresponding to the list of verified transactions (e.g., a reversible Bloom lookup table and any accompanying data corresponding to the list of verified transactions, wherein the verified transactions are included in the block). In addition, the step of distributing data corresponding to the verified transactions to the blockchain network for mining includes creating a commitment transaction for the digital asset in exchange for providing the data corresponding to the list of verified transactions to a network node. For example, a hash tree, a Patricia tree, or another type of radix tree is computed, and the commitment transaction is included.

After distributing and mining the data corresponding to the verified transaction by solving an associated cryptographic puzzle (e.g., a hash puzzle), the mined data is sent back to the transaction verification node, rather than being stored directly on the blockchain by the network nodes. The mined data can be assembled into (large) blocks and stored on storage entities and/or distributed storage systems that are specifically configured to store large amounts of data. As previously described, this enables the verification nodes to construct large blocks to be stored on the storage entities, without the need for network nodes to construct and store large blocks and transmit such blocks over the blockchain network. In addition, the described architecture allows the use of large storage entities dedicated to storing large and growing blockchains.

The mined data received from the blockchain network may include block headers corresponding to the verified transactions. The mined data may also include transactions for digital assets in exchange for assembling the blocks based on the mined data. In addition, the method may include a requirement to wait for a time period t associated with a minimum number of blocks before receiving the digital assets. This provides an incentive scheme for providing verification nodes. A minimum time is required before receiving digital assets, network nodes are incentivized to propagate skeleton blocks (including payments) to a series of nodes, and nodes are incentivized to propagate skeleton blocks to other nodes.

The step of assembling the blocks based on the mined data may involve assembling large blocks, each block having, for example, a size of at least 2, 4, 6, 8, 10, 50, 100, 500, 1000, or 10,000 megabytes. Although the upper limit will increase over time, a nominal upper limit value of 1 PB may be specified. Each block may include, for example, at least 5,000, 10,000, 500,000, 100,000, 500,000, or 1,000,000 transactions. Although the upper limit will increase over time, a nominal upper limit value of 10 ¹² transactions per block may be specified. As previously described, the methods, nodes, and blockchain network architectures described herein enable large blocks to be constructed and stored on storage entities without the need for network nodes to construct and store a large number of transactions. This enables the system to cope with significantly increased transaction rates.

In the methods described herein, a block may be modified to include a block header that includes a random number provided by a network node. That is, a transaction verification node may be used to process a block that includes a block header that includes a random number provided by a network node when receiving a resolved transaction from a blockchain network. This constitutes a change to the block header, so that the network node may select or randomly generate a number to be inserted into the block header. This helps ensure that even if many network nodes select the same transaction list, the network nodes do not compete with each other when attempting to mine the same block.

The storage entity for storing the aforementioned large blocks of data can be shared among multiple transaction verification nodes on the blockchain network, and the multiple transaction verification nodes form a super node on the blockchain network, wherein the shared storage entity is a public storage node, distributed storage, or a combination of the two. This architecture enables the formation of a super node on the blockchain network and allows the provision of dedicated storage facilities to store blockchains and serve blockchain networks.

In view of the above, a super node of a blockchain network is also provided, and the super node includes:

A plurality of validating nodes as previously described; and

a shared storage entity for storing said blockchain,

Wherein, the shared storage entity is a public storage node, a distributed storage or a combination of the two, and

The blocks assembled by the multiple nodes are sent to and stored in the shared storage entity, whereby the shared storage entity maintains the blockchain.

This architecture is better suited to handle the large block sizes required to achieve the desired increase in transaction rates, which is the purpose of the methods and configurations described herein. For example, the shared storage entity may be configured to have a storage capacity of at least 100G, and more preferably at least 1, 10, 100, or 1000GB. Although the upper limit will increase over time, a nominal upper limit value of 10 ⁶ TB or even 10 ⁶ YB may be specified.

In terms of the overall network architecture, a blockchain network comprising a plurality of such supernodes may be provided. The supernodes may be connected (but not overlapping) on the blockchain network, and a shared storage entity of each supernode is used to store a copy of the blockchain. A supernode actually comprises a group of nodes that form a pool that serves as a supernode. In order to maintain the distributed nature of the blockchain, there should advantageously be a certain number of such supernodes (e.g., at least 10, 50, 100, or 1000, and optionally less than 100,000,000).

Bloom Filter and Invertible Bloom Lookup Table (IBLT)

In this section, we summarize the properties of so-called Bloom filters and an extension of these filters called reversible Bloom lookup tables.

In its simplest form, a Bloom filter is an array. The array has two parameters M and k associated with it. M is the number of bits in the array, and k is the number of different hash functions H _k , so

Where S ₁₆ is the space of hexadecimal strings on which the hash function is applied. To determine whether transaction Tx ₀ belongs to the set targeted by the Bloom filter, we need to calculate H1(Tx ₀ )…Hk(Tx ₀ ) and then check whether the corresponding bit in the array is set to 1. Figure 9 shows the workflow for creating a Bloom filter.

If one or more are no, then Tx ₀ is definitely not in the queried set. However, Bloom filters do allow for false positives. This is because the probability that the hash function changes a bit to 1 is p = 1/|size of array| = 1/M. Therefore, there is no so-called given hash function used. Set the bit to 1.

Therefore, if there are k hash functions, the probability that a given bit is not set to 1 is

If n elements need to be inserted, the probability becomes

An obvious drawback of a Bloom filter is that it neither tracks nor maintains any particular order. Clearly, if we wish to maintain an index of the items being filtered, we need to extend the functionality of the filter. This is where reversible Bloom filters (IBFs) and reversible Bloom lookup tables (IBLTs) come into play.

Instead of just activating bits in the array, the XOR sum of the key, the hash value (as before) and the total counter is stored in each field of the IBF. The process is illustrated in Figure 10, which shows a workflow illustrating how a transaction is encoded in an IBF/IBLT.

application

We assume that there are two nodes _N1 and _N2 , maintaining pools _m1 and _m2 respectively. Each pool contains elements from the hexadecimal string S _16. We further assume that the pools follow the ordering convention proposed by Andresen and outlined earlier in this specification.

Now, _N1 sends _m1 to _N2 . _N2 can now handle the collection reconciliation in two ways:

1) Calculate the set difference by △m=m ₂ -m ₁ (see (David Eppstein, 2011), (Michael T. Goodrich, 2011))

2) Iterate the transactions in _m2 and check whether the transaction exists in the storage pool

We see that IBLT can be used for at least two purposes:

1) Let nodes assemble mined blocks based on transactions already in their storage pool, helping to identify and retrieve blocks they do not have.

2) Maintain a certain level of synchronization between storage pools belonging to different nodes.

Pruning on a distributed storage pool

As mentioned above, the future of blockchain technology relies, at least in part, on the proposal of a new architecture that can increase the number of issued transactions per second. One requirement of this new architecture is to remove the current restrictions on block size limits. In this case, local storage pools cannot provide sufficient storage capacity, so a new model of distributed storage pools (DMPs) needs to be designed. The modified architecture for managing distributed storage pools provides the following capabilities:

Global consensus on the validity of transactions stored in a distributed storage pool;

Consistency and integrity of stored information;

Fast read and write operations; and

Security mechanisms against routing and storage attacks in distributed storage pools.

Once a transaction is included in a newly mined block, the corresponding copy of the DMP no longer needs to be stored. In this section, we extend the previous framework by providing a protocol for securely deleting transactions from DMPs.

We provide an overview of the DMP architecture and distributed hash table operations. We also introduce a new mechanism to reach a distributed consensus to prune transactions inside the DMP. In addition, we introduce different policies for resolving data inconsistencies inside the DMP, including:

Transactions can be marked as removed but still stored locally in the storage pool node;

The same transaction can be marked as removed in one storage pool node and still be available in another storage pool node;

A blockchain reorganization may require storage of transactions that were previously removed from the storage pool.

A single node can be viewed as a cluster of nodes providing a distributed storage pool (DMP). The proposed DMP relies on a distributed hash table structure deployed in a network consisting of individual trust relationships between honest nodes. The connected set of nodes is built on a collection of routing and application-level information. No central authority is involved in the issuance or storage of trust certificates: each node maintains its own record of trusted peers.

Malicious entities need to join the network to perform some form of attack. For example, Sybil Attacks focus on creating a large number of false identities to disrupt the system [John R. Douceur, The Sybil Attack, First International Workshop on Peer-to-Peer Systems, Springer-Verlag, London, UK, 2002]. Sybil nodes connected to the network may interrupt or delay legitimate routing queries and propagate incorrect routing information. The distributed hash table routing protocol described in this paper has sublinear time and space complexity and is based on the following assumptions:

Nodes cannot distinguish between honest and malicious nodes;

Most honest nodes have more connections with other honest nodes; and

Each node is responsible for storing information about key space partitions.

The distributed hash table protocol provides two main functions:

update() is used to build the routing table and insert the key in each distributed hash table node; and

A DHT node x uses GET(x, k) to look up the target key-value record represented by key k.

Each DHT node x is typically identified by a public key P _x and the current IP address addr x . This information is securely linked to the record sign _x (P _x , addr _x ), where sign _x () represents a signature with the corresponding private key. Node IDs are stored in the DHT using signed records. When a node changes location or receives a new IP address, a new record [P _x , addr _x ] must be stored in the DHT. Malicious nodes may insert incorrect key-value pairs. The GET method is responsible for verifying the signature in the returned key-value record.

The data routing network can be represented as an undirected graph. A malicious edge connects a malicious node to an honest node, while an honest edge connects two honest nodes. While creating an arbitrary number of Sybil identities is computationally affordable for a malicious entity, creating a malicious edge requires convincing an honest node to establish a trusted link to one of the Sybil-controlled identities. If there is no sparse cut that divides the honest region into two, a short random walk starting from an honest node may end at an honest node.

A node's routing table can be built by initiating independent random walks in the network. The node will collect one or more random key-value records from the receiving nodes on each walk. This routing information is used in the lookup process: if the local table does not contain the key in question, the lookup message is multicast to the set of nodes. One or more of the receiving nodes may have the queried key-value record stored locally. No direct requests to other nodes' tables are sent. Therefore, a malicious node cannot arbitrarily increase the number of incorrect entries in the remote table.

The keys are not stored in a shared space using deterministic functions (such as hashing) to prevent malicious nodes from brute-forcing the expected output and inserting a false key between two honest keys. Therefore, no function for measuring the distance between two keys will be defined. Each node x builds a long-distance routing table that contains pointers to other nodes whose IDs are evenly distributed in the key space. Pointers are stored by initiating random walks and collecting random IDs from the end nodes. The short-distance routing table contains key-value records. These keys are closest to the IDs of the nodes according to a predefined order.

The routing fabric is able to manage: (a) node failures, i.e., redundancy in the routing table provides alternative paths for retrieving key values; (b) key changes, i.e., adding or removing keys causes inconsistencies between key distribution and long-distance pointer distribution in the DHT space; and (c) changes in network connectivity, i.e., adding or removing trusted connections does not affect routing performance as long as no sparse cuts partition the honest network and no endpoints become malicious. Synchronization and availability of new keys is provided through periodic UPDATE calls.

The concept of trusted connections is critical to providing security and integrity in the DHT architecture. New edges can be created in the network using two different mechanisms.

Subjective ties are based on social or professional relationships between peers: for example, friendship, contracts, and trust with globally or locally known entities. For example, nonprofit organizations, research institutions, government entities, and private companies that sign a specific agreement can all be trusted relationships.

Transitive connections are based on logical properties and trust to increase the number of random paths on the network. The increased randomness in the hop-by-hop walks makes it harder for malicious nodes to exploit vulnerabilities in malicious edges. Signed messages can be exchanged between peers to learn about trusted connections between other nodes. Nodes can decide to assign different weights to their trusted connections in order to adjust the probability of selecting a single hop in a random walk, or to initialize new transitive connections. Each node is responsible for its own weight. Honest nodes will adjust the weight of their connections based on the routing behavior of their peers: if the key should fall into its short-distance routing table, the node may not be able to provide a valid key-value record. Therefore, new random walks can be organized through weak edges, and the influence of malicious nodes will decrease over time.

The DMP functionality is provided by a cluster of dedicated nodes. In the cluster, the validator nodes collect and verify new incoming transactions, while the storage pool nodes are responsible for persistent storage of verified transactions. Figure 11 shows the design of a storage pool cluster with validator nodes v and storage pool nodes s. The functionality of these two types of nodes can be provided by the same physical entity or embedded in different entities.

DMP provides two basic operations: storing and retrieving verified transactions. Given a transaction t _i , an identifier id(t _i ) can be assigned as id _i =H(t _i ), where H represents a hash function. The identifier will be used as a key in the DMP. Using a specific multicast address, transactions can be received from multiple validators in the cluster. Each validator will then verify the transaction independently. However, the underlying DHT structure does not embed transaction IDs in the key space to prevent key cluster attacks. Therefore, transactions can be stored in random nodes of the DMP. For a given transaction or set of transactions, the validator node can send an R storage (RSTORE) message request to a random storage pool node, where R is the data replication factor.

Assume that N validators will independently verify transaction t _i . Each validator v maintains a local database called V _DB . Each entry of the database contains an identifier of transaction id _i and a binary decision for verification. Each validator will send a STORE (t _i , id _i ) or REJECT (id _i ) message to the DMP based on the local decision.

Each storage pool node s will store _ti only if it receives N/2+1 STORE messages. Alternatively, once a node receives the first STORE message, it activates a timer to receive N/2 more STORE messages required to reach the quota. If it times out and does not receive enough confirmations, the transaction will be discarded. Each storage pool node _sm will also keep the number of received confirmations of _ti, N _i,m > N/2+1.

Retrieving a verified transaction _ti requires broadcasting the message QUERY(id _i ). The storage pool node responsible for storing _ti will send the message DATA(t _i ) to the query node. If M storage pool nodes are required to store _ti , the query node should receive at least M/2+1 data messages to consider the consistency of the stored transactions. In addition, the query node can query N/2+1 random validators with a VALIDITY REQ(t _i ) message to evaluate the validity of _ti based on the information stored in V _DB .

A scalable architecture for managing increasing transaction activity requires a storage infrastructure that operates with large amounts of data. For example, a transaction rate of 10K txn/s and an average transaction size of 1KB requires storing approximately 27TB of blocks per month.

In the M Network cluster, two different storage architectures (or their combination) are proposed. The first architecture is suitable for larger entities that own and maintain multiple verification nodes, all of which have access to centralized storage facilities. In contrast, the fully decentralized architecture is suitable for single nodes with sufficient storage capacity that wish to join a shared and distributed storage pool.

In the following, we use the following terms to represent transaction dependencies:

A transaction can spend the output of a previous transaction called the Parent transaction;

Transactions can create outputs for subsequent transactions called child transactions.

Transaction dependencies or chains of transactions can be created to represent complex transaction workflows that require children to be added before parents. When a transaction chain is transmitted over the network, the order in which data arrives may not match the order in which data is transmitted. In this case, the receiving node uses an OrphanTransaction Pool to store transactions that reference unknown parent transactions. Once the parent is known, any orphans that reference UTXO created by said parent are released from the Orphan Pool and recursively revalidated. The entire transaction chain is then reconstructed in the correct order and included in the transaction pool. Therefore, when a new transaction is added to the storage pool, it has no children in the storage pool because those children will be orphans.

To prevent Denial-of-Service attacks, the number of orphan transactions stored in memory is limited. If the number of orphan transactions in the pool exceeds the maximum allowed, randomly selected orphan transactions will be evicted from the pool until the pool size returns to within the limit.

The local storage pool contains a collection of transactions that is consistent with the node's view of the blockchain. Transactions can be removed from the local storage pool for one of the following reasons:

Transactions in mined blocks are published. When a new block is verified, its transactions are removed from the storage pool.

Blockchain reorganization (reorg). If a block is disconnected from the blockchain, its transactions are moved back to the storage pool.

Conflict with transactions in the block (Double Spending). When validating a new block, all transactions that conflict with transactions in that block and all their related relationships will be removed from the storage pool.

Additionally, depending on the configuration of the local storage pool, the following additional scenarios may be considered:

Manual pruning;

expiry; and

Local storage size limit.

When deleting a transaction and its children, the complete set of transactions must be computed before performing the actual removal, otherwise the storage may be left in an inconsistent state where parent transactions cannot be retrieved. All ancestors of a given transaction marked for removal need to be updated. Therefore, intermediate transactions in the chain cannot be removed before the state of all dependent transactions has been updated.

In case of a reorganization, new transactions may have pool child transactions, leading to inconsistent state, as there may be descendant transactions of the transaction from the disconnected block. In this case, the out-of-block descendants of the in-block transaction in the pool must be checked. The function UpdateForDescendants is used to update the descendants of a single transaction added to the pool and can have child transactions in the pool.

As mentioned before, the winning network node sends the block skeleton back to the validator node. The block skeleton consists of a random number, an invertible bloom filter lookup table (IBLT), and a coinbase transaction. Based on this, the validator node can:

1. Ordering transactions according to a specific set of rules;

2. Assemble newly mined blocks;

3. Continue to store the block in its own memory;

4. Propagate the skeleton to other new full nodes.

Therefore, the verification node is aware of the newly published transactions. Each verifier node can independently send a delete message containing a list of transaction IDs to the DMP. A verifier can only send delete requests for previously verified transactions (check_validator_send is enabled). This information is stored in the database V _DB (Validator Database). According to this model, storage pool nodes can discard delete requests received from verifiers who have not verified transactions (check_validator_rcv is enabled). This information is stored in the database S _DB (Storage Database). The two check_validator_send and check_validator_rcv options can be hard-coded or independently configured. The delete message is signed with the private key of the verifier. The message must contain a random number or a timestamp to avoid spreading fraudulent copies that may affect the consensus protocol.

Figure 12 shows a validator node v _n (0≤n<N) that receives updates about newly mined blocks. After validating a block, the validator sends a removal request for the published transaction to the DMP. Each storage pool node s _m (0≤m<M) tracks storage requests on the S _DB table and removal requests on the R _DB table. As shown in Figure 12, each storage pool node uses a new table R _DB (Removal Database) to track removal requests for each stored transaction. Given a removal request for transaction ti from a validator n<N of a storage pool node m<M, the local R _DB (m) table will contain a Boolean r _mn value. For example, if N = 100, and each storage pool node stores 10M transactions every 10 minutes for 3 days, the size of R _DB (m) can be limited to: 100·10 ⁷ ·3·24·6=432GB. Note that this number represents the extremely unlikely worst-case scenario, where all transactions stored locally in the past 3 days have not been confirmed and each table entry requires 1 byte. Table sizes at least one or two orders of magnitude smaller (e.g., around 4GB) would likely represent a more realistic situation. However, since storage pool nodes have the required high storage capacity, we do not impose any limits on the maximum size of local tables.

Let ^NMAX >N be the maximum number of validators that can send a store or delete message to the cluster. Depending on the chosen distributed architecture, ^NMAX may correspond to the total number of validators in the network or it may be proportional to the size of a particular DMP cluster. The size of a DMP cluster is given by the number of storage pool nodes and/or the total amount of transactions that can be stored locally.

We introduce some criteria to determine whether a transaction _ti stored in a local storage pool node _sm within the cluster should be deleted:

At least N*＜N ^MAX validators send deletion requests. Validators can actually promote N*, depending on the check_validator_send and check_validator_rcv settings. If one or both options are set, N*≈N (approximately) can be imposed. If no options are set, it is recommended to use N*＞N in order to achieve the highest possible consensus.

_ti depends on a previously removed transaction _tj . If both transactions are stored in _sm , and _tj is safely removed, then _ti can also be safely removed (if consensus is reached to remove the transaction, the dependent transaction must be considered invalid and marked as removed). No signaling is required to other storage pool nodes.

We do not force the DMP to store a chain of transactions in a single storage pool node to prevent specific denial of service attacks. Therefore, a storage pool node may be in an inconsistent state due to the state of individual transactions in the chain. As long as the nodes that need data related to the chain of transactions learn about the inconsistency by issuing specific query requests to the DMP, there will be no problem. For example, a node requests transaction _ti from the DMP. This transaction depends on a second transaction _tj . After querying the DMP for the second transaction, the node finds out that _tj is no longer stored. Therefore, the node is responsible for considering _ti unavailable even if it is still stored in some storage pool nodes of the DMP. Long-term inconsistencies will be resolved by permanent pruning.

If consensus is reached, transaction t _i will be locally marked as removed. However, for security reasons, the transaction may not be physically deleted. We introduce two criteria to determine when a transaction should be physically removed from s _m :

At least N**≥N* validators have sent pruning requests. The value of N** should be high enough to consider pruning consensus fully safe.

The amount of time ΔT _i > ΔT* has passed since the last deletion request was received, and no further data requests for t _i have been forwarded. The value of ΔT* should be high enough that the probability of a reorganization event is negligible. According to the 6-block confirmation rule, a transaction can be safely considered accepted by the network approximately 1 hour after it is published in a newly mined block.

However, depending on the storage limit of _sm , _ti can be removed even if the above criteria are not met. Transactions can be sorted in descending order _ΔTi and selected for permanent deletion. Transactions that belong to the chain and are stored in _sm must be deleted immediately. There is no need to notify validators about the pruning of the transaction chain. Nodes that need data related to the transaction chain learn about the inconsistency by issuing specific query requests to the DMP.

Storage pool nodes can collect data requests for transactions that have been marked as removed. Receiving multiple data requests for a single transaction can have different implications:

Denial of service attacks that retain useless data;

Blockchain reorganization and the need to transfer transactions back to the DMP.

The storage pool node does not take any action due to the data request, as the request may not be legitimate. The data request counter can be used to mark removed transactions as candidates for recovery. These candidates can still be considered for physical pruning. In the case of a priority policy for physical pruning, a lower priority can be assigned to candidate transactions for recovery.

Transactions marked as removed but still stored locally can only be recovered if they receive a recovery message from a validator. If a chain reorganization occurs, it is the responsibility of the validator to signal this to the DMP. Storage pool nodes need to collect a certain number of unique recovery messages to make transactions available again. We recommend a number of messages as a function of the N, N*, and/or N** parameters. For example, the minimum number of recovery messages could be equal to the average between N and N*.

Table 1 summarizes the configurable parameters used by the pruning protocol.

Table 2: Configurable parameters.

parameter describe N A set of validators responsible for verifying a single transaction. N ^MAX The full set of validators for the data cluster. N* The minimum number of validators required to trigger a proposed reversible pruning. N** The minimum number of validators required to trigger physical pruning of a transaction. M A collection of storage pool nodes in a data cluster. ΔT* The time window that triggers physical pruning of transactions.

The protocol governing transactions consists of the following messages.

Received(t _i ). The validator callback is triggered when a new unverified transaction t _i is received.

Store (t _i , id _i ). The verifier stores the request for a valid transaction t _i and key id _i .

Store (id _i ). The validator stores the optimization request for the valid transaction key id _i .

Query(id _i ). A request by a general node for a transaction with key id _i .

Data (t _i ). The storage pool node’s response to the query request for transaction t _i .

VALIDITY_REQ(t _i ). The general node requests to recheck the validity of transaction t _i .

VALIDITY_ACK (t _i ). The validator’s response to the validity request for transaction t _i .

The extensions shown here require the following additional messages.

Remove(id _i ). The validator requests to remove the transaction identified by id _i .

Restore(id _i ). The validator requests that the transaction identified by id _i be restored after a chain reorganization.

Removed(id _i ). The storage pool node's reply to a query request for the removed transaction identified by id _i . If the information is still available, this message may contain the number of removal messages received by the storage pool node for the transaction identified by id _i .

FIG13 shows a complete state diagram of a transaction identified by id _i based on a verifier's request. Depending on the type (i.e., store, remove, restore) and number (e.g., N, N*, N**) of messages received from the verifier, the transaction can change state to available, removed, or physically deleted. It is not possible to restore a transaction state of "physically deleted". However, data may still be recoverable based on the capabilities provided by the local file system and operating system.

It should be noted that the above embodiments illustrate rather than limit the present invention, and those skilled in the art will be able to design many alternative embodiments without departing from the scope of the present invention as defined by the appended claims. For example, it should be understood that users can instead use the methods and systems described herein to exchange other resources (e.g., information, contracts). The smart contract itself can be stored outside the blockchain or in one or more transactions.

In the claims, any reference signs in brackets shall not be construed as limiting the claims. The words "comprising" and "comprises" etc. do not exclude the presence of other elements and steps as a whole, even if these elements and steps are not listed in any claim or in the specification. In this specification, "comprising" means "includes or consists of" and "comprising" means "including or consists of". The singular reference of an element does not exclude the plural reference of these elements and vice versa. The present invention may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by the same component of hardware. The indisputable fact that certain methods are enumerated in mutually different dependent claims does not mean that the combination of these methods cannot achieve beneficial results.

References

An Integrated World.(n.d.).Retrieved fromhttps://www.anintegratedworld.com/whats-in-a-block/

David Eppstein,M.T.(2011).What's the Difference? EfficientSetReconciliation without Prior Context.ACM.

maidsafe.(n.d.).Retrieved from github.com:https://github.com/maidsafe/Whitepapers

Michael T.Goodrich,M.M.(2011).Invertible Bloom LookupTables.Communication,Control,and Computing(Allerton),2011 49th AnnualAllertonConference on.

NebulousLabs.(n.d.).Retrieved from github.com:https://github.com/NebulousLabs/Sia

O(1)Block Propagation.(n.d.).Retrieved from github.com:https://gist.github.com/gavinandresen/e20c3b5a1d4b97f79ac2

Wikipedia.(n.d.).Retrieved fromhttps://en.wikipedia.org/wiki/Distributed_hash_table

Wilkinson et al.(2016,December 15).Retrieved fromhttps://storj.io/storj.pdf

John R.Douceur.The Sybil Attack.First International Workshop onPeer-to-Peer Systems.Springer-Verlag,London,UK,2002

Claims (28)

1. A computer-implemented method for a node of a blockchain network, the computer-implemented method comprising:

Storing a plurality of transactions forming at least a portion of a distributed storage pool of transactions waiting to be mined into a block of a blockchain; and

Receiving a delete request from a validation node of the blockchain network, the delete request identifying one or more transactions already included in a newly mined block, the delete request indicating that the one or more transactions should be deleted from the distributed storage pool;

The method further includes marking the transaction as removed from the distributed storage pool when a first threshold number of delete requests for the transaction are received;

The method further includes physically removing the transaction from the distributed storage pool when a second threshold number of delete requests for the transaction are received;

Wherein the first and second threshold number of delete requests require a threshold number of different authentication nodes from the blockchain network;

wherein the second threshold is greater than the first threshold.

2. The computer-implemented method of claim 1, wherein the transaction is physically removed from the distributed storage pool only after a threshold time has elapsed since the deletion request was received, during which no further data requests for the transaction were received.

3. The computer-implemented method of claim 2, wherein the threshold time corresponds to at least 1, 2, 3, 4, 5, 6, or more blocks incorporated into the blockchain after the transaction is incorporated into the blockchain, the blockchain being a blockchain that requests deletion from the distributed storage pool.

4. The computer-implemented method of claim 1, wherein the transaction is physically removed from the distributed storage pool in descending chronological order since the deletion request was received, during which no further data requests for the transaction were received.

5. The computer-implemented method of claim 4, further comprising:

When the transaction depends on a previously removed transaction, the transaction is marked for removal from the distributed storage pool.

6. The computer-implemented method of claim 1, wherein the delete request is stored in a database.

7. The computer-implemented method of claim 1, wherein the delete request received from a verifier, the verifier not verifying the transaction, is discarded.

8. The computer-implemented method of claim 7, wherein the transaction is discarded only when a check verifier option is enabled.

9. The computer-implemented method of claim 1, further comprising:

The number of data requests that have been marked as transactions removed from the distributed storage pool is monitored.

10. The computer-implemented method of claim 1, further comprising:

for the transaction marked for removal from the distributed storage pool, after a threshold number of data requests for the transaction are received, the transaction is marked as a candidate for reverting to the distributed storage pool.

11. The computer-implemented method of claim 1, further comprising:

When a request to resume the transaction is received, the transaction is unmarked, thereby indicating removal from the distributed storage pool.

12. The computer-implemented method of claim 1, further comprising:

When a threshold number of resume requests for the transaction are received, the transaction is unmarked, thereby indicating removal from the distributed storage pool.

13. The computer-implemented method of claim 12, wherein the threshold number of recovery requests are from a threshold number of different authentication nodes in the blockchain network.

14. The computer-implemented method of claim 1, further comprising:

In response to a query for a transaction that has been removed from the distributed storage pool, a message is sent indicating that the transaction has been removed from the distributed storage pool.

15. The computer-implemented method of claim 14, wherein the message indicating that the transaction has been removed from the distributed storage pool further indicates: for the removed message, the number of delete messages received.

16. The computer-implemented method of claim 1, wherein the transaction comprises three states with respect to the distributed storage pool: is available; removed, and physically removed.

17. A computer-implemented method for a node of a blockchain network, the computer-implemented method comprising:

Receiving data regarding a newly mined tile, the newly mined tile including a plurality of transactions; and

A deletion request is sent to a distributed storage pool to delete the plurality of transactions from the distributed storage pool according to the method of any one of claims 1 to 16.

18. The computer-implemented method of claim 17, wherein the delete request includes a random number and/or a timestamp.

19. The computer-implemented method of claim 17 or 18, wherein the delete request is signed using a private key of the node.

20. The computer-implemented method of claim 17, wherein the node stores a record of transactions it has verified and only sends delete requests for transactions it has previously verified.

21. The computer-implemented method of claim 20, wherein a delete request is sent for a transaction that the node has previously authenticated only when a check verifier option is enabled.

22. The computer-implemented method of claim 17, further comprising:

a resume request is sent to make the transaction available again in the distributed storage pool.

23. The computer-implemented method of claim 22, wherein the resume request is sent after a blockchain reorganization.

24. The computer-implemented method of claim 17, wherein the delete request is activated by one or more of:

receiving the mined blocks;

Block chain reorganization;

dual spending or other forms of transaction conflict.

25. The computer-implemented method of claim 17, wherein transaction pruning from the distributed storage pool is activated by one or more of:

manually trimming;

Expiration of the transaction;

Transactions in the distributed storage pool reach a storage limit.

26. A computer-readable storage medium comprising computer-executable instructions that, when executed, cause one or more processors to perform the method of any of claims 1-25.

27. An electronic device, comprising:

An interface device;

one or more processors coupled to the interface device;

A memory connected to the one or more processors, the memory storing computer-executable instructions that, when executed, cause the one or more processors to perform the method of any of claims 1-25.

28. A node of a blockchain network for performing the method of any of claims 1 to 25.