CN114489809A - High-throughput many-core data stream processor and its task execution method - Google Patents

Abstract

The present invention provides a high-throughput many-core data stream processor, comprising: a plurality of processing units, which are connected to each other to form an on-chip network structure of the processor; each of the processing units includes a plurality of sub-processing units, and the sub-processing units include Instruction sub-memory and data sub-memory, a plurality of the sub-processing units are arranged in an array structure, and are connected in communication with each other to form a multi-hop network structure of the processing units; a configuration unit is connected in communication with each of the sub-processing units. And a task execution method of the high-throughput many-core data stream processor. Compared with the prior art, the invention has better scalability, simple control logic, and is suitable for a large-scale many-core structure. At the same time, it supports SIMD-MIMD-Systolic mode configuration, scale configuration, regional configuration and other advantages, which is more flexible and suitable for processing in more general application fields.

Description

High-throughput many-core data stream processor and its task execution method

technical field

The invention relates to the field of processor architecture design, in particular to a many-core processor architecture based on a data flow execution mode.

Background technique

A single instruction stream, multiple data stream machine (SIMD) uses one instruction stream to process multiple data streams. Such machines are very effective in the fields of digital signal processing, image processing, and multimedia information processing. The MMXTM, SSE (Streaming SIMD Extensions), SSE2 and SSE3 extended instruction sets implemented by Intel processors can process multiple data units in a single clock cycle. That is to say, the single-core computers used today are basically SIMD machines. Multiple Instruction Streams Multiple Data Streams Machines (MIMD) machines can execute multiple instruction streams simultaneously, each of which operates on different data streams. The latest multi-core computing platforms belong to the category of MIMD. For example, Intel and AMD's multi-core and many-core processors belong to MIMD. In computer architecture, there are two implementation paths for data parallelism: MIMD and SIMD. Among them, MIMD's main manifestations are multi-launch, multi-threading, and multi-core, which can be seen in contemporary designed processors driven by processing power. At the same time, with the rise of multimedia, big data, artificial intelligence and other applications, it has become more and more important to give SIMD processing capabilities to processors, because these applications have a large number of fine-grained, homogeneous, independent data operations, and SIMD is naturally suitable for handle these operations.

Due to the rapid development and iteration of application algorithms and different data processing requirements, in some common applications, some codes are suitable for SIMD structures, and some codes perform better in MIMD. In order to improve the generality of processor chips, some invention patents and papers have proposed configurable implementation methods of SIMD and MIMD. For example, for multi-core structure and many-core structure, multi-core and many-core can be dynamically configured according to the characteristics of the application algorithm. How SIMD and MIMD are implemented. And by controlling the existing SIMD processing structure, the MIMD calculation mode is realized and so on.

However, there are the following problems in the prior art: First, the SIMD-MIMD structure design of large-scale many-core arrays with data flow-oriented execution characteristics is not considered; Poor scalability, when the array scale is large, the control structure is complex; third, the flexibility is poor, the dynamic structure configuration matching of the computing characteristics of the data flow graph nodes is not considered, such as the demand for different granularity SIMD, etc., for the data flow many-core processing There is no effective SIMD-MIMD structure design; fourth, the SIMD-MIMD structure cannot be dynamically adjusted according to the computing characteristics of the data flow graph nodes. Finally, there is no way to dynamically configure into Systolic mode.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention proposes a network-on-chip architecture of a data stream many-core processor, comprising: a plurality of processing units, which are connected in communication with each other to form a network-on-chip structure of the processor; each of the processing units includes a plurality of sub-processing units, The sub-processing unit includes an instruction sub-memory and a data sub-memory, a plurality of the sub-processing units are arranged in an array structure, and are communicatively connected to each other to form a multi-hop network structure of the processing unit; a configuration unit is communicatively connected to each of the sub-processing units .

In the processor of the present invention, when executing a SIMD task of a single instruction stream and multiple data streams, the configuration unit divides the processing unit into a SIMD task group according to task requirements, and sends the instruction code to the processing unit of the SIMD task group. sub-processing unit to perform the SIMD task. The configuration information sent by the configuration unit is received by the processing unit, and all sub-processing units in the processing unit and sub-routers of the sub-processing units are configured by the processing unit according to the configuration information. A sub-processing unit of the processing unit is designated as a data transmission unit, and the data transmission unit loads or stores data for all sub-processing units of the processing unit through memory access commands. The SIMD task group designates a sub-processing unit as the main control unit of the SIMD task group. When a memory request and/or data transmission is performed between the two SIMD task groups, the interactive data is transmitted through the main control unit of each SIMD task group. . The configuration unit includes a shuffle register. When a shuffle operation is performed on the SIMD task group, data exchange is completed by reading the vector data of the shuffle instruction into the shuffle register and writing it into the target sPE.

In the processor of the present invention, when executing MIMD tasks with multiple instruction streams and multiple data streams, the configuration unit configures each sub-processing unit to run independently, and each sub-processing unit receives its own control commands, instructions and operating data, and Process a standalone DFG or part of a DFG. The sub-processing units communicate only through the sub-routers of the sub-processing units.

In the processor of the present invention, when performing a systolic array task, the configuration unit configures the sub-processing units of the plurality of processing units to construct a systolic array, and configures the sub-processing of the last row or last column of the data flow direction of the systolic array The unit is an accumulator.

The present invention also provides a task execution method for executing a data flow program task on the aforementioned many-core processor. The task execution method includes: receiving and compiling a user program, and dividing the user program into program blocks , generate a data flow diagram; judge the execution mode of the program block, mark the program block as single instruction stream multi-data stream mode, multi-instruction stream multi-data stream mode or systolic array mode; configure the execution unit for the program block, The program block is distributed to the configured execution unit; the configured execution unit is set to the task execution mode corresponding to the program block; the program block is executed by the configured execution unit.

Description of drawings

FIG. 1 is a schematic diagram of the architecture of a data stream many-core processor of the present invention.

FIG. 2 is a layout and wiring diagram of an sPE inside a single PE of the present invention.

FIG. 3 is a schematic diagram of domain division of different SIMD configurations of the present invention.

FIG. 4 is a schematic diagram of the sPE data transmission path of the present invention.

FIG. 5 is a schematic diagram of the configuration and instruction path of the sPE of the present invention.

FIG. 6 is a flow chart of the execution of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific implementation methods described herein are only used to explain the present invention, but not to limit the present invention.

The purpose of the present invention is to solve (overcome) the problems (defects) of the above-mentioned prior art such as poor scalability of SIMD-MIMD, impractical data stream many-core architecture, etc., and proposes a highly flexible and configurable SIMD-MIMD-Systolic configuration method and system.

The data stream many-core processor architecture of the present invention can flexibly configure SIMD-MIMD width, scale, area, etc. according to the structure design of the data stream many-core processor, realizes a flexible and configurable data stream many-core architecture according to software commands, and supports Parameters such as SIMD-MIMD can be configured, which improves the execution effect of different application types, thereby improving the energy efficiency ratio; supports the configuration of the systolic array mode of the systolic array. For application types such as matrix multiplication, the execution efficiency of the systolic array is higher; The design and configuration method of the extended data network, instruction network and configuration network can support large-scale structure expansion, simple control structure, and low hardware overhead; support scalable SIMD-scale Shuffle operation, which has high performance for matrix operations execution efficiency.

While conducting research on high-throughput general-purpose data stream many-core processors, the inventor found that the reason for the low processing energy efficiency (performance/power consumption) in the prior art is the underutilization of hardware resources, or the computing features provided by hardware resources. Due to the mismatch with the computing features required by the application, because it is oriented to general data processing, the types of applications encountered will have a variety of data volume, data computing mode and other characteristics. The study of architectural features found that the solution to this defect can be achieved by means of variable SIMD width, variable SIMD computing unit scale, variable MIMD scale, and even systolic arrays. There are already some designs based on the SIMD-MIMD configurable way. Based on this idea, the present invention further explores a design that is more flexible, efficient, highly scalable, and suitable for a data stream many-core processor structure.

In order to support multiple execution modes, the present invention proposes a configurable NoC (Network-on-Chip, Network-on-Chip) architecture. FIG. 1 is a schematic diagram of the architecture of a data stream many-core processor of the present invention. Figure 1 shows the architecture design of the data stream many-core processor, including: DMA, on-chip storage, configuration unit and PE processing unit. sPE), each sPE can perform tasks independently, and has complete instruction execution functions such as fetching, decoding, transmitting, executing, and submitting.

SIMD mode: As shown in Figure 2, when configured in SIMD mode, the star configuration NoC is activated. All sPEs are controlled synchronously by a SIMD control unit, which behaves as a decoupled SIMD architecture. As shown in Figure 2, four data paths are used to transmit data signals, feedback signals, control signals and command signals respectively. Different from the traditional SIMD model, in the data flow many-core processor architecture of the present invention, there are N sPEs in one PE, or the width of SIMD is N, and each sPE has independent instruction memory and data memory. When configured in the selected SIMD mode, the PE is controlled by the SIMD controller of the configuration unit. The instruction code is loaded through the router in the SIMD controller and distributed to each sPE belonging to the specified SIMD group. The execution of each sPE is simultaneously controlled by the SIMD controller. In order to improve the scalability of SIMD width, or the size of sPE in one SIMD group, the present invention attempts to decouple the centralized control mechanism, and the memory request and data transfer between the two SIMD groups are controlled by the designated master SPE in each SIMD group respectively. issue. In other words, for a SIMD group, the data of each sPE is treated as a whole to prevent data movement from being out of sync. Shuffle is a common but special operation in SIMD structures that requires special consideration. Shuffle operations are supported by data exchange between two sPEs, which should use a mesh network to transmit shuffled data. However, if the degree of shuffle is high, it will take a lot of time to exchange data between sPEs. In order to speed up the shuffle operation, the present invention supports shuffle by adding control logic and several registers in the SIMD controller unit. The shuffle instruction controls data exchange by reading vector data into registers in the SIMD controller unit and writing them to the target sPE. In order to reduce hardware resources, the present invention multiplexes the instruction channel of shuffle data.

FIG. 2 is a layout and wiring diagram of an sPE inside a single PE of the present invention. As shown in Figure 2, the data signal line constitutes the Mesh on-chip network structure, the feedback signal line is the ack feedback signal, which is used to control the execution of the data flow graph, the control signal line is the control signal, the command signal line and the command transmission line are composed of SIMD The control unit performs unified control and implementation, completes the configuration of hardware resources and sends the instructions to be executed to each sPE, and receives them through the router sR of each sPE.

MIMD Mode: MIMD Mode is designed to handle irregular data or programs that cannot take advantage of SIMD computing modes. Therefore, for these applications that are not suitable for SIMD computing, pure MIMD mode can be configured, i.e. many-core mode for typical 2D topologies. In MIMD mode, all sPEs are configured as independent cores, run independently, and are not subject to SIMD constraints. As shown in Figure 4, after the SIMD controller is configured, the sPEs communicate only through the sub-router sR. Each sPE has its own control commands, instructions and data, and can process an independent DFG (DataFlow Graph) or a part of a DFG.

SIMD-MIMD Hybrid Mode: Some applications are neither suitable for pure MIMD mode nor high-width SIMD mode, they prefer low-width SIMD or even variable SIMD width during execution for higher energy efficiency. For other applications, although higher SIMD width can achieve higher performance, it also brings higher power consumption, which may not meet some low-power scenarios. In order to adapt to multi-domain scenarios, a hybrid mode is implemented to provide possible choices. Users can choose different SIMD widths and different MIMD ratios according to different needs. For example, the input data may not be large enough to fully utilize all SIMD computing units, or for low power consumption, the user may sacrifice performance in pursuit of higher efficiency, as the wider SIMD may consume higher power, while Does not increase performance proportionally. For larger-scale PE arrays, flexible configuration strategies are provided, which can effectively improve the energy efficiency of local PEs or the entire PE array.

SystolicArray Systolic Array Mode: For some applications, such as matrix multiplication, systolic array is undoubtedly one of the most efficient architectures and has been proven by Google TPU. From another perspective, systolic arrays are a special case of the data flow model. Data flows into the calculation array from the top and left SPMs, in which case the bottom sPE of the bottom row of PEs can be configured as an accumulator. Thus, the hardware architecture possessed by the systolic array is realized.

FIG. 3 is a schematic diagram of domain division of different SIMD configurations of the present invention. As shown in Figure 3, the present invention takes a local 4*4PE array as an example, and the NoC can be configured into different SIMD modes, such as: SIMD-1(①), SIMD-2(②), SIMD-4(③), SIMD -8(④) and SIMD-16(⑤). Figures 4 and 5 describe the physical connections between routers and channels. In order to explain the structure more clearly, the present invention logically divides it into five optional modes:

·SIMD-1 (MIMD) mode, the processor runs under the first-level SIMD controller, forming a typical 4*42DMeshNoC. Each sub-router serves one sPE, and each sPE operates independently. Sub-routers from 1 to 16 are all connected to the aa channel in both X and Y directions, as shown in Figure 4.

·SIMD-2 mode, every two SPEs are used as a SIMD-2PE on the second layer. The Level-2 SIMD controller controls the two designated sPEs to execute the same instruction synchronously. Configuration commands, instructions and data are managed by the Level-2SIMD controller. Since the present invention combines two sPEs of the x-axis, a new channel should be added in addition to the shared existing channel to ensure that both sPEs can receive data from the x-axis at the same time. E.g. Sub-router 2 is connected to channel a, and sub-router 1 is connected to channel b in the X direction. In the Y direction, they are both connected to the a channel. Therefore, 1 and 2 are combined as SIMD-2PE.

· SIMD-4 mode, a SIMD controller contains two secondary controllers, four SPEs. The execution behavior is the same as level 3. It should be noted that the present invention adds a corresponding data path on the y-axis, that is, 2 and 6 are configured to connect the a channel, and 1 and 5 are connected to the b channel in the X direction. In the Y direction, 1 and 2 are connected to a channel, and 5 and 6 are connected to b channel.

·SIMD-8 mode, a 4-level controller manages 8 SPEs simultaneously. The present invention adds data paths on the y-axis to support SIMD-8 mode based on SIMD-4 mode, which means that in the X direction, 1, 2, 3, 4 are connected to a, b, c, and d channels, respectively. 5, 6, 7, 8 are the same. In the Y direction, 1, 2, 3, and 4 are connected to a channel, and 5, 6, 7, and 8 are connected to b channel.

·SIMD-16 mode, the last level SIMD controller controls all 16 sub-routers. As mentioned above, in the X direction, the four sub-routers of each row are connected to four different channels. In the Y direction, the four sub-routers of each column are also connected to four different channels.

Figure 5 shows the configuration path of the PE. For the sub-router sR (sub Router), which channel should be connected is configured by the config controller. In Figure 5, each layer logically has a SIMD controller. However, the area and power cost of the SIMD controller increases as the SIMD width increases. In order to reduce design complexity and improve scalability, the present invention does not set different layer control logics for various SIMD modes. A unified controller is implemented in the PE, which receives the configuration information from the configuration unit, and configures all sub-routers and sPEs, as shown in Figure 5. After the SIMD mode is set, the first sPE among all the sPEs in the SIMD mode is configured as the master and child PE, responsible for memory access and data exchange with other PEs. For example, when configured in SIMD-32 mode, the first child PE will simultaneously issue memory access commands to load or store data for all 32 sPEs. Since all sPEs have the same structure and data arrive at the same time, the sPEs execute instructions in a SIMD fashion.

The specific flow chart and configuration process of the processor of the present invention executing the user program are shown in FIG. 6 . On the basis of compiler compilation and analysis, the user program forms configuration files, instruction codes and data segments for the hardware structure. Through the mapping of the data flow graph, the configuration files, instructions, etc. are sent to each PE and sPE, and the hardware is configured and implement. Specifically include:

Step S01, receiving a user program;

Step S02, the compiler compiles the program;

Step S03, analyzing program application features, forming a coarse-grained program block (block) data flow diagram;

Step S04, judge whether the program block is executed in SIMD mode;

Step S05, if the program block is executed in the SIMD mode, then the program block is marked as the SIMD mode;

Step S06, if the program block is not executed in SIMD mode, then judge whether the program block is executed in Systolic mode;

Step S07, if the program block is not executed in Systolic mode, then the program block is marked as MIMD mode;

Step S08, if the program block is executed in the Systolic mode, the program block is marked as the Systolic mode;

Step S09, the program block controller distributes the program block, and configures the processing unit to be SIMD, MIMD or Systolic mode according to the pre-order judgment step;

Step S11, dynamically schedule blocks to be executed on the processing unit according to the current processing unit configuration state table and the dependencies between the blocks;

Step S12, the processing unit executes the user program in SIMD mode, MIMD mode or Systolic mode according to the allocated block configuration.

Compared with the prior art, the invention has better scalability, simple control logic, and is suitable for a large-scale many-core structure. At the same time, it supports the advantages of SIMD-MIMD-Systolic mode configuration, scale configuration, and regional configuration. It is more flexible and suitable for processing in more general application fields.

The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all Equivalent technical solutions also belong to the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims (10)

1. a high-throughput many-core data stream processor, is characterized in that, comprises: a plurality of processing units, connected in communication with each other to form an on-chip network structure of the processor; Each of the processing units includes a plurality of sub-processing units, the sub-processing units include an instruction sub-memory and a data sub-memory, a plurality of the sub-processing units are arranged in an array structure, and are connected in communication with each other to form a multi-hop network structure of the processing unit; a configuration unit in communication with each of the sub-processing units. 2. The high-throughput many-core data stream processor as claimed in claim 1, wherein when the processor executes the SIMD task of single instruction stream and multiple data streams, the configuration unit divides the processing unit according to task requirements The SIMD task group, and the instruction code is sent to the sub-processing unit of the processing unit of the SIMD task group to execute the SIMD task. 3. The high-throughput many-core data stream processor as claimed in claim 2, wherein the configuration information sent by the configuration unit is received by the processing unit, and the configuration information sent by the configuration unit is received by the processing unit according to the configuration information. All sub-processing units and sub-routers of sub-processing units are configured. 4. The high-throughput many-core data stream processor as claimed in claim 3, wherein a sub-processing unit of the designated processing unit is a data transmission unit, and the data transmission unit has all of the processing units through memory access commands The sub-processing unit performs the loading or storing of data. 5. high-throughput many-core data stream processor as claimed in claim 2, is characterized in that, this SIMD task group designates a sub-processing unit as the main control unit of this SIMD task group, when between two SIMD task groups During memory request and/or data transmission, the interaction data is transmitted through the main control unit of each SIMD task group. 6. The high-throughput many-core data stream processor of claim 2, wherein the configuration unit comprises a shuffle register, and when the SIMD task group is subjected to a shuffle operation, the vector data of the shuffle instruction is read by to this shuffle register and write to the target sPE to complete the data exchange. 7. The high-throughput many-core data stream processor of claim 1, wherein when the processor executes the MIMD task of multiple instruction streams and multiple data streams, the configuration unit configures each of the sub-processing units to be independent In operation, each sub-processing unit receives its own control commands, instructions and operation data, and processes an independent DFG or a part of a DFG. 8 . The high-throughput many-core data stream processor of claim 7 , wherein each sub-processing unit communicates only through a sub-router of the sub-processing unit. 9 . 9. The high-throughput many-core data stream processor of claim 1, wherein when the processor performs a systolic array task, the configuration unit configures the sub-processing units of the plurality of processing units to construct a systolic array, And configure the sub-processing unit in the last row or the last column of the data flow direction of the systolic array as an accumulator. 10. A task execution method for executing a data flow program task on the high-throughput many-core data flow processor as claimed in any one of claims 1-9, wherein the task execution method comprises: Receive the user program and compile it, divide the user program into program blocks, and generate a data flow diagram; Determine the execution mode of the program block, and mark the program block as a single instruction stream multi-data stream mode, a multi-instruction stream multi-data stream mode or a systolic array mode; Configure an execution unit for the program block, and distribute the program block to the configured execution units; Set the configured execution unit to the task execution mode corresponding to the program block; The program block is executed by the configured execution unit.

Images