CN111966402A - Instruction processing method and device and related product - Google Patents

Abstract

The present disclosure relates to a tensor instruction processing method, device and related products. The machine learning device includes one or more instruction processing devices, which are used to obtain tensors to be operated and control information from other processing devices, perform specified machine learning operations, and transmit the execution results to other processing devices through the I/O interface; When the machine learning computing device includes multiple instruction processing devices, the multiple instruction processing devices can be connected through a specific structure to transmit data. Among them, multiple instruction processing devices are interconnected and transmit data through the fast peripheral device interconnection bus PCIE bus; multiple instruction processing devices share the same control system or have their own control systems, and share memory or have their own memory; multiple instructions The interconnection of the processing devices is an arbitrary interconnection topology. The tensor instruction processing method, device and related products provided by the embodiments of the present disclosure have a wide range of applications, and have high processing efficiency and high processing speed for instructions.

Description

Instruction processing method, device and related products

technical field

The present disclosure relates to the field of computer technology, and in particular, to a tensor instruction processing method, device and related products.

Background technique

With the continuous development of science and technology, the use of machine learning, especially neural network algorithms, is becoming more and more extensive. It has been well used in image recognition, speech recognition, natural language processing and other fields. However, due to the increasing complexity of neural network algorithms, the types and quantities of data operations involved continue to increase. In the related art, the efficiency and speed of performing tensor correlation operations on tensor data are low.

SUMMARY OF THE INVENTION

In view of this, the present disclosure proposes a tensor instruction processing method, device, and related products, so as to improve the efficiency and speed of performing tensor-related operations on tensor data.

According to a first aspect of the present disclosure, there is provided a tensor instruction processing apparatus, the apparatus comprising:

The control module is used to parse the acquired tensor instruction, obtain the operation code and operation field of the tensor instruction, and obtain the to-be-operated tensor required to execute the tensor instruction according to the operation code and the operation field , the scalar to be operated and the target address;

an operation module, used for tensors to perform a tensor-to-scalar multiplication operation on the to-be-operated tensor and the to-be-operated scalar, obtain an operation result, and store the operation result in the target address;

The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address.

According to a second aspect of the present disclosure, there is provided a machine learning computing device, the device comprising:

One or more tensor instruction processing devices described in the first aspect above are used to obtain tensors to be operated and control information from other processing devices, perform specified machine learning operations, and transmit the execution results through the I/O interface to other processing devices;

When the machine learning computing device includes a plurality of the tensor instruction processing devices, the plurality of the tensor instruction processing devices can be connected through a specific structure and data can be transmitted;

Wherein, a plurality of the tensor instruction processing devices are interconnected and transmit data through the fast peripheral device interconnection bus PCIE bus to support larger-scale machine learning operations; a plurality of the tensor instruction processing devices share the same control system Or have their own control systems; a plurality of the tensor instruction processing devices share memory or have their own memory; the interconnection mode of the multiple tensor instruction processing devices is any interconnection topology.

According to a third aspect of the present disclosure, there is provided a combined processing device, the device comprising:

The machine learning computing device, universal interconnection interface, and other processing devices described in the second aspect above;

The machine learning computing device interacts with the other processing devices to jointly complete the computing operation specified by the user.

According to a fourth aspect of the present disclosure, there is provided a machine learning chip, where the machine learning chip includes the machine learning network computing device described in the second aspect or the combined processing device described in the third aspect.

According to a fifth aspect of the present disclosure, a machine learning chip packaging structure is provided, and the machine learning chip packaging structure includes the machine learning chip described in the fourth aspect.

According to a sixth aspect of the present disclosure, there is provided a board card including the machine learning chip packaging structure described in the fifth aspect.

According to a seventh aspect of the present disclosure, an electronic device is provided, and the electronic device includes the machine learning chip described in the fourth aspect or the board card described in the sixth aspect.

According to an eighth aspect of the present disclosure, there is provided a tensor instruction processing method, the method being applied to a tensor instruction processing apparatus, and the method includes:

Analyze the acquired tensor instruction to obtain the operation code and operation field of the tensor instruction, and obtain the tensor to be operated, the scalar to be operated and the target required to execute the tensor instruction according to the operation code and the operation field address;

Multiplying the tensor and the scalar to be calculated by the tensor and the scalar to obtain an operation result, and storing the operation result in the target address;

The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes a source address and a destination address of the tensor to be operated.

According to a ninth aspect of the present disclosure, a computer-readable storage medium is provided, and a computer program is stored in the storage medium. When the computer program is executed by one or more processors, the steps of the above-mentioned tensor instruction processing method are implemented. .

In some embodiments, the electronic device includes a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a server, a cloud server, a camera, Cameras, projectors, watches, headphones, mobile storage, wearables, vehicles, home appliances, and/or medical equipment.

In some embodiments, the vehicles include airplanes, ships and/or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical Equipment includes MRI machines, ultrasound machines and/or electrocardiographs.

The tensor instruction processing method, device, and related products provided by the embodiments of the present disclosure include a control module and an operation module, and the control module is used to parse the acquired tensor instruction to obtain the operation code and operation of the tensor instruction. field, and obtain the tensor to be operated and the target address required to execute the tensor instruction according to the opcode and operation field; the operation module is used to multiply the tensor to be operated by a tensor and a scalar to obtain the operation result, and use The result of the operation is stored in the target address. The tensor instruction processing method, device and related products provided by the embodiments of the present disclosure have a wide range of applications, and have high processing efficiency and fast processing speed for tensor instructions.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure, and together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure.

2a-2f show block diagrams of a tensor instruction processing apparatus according to an embodiment of the present disclosure.

FIG. 3 shows a schematic diagram of an application scenario of a tensor instruction processing apparatus according to an embodiment of the present disclosure.

4a and 4b illustrate block diagrams of a combined processing apparatus according to an embodiment of the present disclosure.

FIG. 5 shows a schematic structural diagram of a board according to an embodiment of the present disclosure.

FIG. 6 shows a flowchart of a tensor instruction processing method according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following detailed description. It will be understood by those skilled in the art that the present disclosure may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

FIG. 1 shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. As shown in FIG. 1 , the device includes a control module 11 and an arithmetic module 12 . Among them, the control module 11 is used to analyze the acquired tensor instruction, obtain the operation code and operation field of the tensor instruction, and obtain the tensors to be operated and the operation fields required to execute the tensor instruction according to the operation code and operation field. scalar and destination address tensor tensor. The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address and destination address of the tensor to be operated. The operation module 12 is used for multiplying the tensor to be operated by a tensor and a scalar to obtain an operation result, and store the operation result in the target address. Specifically, the operation module 12 can multiply each element in the tensor to be calculated by the scalar to be calculated to obtain an operation result, and store the operation result in the target address. Optionally, the above-mentioned scalar to be calculated can be an immediate number or scalar register.

Optionally, the target address may be a start address, and the processing module may determine the size of the storage space required for the operation result according to the start address and the size of the operation result, and store the operation result in the determined storage space.

In this embodiment, the control module may obtain the to-be-operated tensors respectively from the addresses of the to-be-operated tensors. The control module can obtain instructions and data through a data input and output unit, and the data input and output unit can be one or more data I/O interfaces or I/O pins. The operation module is used to perform tensor operation on the tensor to be operated according to the tensor operation type, obtain the operation result, and store the operation result in the target address. The tensor instruction processing device provided by the embodiment of the present disclosure has a wide range of applications, and has high processing efficiency and fast processing speed for tensor instructions.

In this embodiment, the operation code may be the part of the instruction or field (usually represented by code) specified in the computer program to perform the operation, and the instruction sequence number, which is used to inform the device that executes the instruction which instruction needs to be executed. . The operation domain can be the source of all data required to execute the corresponding instruction. All the data required to execute the corresponding instruction includes parameters such as the tensor to be operated and the corresponding tensor instruction processing method, or the storage operation tensor, tensor operation Parameters such as type and the address of the corresponding tensor instruction processing method, etc. For a tensor instruction, it must include an opcode and an operation field, where the operation field at least includes the address of the tensor to be operated and the target address. It should be understood that those skilled in the art can set the instruction format of the tensor instruction and the included operation codes and operation fields as required, which is not limited in the present disclosure.

Optionally, there may be one or more source addresses and target addresses of the tensors to be operated in the operation domain, and each source address corresponds to a target address. The number of tensors to be operated on can be one or more. That is to say, the operation module 12 of the present application can simultaneously implement the operation of multiplying one or more tensors to be calculated and the same scalar to be calculated. Further optionally, the number of the scalars to be calculated may also be one or more, each tensor to be calculated is correspondingly set with a scalar to be calculated, and the values of the multiple scalars to be calculated may be the same or different. At this time, the operation module 12 of the present application can simultaneously realize the operation of multiplying one or more tensors to be operated and corresponding scalars to be operated.

Further optionally, the device may include one or more control modules and one or more operation modules, and the number of the control modules and the operation modules may be set according to actual needs, which is not limited in the present disclosure.

Fig. 2a shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. In a possible implementation manner, as shown in FIG. 2a, the operation module 12 may include at least one tensor operator 120, and the at least one tensor operator 120 is configured to perform a tensor operation corresponding to a tensor operation type. Specifically, the tensor operator is configured to multiply each element of the to-be-operated tensor by the to-be-operated scalar to obtain an operation result, so as to realize the multiplication operation of the tensor and the scalar.

Further, the operation module may further include a data access circuit, the data access circuit may obtain the data to be operated from the storage module, and the data access circuit may also store the operation result in the storage module. Optionally, the data access circuit may be a direct memory access module.

In this implementation manner, the tensor operator may include operators such as adders, dividers, multipliers, and comparators that can perform arithmetic operations, logical operations, and other operations on tensors. The tensor operator can set the type and quantity of the tensor operator according to the size of the data volume of the tensor operation to be performed, the type of tensor operation, the processing speed and efficiency of the tensor operation, etc. This is not limited.

FIG. 2b shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. In a possible implementation manner, as shown in FIG. 2 b , the operation module 12 may include a master operation sub-module 121 and a plurality of slave operation sub-modules 122 . Both the master operation sub-module 121 and the plurality of slave operation sub-modules 122 include the tensor operator (not shown in the figure).

The control module 11 is further configured to parse the tensor instruction to obtain multiple operation instructions, and send the tensor to be operated and the multiple operation instructions to the main operation sub-module 121 .

The main operation sub-module 121 performs pre-processing on the to-be-operated tensor, and sends the operation instruction and at least a part of the to-be-operated tensor to the slave operation sub-module; The tensor operator can perform a multiplication operation of the tensor and a scalar to obtain an intermediate result.

The tensor operator of the slave operation sub-module 122 is configured to perform the multiplication operation of the tensor and the scalar in parallel according to the data and operation instructions received from the main operation sub-module 122 to obtain multiple intermediate results, and combine the multiple intermediate results The result is transmitted to the main operation sub-module 121 . The main operation sub-module 121 is further configured to perform subsequent processing on a plurality of intermediate results, obtain operation results, and store the operation results in the target address.

In a possible implementation manner, the tensor instruction and the to-be-operated tensor are split into at least one corresponding tensor operation instruction and at least one sub-tensor according to the dimension of the to-be-operated tensor , wherein the tensor operation instructions and sub-tensors are in one-to-one correspondence, and the plurality of corresponding operation instructions and sub-tensors are respectively sent to the slave operation sub-module for multiplication of tensors and scalars, The intermediate result of multiplying the tensor and the scalar is obtained and sent to the main operation sub-module, and the main operation sub-module combines the intermediate results to obtain the operation result, and stores the operation result in the target address, where the According to the dimension of the tensor is divided into a plurality of corresponding tensor operation instructions can be divided into one more one-dimensional tensor, can also be divided into multiple multi-dimensional tensors, the divided into The dimension of the tensor is less than or equal to the dimension of the acquired tensor to be operated.

Optionally, the above-mentioned tensor and scalar multiplication operation may also be implemented only by an operator in the main processing submodule. For example, when the operation instruction is an operation performed on scalar or vector data, the apparatus may control the main operation sub-module to perform an operation corresponding to the calculation instruction by using the operator therein. When the operation instruction is to perform operations on data with dimensions greater than or equal to 2 such as matrices and tensors, the device may perform operations in a coordinated manner through the master operation sub-module and the slave operation sub-module. For the specific implementation, see the above description.

It should be noted that those skilled in the art can set the connection mode between the main operation sub-module and multiple slave operation sub-modules according to actual needs, so as to realize the architecture setting of the operation module. For example, the architecture of the operation module can be "H"-type architecture, array-type architecture, tree-type architecture, etc., are not limited in the present disclosure.

FIG. 2c shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. In a possible implementation manner, as shown in FIG. 2c, the operation module 12 may further include one or more branch operation sub-modules 123, and the branch operation sub-module 123 is used for forwarding the master operation sub-module 121 and the slave operation sub-module 122 between data and/or operation instructions. Wherein, the main operation sub-module 121 is connected with one or more branch operation sub-modules 123 . In this way, the main operation sub-module, the branch operation sub-module and the slave operation sub-module in the operation module are connected by an "H" type structure, and data and/or operation instructions are forwarded through the branch operation sub-module, saving the need for the main operation sub-module. resource consumption, thereby improving the processing speed of instructions.

FIG. 2d shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. In a possible implementation manner, as shown in FIG. 2d , a plurality of slave operation sub-modules 122 are distributed in an array.

Each slave operation sub-module 122 is connected to other adjacent slave operation sub-modules 122, the master operation sub-module 121 is connected to k slave operation sub-modules 122 in the plurality of slave operation sub-modules 122, and the k slave operation sub-modules 122 are : n slave operation submodules 122 in the first row, n slave operation submodules 122 in the mth row, and m slave operation submodules 122 in the first column.

Among them, as shown in Figure 2d, the k slave operation submodules only include n slave operation submodules in the first row, n slave operation submodules in the mth row, and m slave operation submodules in the first column, that is, The k slave operation submodules are slave operation submodules that are directly connected to the master operation submodule among the plurality of slave operation submodules. The k slave operation submodules are used for data and instruction forwarding between the master operation submodule and a plurality of slave operation submodules. In this way, the plurality of slave operation sub-modules are distributed in an array, which can improve the speed at which the master operation sub-module sends data and/or operation instructions to the slave operation sub-modules, thereby increasing the instruction processing speed.

FIG. 2e shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. In a possible implementation manner, as shown in FIG. 2e , the operation module may further include a tree-type sub-module 124 . The tree-type sub-module 124 includes a root port 401 and a plurality of branch ports 402 . The root port 401 is connected to the master operation sub-module 121 , and the multiple branch ports 402 are respectively connected to the multiple slave operation sub-modules 122 . The tree-type sub-module 124 has the function of sending and receiving, and is used to forward data and/or operation instructions between the master operation sub-module 121 and the slave operation sub-module 122 . In this way, the operation modules are connected in a tree structure through the function of the tree-type sub-module, and the forwarding function of the tree-type sub-module can be used to improve the speed at which the master operation sub-module sends data and/or operation instructions to the slave operation sub-module, thereby improving the The processing speed of the instruction.

In a possible implementation manner, the tree-type sub-module 124 may be an optional result of the apparatus, which may include at least one level of nodes. The node is a line structure with forwarding function, and the node itself does not have the computing function. The node at the lowest level is connected to the slave operation sub-module to forward data and/or operation instructions between the master operation sub-module 121 and the slave operation sub-module 122 . In particular, if the tree-type sub-module has zero-level nodes, the device does not need the tree-type sub-module.

In a possible implementation manner, the tree sub-module 124 may include multiple nodes in an n-ary tree structure, and the multiple nodes in the n-ary tree structure may have multiple layers.

For example, FIG. 2f shows a block diagram of a tensor instruction processing apparatus according to an embodiment of the present disclosure. As shown in FIG. 2f , the n-ary tree structure may be a binary tree structure, and the tree-type sub-module includes 2-layer nodes 01 . The lowermost node 01 is connected to the slave operation sub-module 122 to forward data and/or operation instructions between the master operation sub-module 121 and the slave operation sub-module 122 .

In this implementation manner, the n-ary tree structure may also be a ternary tree structure, etc., and n is a positive integer greater than or equal to 2. Those skilled in the art can set n in the n-ary tree structure and the number of layers of nodes in the n-ary tree structure as required, which is not limited in the present disclosure.

In a possible implementation, the operation domain may also include a tensor operation type.

Wherein, the control module 11 can also be used to determine the tensor operation type according to the operation domain.

In a possible implementation manner, the tensor operation type may include at least one of the following: tensor multiplication operation, tensor and scalar multiplication operation, tensor addition operation, tensor sum operation, and storage of specified values that satisfy operation conditions Operations, bitwise AND operations, bitwise OR operations, bitwise XOR operations, bitwise negation operations, bitwise maximum operations, bitwise minimum operations. The operation condition may include any of the following: bitwise equality, bitwise inequality, bitwise less than, bitwise greater than or equal to, bitwise greater than, and bitwise less than or equal to. The specified value may be a numerical value such as 0, 1, etc., which is not limited in the present disclosure.

Wherein, the operation that satisfies the bitwise equality to store the specified value may be: judging whether the corresponding bits of the first to-be-operated tensor and the second to-be-operated tensor in the to-be-operated tensors are equal, and the When the corresponding bits of the quantities are equal, the specified value is stored; when the corresponding bits are not equal, the value of the corresponding bit of the first tensor to be operated or the second tensor to be operated is stored, or a value different from the specified value such as 0 is stored.

The operation that satisfies the bitwise unequal storage of the specified value can be: judging whether the corresponding bits of the first tensor to be calculated and the second tensor to be calculated in the tensors to be calculated are equal, and the first tensor to be calculated and the second tensor to be calculated When the corresponding bits of are not equal, store the specified value; when the corresponding bits are equal, store the value of the first tensor to be operated or the value of the second tensor to be operated in the corresponding bit, or store a value different from the specified value such as 0.

The operation that satisfies the bitwise less than storage specified value may be: judging the size relationship between the corresponding bits of the first to-be-operated tensor and the second to-be-operated tensor in the to-be-operated tensor, and the value of the first to-be-operated tensor in the corresponding bit is less than When the value of the second tensor to be calculated, the specified value is stored; when the value of the first tensor to be calculated on the corresponding bit is greater than or equal to the value of the second tensor to be calculated, the first tensor to be calculated or the second tensor to be calculated is stored. The value of the corresponding bit of the tensor to be operated, or a value different from the specified value, such as 0, is stored.

The operation that satisfies the bitwise value greater than or equal to the storage specified value may be: judging the size relationship between the corresponding bits of the first to-be-operated tensor and the second to-be-operated tensor in the When the value is greater than or equal to the value of the second tensor to be computed, store the specified value; when the value of the first tensor to be computed on the corresponding bit is less than the value of the second tensor to be computed, store the first tensor to be computed or The value in the corresponding bit of the second tensor to be operated, or a value different from the specified value, such as 0, is stored.

The operation that satisfies the bitwise greater than the storage specified value may be: judging the size relationship between the corresponding bits of the first to-be-operated tensor and the second to-be-operated tensor in the to-be-operated tensor, and the value of the first to-be-operated tensor in the corresponding bit is greater than When the value of the second tensor to be computed, the specified value is stored; when the value of the first tensor to be computed on the corresponding bit is less than or equal to the value of the second tensor to be computed, the first tensor to be computed or the second tensor to be computed is stored. The value of the corresponding bit of the tensor to be operated, or a value different from the specified value, such as 0, is stored.

The operation that satisfies the bitwise value less than or equal to the storage specified value may be: judging the size relationship between the corresponding bits of the first to-be-operated tensor and the second to-be-operated tensor in the to-be-operated tensor; When the value is less than or equal to the value of the second tensor to be computed, store the specified value; when the value of the first tensor to be computed on the corresponding bit is greater than the value of the second tensor to be computed, store the first tensor to be computed or The value in the corresponding bit of the second tensor to be operated, or a value different from the specified value, such as 0, is stored.

In this implementation manner, different operation domain codes can be set for different tensor operation types to distinguish different operation types. For example, the code for "multiplication of tensors" can be set to "mult". The code for "multiplying a tensor by a scalar" can be set to "mult.const". You can set the code for the "addition of tensors" to "add". The code for "summation of tensors" can be set to "sub". The code for "bitwise AND" can be set to "and". The code for "bitwise OR" can be set to "or". The code for "bitwise exclusive-or" can be set to "xor". The code for "bitwise negation" can be set to "not". The code for "bitwise max operation" can be set to "max". You can set the code for "bit-wise min operation" to "min". You can set the code for "storing the specified value 1 operation if bitwise equality is satisfied" to "eq". You can set the code for "Store the specified value 1 operation if bitwise inequality is satisfied" to "ne". The code that "satisfies the bitwise less than store specified value 1 operation" can be set to "lt". The code that "satisfies the bitwise greater than or equal to store the specified value 1 operation" can be set to "ge". The code that "satisfies the bitwise greater than storage specified value 1 operation" can be set to "gt". The code that "satisfies the bitwise less than or equal to store the specified value 1 operation" can be set to "le".

Those skilled in the art can set the operation types and their corresponding codes according to actual needs, which is not limited in the present disclosure.

In a possible implementation, the operation field may also include an input quantity. Wherein, the control module 11 is further configured to determine the input quantity according to the operation domain, and obtain the to-be-operated tensor whose data quantity is the input quantity from the to-be-operated data address.

In this implementation manner, the input quantity may be a parameter representing the data quantity of the tensor to be operated, for example, the length and width of the tensor.

In one possible implementation, a default input amount can be set. When the input quantity cannot be determined according to the operation domain, the default input quantity can be determined as the input quantity of the current tensor instruction, and the to-be-operated tensor whose data quantity is the default input quantity is obtained from the to-be-operated data address.

In a possible implementation manner, as shown in FIGS. 2 a to 2 f , the apparatus may further include a storage module 13 . The storage module 13 is used for storing the tensors to be operated.

In this implementation, the storage module may include one or more of a memory, a cache, and a register, and the cache may include a temporary cache. The tensors to be operated can be stored in the memory, cache and/or registers of the storage module as required, which is not limited in the present disclosure.

In a possible implementation manner, the apparatus may further include a direct memory access module for reading or storing data from the storage module.

In a possible implementation manner, as shown in FIGS. 2 a to 2 f , the control module 11 may include an instruction storage sub-module 111 , an instruction processing sub-module 112 and a queue storage sub-module 113 .

The instruction storage sub-module 111 is used to store tensor instructions.

The instruction processing sub-module 112 is used to parse the tensor instruction to obtain the operation code and operation domain of the tensor instruction.

The queue storage sub-module 113 is used to store an instruction queue, and the instruction queue includes a plurality of instructions to be executed sequentially arranged in an execution order, and the instructions to be executed may include tensor instructions.

In this implementation manner, the instructions to be executed may also include calculation instructions related or unrelated to tensor operations, which are not limited in the present disclosure. An instruction queue can be obtained by arranging the execution order of multiple instructions to be executed according to the receiving time, priority level, etc. of the instructions to be executed, so as to execute the multiple instructions to be executed in sequence according to the instruction queue.

In a possible implementation manner, as shown in FIGS. 2 a to 2 f , the control module 11 may further include a dependency relationship processing sub-module 114 .

The dependency relationship processing sub-module 114 is configured to cache the first to-be-executed instruction in the In the storage sub-module 111 , after the execution of the zeroth to-be-executed instruction is completed, the first to-be-executed instruction is extracted from the instruction storage sub-module 111 and sent to the operation module 12 .

Wherein, the relationship between the first instruction to be executed and the zeroth instruction to be executed before the first instruction to be executed includes: a first storage address range for storing data required by the first instruction to be executed and data required for storing the zeroth instruction to be executed The zeroth memory address range of has overlapping regions. Conversely, if there is no association between the first instruction to be executed and the zeroth instruction to be executed before the first instruction to be executed, it may be that the first storage address interval and the zeroth storage address interval have no overlapping area.

In this way, according to the dependency between the first to-be-executed instruction and the zeroth to-be-executed instruction before the first to-be-executed instruction, after the previous zeroth to-be-executed instruction is executed, the subsequent zeroth to-be-executed instruction is executed. Once the instruction to be executed, the accuracy of the operation result is guaranteed.

In one possible implementation, the instruction format of a tensor instruction can be:

opcode,dst,src,type,NumOfEle

Among them, opcode is the operation code of the tensor instruction, and dst, src, type, and NumOfEle are the operation fields of the tensor instruction. Among them, dst is the target address. src is the address of the tensor to be operated on. When there are multiple tensors to be operated on, src may include multiple addresses of data to be operated on src0, src1, . . . , srcn, which is not limited in the present disclosure. type is the tensor operation type. NumOfEle is the input quantity. Among them, type can be the code of the tensor operation type, such as mult, mult.const, add, sub, eq, ne, lt, ge, gt, le, eq, and, or, xor, not, max, min.

Wherein, when there are multiple tensors to be operated on, the instruction format may include multiple addresses of data to be operated on. Taking the following as an example of including two tensors to be operated on, the instruction format of the tensor instruction may be:

opcode,dst,src0,src1,type,NumOfEle

In one possible implementation, the instruction format of a tensor instruction can be:

type,dst,src,NumOfEle

In one possible implementation, the instruction format of the tensor instruction used for the "tensor multiplication operation" can be set to: mult, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. Multiply the tensors to be operated to obtain the operation result. And store the operation result in the target address dst.

In a possible implementation, the instruction format of the tensor instruction used for the "multiply tensor and scalar operation" can be set to: mult.const, dst, src0, NumOfEle, scale. It means: obtain the NumOfEle-sized to-be-operated tensor from the first to-be-operated data address src0, obtain the to-be-operated scalar scale and the input NumOfEle from the scalar register, multiply the to-be-operated tensor and the to-be-operated scalar, and obtain the operation result , and store the operation result in the target address dst.

In a possible implementation manner, the instruction format of the tensor instruction used for "multiplication of tensor and scalar" can also be set to: mult.const, dst, src0, NumOfEle, scale. It means: obtaining the tensor to be operated with the size of NumOfEle from the first data address src0 to be operated, and obtaining the scalar scale to be operated and the input quantity NumOfEle from the scalar register. Optionally, the type of the first data to be operated can be fixed-point data, floating-point data, 16-bit data, or 32-bit data. The first type of the data to be operated is multiplied by the tensor to be operated and the scalar to be operated to obtain the operation result, and the operation result is stored in the target address dst.

Wherein, the input quantity NumOfEle may be an integer divisible by 64. Of course, in other embodiments, the input quantity NumOfEle may also be an integer divisible by 2, 4, 8, 16, or 32, etc. This is only an example The description is not used to limit the specific value range of the input quantity.

Optionally, the storage module may include an on-chip storage space, which may be an on-chip NRAM, for storing tensor data or scalar data. The storage space pointed to by the source address sr0 and the target address dst may be NRAM. Certainly, in other embodiments, the storage space pointed to by the source address sr0 and the target address dst may also be other storage spaces of the storage module.

Further, the source address sr0 and the target address dst both refer to the starting address, the source address corresponds to a default address offset, the target address corresponds to a default address offset, and the default address offset can be is a multiple of 64 bytes. Of course, in other embodiments, the default address offset may also be an integer multiple of 8 bytes, 16 bytes, 32 bytes, or 128 bytes, etc., which is only used for illustration here, and is not specifically limited. Specifically, the address offset can be determined according to the operation result.

In a possible implementation, the instruction format of the tensor instruction used for the "tensor addition operation" can be set as: add, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The tensors to be operated are added to obtain the operation result. And store the operation result in the target address dst.

In one possible implementation, the instruction format of the tensor instruction used for "summation of tensors" can be set to: sub, dst, src, NumOfEle. It means: obtain a plurality of tensors to be operated with the size of NumOfEle from the address src to be operated, and perform a sum operation on the plurality of tensors to be operated to obtain the operation result. And store the operation result in the target address dst.

In one possible implementation, the instruction format of the tensor instruction for "bitwise AND" can be set to: and,dst,src0,src1,NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. Perform a bitwise AND operation on the tensors to be operated to obtain the operation result. And store the operation result in the target address dst.

In one possible implementation, the instruction format of the tensor instruction for "bitwise OR" can be set to: or,dst,src0,src1,NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. Perform a bitwise OR operation on the tensors to be operated to obtain the operation result. And store the operation result in the target address dst.

In one possible implementation, the instruction format of the tensor instruction for "bitwise XOR" can be set to: xor,dst,src0,src1,NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. Perform a bitwise XOR operation on the tensors to be operated to obtain the operation result. And store the operation result in the target address dst.

In one possible implementation, the instruction format of the tensor instruction for "bitwise negation" can be set to: not,dst,src,NumOfEle. It means: obtain the NumOfEle-sized tensor to be operated from the address src to be operated, and perform a bitwise inversion operation on the to-be-operated tensor to obtain the operation result. And store the operation result in the target address dst.

In a possible implementation, the instruction format of the tensor instruction used for "bitwise maximum operation" can be set as: max, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. Perform a bitwise maximum operation on the tensor to be operated to obtain the operation result. And store the operation result in the target address dst.

In a possible implementation, the instruction format of the tensor instruction used for "bitwise minimum operation" can be set as: min, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. Perform a bitwise minimum operation on the tensor to be operated to obtain the operation result. And store the operation result in the target address dst.

In a possible implementation, the instruction format of the tensor instruction used for "the operation of storing the specified value 1 if bitwise equality is satisfied" can be set as: eq, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The tensors to be operated on are compared bit by bit, and the specified value 1 is stored when the corresponding bits of the first tensor to be operated and the second tensor to be operated are equal to obtain the operation result. And store the operation result in the target address dst.

In a possible implementation manner, the instruction format of the tensor instruction used for "storing the specified value 1 operation if bitwise unequal is satisfied" can be set as: ne, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The to-be-operated tensors are compared bit by bit, and the specified value 1 is stored when the corresponding bits of the first to-be-operated tensor and the second to-be-operated tensor are not equal to obtain the operation result. And store the operation result in the target address dst.

In a possible implementation, the instruction format of the tensor instruction that "satisfies the bitwise less than 1 operation to store the specified value" can be set as: lt, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The tensors to be operated on are compared bit by bit, and the specified value 1 is stored when the value of the first tensor to be operated on the corresponding bit is smaller than the value of the second tensor to be operated, and the operation result is obtained. And store the operation result in the target address dst.

In a possible implementation, the instruction format of a tensor instruction that "satisfies the bitwise greater than or equal to store the specified value 1 operation" can be set as: ge, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The tensors to be operated on are compared bit by bit, and when the value of the first tensor to be operated on the corresponding bit is greater than or equal to the value of the second tensor to be operated, the specified value 1 is stored, and the operation result is obtained. And store the operation result in the target address dst.

In a possible implementation, the instruction format of a tensor instruction that "satisfies the bitwise greater than then store specified value 1 operation" can be set to: gt, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The tensors to be operated on are compared bit by bit, and the specified value 1 is stored when the value of the first tensor to be operated on the corresponding bit is greater than the value of the second tensor to be operated on, and the operation result is obtained. And store the operation result in the target address dst.

In a possible implementation, the instruction format of the tensor instruction that "satisfies the bitwise less than or equal to store the specified value 1 operation" can be set as: le, dst, src0, src1, NumOfEle. It means: obtain the first tensor to be operated with the size of NumOfEle from the first address to be operated src0, and obtain the second tensor to be operated with the size of NumOfEle from the second address to be operated src1. The tensors to be operated on are compared bit by bit, and the specified value 1 is stored when the value of the first tensor to be operated on the corresponding bit is less than or equal to the value of the second tensor to be operated, and the operation result is obtained. And store the operation result in the target address dst.

It should be understood that those skilled in the art can set the operation code of the tensor instruction, the position of the operation code and the operation field in the instruction format as required, which is not limited in the present disclosure.

In a possible implementation manner, the apparatus may be provided in a graphics processing unit (Graphics Processing Unit, GPU for short), a central processing unit (Central Processing Unit, CPU for short), and an embedded neural-network processing unit (Neural-network Processing Unit, abbreviated as NPU) among one or more of them.

It should be noted that, although the tensor instruction processing apparatus is described above by taking the above embodiment as an example, those skilled in the art can understand that the present disclosure should not be limited thereto. In fact, the user can flexibly set each module according to personal preferences and/or actual application scenarios, as long as it conforms to the technical solutions of the present disclosure.

Application example

In the following, an application example according to an embodiment of the present disclosure is given in conjunction with "using a tensor instruction processing apparatus to perform tensor operations" as an exemplary application scenario, so as to facilitate understanding of the flow of the tensor instruction processing apparatus. Those skilled in the art should understand that the following application examples are only for the purpose of facilitating the understanding of the embodiments of the present disclosure, and should not be regarded as limitations on the embodiments of the present disclosure

FIG. 3 shows a schematic diagram of an application scenario of a tensor instruction processing apparatus according to an embodiment of the present disclosure. As shown in Figure 3, the process of processing the tensor instruction by the tensor instruction processing device is as follows:

The control module 11 parses the acquired tensor instruction 1 (for example, the tensor instruction 1 is @opcode#500#101#mult.const#1024), and obtains the opcode and operation domain of the tensor instruction 1. The operation code of tensor instruction 1 is opcode, the target address is 500, and the address of the first tensor to be operated is 101. The tensor operation type is mult.const (multiplication of tensor and scalar), and the input is 1024. The control module 11 obtains the first to-be-operated tensor whose data volume is the input amount 1024 from the address 101 of the to-be-operated tensor, and obtains the input amount 1024 and the to-be-operated scalar from the immediate register. The operation module 12 performs an addition operation on the first to-be-operated tensor and the second to-be-operated tensor to obtain an operation result 1, and stores the operation result 1 in the target address 500 .

Among them, the tensor instruction 1 can be the above @opcode#500#101#102#add#1024 or @add#500#101#102#1024. The processing process of tensor instructions in different instruction formats is similar. Repeat.

For the working process of the above modules, reference may be made to the above related descriptions.

In this way, the tensor instruction processing apparatus can process the tensor instructions efficiently and quickly.

The present disclosure provides a machine learning computing device. The machine learning computing device may include one or more of the above-mentioned tensor instruction processing devices, which are used to obtain the tensors to be operated and control information from other processing devices, and perform specified machine learning operations. . The machine learning computing device can obtain tensor instructions from other machine learning computing devices or non-machine learning computing devices, and transmit the execution results to peripheral devices (also called other processing devices) through the I/O interface. Peripherals such as camera, monitor, mouse, keyboard, network card, wifi interface, server. When more than one tensor instruction processing device is included, the tensor instruction processing devices can be linked and transmitted through a specific structure, for example, interconnected and transmitted through the PCIE bus to support larger-scale neural network operations. At this time, the same control system can be shared, or there can be independent control systems; memory can be shared, or each accelerator can have its own memory. In addition, the interconnection method can be any interconnection topology.

The machine learning computing device has high compatibility and can be connected with various types of servers through the PCIE interface.

FIG. 4a shows a block diagram of a combined processing apparatus according to an embodiment of the present disclosure. As shown in Fig. 4a, the combined processing device includes the above-mentioned machine learning computing device, a general interconnection interface and other processing devices. The machine learning computing device interacts with other processing devices to jointly complete the operation specified by the user.

Other processing devices include one or more processor types among general-purpose/special-purpose processors such as a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processor. The number of processors included in other processing devices is not limited. Other processing devices serve as the interface between the machine learning computing device and external data and control, including data transfer, to complete the basic control of starting and stopping the machine learning computing device; other processing devices can also cooperate with the machine learning computing device to complete computing tasks.

A universal interconnect interface for transferring data and control instructions between machine learning computing devices and other processing devices. The machine learning computing device obtains required input data from other processing devices, and writes it into a storage device on-chip of the machine learning computing device; it can obtain control instructions from other processing devices and write it into the control cache on the machine learning computing device chip; The data in the storage module of the machine learning computing device can be read and transmitted to other processing devices.

FIG. 4b shows a block diagram of a combined processing apparatus according to an embodiment of the present disclosure. In a possible implementation manner, as shown in FIG. 4b, the combined processing device may further include a storage device, and the storage device is respectively connected to the machine learning computing device and the other processing devices. The storage device is used to save data in the machine learning computing device and the other processing devices, and is especially suitable for data that cannot be fully stored in the internal storage of the machine learning computing device or other processing devices.

The combined processing device can be used as an SOC system for mobile phones, robots, drones, video surveillance equipment and other equipment, effectively reducing the core area of the control part, improving the processing speed and reducing the overall power consumption. In this case, the general interconnection interface of the combined processing device is connected to certain components of the apparatus. Some components such as camera, monitor, mouse, keyboard, network card, wifi interface.

The present disclosure provides a machine learning chip, which includes the above-mentioned machine learning computing device or combined processing device.

The present disclosure provides a machine learning chip packaging structure, and the machine learning chip packaging structure includes the above-mentioned machine learning chip.

The present disclosure provides a board, and FIG. 5 shows a schematic structural diagram of the board according to an embodiment of the present disclosure. As shown in FIG. 5 , the board includes the above-mentioned machine learning chip packaging structure or the above-mentioned machine learning chip. In addition to the machine learning chip 389 , the board may also include other supporting components, including but not limited to: a storage device 390 , an interface device 391 and a control device 392 .

The storage device 390 is connected to the machine learning chip 389 (or the machine learning chip in the machine learning chip package structure) through a bus for storing data. The memory device 390 may include groups of memory cells 393 . Each group of storage units 393 is connected to the machine learning chip 389 through a bus. It can be understood that each group of storage units 393 may be DDR SDRAM (English: Double Data Rate SDRAM, double-rate synchronous dynamic random access memory).

DDR does not need to increase the clock frequency to double the speed of SDRAM. DDR allows data to be read out on both the rising and falling edges of the clock pulse. DDR is twice as fast as standard SDRAM.

In one embodiment, the memory device 390 may include four sets of memory cells 393 . Each set of memory cells 393 may include a plurality of DDR4 particles (chips). In one embodiment, the machine learning chip 389 may include four 72-bit DDR4 controllers inside, where 64 bits of the 72-bit DDR4 controllers are used for data transmission and 8 bits are used for ECC verification. It can be understood that when DDR4-3200 particles are used in each group of storage units 393, the theoretical bandwidth of data transmission can reach 25600MB/s.

In one embodiment, each set of memory cells 393 includes a plurality of double-rate synchronous dynamic random access memories arranged in parallel. DDR can transfer data twice in one clock cycle. A controller for controlling the DDR is provided in the machine learning chip 389 for controlling data transmission and data storage of each storage unit 393 .

The interface device 391 is electrically connected to the machine learning chip 389 (or the machine learning chip within the machine learning chip package structure). The interface device 391 is used to realize data transmission between the machine learning chip 389 and an external device (such as a server or a computer). For example, in one embodiment, the interface device 391 may be a standard PCIE interface. For example, the data to be processed is transmitted by the server to the machine learning chip 289 through a standard PCIE interface to realize data transfer. Preferably, when the PCIE 3.0X 16 interface is used for transmission, the theoretical bandwidth can reach 16000MB/s. In another embodiment, the interface device 391 may also be other interfaces, and the present disclosure does not limit the specific expression forms of the other interfaces, as long as the interface device can realize the switching function. In addition, the calculation result of the machine learning chip is still transmitted back to the external device (such as a server) by the interface device.

The control device 392 is electrically connected to the machine learning chip 389 . The control device 392 is used to monitor the state of the machine learning chip 389 . Specifically, the machine learning chip 389 and the control device 392 may be electrically connected through an SPI interface. The control device 392 may include a Micro Controller Unit (MCU). For example, the machine learning chip 389 may include multiple processing chips, multiple processing cores or multiple processing circuits, and may drive multiple loads. Therefore, the machine learning chip 389 can be in different working states such as multi-load and light-load. The control device can realize the regulation of the working states of multiple processing chips, multiple processing and/or multiple processing circuits in the machine learning chip.

The present disclosure provides an electronic device including the above-mentioned machine learning chip or board.

Electronic devices may include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cloud servers, cameras, video cameras, projectors, watches, Headphones, mobile storage, wearables, vehicles, home appliances, and/or medical equipment.

Vehicles may include aircraft, ships and/or vehicles. Household appliances may include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods. Medical equipment may include MRI machines, ultrasound machines and/or electrocardiographs.

FIG. 6 shows a flowchart of a tensor instruction processing method according to an embodiment of the present disclosure. As shown in FIG. 6, the method is applied to the above-mentioned tensor instruction processing apparatus, and the method includes step S51 and step S52.

In step S51, the compiled tensor instruction is parsed to obtain an operation code and an operation field of the tensor instruction, and according to the operation code and the operation field, a waiting list required to execute the tensor instruction is obtained. Operates on the tensor, the scalar to be operated on, and the destination address. The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address.

In step S52, a tensor-to-scalar multiplication operation is performed on the tensor to be operated and the scalar to be operated to obtain an operation result, and the operation result is stored in the target address.

In a possible implementation manner, performing a tensor operation on a tensor to be operated to obtain an operation result may include: using at least one tensor operator to perform a multiplication operation between a tensor and a scalar. Specifically, at least one tensor operator is used to multiply each element of the tensor to be operated by the scalar to be operated to obtain an operation result, so as to realize the multiplication operation of the tensor and the scalar.

In a possible implementation manner, the method may further include: parsing the compiled tensor instructions to obtain multiple operation instructions. Wherein, step S52 may include:

The control module parses the compiled tensor instruction to obtain multiple operation instructions, and sends the to-be-operational tensor and the multiple operation instructions to the main operation sub-module;

The master operation sub-module performs pre-processing on the to-be-operated tensor, and sends the operation instruction and at least a part of the to-be-operated tensor to the slave operation sub-module; The tensor operator can perform the multiplication operation of the tensor and the scalar to obtain an intermediate result;

The tensor operator of the slave operation sub-module executes the multiplication operation of the tensor and the scalar in parallel according to the data and operation instructions received from the master operation sub-module to obtain multiple intermediate results, and combines the multiple intermediate results. transmitted to the main operation sub-module;

The main operation submodule performs subsequent processing on the plurality of intermediate results to obtain operation results, and stores the operation results in the target address.

In a possible implementation, the operation field may also include an input quantity. Wherein, obtaining the tensor to be operated and the target address required to execute the tensor instruction according to the operation code and the operation field may further include: determining the input amount according to the operation field, and obtaining the pending operation whose data amount is the input amount from the address of the data to be operated. Operate on tensors.

In a possible implementation manner, the method may further include: storing a tensor to be operated.

In a possible implementation manner, the scalar to be operated is an immediate value or a scalar register.

In a possible implementation manner, the compiled tensor instruction is parsed to obtain the operation code and operation field of the tensor instruction, which may include:

Store compiled tensor instructions;

Analyze the compiled tensor instruction to obtain the opcode and operation domain of the tensor instruction;

An instruction queue is stored, and the instruction queue includes multiple instructions to be executed sequentially arranged in an execution order, and the multiple instructions to be executed may include compiled tensor instructions.

In a possible implementation manner, the method may further include: when it is determined that the first to-be-executed instruction in the plurality of to-be-executed instructions has an associated relationship with the zeroth to-be-executed instruction before the first to-be-executed instruction, caching the first to-be-executed instruction The instruction to be executed, and after it is determined that the execution of the zeroth execution instruction is completed, the execution of the first instruction to be executed is controlled,

Wherein, the relationship between the first instruction to be executed and the zeroth instruction to be executed before the first instruction to be executed may include: a first storage address interval for storing data required by the first instruction to be executed and a first storage address range required for storing the zeroth instruction to be executed The zeroth storage address range of data has overlapping areas.

In a possible implementation manner, the acquired tensor instructions are compiled to obtain the compiled tensor instructions, which may include:

Generate assembly files based on tensor instructions, and translate assembly files into binary files. Among them, the binary file is the compiled tensor instruction.

It should be noted that although the above embodiments are used as examples to describe the tensor instruction processing method as above, those skilled in the art can understand that the present disclosure should not be limited thereto. In fact, the user can flexibly set each step according to personal preference and/or actual application scenarios, as long as it conforms to the technical solutions of the present disclosure.

The tensor instruction processing method provided by the embodiment of the present disclosure has a wide range of applications, and has high processing efficiency and high processing speed for tensors.

It should be noted that, for the sake of simple description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Because in accordance with the present application, certain steps may be performed in other orders or concurrently. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

In the embodiments provided in the present disclosure, it should be understood that the disclosed systems and devices may be implemented in other manners. For example, the above-described system and device embodiments are only illustrative, for example, the division of devices, devices, and modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules may be combined. Or it may be integrated into another system or device, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices, devices or modules, which may be in electrical or other forms.

Modules described as separate components may or may not be physically separated, and components shown as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present disclosure may be integrated into one processing unit, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, or can be implemented in the form of software program modules.

The integrated modules, if implemented in the form of software program modules and sold or used as stand-alone products, may be stored in a computer-readable memory. Based on such understanding, the technical solutions of the present disclosure essentially or the parts that contribute to the prior art or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a memory, Several instructions are included to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present disclosure. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash disk , Read-only memory (English: Read-Only Memory, referred to as: ROM), random access device (English: Random Access Memory, referred to as: RAM), magnetic disk or optical disk, etc.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by one or more processing devices, the method in any of the foregoing embodiments is implemented. A step of. Specifically, when the above computer program is executed by the processor, the following steps are implemented:

Parse the acquired tensor instruction to obtain the operation code and operation field of the tensor instruction, and obtain the tensor to be operated and the scalar to be operated required to execute the tensor instruction according to the operation code and the operation field and destination address;

Multiplying the tensor and the scalar to be calculated by the tensor and the scalar to obtain an operation result, and storing the operation result in the target address;

The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address.

It should be clear that, in the embodiment of the present application, the implementation manner of each step is the same as the implementation manner of each step in the foregoing method, and for details, reference may be made to the above description, which will not be repeated here.

The foregoing can be better understood in terms of:

Clause 1: A tensor instruction processing apparatus, the apparatus comprising:

The control module is used to parse the acquired tensor instruction, obtain the operation code and operation field of the tensor instruction, and obtain the to-be-operated tensor required to execute the tensor instruction according to the operation code and the operation field Quantity, scalar to be operated and target address tensor;

an operation module, used for tensors to perform a tensor-to-scalar multiplication operation on the to-be-operated tensor and the to-be-operated scalar, obtain an operation result, and store the operation result in the target address;

The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address.

Clause 2: The apparatus of clause 1, the computing module comprising:

A plurality of tensor operators, configured to multiply each element of the tensor to be calculated by the scalar to be calculated to obtain an operation result, so as to realize the multiplication operation of the tensor and the scalar.

Clause 3: The apparatus of Clause 1 or 2, the operation module comprising a master operation sub-module and a plurality of slave operation sub-modules, the master operation sub-module and the plurality of slave operation sub-modules each including the Zhang Quantity calculator;

The control module is further configured to parse the compiled tensor instructions to obtain multiple operation instructions, and send the to-be-operated tensor and the multiple operation instructions to the main operation sub-module;

The master operation sub-module performs pre-processing on the to-be-operated tensor, and sends the operation instruction and at least a part of the to-be-operated tensor to the slave operation sub-module; The tensor operator can perform the multiplication operation of the tensor and the scalar to obtain an intermediate result;

The tensor operator of the slave operation sub-module is configured to perform the multiplication operation of the tensor and the scalar in parallel according to the data and operation instructions received from the master operation sub-module to obtain multiple intermediate results, and combine the multiple intermediate results. The result is transmitted to the main operation sub-module;

The main operation sub-module is further configured to perform subsequent processing on the plurality of intermediate results, obtain operation results, and store the operation results in the target address.

Clause 4: The apparatus of any of clauses 1-3, the operational field further comprising an input quantity,

Wherein, the control module is further configured to determine the to-be-operated tensor according to the source address of the to-be-operated tensor and the input quantity.

Clause 5: The apparatus of any one of clauses 1-4, the scalar to be operated is an immediate value or a scalar register.

Clause 6: The apparatus of any of clauses 1-5, further comprising:

A storage module, configured to store at least one of the to-be-operated tensor and the to-be-operated scalar.

Clause 7: The apparatus of any of clauses 1-6, the control module comprising:

an instruction storage submodule for storing the compiled tensor instruction;

an instruction processing submodule, used for parsing the compiled tensor instruction to obtain the operation code and operation domain of the tensor instruction;

The queue storage sub-module is used for storing an instruction queue, where the instruction queue includes a plurality of instructions to be executed sequentially arranged in an execution order, and the multiple instructions to be executed include the compiled tensor instructions.

Clause 8: The apparatus of any of clauses 1-7, the control module further comprising:

The dependency relationship processing submodule is configured to, when it is determined that the first to-be-executed instruction in the plurality of to-be-executed instructions has an associated relationship with the zeroth to-be-executed instruction before the first to-be-executed instruction, to process the first to-be-executed instruction The execution instruction is cached in the instruction storage sub-module, and after the execution of the zeroth instruction to be executed is completed, the first instruction to be executed is extracted from the instruction storage sub-module and sent to the operation module,

Wherein, the relationship between the first instruction to be executed and the zeroth instruction to be executed before the first instruction to be executed includes:

The first storage address interval storing the data required by the first instruction to be executed and the zeroth storage address interval storing the data required by the zeroth instruction to be executed have an overlapping area.

Clause 9: A tensor instruction processing method, the method being applied to a tensor instruction processing apparatus, the method comprising:

Parse the acquired tensor instruction to obtain the operation code and operation field of the tensor instruction, and obtain the tensor to be operated and the scalar to be operated required to execute the tensor instruction according to the operation code and the operation field and destination address;

Multiplying the tensor and the scalar to be calculated by the tensor and the scalar to obtain an operation result, and storing the operation result in the target address;

The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address.

Clause 10: The method of Clause 9, the computing module comprising:

A plurality of tensor operators, configured to multiply each element of the tensor to be calculated by the scalar to be calculated to obtain an operation result, so as to realize the multiplication operation of the tensor and the scalar.

Clause 11: The method of clause 9 or 10, the operation module comprising a master operation sub-module and a plurality of slave operation sub-modules, the master operation sub-module and the plurality of slave operation sub-modules each including the Zhang Quantity calculator;

The control module parses the compiled tensor instruction to obtain multiple operation instructions, and sends the to-be-operational tensor and the multiple operation instructions to the main operation sub-module;

The master operation sub-module performs pre-processing on the to-be-operated tensor, and sends the operation instruction and at least a part of the to-be-operated tensor to the slave operation sub-module; The tensor operator can perform the multiplication operation of the tensor and the scalar to obtain an intermediate result;

The tensor operator of the slave operation sub-module executes the multiplication operation of the tensor and the scalar in parallel according to the data and operation instructions received from the master operation sub-module to obtain multiple intermediate results, and combines the multiple intermediate results. transmitted to the main operation sub-module;

The main operation sub-module performs subsequent processing on the plurality of intermediate results to obtain operation results, and stores the operation results in the target address.

Clause 11: The method of any of clauses 9-10, the operational field further comprising an input quantity,

Wherein, the control module is further configured to determine the to-be-operated tensor according to the source address of the to-be-operated tensor and the input quantity.

Item 12: The method according to any one of Items 9-11, wherein the scalar to be operated is an immediate value or a scalar register.

Clause 13: The method according to any one of Clauses 9-12, characterized in that the method further comprises:

At least one of the to-be-operated tensor and the to-be-operated scalar is stored.

Item 14: The method according to any one of Items 9 to 13, characterized in that, by parsing the compiled tensor instruction, the operation code and operation domain of the tensor instruction are obtained, including:

The instruction storage submodule stores the compiled tensor instruction;

The instruction processing submodule parses the compiled tensor instruction to obtain the operation code and operation domain of the tensor instruction;

The queue storage submodule stores an instruction queue, and the instruction queue includes a plurality of instructions to be executed sequentially arranged in an execution order, and the multiple instructions to be executed include the compiled tensor instructions.

Clause 15: The method of any of clauses 9-14, further comprising:

When it is determined that the first to-be-executed instruction in the plurality of to-be-executed instructions has an associated relationship with the zeroth to-be-executed instruction before the first to-be-executed instruction, the first to-be-executed instruction is cached in the instruction storage In the sub-module, after the execution of the zeroth instruction to be executed is completed, the first instruction to be executed is extracted from the instruction storage sub-module and sent to the operation module,

Wherein, the relationship between the first instruction to be executed and the zeroth instruction to be executed before the first instruction to be executed includes:

The first storage address interval storing the data required by the first instruction to be executed and the zeroth storage address interval storing the data required by the zeroth instruction to be executed have an overlapping area.

Clause 16: A computer-readable storage medium having stored therein a computer program that, when executed by one or more processing devices, implements the method as described in any one of clauses 9-15 steps in .

The embodiments of the present application have been introduced in detail above, and specific examples are used to illustrate the principles and implementations of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, according to the idea of the present application, will have changes in the specific implementation manner and application scope. In conclusion, the contents of this specification should not be construed as a limitation on the present application.

Claims (10)

1. A tensor instruction processing device, wherein the device comprises: The control module is used to parse the acquired tensor instruction, obtain the operation code and operation field of the tensor instruction, and obtain the to-be-operated tensor required to execute the tensor instruction according to the operation code and the operation field Quantity, scalar to be operated and target address; an operation module, used for tensors to perform a tensor-to-scalar multiplication operation on the to-be-operated tensor and the to-be-operated scalar, obtain an operation result, and store the operation result in the target address; The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address. 2. The device according to claim 1, wherein the operation module comprises: A tensor operator, configured to multiply each element in the tensor to be operated by the scalar to be operated to obtain an operation result, so as to realize the multiplication operation of the tensor and the scalar. 3. The device according to claim 2, wherein the operation module comprises a master operation sub-module and a plurality of slave operation sub-modules, and the master operation sub-module and the multiple slave operation sub-modules both include the the tensor operator; The control module is also used to parse the The tensor instruction obtains multiple operation instructions, and sends the to-be-operated tensor and the multiple operation instructions to the main operation submodule; The master operation sub-module performs pre-processing on the to-be-operated tensor, and sends the operation instruction and at least a part of the to-be-operated tensor to the slave operation sub-module; The tensor operator can perform the multiplication operation of the tensor and the scalar to obtain an intermediate result; The tensor operator of the slave operation sub-module is configured to perform the multiplication operation of the tensor and the scalar in parallel according to the data and operation instructions received from the master operation sub-module to obtain multiple intermediate results, and combine the multiple intermediate results. The result is transmitted to the main operation sub-module; The main operation sub-module is further configured to perform subsequent processing on the plurality of intermediate results, obtain operation results, and store the operation results in the target address. 4. The apparatus according to claim 1, wherein the operation field further comprises an input quantity, Wherein, the control module is further configured to determine the to-be-operated tensor according to the source address of the to-be-operated tensor and the input quantity. 5 . The apparatus according to claim 1 , wherein the scalar to be operated is an immediate value or a scalar register. 6 . 6. The device according to any one of claims 1-4, wherein the device further comprises: A storage module, configured to store at least one of the to-be-operated tensor and the to-be-operated scalar. 7. The device according to any one of claims 1-4, wherein the control module comprises: an instruction storage submodule for storing the tensor instruction; The instruction processing submodule is used to parse the tensor instruction to obtain the operation code and operation domain of the tensor instruction; The queue storage submodule is used to store an instruction queue, the instruction queue includes a plurality of instructions to be executed sequentially arranged in an execution order, and the multiple instructions to be executed include the tensor instructions. 8. The apparatus according to claim 7, wherein the control module further comprises: The dependency relationship processing submodule is configured to, when it is determined that the first to-be-executed instruction in the plurality of to-be-executed instructions has an associated relationship with the zeroth to-be-executed instruction before the first to-be-executed instruction, to process the first to-be-executed instruction The execution instruction is cached in the instruction storage sub-module, and after the execution of the zeroth instruction to be executed is completed, the first instruction to be executed is extracted from the instruction storage sub-module and sent to the operation module, Wherein, the relationship between the first instruction to be executed and the zeroth instruction to be executed before the first instruction to be executed includes: The first storage address interval storing the data required by the first instruction to be executed and the zeroth storage address interval storing the data required by the zeroth instruction to be executed have an overlapping area. 9. A tensor instruction processing method, wherein the method is applied to a tensor instruction processing device, the method comprising: Analyze the acquired tensor instruction, obtain the operation code and operation field of the tensor instruction, and obtain the tensor to be operated, the scalar to be operated and the operation field required to execute the tensor instruction according to the operation code and the operation field. target address; The tensor performs a tensor-scalar multiplication operation on the to-be-operated tensor and the to-be-operated scalar to obtain an operation result, and stores the operation result in the target address; The operation code is used to indicate that the operation performed by the tensor instruction on the data is a multiplication operation of a tensor and a scalar, and the operation domain includes the source address of the tensor to be operated, the scalar to be operated and the target address. 10. A computer-readable storage medium, characterized in that, a computer program is stored in the computer-readable storage medium, and when the computer program is executed by one or more processing devices, the method according to claim 9 is implemented. A step of.

Images