JPH02287859A - Data processor - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a data processing device, and particularly to a data processing device that maps neural network processing to a plurality of processors and performs information processing using a multilayered neural network algorithm. It is related to.

[Conventional technology]

In recent years, research on neural network systems that simulate the functioning of the brain's nervous system has been attracting attention in various fields such as speech understanding, image understanding, and cognitive science.

Various types of neural network systems of this type have been proposed, each with its own characteristics.

Research on this type of neural network system can be found in Nifu Rosenblatt, "Perceptron", Psychological Review 65 (, 1958), p. 386N.
Page 408 [F. Rosenblatt, “The
perceptron”.

Psychologjcal Review 65
．． pp, 386-408 (1958) is based on the research on perceptrons found in the literature of Ko.

FIG. 7 is a diagram for explaining the concept of a perceptron. As shown in FIG.
M) 3, and each neuron element between each layer 3,
They are connected by a connection 2 whose degree of connection (weight) is variable. Now, the input data to be input to the perceptron is (X
, ), each vector element X is input to each neuron i in the A layer. Furthermore, if the weight of the degree of connection of a variable connection is expressed as wo, then the signal yk input to the neuron k of the B layer is equal to each back 1 health element X of the input from each neuron i of the A layer.

is weighted and integrated by w4□ and input to neuron k of layer B, so the signal yk is yk=ΣwkJ'x
.

It is required as. Furthermore, since the neurons in the B layer are themselves deformed by the function f, the value Zk output from the neuron element k in the B layer becomes Z,=f(yk). In this way, the output Zk to the B-layer neuron is obtained, and the output Zk is further transmitted to each element of the 0-layer neuron. Since various models can be realized by using various types of functions f of the neuron elements themselves, various systems have been proposed.

The signal entering the neuron in layer 0 is transformed in the same way as the signal entering the neuron in layer 13. In FIG. 7, a three-layer network is shown to avoid complexity, but Parcellon generally has a structure having a multi-layer network.

Now, the state of each element of the CM neuron is expressed as a vector (Z,
), and if the vector (○□) is the desired target output output from the perceptron, then in order to obtain the desired output (O8) vector, optimize the parameters of the weights w, 4 and the function f. Otherwise, generally (Z,)
The output of the vector does not match the desired output (○,) vector.

One method for optimizing the parameters of the loads w, 4, the function f, etc. is, for example, a method of minimizing the squared error. This is the (Z,) of the output of the C layer of the system output.
In order to optimize and determine the weight w,6 in order to match the vector with the target output (0,) vector, the weight w1 is
The load wL5 is changed so as to minimize the squared error d; d=Σ(o, -z,)2. The operation of changing the load W work J7 in this way is called learning.

By repeating a large number of learnings for various input patterns (xj) and objective vectors (O8), and determining the load WiJ by learning to minimize the squared error d, m types of input data can be obtained. It is possible to construct a perceptron system that can classify data into m' types of data.

Since the perceptron has a predetermined classification function by determining the weights optimized in this way, it is possible to classify various miscellaneous input vectors into one category. That is, the perceptron enables pattern recognition.

[Problem to be solved by the invention]

By the way, as described above, a perceptron type multilayer neural network system has a pattern recognition function in principle. However, in order to create a configuration that can recognize a large number of patterns, it is necessary to increase the number of elements constituting the neural network (the number of neurons and the number of connections) to an extent. From experience,
It is known that when the number of recognized buttons is N, the number of neurons at least 1°ON or more is required.

An increase in the number of neurons also increases the number of learning operations, increasing the risk that the learning process for minimizing the squared error d will reach a local minimum, from which it will be impossible to escape. For this reason, there are limits to simply increasing the scale of multilayer neural networks. In addition, in order to configure such a system, the learning time required to operate the system increases, and furthermore, as the scale of the neural network increases, there are many problems in information convergence when determining the loads in the network.

The present invention has been made to solve the above problems.

An object of the present invention is to configure a large-scale neural network system while avoiding problems such as increased learning time and convergence.

Another object of the present invention is to provide a solution to the problem of constructing a neural network system in terms of an algorithm, hardware for executing it, and rough l-ware for executing the algorithm on the hardware. It is in.

Furthermore, another object of the present invention is to provide a data processing apparatus that maps work processing to a neural network 1 to a plurality of processors and performs information processing using a multilayer neural network algorithm.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[Means to solve the problem]

In order to achieve the above object, the data processing device of the present invention includes:
A plurality of processor elements, a plurality of memory elements, a plurality of processor elements and a plurality of memory elements are temporally and spatially divided into two-dimensional groups, and the processing of each layer of the neural screw l-work is made to correspond to each group. , a control management unit that controls information processing performed by processor elements and memory elements in each group, and maps neural network processing to multiple processors to perform information processing using a multilayered neural network algorithm. Features.

[Effect]

According to the above means, the data processing device includes a plurality of processor elements, a plurality of memory elements, and a neural network for each group obtained by dividing the plurality of processor elements and the plurality of memory elements two-dimensionally in time and space. A control management unit is provided that controls information processing performed by processor elements and memory elements in each group by associating processing in each layer of the network. In this data processing device, a control management unit uses a plurality of processor elements and a plurality of memory elements to perform control to map neural network processing to a plurality of processors, and uses a multilayered neural network algorithm to control the mapping of neural network processing to a plurality of processors. Perform processing.

That is, in order to perform information processing using a multilayer neural network algorithm in a data processing device, a system for data processing is constructed by introducing the concept of structuring into each processing process of the multilayer neural network. With such a system configuration of structured processing processes, a learning algorithm can be executed on each neural screw 1-work in each structured processing process. In order to realize data processing on a structured neural network, in a data processing device, each processing process by the hardware and software system is divided into two dimensions in time and space, and the processing process is , execute each neural network process. This processing is controlled by the control management section.

This allows you to partition any neural network and
Connection processing between neurons between each layer of the divided network can be mapped and processed by one or more processors. Furthermore, the results of the divided and executed processing can be sequentially processed and used in combination with techniques such as symbolic processing. That is, the processing of the neural network becomes equivalent to the □ processing of one subroutine in language processing, and the processing of the neural network can be easily executed. Therefore, for example, the insufficiency of processing pointed out in the processing of Neural Screw 1 to Work, such as a method for dividing a problem and a method for processing a combinatorial structure by memory, is resolved.

〔Example〕

Hereinafter, one embodiment of the present invention will be specifically described using the drawings.

Note that throughout the explanation of the embodiments, the same elements are given the same reference numerals, and repeated explanations thereof will be omitted.

First, before specifically explaining a data processing device according to an embodiment of the present invention, in order to make the content easier to understand, we will explain the principle using an example where a recognition system is constructed by combining neural networks of subsets. explain.

FIG. 8 is a block diagram illustrating the concept of the system configuration of a hierarchical neural network.

Here, multiple input data given to the recognition system are (X,
), then dividing this (X, ) into subsets,
Create a neural network (subset neural network) that identifies each element for each subset. By dividing the input data into subsets,
For example, when the number of constituent elements becomes n-1, the number of neurons in the network can be n-1, the number of connections between neurons is n-2, and the learning time is reduced. Therefore, when configuring a system using this type of neural network, input data is divided into subsets, a subset neural network (sub-neural network) is created that identifies each element for each subset, and each sub-neural network is combined. and configure the system.

In FIG. 8, one recognition system is constructed by combining the sub-neural networks learned for each subset. In this system configuration, AM is a neuron layer that receives input data. The B layer is a sub-neural network 4 that recognizes a subset of input data.
It consists of The 0th layer is a glue network 5 for selecting the output of the B layer network and obtaining the desired output. This glue network 5 is a network that connects multiple networks. Hereinafter, a network that connects multiple networks is called a glue network.

In the network having the system configuration shown in FIG. 8, the output of the sub-neural network 4 is determined by the glue network 4, and all input data (X,) is recognized. However, it is known that the probability of correct recognition with this system configuration is extremely low.

The reason for this is that although the sub-neural network 4 gives a correct output for the learned subset X8, it is not guaranteed what kind of output it will give for the unlearned subset 7. Furthermore, the output of each sub-neural network is the result of recognition using the weights learned in the subset set, and has the property that no new information that would strengthen discrimination can be extracted from itself. What should be noted in such a system configuration is that the system configuration is to obtain the target value vector (0□) of the network mentioned above, but this target value (O) is It is set in advance, and there is no necessity for (O4) to take such a value.

The only information that can be extracted from the output from the sub-neural network is how strongly the ``learned information'' has been recognized. Therefore, it is difficult to sufficiently discriminate all input data (X,) using only the glue network selection mechanism that selects the output of the sub-neural network.

In order to overcome the drawbacks of the system configuration of the neural network shown in FIG. 8, for example, there is a method of configuring the system as follows.

The first method is to train the sub-neural network of the subset so as not to output an erroneous output even for information (7) other than the subset X. That is, learning is performed so that is minimized. In this study,
The amount of calculation is on the order of n'n-"=n-".

The second method is to train the sub-neural network to correspond to various features included in the total input data (X,). As a result of this learning, when the output of the sub-neural network indicates feature extraction information, it becomes possible to recognize patterns in input data by combining the features extracted by the crew network. The output of this glue network is given by the feature vector (b,),
If we convert the characteristics into a mathematical formula, we get the following. where K is the space in which the features are defined,
is equal to or smaller than the space in which Σ is defined. Therefore, if pattern recognition is now possible with Q features, the extraction vector of the glue network is (a
l), the vector corresponding to the target value vector (O9) is ΣaL ΣbLTL. Therefore, the learning is to minimize . This learning operation is divided into a sub-neural network and a glue network, and the output (2,) is referred to and the target value (○,) is achieved.
Transformation is performed to determine vectors (ai,) and (bi), respectively. However, learning about the vector (b,) is difficult in a multilayer neural network without a feedback mechanism. Furthermore, since the feature pattern is already known, it is difficult to converge by learning it.

Therefore, we will use the first method to configure the recognition system and discuss how to improve recognition accuracy.

FIG. 9 is a block diagram illustrating the concept of the system configuration of the split type neural screw 1-work.

In FIG. 9, 6 is a sub-neural network optimized for sub-cell 1- of input data, and 7 is a glue network 1-work. In the system configuration shown in FIG. 9, the sub-neural network is divided into two subsets in order to facilitate understanding and avoid complexity. The glue network 7 corresponds to and is similar to the glue network 5 in FIG. 8, but here it is directly connected to the input data receiving section of the neuron layer of the A layer. In addition, in the system configuration shown in FIG. 9, if the glue screw 1 and work 7 can be made to learn the function of separating the outputs of each of the plurality of sub-neural networks 6, it is possible to combine the plurality of sub-neural networks and input data (XS). can be separated into each.

The processing of the layers of the neural network is done by groups of connection weights between neurons. Now, if each element of the group of connection loads is shown in the form of a matrix, it will be as shown in FIG. The matrix 8 indicating each element is a matrix including a matrix 8a indicating the element value with "O" and a matrix 8b indicating the element value with "1". In this matrix, the connection weights are assumed to be expressed as a symmetric matrix. For example, in the matrix 8 shown in FIG. 10, the matrix 8a whose element values are indicated by "O" is a portion corresponding to the action identified by the subneural network u OI+ in the system configuration of FIG. 8. Also, change the value of the element to “
The matrix indicated by ``1'' is the sub-neural network ``1''
corresponds to the action identified by The fact that both sub-neural networks cannot correctly recognize it is the same as the fact that the action of the hatched portion 8C of matrix 8 in FIG. 10 cannot be regarded as a 0 matrix. The interaction of each sub-neural network in the system configuration shown in Figure 9 is
When displayed in a matrix as shown in FIG. 0, a matrix 9 as shown in FIG. 11 is obtained. In the matrix 9 in FIG. 11, the G portion corresponds to the action of the glue network 7 (FIG. 9). F part is glue network 7 and sub neural network 6
0 in the system configuration of Figure 9.
Corresponds to the interaction part of the layer. Further, the E portion is a 0 matrix in a split system configuration as shown in FIG. In other words, the conditions for creating a system for correct recognition by the neural network system with the system configuration shown in Figure 9 are that when the input data (X8) is artificially divided into subsets, the interaction between the subsets can be separated; This is equivalent to ``the discrimination effect of each element inside the subset is not affected by''.

If we replace the interaction between subsets with the term feature extraction, the system configuration shown by the matrix in Figure 11 is a neural network "0" using information called features.
This is the same as constructing a system by separating similar components between the neural network tl I 11 and the neural network tl I 11.

By the way, glue networks can also be divided in the same way as subneural networks. Therefore, the neural network system can be configured as a combination of sub-neural networks that are each divided into sub-neural networks.

Hereinafter, hardware that executes processing of a neural network system configured as a combination of sub-neural networks will be described.

In neural network processing, when the connection weights between neurons are arranged in a memory and the intensity of the signal input to each neuron is processed by a calculation unit, the calculation becomes an inner product calculation performed by matrix calculation or the like. Therefore, high-speed processing is possible using a vector processor or a parallel processor.

Of course, the speed is slow, but it is possible to perform arithmetic processing on a general-purpose computer. In either case, if you divide the neural network into subneural networks,
In addition to saving calculation time, it is possible to save memory area by holding data such as connection loads that are currently unnecessary on the disk device.

For this reason, the hardware that realizes neural network processing is memory and arithmetic units.

An external storage device and a control circuit to control these computing elements are required. In particular, since the amount of calculation becomes enormous, a system configuration using a parallel architecture is required.

In a neural screw 1-work with a multilayer structure, neuron processing in each layer can be calculated independently, so the correspondence between neurons and processors can be made variable by using processor elements in a time-division manner. In this case, a system control section (management control section) is provided to control the time-division use of the processor elements and also control to vary the correspondence between the neuron elements and the processor elements.

By providing such a system control unit, processing of subneural networks having different numbers of neurons can be realized with hardware having the same number of processor elements.

In this embodiment, in order to develop an arbitrary neural network system on actual limited hardware, a data processing device is configured with the following elements. That is, ``(1) Multiple processor elements and memory elements at -20
= A hardware management unit that is divided in two-dimensional format in the time and space directions; (2) Processing unit elements of each processor element and each memory element that are divided in two-dimensional format in the time and space directions (hereinafter referred to as (3) each processor element and each memory element divided in two-dimensional form in the time and space directions; (4) the contents of the memory element attached to the processor element;
(5) a logic unit that writes data to a main memory that can be commonly accessed by each processor (referred to as common memory in software terms); (5) a logic unit that reads and writes data from the common memory to memory elements attached to the processor; 6) A processing unit that serially processes data on the common memory and stores the results in the common memory, and a data processing device is constituted by hardware means and software means.

In a data processing device constituted by such hardware means and software means, processing corresponding to the processing algorithm of sub-neural network and Groene [Hewerk] is performed.

Information from multiple neural networks is stored in a common memory (main memory). If it does not fit into the main memory, part of the neural network information is stored on the disk device.

When processing one neural network (hereinafter abbreviated as network), 111 minutes of network information is transferred to the memory group. Assuming that the number of processor elements is , and the number of neurons in one layer of network is Q, the processor elements are time-divided into (II/]0+1=k.

The memory group contains signal data input to the processor.

Connection load data is stored. The aforementioned logic unit (5) is used for this processing. The number of these data is P
Assuming 10P*n, these are ((p10p*p)/k
)+1. Note that here, in order to simplify the explanation, it is assumed that (number of memories)=(number of processor elements). Therefore, in addition to being simply partitioned, p+p*R pieces of data are stored in k pieces of memory elements like data on the main memory that is interleaved with k pieces.

Then, each memory element and processor element are connected by the logic unit (3) described above, and processing is started by the hardware management unit (1).

When each process in the elements of the processing unit divided in two-dimensional format in the time and space directions is completed, the data in each memory element is written onto the main memory by the logic unit (4).

This series of processing is software management means (2)
, and the processing of one sub-neural network and glue network is completed. Based on the results of these processes, the processing unit (6) makes various determinations and outputs the final recognition results onto the main memory.

The principle of the system configuration for performing neural network processing has been described above.Next, an embodiment of a data processing apparatus configured based on the principle of the system configuration of this neural network system will be described.

FIG. 1 is a block diagram showing the configuration of a data processing device according to an embodiment of the present invention. The block diagram of the data processing device shown in FIG. This is a block diagram shown.

In FIG. 1, 10 is a main memory, and 11 is a data transfer circuit including a memory requester. A memory element 12 stores data of each processing unit, and a plurality of memory elements are provided corresponding to the spatially divided processing units. 13 is a switching circuit, 14 is a processor element, and a plurality of them are provided. Further, 15 is a controller 1-roller that controls connections between a plurality of processor elements and memory elements by the switching circuit 13, and 16 is a processor element 1.
17 is a mask processor that controls the entire hardware.

The relationship between the master processor 17 and the main memory 10 is similar to the relationship between the general-purpose processor and the main memory in the Neumann architecture, and they perform similar operations. That is, the program reads the program on the main memory, executes the read program sequentially in order, and processes the data on the main memory.

In the system configuration shown in FIG. 1, the memory element 12 appears to be an (n+1)-way interleaved local memory when viewed from the mask processor 17. The data transfer circuit 11 provides (n+1) data transfer paths to the system. These local memory functions and data transfer bus functions are managed by instructions executed by the master processor.

The controller 15 includes (n+1) processor elements 14
Manage startup and processing completion. Further, the connection relationship between each processor element 14 and memory element 12 is managed by controlling the switching circuit 13. Activation of the processing system of each processor element by the controller 15 is performed by an instruction executed by the master processor 17.

The relationship between processor element 14 and memory element 12 is
The relationship is also similar to the relationship between the general-purpose processor and main memory in the Neumann architecture.

When viewed from the mask processor 17, the instructions of the processor 14 are data.

FIG. 2 is a diagram showing an example of a memory map illustrating data allocation in each memory element. An example of how to use each memory element 12 will be described with reference to FIG. In FIG. 2, 20 is an area for storing instructions of the processor, 21 is an area for holding the state of neurons in the previous stage,
Reference numeral 22 denotes an area that holds connection weights between neurons in the previous stage and neurons in the processing target stage. The connection weight area 22 is stored in the form of an n×n matrix. P regions 23 similar to this joint load region 22 are secured, and this is a value identified by the parameter t. The parameter t is a value used when using a group of processors in a time-sharing manner.

24 is an area in which the processing results of the processor are written.

The operation will be explained again with reference to FIG. For example, in one calculation cycle, the processor element PE calculates the state of the neuron to be processed with the i number. Assume that the switching circuit 13 couples the processor element PEL and the memory element M at a certain timing. At this time, in the processor element PEL, the previous stage neuron part i and the connection weight part (i, i) are integrated. Letting the number of neurons in the previous stage be N, divide this into n parts and assign numbers 1, 2, . . . to each part. '3. ..., n, and this number designates a population of neurons. At the next timing, when the processor element PEi and the memory element ML+□ are coupled, the previous stage neuron section (i+1) and the coupling weight section (i, i+1) are integrated. Similarly, below,
The connection relationship between the processor element PE and each memory element is sequentially switched, and the memory element Mn is followed by the memory element M.
. , processor element PE and memory element Mi
Similar calculations are repeated until -2 is connected.

In this way, the state of the second processing target of number i is calculated in the processor element PEi.

If the number of neurons in the neuron layer to be processed is larger than the number of processors, 1 is added to the value obtained by dividing the number of neurons by the number of processors, and this number is set as P. According to the connection between the aforementioned processor element and memory element P times, Repeat the process. This performs calculations for all neurons. Such time-sharing control of each processor element is performed by each processor element 14 from the controller 15. This is done by instructions to the switching circuit 13. Instructions for time-sharing control by the controller 15 are defined by a program processed by the mask processor 17 under the control of the mask processor 17 .

When multiple neuron processes are assigned to the mountable number of processor elements, a fraction occurs unless the number of neurons is an integral multiple of the number of processors.

In order to appropriately perform this fractional neuron processing in the processor element 14, mask information in the mask storage section 16 provided corresponding to each processor element 14 is used. The mask information is generated and set by the controller 15, and is held and used in the mask storage section 16.

In the data processing device configured in this manner, the master processor 17 executes the following instructions to perform neural network processing. That is, (1) processor startup instructions, (2) controller operation instruction instructions, (3) data transfer instructions between main memory and local memory (memory elements), and (4) instructions executed using resources within the master processor. (normal data processing instructions), the processing proceeds by executing each instruction.

The processing of each command in items (1) to (3) will be described below with reference to FIGS. 3 to 5.

FIG. 3 is a block diagram showing the circuit configuration of main parts of the controller and data transfer circuit. The block diagram of FIG. 3 shows the main memory 10. Master processor 17
．． Controller 15. 2 is a circuit diagram showing the circuit configuration of a part of the memory 12° and the data transfer circuit 11, and is a circuit diagram showing the circuit configuration of a main part that performs memory request processing. FIG. In the block diagram of FIG. 1, the data line for reading instructions of the mask processor from the main memory 10 and the data line for reading operand data are shown separated, but the circuit diagram in FIG. In this figure, the data lines for reading out instructions are omitted.

In FIG. 3, an instruction read from main memory 10 is set in register 30 through path 10a.

The opcode part of the instruction set in the register 30 is decoded by the decoder 31 and distinguished into the instruction system of item number (4) mentioned above and the other instruction systems. The identification signal from the decoder 31 passes through the path 31a and controls the switching circuit 32, sending out the instruction system of item number (4) onto the path 32a, and sending out the other instruction systems onto the path 32bJ:.

Therefore, the instructions stored on the register 33 are the instructions of the above-mentioned item numbers (1) to (3).

Here, it is assumed that this series of instructions is composed of three fields: ○P, R1, and R2. The fields of this instruction may be augmented. In FIG. 3, register 33
The logic unit centered around is a logic unit that belongs to the controller 15 (FIG. 1). The operation code section OP of the register 33 is decoded by the decoder 34. Depending on the type of instruction, the switching circuits 35 and 36 are controlled by the decoded output of the decoder 34, and the data in the R1 operand part and the R2 operand part are sent to registers 37 to 4, respectively.
Set to 0. The logic section shown below the registers 37 to 40 is the logic section of the data transfer circuit 11 (FIG. 1). Register 37 holds the start address of main memory access, and register 38 holds the end address of main memory access. In addition, the register 39 is stored in local memory (
The register 40 holds the start address of local memory access (memory element), and the register 40 holds the end address of local memory access. Instructing data transfer between the main memory and the local memory requires 4 operand data, so in the processing of the logic section in FIG. 3, 4 operands are set in two instructions to instruct data transfer. Thereafter, when the decoder 34 detects a subsequent instruction to actually start data transfer, the signal value on the path 34a becomes LL I If and the register 4
The value is set to 1°42. The signal value on path 34a is held until the instruction is completed. Also, decoder 3
4 detects an instruction to set a value in the register 37 or 38, the signal value on the path 34b is set to "1" for one machine cycle.
''. The address data of registers 37 and 38 are transferred to adders 44 and 38 by the signals of paths 34 and 34b, respectively.
45 to registers 41 and 42. The data length of the access unit of the main memory or local memory area is set in the register 43, and the adder 44.
The data length of the access unit is added by 45, and the main memory access address and memory access address to be accessed next are respectively generated, and the registers 41 and 42
is set to

These two access addresses are applied via paths 41a, 42a to comparators 46.47 and are compared by comparators 46.47 with the end address of register 39.40. As a result of the comparison, if the main memory access address matches the end address, a signal value "1" is sent onto the path 46a. This signal value is inverted by inverter 48 and sent onto path 48a. The signal on path 48a is sent to decoder 34, which converts the signal value on path 34a to I
f Drop into OIf. Also, from the register 41 to the path 41a
The main memory access address sent via the main memory 10 is sent to the main memory 10. At this time, the signal on path 48a indicates whether the address data is valid or invalid.

The local memory address set on register 38 is first set in register 42 through adder 45 if the instruction on register 33 is a data transfer start instruction. Then, from the next machine cycle onwards, the local memory access address on the register 42 and the address of the data length of the access unit on the register 43 are added to generate an access address. The set signal for register 42 is connected to the signal on path 34a and the signal on path 60a.
This is a logical AND with the above signal. The generated local memory access address is set in the register 42 and sent from the register 42 onto the path 42a. path 42a
The address sent out above is supplied to a comparator 47,
It is compared with the end address of register 40. The comparison result is inverted by an inverter 49 and output. Inverter 4
When the signal value on the path 49a of the output of register 9 is L (I II, it indicates that the generated local memory access address is valid, and when it is OII, it indicates that the address is invalid. The address on the output path 42a is supplied to the memory element (local memory) 12 via the shifter 50, and the address ignoring the lower bits 1 to n pits of the number of memory implementations is the address on the memory element 12.
This is the address to access.

With such an access address, main memory 1° and memory element 12 are accessed, and data read from main memory 1o is written to memory element 12 through path 19. Note that during this writing, the signal on the path 60a is used as a write enable.

On the other hand, the signal on the path 48a supplied from the comparator 46 via the inverter 48 is also used as a trigger for reading the next instruction from the main memory, and is supplied to the control logic section 10b of the main memory 10. . Note that the function of the control logic unit 10b that performs the process of reading instructions from the main memory is the control logic of a normal storage unit, and is not directly related here, so it will not be specifically explained.

By the way, when the command on the register 33 is a controller operation instruction command, the data in the R1 operand section is sent onto the path 61 under the control of the switching circuit 35. The operand pig sent to path 61 is the fourth
The signal is sent to the main circuit of the controller 15 shown in FIG. 4A and FIG. 4B. Note that the path 61 here is the east line.

Further, when the instruction on the register 33 is a processor startup instruction, when the decoder 34 decodes the instruction, a signal is sent from the decoder 34 onto the path 62.

Signals on path 62 are provided to each processor element 14.

FIGS. 4a and 4b are logic circuit diagrams showing the circuit configuration of the main part of the control signal generation section of the controller.

The logic circuit diagram in FIG. 4a is a logic circuit diagram showing the circuit configuration of the control signal generation section of the controller 15 that generates a control signal to the switching circuit 13 that controls which connection relationship between the processor element 14 and the memory element 12 in FIG. 1. It is a circuit diagram. Further, FIG. 4b is a logic circuit diagram showing a circuit configuration of a control signal generation unit of the controller 15 that generates control information (mask information) sent to the mask storage unit 16 that controls invalidation/validity of the operation of the processor element 14. It becomes.

In FIG. 4a, data is set in register 52 indicating the number of phases on path 61a. Here, the number of phases refers to the number of connections between the processor element and the memory element, as shown in the operational diagram of the switching circuit in Figure 5.
ction ) is the number of time intervals for switching. In this embodiment, this number of phases is made equal to the number n of processors. In FIG. 4a, the adder 53 outputs a value obtained by adding the value of the register 54 and IIIII II every machine cycle, and sets the value in the upper register 54 of the register 33. The data sent out from the register 54 is compared with the data in the register 52 by a comparator 55. When both data match, the signal value on the path 55a becomes u I II, and the register 54 is reset. Therefore, the value of the register 54 changes so as to cycle through values between O and the number of phases n. A decoder 56 decodes the value of the register 54 and sends out a "1" on the corresponding signal line of the path 60 (east line), for example, signal line 60b, for one machine cycle. Furthermore, the path 54a that sends out the signal from the register 54 is connected to the switching circuit 13 (the first
(Figure). The switching circuit 13 is controlled by this signal.

FIG. 4b is a logic circuit diagram showing a circuit configuration of a control signal generation section of the controller 15 that generates mask information for controlling whether or not the operation of the processor element 14 is disabled or enabled. In this logic circuit, which processor element 14 (
When it is indicated whether to enable or disable the output of FIG.
7, decoded by a decoder 58, and sent out via a path 59. The decoded output sent from the path 59 is set and held in the mask storage section 16 (FIG. 1) as mask information.

Next, the operation of the processor element will be explained, including the control of invalidation/validity of the processor element based on the mask information set in the mask storage section.

FIG. 6 is a block diagram showing the circuit configuration of main parts of the processor element.

This will be explained with reference to FIG. In the processor element 14, when an instruction is set in the register 70, the operation code part ○P is decoded by the decoder 71, and the operand part R is decoded by the decoder 72. Decoder 71
The decoded output is sent to the arithmetic unit 73 and the memory requester 74 via the path 71a. Detailed operations of the arithmetic unit 73 and memory requester 74 are defined by the decoded output signal.

The decoder 72 decodes data such as register numbers in the operand portion R of the instruction. The output of the decoder 72 is a path 72a
, 72b, the selector 76 and the switching circuit 77 are controlled, respectively, and the connection relationship between the register 78 specified by the operand, the arithmetic unit 73, and the memory requester 74 is controlled. The arithmetic processing here assumes a register-register arithmetic architecture. Furthermore, the signal on the path 73c output from the decoder 72 is
It is issued when instruction processing enters the final stage, and requests the memory requester 75 to read the next instruction. The memory requesters 74 and 75 will issue access requests to the memory elements 12 connected via the switching circuit 13. If the access request is a fetch operation, fetch operand data is sent from the memory requesters 74 and 75 to each of the paths 74a and 75a.

The command is sent to the switching circuit 77 and register 70.

On the other hand, the mask information in the mask storage section 16 set by the controller 15 acts on the decoder 71 to invalidate the processing of the command. That is, the order sent to the arithmetic unit 73 and memory requester 74 is invalidated =39-.

Next, software for operating the hardware of the data processing apparatus configured as described above will be described.

Referring again to FIG. Since the relationship between the mask processor 17 and the main memory 10 is the same as that of a conventional sequential computer, data and instructions are placed in the main memory 10 and processing is executed. The data given here are the states of neurons in the neural network, data on connection weights between neurons, and program data on operation functions of neurons. Memory element 12 is a single local memory from the perspective of mask processor 17.

Therefore, by using the memory requester of the data transfer circuit 11 to transfer the data in the main memory 10'2 to the local memory and starting up the processor elements 14 via the controller 15, each processor element will be activated by the neural network. Simulate behavior.

If the above operation is shown using a programming language, for example, -40= CALL DATATRNSF(X, "O")C
ALL 5TARTPROC (“O”). In this procedure DATATRNSF, area
Indicates that the data is stored starting from the address. Then, procedure 5TARTPR○C instructs the controller 15 to start processing from address O in the local memory.

Since the information regarding the division of the neural network is to set a value in a register in the controller 15, CALLS ETUP (Y, re
gjstor number). In this procedure 5ETUP, Y is an area where set-up information (register set information) taken on the main memory is stored.

CALL DATArRNsF(“q”, Z) reads the processing result stored in the local memory (memory element 12) onto the main memory 10. where q is the location of the result stored by processor element 14 on memory element 12 and 2 is the location of the result stored on memory element 12 by processor element 14;
This is the area above. Therefore, processing of these procedures can be easily performed symbolically using variables or arrays Z.

The present invention has been specifically explained above based on examples, but
It goes without saying that the present invention is not limited to the embodiments described above, and can be modified in various ways without departing from the spirit thereof.

〔Effect of the invention〕

As described above, according to the data processing device of the present invention, a neural network of an arbitrary size is divided into sub-neural networks, and connections between neurons in one layer of the network are divided into one or more sub-neural networks. It can be mapped to processor elements and processed.

Furthermore, the results of the divided and executed processing are processed sequentially,
It can be used in combination with techniques such as symbol processing.

In other words, neural network processing can be viewed like a subroutine in a language, and can be seen as realizing a single function.

The present invention solves problems that have been pointed out in neural network processing, such as the insufficiency of methods for dividing and processing combinatorial structures using memory.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of a data processing device according to an embodiment of the present invention, FIG. 2 is a diagram showing an example of a memory map explaining data allocation in each memory element, and FIG. is a block diagram showing the circuit configuration of the main part of the controller and the data transfer circuit, FIGS. 4a and 4b are logic circuit diagrams showing the circuit structure of the main part of the control signal generation section of the controller, and FIG. Figure 6 is a block diagram showing the circuit configuration of the main part of the processor element; Figure 7 is a diagram explaining the concept of a perceptron; Figure 8 is a hierarchical neural network system. Figure 9 is a block diagram explaining the concept of the system configuration of a divided neural network. Figures 10 and 11 are diagrams showing the interaction between hierarchical and divided sub-neural networks. FIG. 3 is a diagram for explaining the action using a matrix. In the figure, 10...main memory, 11...data transfer circuit, 1
3. Switching circuit, 14... Processor element, 1
5... Controller, 16... Mask storage section, 17
...mask processor.

Claims (1)

[Claims] 1. Each layer of the neural network is divided into a plurality of processor elements, a plurality of memory elements, and each group obtained by dividing the plurality of processor elements and the plurality of memory elements two-dimensionally in time and space. It is equipped with a control management unit that controls information processing performed by processor elements and memory elements in each group by matching processing, and maps neural network processing to multiple processors to perform information processing using a multilayered neural network algorithm. A data processing device characterized by performing the following. 2. In the data processing device that performs information processing using a neural network algorithm according to claim 1, the control management section divides the multilayered neural network processing, and separates each neural network processing from a plurality of processor elements and a plurality of processor elements. A data processing device characterized in that association of memory elements with processing elements is performed temporally or spatially divided. 3. a main memory section, a sequential processing processor section, one or more neuron processing processor sections that simulate the operation of neurons, and a neuron data storage section that holds data and instructions of each neuron processing processor;
a first logic unit that cyclically sequentially changes the connection between each of the plurality of neuron processing processors and each neuron data storage unit; and an invalidation information storage that holds control information for disabling the operation of the neuron processing processor. and a second logic unit that transfers data between a plurality of neuron data storage units and a main memory unit, and maps neural network processing to multiple processors to perform information processing using a multilayered neural network algorithm. A data processing device characterized by performing the following.