JPH05197707A - Information processing system - Google Patents

Abstract

(57) [Summary] [Purpose] To realize an information processing system composed of a neural network capable of high-speed learning. A semiconductor integrated circuit 1050 connected by data buses 1080 to 1084 and a first memory for storing input values
Second memory 1010 for storing a desired value of output
Memory 1012 and a third memory 1 for storing output values
011 and a control circuit 1003 for controlling them, the semiconductor integrated circuit has a plurality of functional blocks for calculating a neuron model, one of the functional blocks being a data bus, It has means for transferring data to at least one of the functional blocks, and means for correcting the weight value between the neuron models with respect to a desired value of a plurality of input values and outputs corresponding to those input values. .

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to voice recognition, image processing,
The present invention relates to an information processing system capable of learning a neural network used for control etc. at high speed. In particular, the present invention relates to an information processing system that is convenient to construct on a wafer scale integrated circuit (WSI) and provides a large-scale neural network with a high-speed learning function in a compact size and at a low price.

[0002]

2. Description of the Related Art Neural networks are applied to information processing such as recognition and knowledge processing. Neural networks can self-organize by learning if they are given inputs and expected output values. Therefore, it does not require a program and its application is expected.

One of the learning algorithms is the parallel distributed processinning by Rumelhart et al.
g) Chapter 8, pages 318-362 and Nikkei Electronics 1987.8.1
0 (No.427) "Backpropagation" introduced on pages 115-121 is known. The back propagation will be described below.

FIG. 2 shows a model of a neuron that constitutes a neural network. A neuron inputs the input x _i from another neuron. Then, each input x _i is weighted by the weight value w _ji , and the internal energy (inner product) u _j , which is the sum thereof, is calculated. This value is converted by, for example, a sigmoid function f, and the output x _j is output. Here, the weight value w _ji is a weight value for the input neuron i in the neuron j.

[0005]

[Equation 1]

[0006]

[Equation 2]

A neural net can be constructed by connecting a plurality of neurons. Then, the weight value w
Various information processing can be performed by changing _ji .

FIG. 3 shows a hierarchical neural network. Although this figure is composed of three layers of an input layer, an intermediate layer, and an output layer, it may be configured to have a plurality of intermediate layers. A mode in which information is processed and outputted in response to an input, and an expected output value for a given input is given from the outside, and a weight value w _ji is given from the output layer side to the input layer side. It has a mode that works in the opposite direction to correct. The modified value Δw _ji of the weight value w _ji at that time is set in the neuron j.

[0009]

[Equation 3]

[0010] Here, n is the number of learnings, α, η
Is a constant. Further, the function δ is determined as follows.

In the neuron k in the output layer, if the expected output value is t _k ,

[0012]

[Equation 4]

In the middle layer neuron j,

[0014]

[Equation 5]

And decide. Here, the function f ′ is a derivative of the function f.

The correction value Δw is determined as described above, and the weight value w is corrected. Then, the above learning is repeated until the output of the output layer is as expected.

In general, research on the elucidation of a neural network and its application is carried out by software simulation on a sequential computer. However, in order to learn a large-scale neural network composed of a large number of neurons, for example, thousands or tens of thousands of neurons, enormous calculation time is required. Also, in general, learning often requires a large number of iterations. Therefore, at present, the capabilities of large-scale neural networks are not well understood.

On the other hand, there is an attempt to calculate a neural network at high speed by using dedicated hardware. One of them, "Basic design of fully digital neuro WSI",
Proceedings of the 1989 IEICE Spring National Convention 7
Figure 4 shows the method announced on pages 291 to 292 of SD-1-5. Here, the WSI is a wafer scale integration.

In FIG. 4, 201 is a neuron calculation unit and 210 is a bus. Each neuron calculation unit 20
1s are connected by a bus. One of the neuron calculation units 201 is selected, and its output value is output to the bus 210
To one after another. Each neuron calculation unit 201
Holds the weight value for the output neuron in the memory, weights the value sequentially input from the bus 210 from the memory, weights the value, and cumulatively adds the products. When outputting to the bus 210, the value is converted by the sigmoid function of the equation (1) and output.
The weighting circuit operates in a time division manner. If each neuron calculation unit 201 outputs all to the bus 210, everything is
(2) can be calculated. Hereinafter, this is referred to as the time division bus coupling method.

By performing the above operation, the neuron model of equations (1) and (2) can be calculated at high speed. Another feature is to implement FIG. 4 on WSI. According to the time division bus coupling method, the weighting circuit is used for time division, and the area of one neuron calculation unit 201 can be greatly reduced. In this way, the area of one calculation unit is made as small as possible, the yield per one calculation unit is improved, and the ratio of the neuron calculation unit 201 which becomes inoperable due to a defect is reduced, thereby achieving the WSI. By using WSI, it becomes possible to realize a large-scale neural network faster and more compactly than before.

In a neural network in which neurons are completely connected, assuming that the number of neurons is N, the number of times of weighting required to perform one calculation of equations (1) and (2) is about N ^2. Becomes In the time-division bus coupling method, N weighting circuits operate simultaneously, so the calculation time is O
(N), and a large-scale neural network can be calculated at high speed.

[0022]

In the time-division bus coupling method described in the prior art, when the above-mentioned back propagation is attempted, the function δ is calculated in the reverse equations (3) to (5). In order to calculate, the weight value at the output destination of the neuron is necessary. For example, in order to calculate δ _j of the neuron j in the intermediate layer, the weight value w _ki for the neuron j in the neuron k in the output layer is required.
However, the calculation unit 201 that calculates the neuron j as described above uses the weight value w _ji for the neuron i in the input layer.
Only hold it. Therefore, equation (5) cannot be calculated in parallel, and δ _k w _kj must be calculated one by one. Therefore, although the time required to perform the forward calculation is O (N), the time required to perform the backward calculation is O (N ² ), and there is a problem that the learning cannot be performed at high speed.

The neural network dedicated hardware described in the above prior art does not have a function for performing high-speed learning, and it is preferably operated as the entire information processing system by connecting it to a host computer. There is no disclosure of the point.

An object of the present invention is to realize an information processing system composed of a neural network capable of high-speed learning, and in particular to realize an information processing system combining a time division bus coupling method and back propagation. is there.

Other objects of the invention will be apparent from the detailed description below.

[0026]

One of the features of the present invention is:
A first aspect of the present invention is to configure an information processing system itself of a neural network having a learning function, and a semiconductor integrated circuit connected by a data bus and a first value storing input value.
, A second memory for storing a desired value of the output, a third memory for storing the output value, and a control circuit for controlling these, and calculates a neuron model. A means for transferring data from the functional block to at least one of the other functional blocks via the data bus, and a weight between neuron models for a plurality of input values and desired values of outputs corresponding to those input values. It has a means to correct.

Another feature is that an information processing system of a neural network having a learning function is constructed by using a wafer scale integrated circuit.

Another feature is that an information processing device of a neural network having a learning function is connected to a host computer, and is controlled by the host computer while the information processing device is operating without intervening the host computer. It has a means.

Still another feature is that the neural network information processing device having a learning function has a first network for performing forward calculation and a second network for performing reverse δ calculation, and each network Keeps the weight values so that it is convenient to perform each calculation in parallel,
These corrections are performed by the respective networks, and the information processing system is configured so that the correction results have the same weight value.

[0030]

INDUSTRIAL APPLICABILITY The present invention is suitable for an information processing system using a dedicated computer such as a backpropagation learning algorithm that performs high speed learning of a neural network that performs forward and backward calculations. In the present invention, the equations (1) and (2) are calculated in the forward network, and the equations (4) and (5) are calculated in the backward network. The weight value between the neurons is held at the same value in the two networks. Further, both the values calculated in each network are necessary as the values necessary for calculating the corrected value of the equation (3). In the present invention, by providing the communication means between the two networks, it is possible to obtain the value necessary for the unit forming each network to correct the weight value, and set the weight value to the same value for each unit. It is possible to modify. In the forward network,
Conveniently constructing equations (1) and (2) in parallel,
The reverse network is conveniently configured to calculate equations (4) and (5) in parallel. The calculation of the equation (3) is also performed in parallel in each unit of each network. Therefore, if the present invention is used,
It can be executed in the calculation time of (N).

Further, since the calculation in the forward direction and the calculation in the reverse direction are calculated in different networks, each calculation unit can be realized in a small area. Therefore, in the case of WSI,
The yield per computing unit can be increased.

Further, the calculation unit for performing the calculation in the forward direction and the calculation unit for performing the calculation in the reverse direction can have the same circuit configuration, and the number of design steps can be reduced. At that time, which calculation unit is to be operated depends on the control signal. Therefore, W
Even in the case of SI, it is sufficient to design and manufacture one type of WSI, so that the cost can be reduced. Further, by preparing a plurality of WSIs and connecting them, a larger-scale neural network can be easily constructed.

[0033]

EXAMPLES The present invention will be described in detail below with reference to examples.

FIG. 1 is a diagram showing an embodiment of the present invention.
In the figure, 120F, 121F, 120B, 121B are calculation units,
23 and 124 are control circuits. Each calculation unit has a control circuit 12
3 or connected to the control circuit 124. Reference numeral 130 is a control signal generation circuit that outputs control signals to the control signal lines 142 to 145. Each of the control circuits 123 and 124 inputs a control signal from the control signal line 142. 140 is an output bus. Reference numeral 128 denotes a buffer, which outputs the output value of each calculation unit to the output bus 140 or separates it from the output bus 140, and controls by the control circuits 123 and 124. Reference numeral 131 is a function conversion circuit, which converts the value input from the output bus 140 using the sigmoid function f of the equation (1) and outputs it. A function conversion circuit 132 converts a value input from the output bus 140 according to the derivative f ′ of the sigmoid function f and outputs the converted value. Reference numeral 125 denotes a selector that outputs one of the output of the output bus 140 or the function conversion circuit 131 or the output of the function conversion circuit 132 according to the control signal of the control signal line 145. 160 is a multiplier, which is a constant η with respect to the output of the selector 125.
Multiply and output. 127 is a selector and a control signal line 144
Output of selector 125 or multiplier 1 depending on the control signal of
Output 60 outputs. Reference numeral 150 is an external input terminal and 141 is an input bus. 126 is a selector, which outputs the selector 127 or the external input terminal 150 according to the control signal of the control signal line 143.
The signal input from is output to the input bus 141. Each calculation unit is connected to the input bus 141 and can input a signal output to the input bus 141. Reference numeral 151 denotes an external output terminal, which outputs a value output to the input bus 141.

FIG. 1 shows the configuration of the present invention for performing the operations of the expressions (1) to (5) at high speed. Details of each calculation unit will be described later with reference to FIGS. 7, 8 and 9. FIG. 5 shows the operation of the configuration of FIG. 1 in detail. First, FIG.
The operation of the present invention will be described below with reference to the operation of the neuron j. The input layer may be simply configured as a layer for temporarily storing input data, or may be configured as a neuro model calculation unit as shown in FIG. The calculation units in FIG. 5 correspond to the calculation units in FIG. 1, and the calculation units 120F, 1 in FIG.
21F, 120B and 121B correspond to each of FIG. The calculation units 120F and 121F configure a forward network that performs a forward operation, and the calculation units 120B and 121B configure a reverse network that performs a backward operation. Here, the forward network is a network that performs calculations of equations (1) and (2), and the reverse network is a network that performs calculations of equations (4) and (5). The equation (3) is calculated in each network.

The calculation unit in the middle layer of the forward network holds the weight value w _ji for the neuron i in the input layer, and the calculation unit in the output layer holds the weight value w _kj for the neuron j in the middle layer. ing. The calculation unit 120F in the middle layer holds the weight value w _ji . Also, in the calculation unit in the middle layer of the reverse network,
The output layer calculation unit in the forward network holds the weight value w _kj for the neuron j in the intermediate layer, and the input layer calculation unit in the backward network calculates the intermediate layer in the forward network. Weight value w that the unit has for neurons in the input layer
holding _ji The calculation unit 120B in the middle layer holds the weight value w _kj . Weight value w _ji forward network held, and w _kj, weights w _ji of backward network holds, w _kj corresponds, the same value.

In the forward network, the outputs x _i , x _j and x _{k of} each calculation unit from the input layer to the intermediate layer and the output layer are determined. The calculation unit 120F inputs the output x _{i of the} input layer, weights it with the weight value w _ji , calculates the internal energy u _j , further performs nonlinear conversion, and outputs the output value x _j . Next, the calculation unit of the output layer of the forward network inputs the output x _j of the _hidden layer, weights it with the weight value w _kj , and outputs the internal energy u j.
_k is calculated, nonlinear conversion is further performed, and the output value x _k is output. Each computing unit of the forward network sends a respective output value x _i , x _j , x _k to the corresponding computing unit of the reverse network. Calculation unit 1
20F sends its output value x _j to the calculation unit 120B. Next, each calculation unit of the middle layer and the output layer of the forward network outputs to each corresponding calculation unit of the reverse network the derivative x _j ′ of each output value x _j , x _k ,
send x _k '. The calculation unit 120F outputs the derivative x _j '(= f' (u _j )) of the output value x _j to the calculation unit 120B. Then, if the output value x _k of the output layer of the forward network is different from the expected value t _k , the weight value needs to be modified.
The operation will be described below.

At the output layer of the reverse network,
According to the equation (4), δ _k is calculated and output to each calculation unit in the intermediate layer of the backward network. The x _k ′ necessary for calculating the equation (4) has already been input. In the backward network, each calculation unit in the middle layer inputs δ _k ,
Using the value, the correction value Δw _kj is calculated according to the equation (3), and the weight value w _kj is changed. Further, each calculation unit in the output layer of the reverse network sends the respective output value δ _k to the corresponding calculation unit in the output layer of the forward network. Thereby, the calculation unit of the output layer of the forward network uses the modified value Δw _{kj according to the} equation (3).
And the weight value w _kj can be changed.

Next, the output δ _k of each calculation unit in the output layer of the reverse direction network is output, and the calculation unit in the middle layer of the reverse direction network which inputs it outputs δ _k and the weight value w.
Multiplies with _kj , determines δ _j according to equation (5), and outputs it. X _j 'required to calculate equation (5) is the calculation unit 1
I have already entered from 20F. When the calculation unit in the input layer of the reverse direction network inputs δ _j , the value is used to calculate the correction value Δw _ji according to the equation (3) and change the weight value w _ji . Also, each computation unit in the middle layer of the reverse network sends its respective δ _j to the corresponding computation unit in the middle layer of the forward network. The calculation unit in the middle layer of the forward network is
The correction value Δw _ji can be calculated by the equation (3) and the weight value w _ji can be changed.

As described above, the present invention provides (1)
Calculate x, x '(= f' (u)) in the equations (2), (4) and (5) in a forward network which is convenient for calculating it at high speed, and calculate δ in the equations (4) and (5). In the reverse network, which is convenient to compute it fast. Then, Δw in the equation (3) is calculated for each weight value w held in each calculation unit of each network. In the calculation unit of the forward network, δ is needed to calculate the correction value Δw, and in the calculation unit of the reverse network δ
X and x ′ are needed to calculate Δw and Δw. Therefore, it is necessary to communicate data between the calculation units corresponding to each network, but the amount of data to be communicated corresponds to the number of neurons, and the transfer time required for this is O (N). Can be done in a short time.

In this figure, the intermediate layer is described as one layer, but the learning can be similarly performed for a neural network composed of a plurality of intermediate layers.

Next, a method of realizing the above operation on hardware will be described with reference to FIG. FIG. 6 is a calculation flow chart showing the flow of the above operation, and shows the calculation procedure of the calculation units of the forward network and the backward network. The calculation unit j of the forward network is shown in FIG.
And the calculation unit 120F of FIG. 5 and the calculation unit j of the reverse network is the calculation unit 120B of FIG. 1 and FIG.
Is. Hereinafter, FIG. 1 will be described with reference to FIG.

Each calculation unit holds in memory the weight value for the input neuron. First, the weight value w
Write the initial value of to each calculation unit. Calculation unit 12
The weight value w _ji for the neuron i in the input layer is written in 0F. The weight value w _kj which is the same as the weight value for the neuron j in the intermediate layer held by the calculation unit k in the output layer of the forward network is written in the calculation unit 120B.
-Operation 1 Next, the selector 126 is connected to the external input terminal 150 side on the input bus 141.
And output the input value from the external input terminal 150 to each calculation unit i in the input layer of the forward network. -Operation 2 The control signal generation circuit 130 sets the selector 125 to output the output bus 140, sets the selector 127 to output the output of the selector 125, and sets the selector 126 to the selector 127.
The control signal is output to the control signal lines 143 to 145 so as to output the output. Next, the control signal generation circuit 130 outputs the address of the calculation unit i of the input layer of the forward network to the control signal line 142. The control circuit 123 decodes the address, and if the address matches the address of the calculation unit, the buffer 128 is enabled,
The output value x _i of the calculation unit i in the input layer of the forward network is output to the output bus 140. The calculation unit i in the input layer of the forward network only outputs the input value.
The calculation unit j in the middle layer of the forward network sequentially inputs the outputs x _i of the input layer of the forward network from the input bus 141, weights them with the weight value w _ji , and cumulatively adds the products. When the control signal generation circuit 130 specifies all the addresses of the calculation unit i in the input layer of the forward network, each calculation unit in the intermediate layer can perform the calculation of equation (2), and the output value u _j thereof is determined.

Next, the control signal generation circuit 130 operates the selector 125.
Is set to output the output of the function conversion circuit 131, and then the address of the calculation unit in the middle layer of the forward network is output to the control signal line 142. The addressed computing unit outputs its output value u _j on the output bus 140. The non-linear conversion circuit 131 performs the conversion of the equation (1), and x _j (=
f (u _j )) is output. x _j is broadcast by the input bus 141 to the computing unit k in the output layer of the forward network. The output layer calculation unit k is the input bus 14
The output x _{j of the} intermediate layer is sequentially input from 1 and weighted with the weight value w _kj for that, and the products are cumulatively added. When the control signal generation circuit 130 specifies all the addresses of the calculation unit j in the middle layer of the forward network, each calculation unit k in the output layer can perform the calculation of equation (2), and its output value u _k is determined. .. -Operation 3 Next, the control signal generation circuit 130 outputs the address of the calculation unit k in the output layer of the forward network to the control signal line 142. The addressed computing unit outputs its output value u _k on the output bus 140. After that, it is converted into x _k (= f (u _k )) by the non-linear conversion circuit 131, and passes through the selector 125, the selector 127, and the selector 126, and the input bus 1
It is output to 41. The output result can be read from the external output terminal 151. Then, if the output value x _k of the output layer matches the expected output value t _k , the learning ends.
If they do not match, the following operation is performed. -Operation 4 The control signal generation circuit 130 outputs the output of the selector 125 to the output bus.
It is set to the 140 side and the output of the selector 127 is set to the multiplier 160 side. The selector 127 is set on the multiplier 160 side, and the selector 126 outputs the output of the selector 127. Next, the control signal generation circuit 130 outputs the address of the calculation unit i of the input layer of the forward network to the control signal line 142. The output x _{i of the} addressed computing unit _i
Is output to the multiplier 160 through the selector 125, multiplied by a constant η, and converted into η · x _i . Then, η · x _i is output to the input bus 141 through the selector 127 and the selector 126. The calculation unit i in the input layer of the backward network takes in the value of η · x _i when it is output to the input bus 141. Next, the control signal generation circuit 130 operates the selector 1
The output of 25 is the function conversion circuit 131 side, and the selector 127
The output of is set to the multiplier 160 side. The control signal generation circuit 130 outputs the address of the calculation unit j in the middle layer of the forward network as the control signal 142. The output u _j of the address-specified calculation unit _j is the function conversion circuit 1
It is converted into x _j by 31 and passes through the selector 125 and the multiplier
It is output to 160, multiplied by a constant η, and converted into η · x _j .
Then, through the selector 127 and the selector 126, the input bus 14
Output η · x _j to 1. The calculation unit j in the middle layer of the backward network inputs the corresponding η · x _j . For example, the output value u _j of the calculation unit 120F is the function conversion circuit 1
31 and multiplier 160 convert to η · x _j and input bus 1
It is output to 41. At that time, the calculation unit 120B
-Capture x _j . -Operation 5 Next, the control signal generation circuit 130 sets the output of the selector 125 to the function conversion circuit 132 side and the output of the selector 127 to the selector 125 side. Control signal generation circuit 130
Outputs the address of the calculation unit j of the middle layer and the calculation unit k of the output layer of the forward network to the control signal 142. The output u _{j of} the addressed computing unit,
The u _k is converted into x _j ′ and x _k ′ by the function conversion circuit 132, passes through the selector 125, the selector 127, and the selector 126, and is output to the input bus 141. The calculation unit of the reverse network middle layer and the output layer can input the corresponding x _j ′, x _k ′. For example, the output value u _j of the calculation unit 120F
Is converted into a function conversion circuit 132x _j ', which is output to the input bus 141. At that time, the calculation unit 120B takes in x _j '. -Operation 6 In the next operation, the difference (t _k) between the output value t _k expected of the calculation unit in the output layer of the reverse direction network and the output value x _{k obtained} earlier is calculated.
-X _k ) from outside the network. It may be done by the host computer or (t _k −
A circuit may be provided for calculating x _k ). The calculation unit k in the output layer of the backward network calculates δ _k in equation (4). -Operation 7 Next, the control signal generation circuit 130 sets the selectors 125 to 127 so that the output of the output bus 140 is multiplied by η by the multiplier 160 and output to the input bus 141. Then, the address of the calculation unit k in the output layer of the backward network is output to the control signal 142. The output δ _k of each addressed computational unit is η multiplied by a multiplier 160 and the input bus 1
Η · δ _k is output to 41. At this time, the calculation unit k of the output layer of the forward network outputs the output η of the calculation unit k of the backward network corresponding to itself to the input bus 141.
・ If δ _k is output, capture it internally. -Operation 8 Next, the control signal generation circuit 130 causes the value of the output bus 140 to be output to the input bus 141 through the function conversion circuit 131.
After setting the selectors 125 to 127, the address of the calculation unit j in the middle layer of the forward network is output to the control signal 142. The addressed calculation unit j outputs the already calculated internal energy value u _j . as a result,
The function-converted value x _j is output to the input bus 141. In each calculation unit of the output layer of the forward network, the modified value Δw _kj of the weight value w _kj for the input value x _j
Is calculated according to the equation (3), and the weight value w _kj is corrected.
-Operation 9 Next, the control signal generation circuit 130 causes the selectors 125 to 127 to output the value of the output bus 140 to the input bus 141 as it is.
After setting, the address of the calculation unit k in the output layer of the reverse network is output to the control signal 142. The addressed calculation unit outputs the already calculated δ _k . As a result, Δ _k passes through the output bus 140, the selector 125, the selector 127, and the selector 126 and is output to the input bus 141. In each calculation unit in the middle layer of the backward network, the correction value Δw _kj of the weight value w _kj for the input value δ _k
Is calculated according to the equation (3), and the weight value w _kj is corrected.
-Operation 10 Next, the control signal generation circuit 130 specifies the address of the calculation unit k in the output layer of the reverse network. The addressed calculation unit outputs the already calculated δ _k . As a result, Δ _k passes through the output bus 140, the selector 125, the selector 127, and the selector 126, and is output to the input bus 141. In operation 10 each of the calculation units in the middle layer of the backward network uses the modified weight value w _kj ,
Δ _j is calculated according to the equation (5). -Operation 11 Next, the control signal generation circuit 130 sets the selectors 125 to 127 so that the output of the output bus 140 is multiplied by η by the multiplier 160 and output to the input bus 141. And the control signal to the address of the calculation unit in the middle layer of the reverse network.
Output to 142. The output δ _j of each addressed calculation unit is multiplied by η by the multiplier 160 and the selector 12
7. η · δ _j is output to the input bus 141 through the selector 126. At this time, if the output η · δ _j of the calculation unit j of the reverse network corresponding to itself is output to the input bus 141, the calculation unit j of the middle layer of the forward network takes it in. -Operation 12 Next, the control signal generation circuit 130 sets the selectors 125 to 127 so that the value of the output bus 140 is output to the input bus 141, and then the calculation unit i of the input layer of the forward network.
The address of is output to the control signal 142. The addressed computing unit outputs x _i . As a result, x _i is output to the input bus 141. In each calculation unit of the middle layer of the forward network, the correction value Δw _ji of the weight value w _ji for the input value x _{i is} calculated according to the equation (3), and the weight value w _ji is corrected. -Operation 13 Next, the control signal generation circuit 130 selects the selectors 125 to 127 so that the value of the output bus 140 is output to the input bus 141 as it is.
, And outputs the address of the calculation j of the intermediate layer of the reverse direction network to the control signal 142. The addressed calculation unit outputs the already calculated δ _j . As a result, δ _j is output bus 140, selector 125, selector 12
7, output to the input bus 141 through the selector 126. In each calculation unit of the input layer i of the backward network, the correction value Δw _ji of the weight value w _ji for the input value δ _{j is} calculated according to the equation (3), and the weight value w _ji is corrected. −
Operation 14 After that, the above learning is repeated for the input until the output value x _k of the output layer reaches the expected value t _k .

According to the present invention, the calculation of x and δ in the forward and backward directions operates in parallel in each layer, and the amount of data communication required for the calculation is proportional to N. Therefore, it takes O (N) time. It is possible to learn fast.

Next, details of the calculation units forming the forward network and the backward network will be described with reference to FIGS. 7 and 8.

FIG. 7 is a diagram showing a calculation unit forming a forward network. In FIG. 7, 501 is a memory for storing the weight value w _ji and αΔw _ji (n) in the equation (3). Reference numerals 503 and 504 are adders, and 502 is a multiplier. 51
1, 512 are latches. Reference numeral 520 is a multiplier for multiplying the previous correction value Δw _ji (n) in the equation (3) by α. 531 to 534 are selectors. 540 is a connection terminal to the input bus 141. 55
Reference numerals 0 to 557 are control signal lines for control signals output by the control circuit 123, which are the selector 534, the memory 501, and the latch 51.
1, selector 531, selector 533, selector 532, latch 51
2. Control the buffer 128. 541 is a connection terminal to the output bus 140. The connection relationship will be described below.

The latch 511 holds a signal input from the input bus connection terminal 540. The selector 531 inputs the weight value w _ji which is the output of the memory 501 and the output of the latch 511. The multiplier 502 inputs the output of the selector 531 and the input bus connection terminal 540. The adder 503 is the output of the multiplier 502 and the memory 50.
Input αΔw _ji (n) which is the output of 1. The selector 532 inputs the output of the multiplier 502 and the output of the adder 503. Adder
504 inputs the output of the selector 532 and the output of the selector 533. The latch 512 holds the output of the adder 504. buffer
128 inputs the output of latch 512, and output bus connection terminal 541
Output to. Selector 533 is the output of latch 512 and memory 50
The weight value w _ji that is the output of 1 is input. The multiplier 520 inputs the output of the adder 503. The selector 534 inputs the signal input from the input bus connection terminal 540 and the output of the adder 504.
The memory 501 outputs the output of the selector 534 to the weight value w _ji side, the multiplier
The output of 520 is input as write data on the αΔw _ji (n) side. Further, the memory 501 inputs the address at the time of access from the control circuit 123.

When writing the initial value of the weight value w _ji of the operation 1, the initial value is given to the input bus connection terminal 540 and the selector 5
If the output of 34 is the input bus connection terminal 540 side, the memory 501
Can be written on.

The calculation of equation (2) is performed as follows. The output of the selector 531 is set to the weight value w of the memory 501.
_ji Output side. The input value x _i input to the input bus connection terminal 540 is input. The product w _ji x _i is performed by the multiplier 502. The selector 532 outputs the multiplier 502 side. selector
The 533 outputs the latch 512 side. The adder 504 adds the output of the latch 512, which is the cumulative addition result up to that point, and the product w _ji x _i , and writes the result into the latch 512. The above operation is performed one after another, with respect to the input value x _i input to the input bus connection terminal 540, the weight value w
Equation (2) can be calculated by changing _ji .

The ηδ _j input from the reverse network is input from the input bus connection terminal 540 and written in the latch 511.

The calculation of equation (3) is performed as follows. The selector 531 outputs the latch 511 side. The input value x _i input to the input bus connection terminal 540 is input. The multiplier 502 calculates ηδ _j x _i . The adder 503 adds the output αΔw _ji (n) of the memory 501 and the output ηδ _j x _i of the multiplier 502 to calculate a corrected value Δw _ji (n + 1). The selector 532 outputs the adder 503 side. Selector 53
3 outputs the weight value w _ji output side of the memory 501. Adder 50
4 adds the output [Delta] w _ji of the previous weight value w _ji of the output of the selector 533 (n) and a selector 532 (n + 1), the weight value w _ji that fixes
Output (n + 1). The selector 534 outputs the adder 504 side. Further, the output Δw _ji (n + 1) of the adder 503 is multiplied by α by the multiplier 520. Then, the corrected weight value w _ji (n + 1) of the output of the selector 534 and the output αΔw of the multiplier 520 are stored in the memory 501.
Write _ji (n + 1). All the weight values w _ji (n + 1) can be corrected by successively performing the above operation for the input value x _i input to the input bus connection terminal 540.

FIG. 8 is a diagram showing a calculation unit forming a reverse network. In FIG. 8, 601 is a memory for storing the weight value w _kj and αΔw _kj (n) in the equation (3). Reference numerals 603 and 604 are adders, and 602 is a multiplier. 611
~ 615 is a latch. Reference numeral 620 is a multiplier for multiplying the previous correction value Δw _kj (n) in the equation (3) by α. 631-634,636 are selectors. 640 is an input bus connection terminal. 650 ~
Reference numerals 659, 661, and 662 are control signal lines for control signals output from the control circuit 124. Each of them is a selector 636, a latch 613, a latch 612, a latch 611, a memory 601, a selector 631, and a selector 631.
632, selector 634, selector 633, latch 614, latch 61
5, control the buffer 128. 641 is an output bus connection terminal. The connection relationship will be described below. The latches 611 to 613 input the signal input from the input bus connection terminal 640. The selector 631 inputs the weight value w _kj output of the memory 601, the output of the latch 611, and the output of the latch 612. The selector 632 inputs the signal input from the input bus connection terminal 640, the output of the latch 613, and the output of the latch 614. Multiplier 602 is selector 63
Input the output of 1 and the output of selector 632. The adder 603 inputs the αΔw _kj (n) output of the memory 601 and the output of the multiplier 602. The selector 633 inputs the output of the adder 603 and the output of the multiplier 602. The adder 604 inputs the output of the selector 634 and the output of the selector 633. The latch 614 inputs the output of the adder 604. Selector 634 is the output of latch 614 and memory 60
The weight value w _kj output of 1 is input. The multiplier 620 is an adder
Input the output of 603. Selector 636 is an input bus connection terminal
The signal input from 640 and the output of the adder 604 are input. The memory 601 outputs the output of the selector 636 to the weight value w _kj side and the multiplier 62.
The output of 0 is input as write data on the αΔw _kj (n) side. Further, the memory 601 inputs the address at the time of access from the control circuit 124. The latch 615 inputs the output of the multiplier 602. The buffer 128 inputs the output of the latch 615 and outputs it to the output bus connection terminal 641.

When writing the initial value of the weight value w _kj , the initial value can be given to the input bus connection terminal 640 and the output of the selector 636 can be written on the input bus connection terminal 640 side.

The η x _j input from the reverse network is input from the input bus connection terminal 640 and written in the latch 611. x _j '(= f' (u _j )) is also input from the input bus connection terminal 640 and written in the latch 612.

Further, the difference (t _k −
x _k ) is input from the input bus connection terminal 640 and written in the latch 613. The output layer calculation unit performs the following operations to calculate δ _k .

The selector 631 outputs the latch 612 side. The selector 632 outputs the latch 613 side. The multiplier 602 multiplies the output value x _k ′ (= f ′ (u _k )) of the latch 612 and the output value (t _k −x _k ) of the latch 613, and outputs δ _k of the equation (4).
Latch 615 takes the output δ _k of multiplier 602 and outputs it to buffer 1
Output to 28. Δ _k is output to the output bus connection terminal 641.

In the calculation unit of the output layer, δ _{j in} equation (5)
Perform the following operations to calculate

The selector 631 outputs the weight value w _kj side of the memory 601. The selector 632 outputs the input bus connection terminal 640 side. The output δ _{k of the} output layer is input to the input bus connection terminal 640. The multiplier 602 outputs the product δ _k w _kj . selector
633 outputs on the multiplier 602 side. Selector 634 has latch 614
Output side. The adder 604 adds the output of the latch 614, which is the cumulative addition result up to that time, and the product δ _k w _kj , and the latch 614
Write in. Input bus connection terminal 64
For input value δ _k input to 0, Σδ _{k in} Eq. (2)
w _kj can be calculated. Next, the selector 631 outputs the latch 612 side. The selector 632 outputs the latch 614 side. The multiplier 602 multiplies the output x _j '(= f' (u _j )) of the latch 612 and the output Σδ _k w _kj of the latch 614 and outputs δ _j . The latch 615 takes in the output δ _j of the multiplier 602 and outputs it to the buffer 128. Δ _j is output to the output bus connection terminal 641.

The calculation of the equation (3) is performed as follows in the intermediate layer of the backward network. selector
631 outputs the latch 611 side. The input value δ _k to be input to the input bus connection terminal 640 is input. By the multiplier 602, η
Calculate x _j δ _k . By the adder 603 performs addition of the output ηx _j δ _k of the output of the memory 601 αΔw _kj (n) and the multiplier 602 calculates a correction value Δw _kj (n + 1). The selector 633 outputs the adder 603 side. The selector 634 outputs the weight value w _kj output side of the memory 601. The adder 604 determines the previous weight value w _kj (n) of the output of the selector 634 and the output Δw of the selector 633.
_kj (n + 1) is added and the corrected weight value w _kj (n + 1) is output. The selector 636 outputs the adder 604 side. The output Δw _kj (n + 1) of the adder 603 is multiplied by α by the multiplier 620. Then, the corrected weight value w _kj (n + 1) of the output of the selector 636 and the output αΔw _kj (n + 1) of the multiplier 620 are written in the memory 601. All the weight values w _kj (n + 1) can be corrected by successively performing the above operation on the input value δ _k input to the input bus connection terminal 640. The weight value w _ji can be similarly corrected in the input layer of the backward network.

As described above, according to the present invention, the calculation unit of the output layer of the forward network and the calculation unit of the intermediate layer of the reverse network have the same weight value w _kj (n), but the modification is to adjust the weight value. Computation unit with has each done. The corrected weight value w _kj (n + 1) becomes the same value. Similarly, the calculation unit in the middle layer of the forward network and the calculation unit in the input layer of the reverse network have the same weight value w _ji (n), but the correction is performed by each calculation unit having that weight value. The corrected weight value w _ji (n + 1) becomes the same value.

Next, FIG. 9 will be described. FIG. 9 shows the calculation unit of FIG. 8 with a selector 635 and a control line 660 for controlling the selector 635 added. 122 is a control circuit. The selector 635 inputs the output of the latch 614 and the output of the multiplier 602. Latch 61
5 inputs the output of the selector 635. Others are the same as those in FIG. If the output of the selector 635 is on the multiplier 602 side, the same operation as in FIG. 7 can be performed, and the calculation unit of the reverse network can be realized. Further, the output of the selector 635 is set to the latch 614 side, and the latch 51 shown in FIG.
1, selector 531, multiplier 502, adder 503, selector 53
2, adder 504, latch 512, selector 533, multiplier 520,
The selector 534, the memory 501, the latch 611 and the selector 6 in FIG.
31, multiplier 602, adder 603, selector 633, adder 604,
If it corresponds to the latch 614, the selector 634, the multiplier 620, the selector 636, and the memory 601, then it can be used as a forward network calculation unit. That is, in order to carry out the present invention, it is not necessary to design the two calculation units shown in FIGS. 7 and 8, but the calculation unit shown in FIG. 9 may be designed. In such a case, the design man-hour can be reduced significantly.

As described above, according to one embodiment of the present invention, since it can be realized on an integrated circuit, it is possible to realize a neural network for learning at high speed.

Next, FIG. 10 will be described. In FIG. 10, 1 and 2 are wafers, and 700 is a functional block. 710-71
2 is a selector. Reference numeral 701 denotes a control signal generation circuit, which has a function of controlling the selectors 710 to 712 added to the function of the control signal generation circuit 130 of FIG. 720 is an input terminal and 730 is an output terminal. 721 to 725 are control signal lines. The functional block 700 can be configured in FIG. The network in wafers 1 and 2 corresponds to FIG. The selector 711 inputs the signals input from the external output terminal 151 and the input terminal 720 of the wafer 1 in FIG. 1 and outputs the signals to the external input terminal 150 of the wafer 2 in FIG. The selector 710 inputs the signal input from the input terminal 720 and the external output terminal 151 in FIG. 1 of the wafer 2 and outputs the signal to the external input terminal 150 in FIG. 1 of the wafer 1. The selector 712 inputs the external output terminal 151 of the wafer 1 and the external output terminal 151 of the wafer 2 and outputs them to the external output terminal 730. The control signal generation circuit 701 outputs control signals to the selectors 710 to 712 through the control signal lines 723 to 725. Further, the control signal lines 721 and 722 control the wafers 1 and 2. The signal on the input bus 141 of the wafer 1 can be sent to the input bus 141 of the wafer 2 through the selector 711. Similarly, the wafer 2 can be sent to the wafer 1. Further, a signal input to the external input terminal 720 can be sent to the input bus 141 of the wafers 1 and 2 through the selectors 710 and 711.

As described above, wafers can be easily connected, and a very large-scale neural network can be learned at high speed.

Further, since the wafer 1 constitutes the forward network and the wafer 2 constitutes the reverse network, the weight values in the equation (3) can be corrected simultaneously in each network, and therefore the speed can be further increased. Can learn.

Further, in this figure, the connection of two wafers is shown, but the connection of three or more wafers can be made by a simple correction.

Further, the structure of FIG. 1 may be directly formed on the wafer. Further, each network can be configured by a plurality of wafers.

Next, FIG. 11 will be described. In FIG. 11, 880 and 890 are functional blocks. 801-803,812-814
Is a selector. The function block 880 can be configured in FIG. 7 and the function block 890 can be configured in FIG. Further, the function blocks 880 and 890 can be configured in FIG. 850,8
60 is an output bus, and 851 and 861 are input buses. 160a and 160b are multipliers. In the figure, the network on the left corresponds to the forward network, and the network on the right corresponds to the reverse network. Although the control signal generation circuit and the control signal line are not shown in the figure, each functional block and each selector can be controlled by using the same configuration as that in FIGS. 1 and 10.

In the forward network, each functional block
The output of 880 is output to output bus 850. Function conversion circuit 13
1, 132 input the output bus 850, perform function conversion in each, and output. The selector 801 inputs the output of the output bus 850 and the outputs of the function conversion circuits 131 and 132, and outputs one of them. The selector 802 inputs the output of the selector 801 and the output bus 860 of the reverse network. The multiplier 160a receives the output of the selector 802, multiplies it by a constant η, and outputs it. The selector 803 inputs the output of the selector 801 and the output of the multiplier 160 and outputs them to the input bus 851. Each function block 880 can input the value output to the input bus 851. In the reverse network, the output of each functional block 890 is output to output bus 860. The selector 812 inputs the output bus 860 and the output of the forward network selector 801.
The multiplier 160b receives the output of the selector 812, multiplies it by a constant η, and outputs it. Selector 814 is the output of selector 812 and the multiplier
Input the output of 160b. The selector 813 inputs the output of the output bus 860 and the output of the selector 814 and outputs it to the input bus 861.

In the configuration of this figure, the functional block 880 forming the forward network can take in the value of the output bus 860 of the backward network multiplied by η by the multiplier 160a. Similarly, the functional block 890 forming the reverse network can take in the value of the output bus 850 of the forward network by η times by the multiplier 160b. Therefore, the operation of communicating ηx _j from the forward network to the reverse network of operation 5 in FIG. 6 can be performed in operation 3. In operation 3, when each functional block broadcasts the output x _j to other functional blocks, it is multiplied by η by the multiplier 160b at the same time,
It can be output to the input bus 861 of the reverse network. Similarly, the act of communicating ηδ from the reverse network to the forward network of acts 8 and 12 can be performed in acts 7 and 11. As described above, by stealing the output value to the bus of the other network, the time required for the operations 5, 8, and 12 in FIG. 6 can be omitted. Therefore, learning can be performed at higher speed.

Next, the neurocomputing system according to the present invention will be described. The following embodiment is one mode in which the information processing apparatus described so far is pursued to a practical level. First, the configuration is described once with reference to FIGS. 12 to 19, and then the operation and the control required therefor are described with reference to FIGS. 20 to 26. It should be noted that although there are some points that overlap with the above-described embodiments, they will be described once.

FIG. 12 is a diagram showing the entire system. In FIG. 12, 1000 is a host computer, 100
A digitizer 1 is one of the input devices of the host computer 1000. Reference numeral 1003 indicates a control board. 1030 is a WSI board, 1031 is a WSI board,
1050 is a wafer, and the wafer 1050 is the WSI substrate 10.
By bonding with WSI substrate 31 and connecting the WSI substrate 1031 to the WSI board, it is possible to supply power to the wafer 1050 and input / output signals to / from the outside. Reference numeral 1080 is a mode signal line, 1081 is an address signal line, 1082 is a clock signal line, 1083 is an input data line, and 1084 is an output data line. Control board 1003 and each W
It is connected to the SI board 1030 by an internal bus 1020 of 1080 to 1084. Control board 10
Reference numeral 1010 denotes an input layer memory, 1011 an output layer memory, 1012 a teacher signal memory, 1013 a sequence RAM, and 1002 a clock oscillator. 1021
Indicates an external bus. The host computer 1000 can access the memories 1010 to 1013 on the control board 1003 through the external bus 1021 and can arbitrarily read and write. 1085
Is a decoder, 1040 is an output buffer, and 1041 is an input buffer. The input layer memory 1010 and the teacher signal memory 1012 can each receive the address signal line 1081 and send the contents of the memory to the input data line 1083 through the output buffer 1040. Also,
Each output buffer 1040 is controlled by the decoder 1085. The decoder 1085 is the mode signal line 1
080 and the address signal line 1081 are input and decoded. The sequence RAM 1013 generates a mode signal and an address signal. Reference numeral 1086 denotes a pointer, which inputs the clock sent from the pulse generator 1002 through the clock signal line 1082 and increments the output by one. By repeating this, the address of the instruction written in the sequence RAM 1013 is designated. Sequence RAM 101
At each address of 3, the commands of the mode signal and the address signal to be executed are sequentially written in advance. Also, 10
Reference numeral 51 is a selector that sends a command output from the sequence RAM 1013 or the external bus 1021 to the mode signal line 10.
80 and the address signal line 1081. 1052
Is the output data line 1084 or the external bus 1 at the selector.
The value output from 021 is output to the input data line 1083.

The teacher signal of the output layer is written in the teacher signal memory 1012. The input layer memory 1010 serves as an input layer neuron, and the value of the input layer neuron is written therein. The output layer memory 1011 inputs the output data line 1084 and the address signal line 1081 through the input buffer 1041 and takes in the output of the output layer neuron.

The WSI board 1030 uses the mode signal line 10
80, address signal line 1081, clock signal line 108
2. The input data line 1083 is connected to the wafer 1050 through the input buffer 1041, and the signal output from the wafer 1050 is output to the output data line 1084 through the output buffer 1040 controlled by the decoder 1085. In addition, the decoder 1085 uses the mode signal line 10
Decoding is performed by 80 and the address signal line 1081. In addition, from the host computer 1000 to the wafer 10
When writing to the latch, memory, etc. in 50, it is performed when the own board address is specified.

FIG. 13 shows the structure of the wafer 1050. Wafer 1050 has 1060 N blocks and 10
61 IO blocks, 1062 B blocks, 1063
Of P blocks. The blocks are arranged at equal intervals in the vertical (column) direction and the horizontal (row) direction. The IO block 1061 is arranged on the lower side of the center row of the wafer 1050,
Seven B blocks 1062 are arranged in the other central columns. Input / output to / from the outside is performed by the IO block 1061 on the lower side. N blocks 1060 are arranged in columns other than the central column, and P blocks 1063 are arranged in the peripheral portion. The P block 1063 is for performing quality control in the semiconductor process of the wafer. The N block 1060 includes a neuron block, and the B block 1062 includes
Equipped with a function conversion table. The IO block 1061 has a function for performing input / output with the outside in addition to the function of the B block 1062. For details of the IO block 1061, the B block 1062, and the N block 1060, see FIG.
6, and FIG. 17 will be described.

FIG. 14 is a diagram showing the configuration and arrangement of signal lines in the wafer 1050. A bonding pad 1201 is provided on the lower side of the IO block 1061 and connects the signal line to the outside by a bonding wire. Each N block 1
Regarding the signal line to 060, first, in the IO block 1061 and each B block 1062, a signal is transmitted in the vertical direction,
Further, it sends a signal in each lateral direction. The output of the neuron block in the N block 1060 is first collected in the IO block 1061 and the B block 1062 in that row, and the function conversion is performed therein, and the IO block 1 in the lower side for inputting / outputting is performed.
061, and output to the outside.

FIG. 15 shows each block 1 of the wafer 1050.
Method for supplying power to 060 to 1062 and WSI substrate 10
The structure of 31 is shown. In FIG. 15, 1301 to 130
3 is a bonding pad, 1304 is a bonding wire, and 1305 is a connector. Reference numeral 1310 is a power supply bus for supplying power to the blocks 1060 to 1062. By connecting the connector 1305 to the WSI board 1030, the WSI board 103 for power and signal lines can be connected.
Connect to 1. The power supply line is bonded to the bonding pad 130.
1, 1304 through the WSI substrate 1031 and drawn to the collector 1305. Further, the signal line is pulled out from the bonding pad 1303 to the connector 1305. In the wafer 1050, the power supply buses 1310 are provided in a grid pattern between the blocks 1060 to 1062, and the blocks 1060 are provided.
1062 to power bus 1310. Power bus 1
A bonding pad 1302 is provided at the end of 310. Bonding pads 1302 on the wafer 1050 and bonding pads 1301 on the WSI substrate 1031,
By connecting 1304 with the bonding wire 1304, power can be supplied to each of the blocks 1060 to 1062. In addition, FIG.
4 bonding pad 1201 and WSI substrate 1031
By connecting the upper bonding pad 1303 with the bonding wire 1304, the internal bus 1020 and the IO block 1061 in FIG. 12 can be connected.

FIG. 16 shows the B block 1062. IO
The block 1061 has an input / output function with the outside in addition to the function of the B block 1062. In FIG. 16, 1070 is a conversion table, 1071 is a control circuit, and 1072-107.
6 is a selector. The IO block 1061 and the B block 1062 are connected to the lower side from the external or IO block 1061 or the B block 1062 from the mode signal line 1080, the address signal line 1081, and the clock signal line 1.
082 is input and is connected to the upper side of the B block 1062.
And to the N block 1060 connected to the side.
The input data line 1083 is input from the lower side, and the selector 1
073, the conversion table 1070, the selector 1074, and the selector 1076 have a path for outputting to the left and right sides. Also, without passing through the conversion table 1070, the selector 107
It is also possible to output only to 6 and to the left and right sides. Output data lines 1084 input from the N blocks 1060 connected to the left and right are selectors 1072, selectors 1073, conversion tables 1070, selectors 1074, and selectors 10.
It has a path for passing to B block 1062 or IO block 1061 connected to the lower side or output to the outside through 75. Further, the output data lines 1084 input from the left and right can pass through the selector 1072 and the selector 1075 and output on the lower side. The selectors 1072 to 1076 are controlled by the control circuit 1071. Control circuit 107
1 receives the mode signal line 1080 and the address signal line 1081 and generates control signals for the selectors 1072 to 1075. The configuration of the control circuit 1071 will be described later with reference to FIG. Further, the conversion table 1070 is
For writing, it is connected to the address signal line 1081 and the input data line 1083.

FIG. 17 shows the N block 1060. Figure 1
N block 1060 shown in FIG. 7 is arranged on the right side of the center of wafer 1050. The N block 1060 arranged on the left side is symmetrical in the horizontal direction. In FIG. 17, reference numeral 1100 indicates a neuron block.
1101 and 1103 are selectors and 1102 is a control circuit. Six neuron blocks 1100 are mounted in each N block 1060. Each neuron block 1
100, mode signal line 1080, address signal line 10
81, clock signal line 1082, input data line 1083
Connect. The control circuit 1102 uses the address signal line 108
1 is decoded, the selector 1101 selects one from the outputs of the six neuron blocks 1100 in the N block 1060, and outputs it to the selector 1103. The other input of the selector 1103 is the output data line 108 output from the N block 1060 connected to the right side.
It is 4. The output of the selector 1103 is N block 1060 or B block 1062 or I connected to the left side.
Connect to the O block 1061. The control circuit 1102 controls the selector 1103.

FIG. 18 shows the details of the neuron block 1100. This figure has almost the same configuration as that shown in FIG. In FIG. 18, reference numerals 1121 to 1125 denote latches. 1130 is a multiplier, 1131 is an α multiplier,
1132 indicates an η multiplier. Reference numerals 1140-1114 denote adders. Reference numerals 1150 to 1152 denote selectors. 11
Reference numeral 60 denotes a memory, and 1161 denotes a pointer that points to an address of the memory. The memory 1160 stores the address of the neuron block 1100 of the connection source, its connection coefficient w, and the previous correction value Δw. Reference numeral 1170 represents a control circuit. The latch 1121 is used only when operating as the reverse network side, and captures f ′ (u) in the equations (4) and (5) sent from the input data line 1083. The latch 1122 takes in the learning signal δ in the equation (3) sent from the input data line 1083 when it operates as the forward network side, and sends it from the input data line 1083 (3 when it operates as the reverse network side). ) Take in x in the equation. The latch 1123 is used only when it operates as the output layer of the backward network, and captures the teacher signal t in the equation (4) sent from the input data line 1083. Selector 1150 is memory 1
160 coupling coefficient w, input data line 1083, latch 1
121, a latch 1122, and a latch 1124 are input, and two of them are selected and output to the multiplier 1130. The multiplier 1130 multiplies the input value and outputs it. Selector 1
151 is the η multiplier 1132, the coupling coefficient w of the memory 1160, the multiplier 1130, the latch 1122, and the latch 112.
3, input the latch 1124, select two of them,
Output to the adder 1140. The adder 1140 adds or subtracts an input value and outputs it. The latch 1024 captures the output of the adder 1140. The selector 1152 is
Input the latch 1124 and the multiplier 1130, and
One of them is selected and output to the latch 1125. Latch 112
5 is output to the output data line 1084 through the selectors 1101 and 1103 of FIG. The α multiplier 1131 inputs the previous correction value Δw of the memory 1160, multiplies it by a constant α, and outputs it. The adder 1141 inputs the α multiplier 1131 and the multiplier 1130, adds them, and outputs them. η multiplier 1
132 multiplies the output of the adder 1141 by a constant η and outputs the result. The memory 1160 fetches the corrected coupling coefficient w of the output of the adder 1140 and the corrected value Δw of the output of the adder 1141. The control circuit 1170 uses the mode signal line 1080,
The address signal line 1081 and the clock signal line 1082 are input to control selectors and latches in the neuron block 1100, and memory writing.

Although the WSI configuration has been described above, it can also be configured by a chip as shown in FIG. In FIG. 19, 1400 is a chip board, 1401 is an N chip, 1403 is a B chip, 140
Reference numeral 4 represents a connector. By replacing the chip board 1400 with the WSI board 1030 in FIG. 12,
A neurocomputing system can be similarly constructed. Further, a configuration in which chips and WSI are mixed is also possible. N chip 1401 is N block 1060
It has the same function as B chip 1403 and B block 10
It has the same function as 62. The N chip and the B chip may be connected on the chip board 1400 in the same manner as the wafer 1050.

Learning is performed by repeating various operations in the system configuration of FIG. 12 described above.

FIG. 25 is a diagram showing a part of the mode signal output to the mode signal line 1080 of FIG. The simple operation contents for each mode signal are shown. The system of FIG. 12 performs various operations according to the mode signal of this figure. Set the mode signal line 1080 to 1, 3, 5, 8, 9, 1
By setting to 1, 13, 15, 17, and 19, initial values of each latch and memory can be written. The mode signals 24 to 32 are used during the learning operation. The order of setting will be described later with reference to FIG.

The operation of the system of FIG. 12 will be described with reference to FIG. 25, and along with that, the sequence RAM 1013 of FIG. 12, the control circuit 1071 of FIG. 16, and FIG.
The functions of the control circuit 1102 and the control circuit 1170 in FIG. 18 will be described.

First, the IO block 10 will be described with reference to FIG.
61, B block 1062, N block 1060 and N
An address for accessing the neuron block 1100 in the block 1060 will be described. The address of each neuron block 1100 during the learning operation operates with a relative address. WSI is likely to be unable to use all neuron blocks 1100 due to defects. Further, in the present invention, the neuron block 11 forming the forward network and the backward network is
00 has a one-to-one correspondence. Therefore, each neuron block 1100 can be set to any relative address. When setting a relative address, access with an absolute address. The wafer 105 is shown in FIG.
The absolute address of each block within 0 is shown. Also, FIG.
The bit structure of the absolute address is shown in (b). Wafer 10
A total of 288 neuron blocks 1100 in 50
Therefore, the absolute address in the wafer is 9 bits. Also, a board address of 4 bits is added to it.
The board address is necessary for the decoder 1085 of FIG. 12, and does not need to be sent into the wafer 1050. When connecting many boards, increase the bits of the board address. Of the in-wafer absolute address, 3 bits indicate a row address, 1 bit indicates a right / left address, 2 bits indicate a column address, and 3 bits indicate a neuron block address in an N block. The row address indicates rows 0 to 7 from the lower stage of the wafer 1050. The right / left address indicates right or left from the center. The column address indicates which column. N
In-block neuron block address is N block 1
It shows any of the 6 neuron blocks in 060. The neuron block address of the address in the N block is shown in FIG. Further, FIG. 20C shows an absolute address in the chip board 1400.

FIG. 21 shows the control circuit 1071 of FIG. Further, the control circuit 1102 of FIG. 17 is shown in FIG.
3 shows the control circuit 1170 of FIG. First, the address decoding method will be described with reference to these three figures.

In FIG. 21, reference numeral 1501 is a column decoder for decoding the column address of the absolute address. 150
Reference numeral 2 denotes a row decoder which decodes an absolute address row address. Reference numeral 1503 denotes a decoder for decoding the mode signal line. Reference numerals 1506 to 1507 denote latches. 15
Reference numeral 04 is an inverter circuit, and reference numeral 1505 is an AND circuit.
Reference numerals 1511 to 1512 denote subtraction circuits, which are address signal lines 1
Relative address and latches 1506, 15 sent to 081
The value written in 07 is subtracted, and the sign of the result is output. Reference numerals 1509 to 1510 denote selectors that select and output either the decoding result by the absolute address or the decoding result by the relative address.
The control line connected from the decoder 1503 to the selectors 1509 and 1510 selects either an absolute address or a relative address. Reference numeral 1520 indicates an N block select line.

In FIG. 22, 1607 is a decoder for decoding the mode signal line 1080. Reference numeral 1604 is a latch that can capture the output of the input data line 1083. The control is performed by the decoder 1607. 1603
Is a subtraction circuit, which subtracts the relative address sent to the address signal line 1081 and the value previously written in the latch 1604, and outputs the sign of the result. 1601
Reference numerals 1602 denote selectors. The selector 1601 selects and outputs either the N block select line 1520 which is the decoding result of the absolute address of the column address or the decoding result of the relative address. Selector 1602
Selects the absolute address decoding result and the relative address decoding result in the N block. The control line connected from the decoder 1607 to the selectors 1601 and 1602 selects either an absolute address or a relative address. The N block select line 1520 is twisted and connected to the adjacent block by the N block 1 in each column.
This is because 060 can have the same pattern.
As a result, the semiconductor mask and the number of design steps can be reduced. 1605 is a decoder for the address signal line 108
Of 1, the addresses in N blocks of absolute addresses are decoded. An encoder 1606 inputs the result of decoding the address in each of the six neuron blocks 1100.

In FIG. 23, reference numeral 1701 designates a latch which fetches its own relative address output to the input data line 1083. In addition to the relative address, whether it operates as the forward network side, the reverse network side, and if it operates on the reverse network side, whether it is the middle layer or the output layer. Also captures information indicating. Reference numeral 1703 is a comparison circuit, which compares the relative address output to the address signal line 1081 with the relative address of the latch 1701 written in advance,
Whether or not they match is sent to the decoder 1702. The result is also output to the decoder 1605 in FIG. Reference numeral 1704 is a comparison circuit, which compares the combination source address of the memory 1160 of FIG. 18 with the relative address output to the address signal line 1081. 1702 is a decoder, which is a comparison circuit 1703, a latch 1701, and a mode signal line 108.
1, the decoder 1605 of FIG. 22, the clock signal line 108
2, the comparator circuit 1704 and the selector 1601 in FIG. 22 are input to control the selector, the latch, the memory, the pointer, and the latch 1701 in the neuron block 1100.

Although only the items necessary for explaining the operation are shown in FIG. 25, a mode signal for performing writing / reading of a latch, a memory or the like is also set for inspection.

First, as an initial setting, a relative address and two bits of information indicating a forward network or a backward network, and an intermediate layer or an output layer are set as absolute addresses by the control circuit 1170 of each neuron block 1100 shown in FIG. Write to the latch 1701 of The host computer 1000 causes the selector 105 of FIG.
1, 1052 are on the external bus 1021 side. The mode signal line 1080 is set to 17 and the absolute address of the neuron block 1100 to be accessed is set to the address signal line 108.
Set to 1. Input the data you want to write 1083
Set to. In the WSI board 1030, the decoder 10
The board address is decoded by 85, and control is performed so that the latch of the wafer 1050, the writing of the memory, and the like are performed only when the address is its own.

In the operation based on the absolute address, the row decoder 1502 of FIG. 21 decodes the row address, and if the row is selected, it can be known. Also,
The column decoder 1501 selects a column address, and
One of the four N blocks 1060 in the row direction is connected to the N block select line 152 connected to it.
Select by 0. Further, the right / left bit is used to select the right or left N block 1060. The AND circuit 150 calculates the result of the row decoder 1502 and the right / left address.
The AND is performed by 5 and the enable signal of the decoder 1501 on the left side is obtained. Also, the result of the row decoder 1502 and the right /
An AND circuit 1505 ANDs the left address logically inverted by the inverter circuit 1504, and the result is used as an enable signal for the decoder 1501. In the selected N block 1060, the address in the N block is decoded by the decoder 1605 in FIG. 22, and the result is output to the control circuit 1170 in FIG. 23 in each neuron block 1100. Also, the selector 16
The selection result of the N block 01 is also output to the control circuit 1170. Decoder 1702 of select line control circuit 1170
When the mode signal is 17 and the self is selected by the control of 1, the value output to the input signal line 1083 is fetched in the latch 1701. By the above operation, data is written in the latches 1701 of all neuron blocks 1100. However, the relative addresses within the same wafer increase from the smallest absolute address to the largest. There may be an empty number. This is related to the control method of the output data line 1084 described below.

Next, FIG. 22 for all N blocks 1060.
The latch 1604 is set. The mode signal is set to 15, and the largest value of the relative addresses set in the neuron block 1100 in the N block 1060 is written in the latch 1604. Host computer 1000
Thus, the mode signal line 1080 is set to 15 and the absolute address of the N block 1060 to be accessed is set to the address signal line 1081. It is not necessary to set the neuron block address in the N block. The value to be written is set in the input data line 1083. Output data line 108 in N block 1060 during operation by relative address
4, that is, the control of the selector 1103 in FIG. 17, is performed by the subtraction circuit 1603 in FIG.
It is performed by the sign bit of the result of subtracting the latch 1604 from 81. The latch 1604 includes the N block 1060.
Since the maximum relative address of the inner neuron block 1100 is written, the neuron block 1100 outside the N block 1060 is latched by the latch 1604.
It is set to a relative address larger than the value written in. Therefore, the address signal line 1081 is
When the value is larger than 04, the selector 1 of FIG.
103 selects the output data line 1084 connected to the adjacent N block 1060.

Next, the maximum relative address set in the neuron block 1100 of that row is written in the latches 1507 of FIG. 21 of all IO blocks 1061 and B blocks 1062. By the host computer 1000,
Set mode signal line 1080 to 11 and access
The absolute address row address of the O block 1061 or the B block 1062 is set in the address signal line 1081. It is not necessary to set the right / left address, the column address, and the address in the N block. Input data line 10
Set to 83.

IO block 1061, B block 106
Control of the output data line 1084 in FIG.
21 is controlled by the subtraction circuit 151 shown in FIG.
2 from the address signal line 1081 to the latch 1507.
It is performed by the sign bit of the result of subtracting. Latch 1507
Since the maximum relative address of the neuron block 1100 in that row is written in, the latch block 1100 is latched in the neuron block 1100 above that block.
A relative address larger than the value of 507 is set.
Therefore, when the address signal line 1081 has a value larger than that of the latch 1507, the selector 1075 in FIG.
To select the output data line 1084 connected to the upper B block 1062.

Next, the maximum relative address set in the neuron block 1100 on the right side of the row is written in the latches 1506 of all IO blocks 1061 and B blocks 1062 in FIG. The mode signal line 1080 is set to 13 by the host computer 1000, and the absolute address row address of the IO block 1061 or B block 1062 to be accessed is set to the address signal line 1081. It is not necessary to set the right / left address, the column address, and the address in the N block. The value to be written is set in the input data line 1083. Control of the selector 1072 of FIG. 16 is performed by the subtraction circuit 1511 of FIG. 21 according to the sign bit obtained by subtracting the latch 1506 from the address signal line 1081. Since the maximum relative address of the neuron block 1100 on the right side of the row is written in the latch 1506, the left neuron block 11
A relative address larger than the value of the latch 1506 is set in 00. Therefore, the address signal line 1081
16 is larger than the latch 1506,
Selector 1072 selects the output data line 1084 connected to the left side.

Next, the conversion table 1070 of FIG. 16 of the IO block 1061 and B block 1062 is written. The conversion table 1070 of the wafer 1050 forming the forward network performs the f conversion of the equation (1), and the conversion table 1070 of the wafer 1050 forming the reverse network performs the f ′ conversion of the equations (4) and (5). .

The mode signal line 1080 is set to 9 by the host computer 1000, and the address signal line 108 is set.
IO block 1061 and B block 1 that access 1
Set the row address of the absolute address of 062. The address to be accessed in the conversion table 1070 is set to the bits of the right / left address, the column address, and the address in the N block. If 7 bits is insufficient, the number of bits of the address signal line 1081 is increased. Input data line 108
Set the value you want to write to 3. IO block 106
1. In each B block 1062, the row decoder 1 of FIG.
The row address is decoded by 502, and if the row address is selected, the value output to the input data line 1083 is written in the conversion table 1070.

Next, the output of the input layer neuron is written in the input layer memory 1010 of FIG. 12 and the teacher signal is written in the teacher signal memory 1012. Generally, since the neural network learns various patterns, the input layer memory 1010
Is segmented for each pattern. Similarly, the teacher signal memory 10 for writing the teacher signal corresponding to each pattern
12 is also segmented. Change the segment to be accessed for each pattern. The host computer 1000 sets the mode signal to 1 and 3 respectively, sets the address to the address signal line 1081, and writes the value to each memory.

Next, the neuron block 110 to be used
The initial value of the coupling coefficient is written in the memory 1160 of 0. The mode signal is set to 19 and the address signal line 108 is set first.
1 sets the address of the neuron block 1100 that accesses 1 and then sets the memory 1 in the address signal line 1081.
Set the write address of 160 and input data line 10
The value to be written in 83 is set. As described above, the address signal line 1081 may be used for time division, or the pointer 1161 may be set to 0 at the beginning to write from the address 0, and the pointer may be sequentially advanced by 1 to write to the memory 1160.

In the memory 1160, the relative address of the neuron block 1100 to be connected, the connection coefficient w with it, and the previous correction value Δw are written. However, the initial value of Δw may be 0. Further, the learning operation is interrupted halfway, the intermediate result is read out to the host computer 1000, and
When resuming learning, the required value will be written accordingly. Also, the initial values of the corresponding coupling coefficients of the forward network and the backward network must be the same.

Next, as shown in FIG. 24, the sequence RA
A series of mode signals for learning and a relative address of the neuron block 1100 for outputting are written in M1013.

FIG. 26 shows an example thereof. FIG. 26 shows an example of the sequence RAM 1013 for learning a neural net having three input layer neurons, two intermediate layer neurons, and two output layer neurons. FIG. 26 shows learning of one pattern, but when there are two or more patterns, it is necessary to change the segments of the input layer memory and the teacher signal memory each time the pattern is changed.

The learning operation will be described below with reference to FIG. The relative addresses of the input layer neurons are 0 to 2, the relative addresses of the intermediate layer neurons are 3 to 4, and the relative addresses of the output layer neurons are 5 to 6. Two boards are used with a board address forming the forward network of 3 and a board address forming the reverse network of 4.

Since the reverse network does not require input layer neurons, a total of 8 neuron blocks 1
Use 100. The board addresses of the input layer memory and the teacher signal memory are 0 and 1, respectively.

First, the pointer 1086 is reset.
The sequence RAM 1013 stores the command and the address written in the address 0, respectively, in the mode signal line 108.
0, and output to the address signal line 1081. Then, the mode signal line 1080 is set to 24, and the address 5 is set to the address signal line. Also, the board address is set to 1. The board address is decoded by the decoder 1085, and the output of the teacher signal memory 1012 is output to the input data line 1083 through the output buffer 1040. The teacher signal memory 1012 has the address signal line 108.
The value of address 5 designated by 1 is output. The neuron block 1100 whose relative address in the output layer on the reverse direction network side is 5 takes in its own teacher signal output to the input data line 1083 to the latch 1123 in FIG. Similarly, the same operation is performed for the neuron block 1100 in the output layer of the backward network having the relative address of 6.

Here, in order to write data in a latch, a memory, etc., an address hold time or write data hold time is required. Therefore, the clock signal line 108
The two clock signals make the timing of the write enable signals to them.

Next, when the pointer 1086 inputs a clock from the clock signal line 1082, the pointer 1086 of FIG. 12 advances by 1 and moves to the next instruction. Mode signal line 1
080 is set to 25, and the pointer 1161 in FIG. 18 is reset. In addition, the neuron block 11 of FIG.
The contents of the latch 1124 in 00 are also reset.

The next instruction performs a forward operation. The mode signal line 1080 is set to 26, and the address signal line is designated by the neuron block 1100 from the input layer toward the output layer in ascending order of relative address. The board address is 0 in the input layer, and the middle layer,
Board address 3 is set in the output layer. The designated input layer memory 1010 outputs the value to the input data line 1083 and broadcasts it to all neuron blocks 1100. In the neuron block 1100, 2
The mode signal line 1080 set to 6 is decoded by the decoder 1702 of FIG. 23, and each selector of the neuron block 1100 of FIG. 18 is controlled. Selector 115
0 is the coupling coefficient w of the memory 1160 and the input data line 108.
3 is output to the multiplier 1130. The multiplier 1130 performs synapse calculation, that is, multiplication between the coupling coefficient w and the output x of the neuron. The selector 1151 is the multiplier 11
30 and the latch 1124 are output to the adder 1140. The output of the adder 1140 is written in the latch 1124. The selector 1152 selects the selector 1124. Latch 1
125 takes in the value. The control circuit 1170 of FIG.
Comparing circuit 1704 compares the connection source address of memory 1160 with the address output to address signal line 1081, and when they match, the outputs of latch 1124 and multiplier 1130 are added, and the result is latched. 11
Write to 24. Then, the value of the pointer 1161 in FIG. 18 is advanced by one. In this way, the address signal line 1 is sequentially
The relative address of the output source sent to 081 and the output sent to the input data line 1083 are input, and cumulative addition is performed. The memory 1160 is written in the ascending order of the connection source address. The address signal line 1081 and FIG.
The relative address written in the latch 1701 of No. 3 is compared by the comparison circuit 1703, and if they match,
The result of cumulative addition, that is, the internal energy value u of the latch 1125 is sent to the selector 1072 of FIG. 16 through the selectors 1101 and 1103 of FIG. 17 and the output data line 1084. IO block 1061, B block 1062
21 of FIG. 21, when the mode signal is set to 26, the selector 1073 selects the selector 107.
The second side is selected, and the selector 1074 selects the conversion table 107.
Select 0. Further, the selector 1076 selects the input data line 1083 side so that the input data line 1083 sent from the lower side is sent to the N block 1060 on the side as it is. The selectors 1072 and 1075 control the address signal line 1081 by the method described above. As a result, the neuron block 11 whose relative address is specified
The internal energy value u, which is the output of 00, is sent to the conversion table 1070 of that row, converted into the output x of the neuron, and sent to the control board 1003 of FIG. 12 through the output data line 1084. Selector 105 of FIG.
2 outputs the value of the output data line 1084 as it is to the input data line 1083.

On the reverse network side in this mode, the address signal line 1081 in FIG.
When its own relative address is output, the value of the input data line 1083 is fetched in the latch 1122.

In the next operation, the mode signal line 1080 is set to 2
7, and the relative addresses of the intermediate layer and the output layer are sequentially set in the address signal 1081. Forward network neuron block 110 with relative addresses
0 again sends the contents of the latch 1125 of FIG. 18, that is, the internal energy value u to the IO block 1061 and B block 1062 of FIG. In mode 27, the selectors 1074 of the IO block 1061 and B block 1062 select the selector 1072. Therefore, the internal energy value u is output to the lower output data line 1084 as it is. The decoding method of the address signal line 1081 is similar to that of the mode 26. The internal energy value u is
Output data line 10 through the control board of FIG.
It is output to 84. On the other hand, in the reverse direction IO block 1061 and B block 1062, the selector 1
073 to the input data line 1083 side, selector 1074
To the conversion table 1070 side and the selector 1076 to the selector 1074 side. The internal energy value u sent by the input data line 1083 is converted into the conversion table 107.
By 0, f ′ conversion is performed to f ′ (u), which is output to the side N block 1060 through the input data line 1083. In the reverse direction network neuron block 1100 of FIG. 18, if the address output to the address signal line 1081 and its own relative address match, f ′ (u) output to the input data line 1083 is latched 11.
Take in 21.

Next, the mode signal line 1080 is set to 28. The neuron block 1100 in the output layer of the backward network sets the selector as follows. In FIG. 18, the selector 1151 includes latches 1122 and 11
23 is output to the adder 1140. (T−x) is calculated by the adder 1140 and written in the latch 1124. The selector 1150 outputs the latch 1124 and the latch 1121 to the multiplier 1130. The multiplier 1130 causes (t−x) of the latch 1124 and f ′ (u) of the latch 1121.
And the learning signal δ of the output layer is calculated. Selector 1152 outputs the output of multiplier 1130 to latch 1125. The learning signal δ is written in the latch 1125.

In the next instruction, the mode signal line 1080 is set to 2
5, the pointer 1161 in FIG. 18 is reset. Further, the latch 1124 in the neuron block 1100 of FIG. 18 is reset.

In the next instruction, the mode signal line 1080 is set to 2
9 and set the relative address of the output layer to the address signal line 1
It is set to 081 in ascending order. The neuron block 1100 of FIG. 18 of the backward network specified with the relative address sends the contents of the latch 1125, that is, the learning signal δ of the output layer to the IO block 1061 and the B block 1062 of FIG. IO block 1 in mode 29
061, selector 107 of B block 1062 in FIG.
Since 4 selects the selector 1072, the learning signal δ is directly output to the output data line 1084. The decoding of the address signal line 1081 is the same as in mode 26. The learning signal δ corresponds to the control board 1 of FIG.
It is output to the input data line 1083 through 003. In the IO block 1061 and B block 1062 of the forward network, the selector 1076 of FIG. 16 is set to the input data line 1083 side connected to the lower side. The learning signal δ sent from the reverse network is output to the side N block 1060. In the neuron block 1100 of FIG. 18 of the forward network, if the address output to the address signal line 1081 and its own relative address match, the learning signal δ output to the input data line 1083 is fetched in the latch 1122.

In the neuron block 1100 in the intermediate layer of the reverse network, the selector 1150 outputs the coupling coefficient w of the memory 1160 and the input data line 1083 to the multiplier 1130. The multiplier 1130 multiplies the coupling coefficient w by the learning signal δ. The selector 1151 outputs the multiplier 1130 and the latch 1124 to the adder 1140. The latch 1124 takes in the output of the adder 1140. By the comparison circuit 1704 of FIG.
The source address and the address signal line 1081 output by 0
Are compared, and when they match, the outputs of the latch 1124 and the multiplier 1130 are added and written to the latch 1124. Then, the value of the pointer 1161 is advanced by one. In this manner, the relative address of the neuron block 1100 as the output source and its output, which are sequentially sent to the address signal line 1081 and the input data line 1083, are input, and Σw · δ
The cumulative addition of is performed.

In the next instruction, the mode signal line 1080 is set to 3
Set to 0. The neuron block 1100 in the output layer of the backward network sets the selector as follows.
In FIG. 18, the selector 1150 outputs the latch 1124 and the latch 1121 to the multiplier 1130. Multiplier 1
130, Σw · δ and f ′ (u) of the latch 1124
And the learning signal δ of the intermediate layer is calculated. Selector 1152 outputs the output of multiplier 1130 to latch 1125. The learning signal δ is written in the latch 1125.

Next, the mode signal line 1080 is set to 29, and the learning signal δ of the intermediate layer is set to all neuron blocks 110.
Broadcast to 0. The neuron block 1100 of the forward network compares the value of the address signal line 1081 with its own relative address written in the latch 1701 of FIG. 23 by the comparison circuit 1703. Take in.

When the number of intermediate layers is two or more, after that, the command of the mode 30 is again given, the learning signal δ is calculated, and then the mode 29 is sent to the forward network.

In the next instruction, the mode signal line 1080 is set to 2
5, the pointer 1161 in FIG. 18 is reset.

Next, the mode signal line 1080 is set to 31, and the address signal lines 1081 are designated from the input layer toward the intermediate layer in ascending order of relative address. With respect to the input layer, it is output from the input layer memory 1010. With respect to the intermediate layer, the neuron block 1100 in the intermediate layer of the forward network calculates its internal energy value u by IO block 1
061, f conversion is performed by the conversion table 1070 of the B block 1062, and the value is broadcast. Each selector of the neuron block 1100 of FIG. 18 of the forward network is controlled as follows. The selector 1150 outputs the latch 1122 and the input data line 1083 to the multiplier 1130. The multiplier 1130 multiplies the learning signal δ of the latch 1122 by the output x of the neuron. The selector 1151 uses the coupling coefficient w of the memory 1160 and the η multiplier 11
32 is output to the adder 1140. Also, the α multiplier 11
31 inputs the previous correction value Δw ′ of the memory 1160,
Multiply by α and output αΔw ′. The adder 1141 outputs the output δ · x of the multiplier 1130 and the output αΔ of the α multiplier 1131.
w ′ is input, added, and output to the η multiplier 1132.
The output of η multiplier 1132 is a modified value. Memory 11
The value before multiplication by η is written in Δw ′ of 60.

[0122]

[Equation 6]

[0123]

[Equation 7]

Since the constant η has a wide range of values, the number of bits of Δw ′ written in the memory 1160 can be saved by doing so. The adder 1140 corrects the coupling coefficient w and writes the correction result in the memory 1160. The above correction operation is performed by the comparison circuit 1704 of FIG. 23 when the coupling source address of the memory 1160 and the address signal line 1081 match.

Next, the mode signal line 1080 is set to 25, and the pointer 1161 in FIG. 18 is reset.

Next, the mode signal line 1080 is set to 32, the relative address of the neuron block 1100 in the output layer of the backward network is set to the address signal line 1081, and the learning signal δ is broadcast. If there are two or more intermediate layers, the learning signal δ of the intermediate layer is also broadcast. Each selector of the neuron block 1100 of FIG. 18 of the backward network is set in the same manner as the correction of the coupling coefficient w of the forward network of the mode 31. The multiplier 1130 multiplies the output x of the neuron of the latch 1122 by the learning signal δ output to the input data line 1083 to obtain the η multiplier 1132 and the adder 114.
The corrected value Δw ′ and the corrected coupling coefficient w are calculated by 0, the α multiplier 1131 and the adder 1141, and the memory 11
Write to 60.

The learning operation can be performed by changing the segments of the input layer memory and the teacher signal memory for each pattern for the above operation. Further, the output layer memory 1011 captures the output of the output layer neuron of the forward network which is output to the output data line 1084.
The host computer 1000 can read the output layer memory 1011 from the external bus 1021 and check whether the output is as expected.

As an example of the application shown in FIG. 12, for example, the digitizer 1001 shown in FIG. 12 inputs handwritten characters, the coordinates of the characters input by the host computer 1000 are normalized, and the handwritten characters are recognized by the input of the neural network. You can create a system that does. In the system of FIG. 12, p input layers and an intermediate layer q
The time required for one learning of one pattern of a multi-layered neural network of 1 and output layer r is almost the same as

[0129]

(8) (2p + 4q + 5r) · t (s) (8) For example, p is 256 (16 dots × 16 dots), q is 100, r is 26, and t is 10 for handwritten character recognition.
Assuming 0 ns, one learning per pattern is about 0.1 m
s can be done. Even if it takes 1000 times to learn one character, it is possible to learn a 10-character pattern in only 1 second.

[0130]

According to the present invention, a neural network can be learned at high speed. Further, it is convenient to form a wafer scale integrated circuit, and it is also possible to form a plurality of wafers. Also, the number of designing steps is small. Therefore, a large-scale neural network can be calculated at high speed, and a compact wafer-scale integrated circuit for a neurocomputer can be manufactured.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of the present invention.

FIG. 2 is a diagram showing a neuron model forming a neural net.

FIG. 3 is a diagram showing a three-layer hierarchical neural network.

FIG. 4 is a diagram showing dedicated hardware for calculating a neural network at high speed.

FIG. 5 is a diagram for explaining the details of the operation of the present invention.

FIG. 6 is a diagram showing a flow of calculation according to the present invention.

FIG. 7 is a diagram showing details of an example of a functional block used in the present invention.

FIG. 8 is a diagram showing details of an example of a functional block used in the present invention.

FIG. 9 is a diagram showing details of an example of a functional block used in the present invention.

FIG. 10 is a diagram showing another embodiment of the present invention.

FIG. 11 is a diagram showing another embodiment of the present invention.

FIG. 12 is a diagram showing an entire neurocomputing system of the present invention.

FIG. 13 is a diagram showing a structure of a wafer of the neuro-computing system of the present invention.

FIG. 14 is a diagram showing a configuration and arrangement of signal lines on a wafer.

FIG. 15 is a diagram showing a method of supplying power to each block of a wafer and a configuration of a WSI substrate.

FIG. 16 is a diagram showing a B block (IO block).

FIG. 17 is a diagram showing N blocks.

FIG. 18 is a diagram showing details of a neuron block.

FIG. 19 is a diagram illustrating a case where the system of the present invention is configured by a chip.

FIG. 20 is an IO block, B block, N block, N
It is a figure explaining the address for accessing the neuron block in a block.

FIG. 21 is a diagram showing the control circuit of FIG. 16;

22 is a diagram showing the control circuit of FIG. 17. FIG.

23 is a diagram showing the control circuit of FIG. 18. FIG.

FIG. 24 is a diagram illustrating a sequence RAM.

FIG. 25 is a diagram showing an example of a mode signal and its operation.

26 is a diagram showing an example of an instruction to write to the sequence RAM of FIG. 24. FIG.

[Explanation of symbols]

120F, 121F, 120B, 121B ... Calculation unit, 123, 124 ... Control circuit, 125-127 ... Selector, 128 ... Buffer, 130 ... Control signal generation circuit, 131, 132 ... Function conversion circuit, 140 ... Output bus,
141 ... Input bus, 142-145 ... Control signal line, 150 ... External input terminal, 151 ... External output terminal, 160 ... Multiplier.

Front page continuation (72) Masayoshi Yagyu, 1-280, Higashi Koikekubo, Kokubunji, Tokyo, Central Research Laboratory, Hitachi, Ltd. (72) Minoru Yamada, 1-280, Higashi Koikeku, Tokyo Kokubunji City, Central Research Laboratory, Hitachi Ltd. (72) Inventor Katsunari Shibata 1-280, Higashi Koigokubo, Kokubunji, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd.

Claims (8)

[Claims] 1. A semiconductor integrated circuit connected by a data bus, a first memory for storing an input value, a second memory for storing a desired output value, and a third memory for storing an output value.
And a control circuit for controlling them, the semiconductor integrated circuit has a plurality of functional blocks for calculating a neuron model, and one of the functional blocks is provided by the data bus. Means for transferring data to at least one of the other functional blocks, and means for modifying the weight value between the neuron models for the plurality of input values and the desired value corresponding to those input values. An information processing system having. 2. At least one connected by a data bus
One wafer-scale integrated circuit and a first memory for storing input values
An information processing system comprising: a memory, a second memory for storing a desired value of output, a third memory for storing an output value, and a control circuit for controlling these; Has a plurality of functional blocks for calculating a neuron model, and one of the above functional blocks
One is means for transferring data to at least one of the other functional blocks by the data bus, and a plurality of the input values and the desired values corresponding to those input values, between the neuron models. An information processing system comprising means for modifying a weight value. 3. At least one connected by a data bus
One wafer-scale integrated circuit and a first memory for storing input values
An information processing system comprising: a memory, a second memory for storing a desired value of output, a third memory for storing an output value, and a control circuit for controlling these; Has a plurality of functional blocks for calculating a neuron model, and one of the above functional blocks
One has means for transferring data to at least one of the other functional blocks by the data bus, and the first layer in which the intermediate layer, the output layer and the output thereof are determined by at least one of the functional blocks. A second hierarchical network in which the output layer, the intermediate layer, and the output thereof are determined by at least one of the functional blocks, and the first hierarchical network and the second hierarchical layer are configured. The functional blocks of the dynamic network have a one-to-one correspondence, and have means for modifying the coupling coefficient between the neuron models with respect to the plurality of input values and the desired values corresponding to the input values. A characteristic information processing system. 4. The information processing system according to claim 3, wherein the plurality of functional blocks of the wafer scale integrated circuit are externally accessed by an absolute address physically determined and a logical address externally set. 5. The first hierarchical network and the second hierarchical network.
The above-mentioned two functional blocks corresponding to one-to-one of the hierarchical network of
The information processing system according to claim 4, wherein the association is determined based on the logical address. 6. A first memory for storing an input value, a second memory for storing a desired output value, a third memory for storing an output value, and a plurality of functional blocks, which control these. An information processing system including a control circuit, the functional block having an information processing device for calculating one neuron model, wherein the information processing device is connected to a host computer, and the host computer is an initial setting of the information processing device. When the information processing apparatus is operated, the output value of each neuron model is obtained from the third memory that stores the output value, and during the operation, there is provided means for controlling without intervening the host computer. An information processing system characterized by the above. 7. A first memory for storing an input value, a second memory for storing a desired output value, a third memory for storing an output value, a plurality of functional blocks, and controlling them. An information processing system including a control circuit for controlling a neuron model, wherein the functional block is an information processing system having an information processing device that calculates one neuron model, wherein the information processing device is connected to a host computer, It has a first function of calculating and a second function of calculating a learning signal value of a neuron model, and is configured to perform either one of the first function and the second function during operation. Then, the host computer controls the function block to operate as the first function or the second function, or to set each of the function blocks. The information processing system characterized by having steps. 8. An input layer for inputting a signal from the outside and holding and outputting the signal, a first intermediate layer for inputting a signal, calculating and outputting an output value of a neuron model in a neuron calculation unit, and inputting a signal And a correction means for correcting the weight value of the neuron model in the first output layer, which outputs and outputs the output value of the neuron model in the neuron calculation unit,
A signal is input from the outside, the signal is processed in the order of the input layer, the first intermediate layer, and the first output layer, and the first network that determines the output value of the neuron model and the learning signal are calculated and output. A second output layer, a second intermediate layer for calculating and outputting the output value of the correction means and the learning signal value, and inputting an expected output value for a given input, A second network that processes a signal in the order of the output layer and the second intermediate layer to determine a learning signal value; and a communication unit that mutually communicates data between the first network and the second network. The communication means communicates the output value of the neuron model and the learning signal value between the first network and the second network, and the correction means outputs the output of the neuron model. And by a value proportional to the product of the learning signal value is added to the weight value of the neuron models, the information processing system characterized by having means for modifying the weighting value.