JPH04237364A - Neuro computer - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention multiplies a plurality of input signals by their corresponding weights, sums them, and then calculates the
This field relates to a neurocomputer that processes data using a neural network that combines multiple neurons that output signals converted by nonlinear functions.

The weight applied to each input signal can take various values depending on the structure and scale of the neural network and the data being handled, so the amount of hardware increases if a certain level of accuracy is to be maintained as a whole. There is a need for a technique that reduces the increase in the amount of hardware and improves the accuracy of calculations related to weights.

0003

[Prior Art] Many conventional neurocomputers perform software simulations on general-purpose computers. Therefore, weights can be represented as floating point numbers, and the range of possible values for weights is wide enough that there is no need to worry about restrictions. However, when implementing a neurocomputer in hardware, expressing the weights in floating point numbers requires a large amount of hardware to store and calculate the weights, which is one of the purposes of implementing a neurocomputer in hardware. It becomes difficult to downsize. Therefore, it is necessary to store the weights in fixed-point representation or as analog values.

0004

However, in the prior art, when the weights are stored as fixed-point representations or analog values, the possible range of the weights is limited if the precision is kept constant. Conversely, if the range of possible values for weights is widened, there is a problem in that accuracy decreases. The present invention aims to solve the above-mentioned problems and to improve the calculation accuracy of a neurocomputer without increasing the hardware size too much.

[0005]

[Means for Solving the Problems] FIG. 1 is a diagram illustrating the principle of the present invention. As shown in Figure 1 (a), neuron 1
0 receives an input signal from the input terminal or the neuron 10 at the previous stage, performs calculations according to the weights stored in advance in the weight memory 11, and outputs the results. In the present invention, each neuron 10 is configured as shown in (b) of FIG.

In (b) of FIG. 1, x1, x2,
..., xn is the input signal from the previous stage neuron, W1
, W2,..., Wn are weights to be multiplied by each input signal, k is a certain coefficient, and f( ) represents a nonlinear function such as a sigmoid function. Further, 12 represents a product-sum calculation section, 13 represents a coefficient multiplication section, and 14 represents a nonlinear transformation section.

Weights to be multiplied by each input signal x1 to xn are given from a weight memory 11. The sum of products calculation unit 12 is
Weight W corresponding to each input signal x1 to xn
Multiply by 1 to Wn and calculate the sum. The coefficient multiplier 13 multiplies the output of the sum-of-products calculator 12 by a coefficient k predetermined for each neuron 10.

The nonlinear transformation section 14 transforms the multiplication result of the coefficient multiplication section 13 using a nonlinear function such as a sigmoid function f, and outputs the transformation result. The output of the neuron 10 is f(kΣWj xj ) (where Σ represents the sum of j=1 to n).

The coefficient k is a value determined in advance according to the weight for the input of each neuron 10 determined by learning, and is, for example, the value of the largest absolute value among the weight data for that neuron 10. The weights stored in the weight memory 11 are obtained by normalizing the weight data determined during learning by a coefficient k.

Alternatively, some coefficients are prepared in advance, and among the weight data for the neuron 10, the one that is larger than the one with the largest absolute value and closest to that value is selected as the value of the coefficient k. It may also be used.

Similar to the weight memory 11, this coefficient k is stored in an external memory of the neuron 10 and given to each neuron 10 at the time of initialization or after the product-sum operation. Alternatively, it may be fixed and stored in each neuron 10 in advance.

Weight data normalized by the coefficient k is stored in the weight memory 11 as a digital value in a fixed-point representation, and the weight data is stored in the corresponding neuron 10 before or during the calculation.
give to The weight data may be stored as an analog value, and the sum-of-products calculation unit 12 may multiply the input by the analog value.

[0013]

[Operation] In the neuron 10, a coefficient multiplier 13 is provided which multiplies the sum of products by a certain coefficient k after the sum of products is calculated by the sum-of-products calculation unit 12 and before the nonlinear conversion unit 14 performs transformation using a nonlinear function.

For example, if the absolute value of the weight is limited to 1 or less, and if the largest value among the weight data for the neuron 10 is the coefficient k, then the weight data is
You can normalize with that number. In this way, the weight with the largest absolute value can make maximum use of the accuracy. The accuracy of other weight data for this neuron 10 will not deteriorate as long as the absolute values do not differ much.

Instead of using the largest weight data as a coefficient, several coefficients are prepared in advance, and the one that is larger than the largest weight data and closest to it is used as a coefficient. It may be selected as the value of k. The weight data is normalized by the coefficient k, and the normalized weight data is stored in the weight memory 11.

This is equivalent to the generalized method of floating point representation for the weight with the largest absolute value. As long as the other weight data for this neuron 10 are not much different in size from the largest weight, the accuracy will not drop much.

[0017]

Embodiment FIG. 2 is a system configuration diagram of an embodiment of the present invention. In FIG. 2, 20 is a controller that generates a control pattern to control the neural network and performs overall control such as switching inputs, 21 is an external memory that stores weight data, coefficients, etc., and 22 is a controller that time-divides external inputs. Input selection units 23a and 23b for input into a common analog bus are analog neuroprocessors (hereinafter referred to as ANP) corresponding to the neuron 10 shown in FIG.
represents.

[0018] The system shown in Fig. 2 has the following reasons: ■ high-speed processing is possible, ■ circuit size can be reduced, and ■ external inputs such as sensors are analog.
It is configured to process input signals using analog values. The weight data and coefficients are 16-bit digital values.

ANP23a is a neuron in the middle layer, AN
P23b is a neuron in the output layer and constitutes a hierarchical network. Since the input layer does not require conversion processing,
No processor is used. ANP23a, 23b are
Analog signals are time-divisionally input from the analog bus, and after a product-sum operation is performed, they are multiplied by a certain coefficient, and the analog signals converted by the sigmoid function circuit are output to the analog bus.

Each ANP 23a,
23b stores the corresponding coefficients and weights. When starting processing, the input selection unit 22 controls input switching, reads the coefficient k and weight data corresponding to each input from the external memory 21, and selects each ANP2.
3a and 23b. In addition, each ANP23a, 23
A control signal for activation is given to b.

Each ANP 23a, 23b has a configuration as shown in FIG. 3, for example. FIG. 3 is a block diagram of an analog neuroprocessor according to an embodiment of the present invention. In FIG. 3, 30 is an external capacitor such as a capacitor, 31 is an analog data processing block, 32 is a weighted data loading processing block, 33 is a control block, 34 is a multiplication type DA converter that weights the input signal, 3
5 is an adder, 36 is a multiplication type DA converter that multiplies coefficient k,
37 represents a sigmoid function circuit that performs nonlinear transformation.

The ANP 23 shown in FIG. 3 is composed of, for example, an IC chip, and is a model of a neuron realized almost directly using a digital-analog mixed circuit. The inside of the chip can be roughly divided into analog data processing blocks 31,
It consists of a weight data loading processing block 32 and a control block 33 that controls the entire chip.

[0023] The weights are determined by learning by a software simulator using a general-purpose computer. Then, 16-bit fixed-point data is written into the external memory 21 shown in FIG. 2.

When the analog input signal x1 comes in from the input line, the digital weight W1 corresponding to that input is loaded. When these pass through the multiplication type DA converter 34, a product W1 x1 of these is obtained. The adder 35 and the external capacitor 30 constitute an integrator. The output of the multiplication type DA converter 34 is connected to the external capacitor 30 of the integrator.
is stored as an electric charge. Next, input signal x2 and weight W2 are input and processed in the same way. This is xn, Wn
Repeat until. This causes the total to be stored in the external capacitor 30.

When the output voltage of the integrator is output, the coefficient k written as 16-bit fixed point data from the external memory is loaded. When these pass through the multiplication type DA converter 36, the product of these is obtained. This is input into a sigmoid function circuit 37, and a non-linearly converted voltage is output. These series of processes are controlled through the control block 33 by external control signals. Since the weight data loading processing block 32 and the control block 33 can be easily constructed from the above functional explanation, detailed explanation thereof will be omitted.

FIG. 4 shows an example of setting coefficients according to an embodiment of the present invention. The model of the neuron 10 shown in FIG. 4(A) has four inputs x1 to x4 with weights W1 to
This is a model that calculates by multiplying by W4. This weight is determined, for example, by a software simulator using a general-purpose computer, using a method such as a backpropagation learning method. Note that θ in the figure is a threshold value, although it was omitted in the previous explanation.

The coefficients are determined by the procedure shown in (b) of FIG.
Perform according to a) to (c). (a) The weights W1 to W4 are determined by learning using a software simulator. (b) Maximum weight (in this example, W2 = +10.2
Divide each weight by 46) and normalize it. (c) The value used for this normalization + 10.246 is the coefficient k
Set as .

FIG. 5 shows an analog neuroprocessor (A
NP) shows another embodiment block diagram. In FIG. 5, the same reference numerals as in FIG. 3 correspond to those shown in FIG. 50 represents a coefficient storage section that stores several predetermined coefficients.

In the embodiment shown in FIG. 3 described above, the maximum weight value is the value of the coefficient k, as shown in FIG. 4, but in the embodiment shown in FIG. ，
Prepare some things like... Once the weights have been determined through learning by the software simulator, the one that is larger than the maximum weight and closest to the maximum value is selected from among the coefficients k1, k2,... This is the coefficient used for multiplication.

The ANP 23 selects the coefficient determined during learning from among the coefficients stored in the coefficient storage unit 50 using a control signal or a selection signal from the weight data bus.
The signal is supplied to the multiplication type DA converter 36. Other processing is similar to the example in FIG. 3.

FIG. 6 shows an analog neuroprocessor (A
NP) shows another embodiment block diagram. In FIG. 6, the same reference numerals as in FIG. 3 correspond to those shown in FIG. 60 represents a multiplier that multiplies analog values by analog values.

In this example, the weights are not stored as digital values, but as analog values. The coefficient k is 16
It is a digital value of bits. Analog input signal x1
, x2 , ... and weight data W1, W based on analog values
2, . . . are multiplied by the multiplier 60.

[0033]

[Effect of the invention] As explained above, according to the present invention,
A neurocomputer that can miniaturize the hardware by storing weights in fixed-point representation or as analog values, and can widen the range of values that weights can take without significantly reducing accuracy. can be realized.

[Brief explanation of the drawing]

FIG. 1 is a diagram explaining the principle of the present invention.

FIG. 2 is a system configuration diagram of an embodiment of the present invention.

FIG. 3 is a block diagram of an embodiment of an analog neuroprocessor according to the present invention.

FIG. 4 is a diagram showing an example of setting coefficients according to an embodiment of the present invention.

FIG. 5 is a block diagram of an embodiment of an analog neuroprocessor according to the present invention.

FIG. 6 is a block diagram of an embodiment of an analog neuroprocessor according to the present invention.

[Explanation of symbols]

10 Neuron 11 Weight memory 12 Product sum calculation section 13 Coefficient multiplier 14　　　　Nonlinear conversion section x1 ~ xn Input signal W1 ~ Wn Weight k coefficient f( ) Nonlinear function

Claims (5)

[Claims] Claim 1: A plurality of neurons (10) that output a signal transformed by a nonlinear function after multiplying a plurality of input signals by weights corresponding to each one and calculating the sum thereof, and a plurality of neurons (10) for outputting signals transformed by a nonlinear function. In a neurocomputer having a weight memory (11) for storing weight data, the neuron (10) converts input signals by a nonlinear function when or after the sum is calculated by multiplying each input signal by a weight. 1. A neurocomputer characterized by comprising means (13) for multiplying a signal during or after summation by a predetermined coefficient in relation to a weight determined during learning. 2. The neurocomputer according to claim 1, further comprising a memory for storing the coefficients, and multiplication of signals during or after the summation is performed by the coefficients set in the memory. Neurocomputer. 3. The neurocomputer according to claim 1 or 2, wherein a plurality of numerical values are prepared in advance as the coefficients, and one coefficient is selected from among the numerical values. . 4. The neurocomputer according to claim 1, claim 2, or claim 3, wherein the weight memory (
11) A neurocomputer characterized in that weight data is stored as a digital value expressed in a fixed-point representation, and the weight data is provided as a digital value to each neuron (10). 5. The neurocomputer according to claim 1, claim 2, or claim 3, wherein the weight data is stored as an analog value, and the weight data is given to each neuron (10) as an analog value. neurocomputer.