JPH03237557A - Neural network simulator and computer system - Google Patents

Abstract

PURPOSE:To enable a user to easily utilize a neural network constructed by a neural network simulator for another processing program by converting the neural network into a high level language source program and storing the converted program. CONSTITUTION:Information for constructing a neural network is read out from a neural network describing language file 40 at first. Then the neural network is constructed in a main storage based upon the read information. For instance, the neural network shown in the figure is constructed. The characteristics, states, etc., of the constructed neural network are changed by a learning means or the like. Then the neural network stored in the main storage is converted into a high level language (C language e.g.) source file 42 equivalent to the original neural network, the converted source file 42 is stored and information necessary for referring variables, functions, etc., defined in the file 42 from other modules is stored as a header file 42 independently of the file 42.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to a neural network simulator and a computer system, and particularly relates to a method for storing a constructed neural network.

[Conventional technology 1] Neural networks have come into practical use, and neural network simulators have been proposed as various devices for simulating neural networks (magazine ``Nikkei AI
J 1987.10.26 Appendix, ``Neural network business has started its run-up j).

Traditionally, neural network simulators are
There are neural network simulators (software simulators) that are software that runs on computer systems, and dedicated coprocessor cards (accelerators) that are incorporated into other computer systems for high-speed simulation processing.

FIG. 2 shows an example of an implementation configuration of a software simulator. In FIG. 2, user l provides information necessary for the simulation (for example, the structure of the neural network,
data, various parameters, etc.) 2 is input into the software simulator 4 on the host computer 3, the software simulator 4 internally constructs a neutral network 5, performs further calculations if necessary, and changes the state of the neutral network 5. Thereafter, information 6 on the structure and characteristics (for example, inter-unit connections, synaptic loads, etc.) of the neutral network 5 is stored as a data file 7 in an auxiliary storage device 8 or the like.

Further, FIG. 3 shows an example of an implementation configuration using a coprocessor card. In FIG. 3, a user 1 constructs a neural network 5 on a cobrocessor card 10 using an interface program 9 of a host computer 3. Further, the structure and characteristics of the neural network 5 are also saved through the interface program 9.

Software simulator 4 and Cobrosesa card 1
Regardless of which device is used, the procedure for using a neural network simulator from the user's point of view is the procedure for inputting the information necessary for simulation into the neural network simulator (step ■), and the inputting of the results from the neural network simulator into the neural network. The structure/characteristics of the system are extracted and saved in a file, etc. (step ■).

In step 2, a neural network description language is often used to input information to the neural network simulator. Specifications for this language are unique to each neural network simulator. In addition, in step (2), the structure, characteristics, and other data of the neural network are saved in the auxiliary storage device 8, etc. according to a specific format, but the saved data is generally in the form of a data file, and the format is also Each simulator uses its own format.

By the way, neural networks are constructed as a means of realizing specific information processing, so once the structure and characteristics are obtained, a neural network with that structure and characteristics can be incorporated into another program as a module and used. There are many things.

Examples of its implementation are shown in FIGS. 4 and 5.

As shown in FIG. 5 (A>), a blank neural network module 12 is provided in the processing program 11, and upon initialization, a neural network module 13 is constructed and initialized by reading the data file 7.
The processing program 11 is completed as shown in Figure (B) (step ①00). When a processing program is executed, data is input to the constructed neural network 14 and calculation is executed according to the processing stage (step 101). When processing using such a neural network module 12 is completed, the processing Post-processing such as outputting the results is performed (step 102).

[Problem to be solved by the invention] However, as mentioned above, the language for executing the simulation and the structure and characteristics of the neural network obtained by the simulation may be unique to each software simulator or Cobrosessor card. many. Due to this lack of versatility, when incorporating a neural network into other processing programs that are often written in high-level languages, a module that reads the data file 7 and constructs and initializes the neural network, Users must create their own modules that allow neural networks to perform calculations.

Some conventional neural network simulators solve the drawback of leaving the creation of the module to the user by providing the user with a module having the above-mentioned functions in the form of a source program or library. However, whether such functionality is provided in the form of a source program or in the form of a library, other processing programs will read the data file and execute the construction and initialization of the neural network at the time of initialization. Therefore, to run a program, multiple files, an executable file and a data file, are required, which complicates file management, and the data file is read at the time of program initialization to construct and initialize the neural network. It has the disadvantage that it takes time to initialize.

The present invention has been made in consideration of the above points, and aims to provide a neural network simulator or computer system in which a constructed neural network can be easily used in other processing programs.

[Means for solving the problem] In order to solve the problem, in the first invention,
In a neural network simulator that constructs a neural network and saves it as a file, the constructed neural network is now converted into a high-level language source program with equivalent functionality and saved.

Further, in the second aspect of the present invention, the neural network I-network constructed by the neural network simulator is converted into a high-level language source program having equivalent functions and stored.

[Operation] In order to make it easier to use the neural network constructed by the neural network simulator in other processing programs, the constructed neural network must be converted into an equivalent program in a general high-level language and saved. Just leave it there.

Therefore, in the first aspect of the present invention, a function of converting and storing a high-level language program is provided as a function of the neural network simulator.

Further, in the second invention, a computer system separate from the neural network simulator converts the neural network constructed by the neural network simulator into a source program in a high-level language having equivalent functions and stores the same. .

In this way, other processing programs can link this source program and use this program immediately.

[Example] Hereinafter, an example of the present invention will be described with reference to the drawings.

Note that, in order to make the embodiment easier to understand, the explanation below will focus on a model of a neural network. Therefore, before explaining the neural network simulator, first, this neural network model will be explained.

Here, Figure 6 shows the neural network unit (
Explanatory diagram of the processing of cells (also called neurons), No. 7
The figure is a characteristic diagram showing the input/output characteristics of functions used in unit processing, Figure 8 is an explanatory diagram of the input/output characteristics of the neural network to be constructed, and Figure 9 is the neural network constructed according to this example. FIG. 2 is a block diagram showing a network.

As shown in Figure 6, each unit, which is an element constituting a neural network, receives weighted input wi*ui (i is 0-N, wi is synaptic weight) from other N+1 units via synaptic connections. , ui is the input itself), and for the value X, which is the summation net plus the self-bias, ) is applied to the output value f(net
+bias).

An example of a hierarchical (three-layer) neural network using four units having such input/output characteristics is shown in FIG. 9, which processes exclusive OR.

However, logical values "0" and "1" as shown in FIG.
”, but as shown in FIG. 8(B), the values rO, 9, and r
This is to perform exclusive OR processing on −0 and 9J.

In Fig. 9, each unit 20 to 24 is assigned a number from 0 to 4 (total number of units - 1) for identification, starting from the input unit, and this identification number is used to identify a specific unit in the program. It is used when referring to the output of . In FIG. 9, numbers 1 to 6 are also given to 30 to 35 (also called synaptic links) for reference.

Next, the neural network simulator of this embodiment will be explained with reference to the drawings.

This fruit! As in the past, the simulator in this example is either a neural network simulator as software running on a computer, or a dedicated coprocessor card that is incorporated into another computer for high-speed simulation processing. Therefore, in terms of hardware, it includes the usual elements of a computer system. In the case of a coprocessor card, it further includes an interface part.

FIG. 1 shows the processing of the neural network simulator of this embodiment.

When the neural network simulator starts a series of processes, it first reads information for constructing a neural network from the neural network description language file 40 (steps 151 and (52). For example, the input shown in FIG. 8(B) Loads the output characteristics, the structure of the neural network such as the number of layers, the number of units in the input and output layers, data, various parameters, etc.

Then, a neural network is constructed on the main memory based on the read information (step 153). As a result, for example, a neural network as shown in FIG. 9 is constructed.

Here, the constructed neural network changes its characteristics or state through learning or other means,
This process is omitted if unnecessary (step 154).
．． 155).

Next, the neural network stored in the main memory is converted into a source file 42 of an equivalent high-level language (for example, C language) and saved, and the variables, functions, etc. defined in this source file 42 are Information necessary for reference from other modules (processing programs) is saved as a header file 41 separately from the source file 42 (step 156).

This embodiment is characterized in that the neural network thus constructed is converted into an equivalent high-level language source program (consisting of source files and header files) and saved.

Therefore, such conversion and storage processing (step 156) will be described in further detail with reference to the drawings.

Here, FIG. 10 is a detailed flowchart of such conversion and storage processing, and FIG. 11 is a header file 4 written in C language created for the neural network shown in FIG. 9.
FIG. 12 is an explanatory diagram showing a source file 42 written in C language created for the neural network shown in FIG.

When the neural network simulator starts this conversion/save process, it first retrieves pre-stored sample files for the header file and source file (step 200).

Figures 11 and 12 show the completed header file and source file, respectively, but each sample file contains a variety of specific data that specifies the neural network, compared to what is shown in these figures. is missing, the variable array is undefined, and incomplete. For example, the specific value (definition data) of the constant definition (character string replacement) command #define shown in FIG. 11 is not inserted in the sample file.

Next, the neural network simulator extracts layer information such as the number of layers, the number of units belonging to each layer, and the number of input synapses each unit has from the constructed neural network (step 201). An identification number is attached from there to the output side (step 202).

Then, data regarding the extracted layer information is inserted in a predetermined format at a predetermined position in each sample file, and a variable array is prepared according to the extracted layer information (step 203). For example, in the case of the neural network shown in Figure 9, a constant definition (string replacement) command line r #d that replaces the string variable NINPUT, which indicates the number of units in the input layer, with a value, which is included in the sample header file.
``efine NlNPu ding'' has 2 added, which is the number of incoming crab nits.r #define NINPLIT
It will be 24. Further, for example, in the case of the neural network shown in FIG. 9, the array 1ayer[], which is included in the sample source file and represents the characteristics of the layers, has three elements corresponding to the number of layers, which is three.

When the process of completing the sample header file and sample source file for such layer information is completed, the neural network simulator will process the synapse information such as information indicating which units the synapse connects and synapse weight information. Information is retrieved from the constructed neural network (step 204). Thereafter, an identification number is assigned to each scenario from the input side to the output side (step 205).

Then, data regarding the extracted synapse information is inserted in a predetermined format at a predetermined position in each sample file, and a variable array is prepared according to the extracted synapse information (step 206). In the case of the neural network shown in the figure, the array weight[] that represents synaptic weighting is included in the sample source file.
, the weighting of each synapse 30 to 35 is the coefficient ■, -1, -1
, 1, 1 above in a predetermined format.

Next, the neural network simulator extracts other information other than the layer information and synaptic information from the constructed neural network, such as the gradient gain for the function f (x) that defines the input-output characteristics of the unit (step 207). Then, data regarding the extracted other information is inserted in a predetermined format at a predetermined position in each sample file, and a variable array is prepared according to the extracted other information (step 208). For example, in the case of the neural network shown in Figure 9, a constant definition is created that replaces the string GAIN that specifies the gain for the function f (x), which is included in the sample header file, with a value. #defne
GAIJ is r#defineGAIN (5,0OOOO
It becomes Oe+00h.

In this way, when the header file 41 and source file 42 are completed by performing processing such as inserting data into the sample header file and sample source file, the completed header file 41 and source file 42 are saved in the auxiliary storage device ( Step 209>.

In addition, if calculations are required for processing such as inserting data into the sample header file and sample source file,
Calculations are being performed as appropriate.

Next, a specific example of the header file 41 shown in FIG. 11 will be explained.

As described above, this header file 41 describes definitions of various constants used in the source file 42 and external definitions of variables and functions referenced from other modules (processing programs).

This header file 41 has a comment 250 indicating that it is a header file, and a plurality of constants 251 used internally in the header file 41 and source file 42 are described. Here, the constant ON relates to one value of the input to the neural network, and the constant OFF relates to the other input value.

Detailed explanations of other constants will be omitted.

Next, declare a structure that manages information about the N-layer structure of the neural network (typedef declaration> 252
is described. This declaration holds information such as the number of the unit belonging to the layer and from which layer it receives input. In order to be compatible with neural networks having more complex structures, various information other than these is held in the structure layer, but a detailed explanation thereof will be omitted.

Next, a plurality of constants and macro definitions 25 used internally in the header file 41 and source file 42 are defined again.
3 is described. The constant and macro definitions here are:
Since it specifies the structure of a neural network in detail, the meaning of each is as follows.

NINPUT Number of incoming crab nits Nt (IDDE
N Number of intermediate units N [)U 'r P U
r Number of output units NUNIT Total number of units NWEIGHT Total number of synapses bias(un
itno) Macro definition for referring to unit bias NLAYER Number of layers GAIN Gradient of unit input/output characteristic function Finally, various external definitions 254 are described.

The meanings of these external definitions are as follows.

float unit[] Variable that holds the output of the unit float veight[] Variable that holds the E/nub load float *1unit Variable that holds the output of the input unit float *hunit Variable that holds the output of the intermediate unit float *unit Output unit A variable that holds the output of void activate() A function that causes the biral network to execute calculations. Note that this header file 41 includes the file names net,
An h is attached.

Next, a specific example of the source file 42 shown in FIG. 12 will be explained.

As described above, this is a processing program that has functions equivalent to the neural network constructed by the neural network simulator (see FIG. 9).

After the comment 3゜O indicating that it is a source file, a compiler control line 301 is written that reads macro files math and h for mathematical functions so that pre-prepared mathematical functions can be used. . Furthermore, a compiler control line 302 for reading a header file (file name net, h) 41 saved at the same time as this source file 42 is written.

Next, an array (1ayer[l) 303 that holds structures t and ayer, which are external variables, is described. Each element of the array manages data for one layer, and corresponds to data for the input layer, intermediate layer, and output layer in order from the first element.

Additionally, an array (unit[
]>]34 are described. Here, the variable 5UNI which specifies the number of elements in the array is specified in the header file 4 mentioned above.
This is a constant indicating the number of units in the entire neural network defined as 1.

Next, variables 305 to 307 are described to facilitate reference to the values of input units, intermediate units, and output units. Each of these variables is a pointer pointing to the address of a specific element of the array unit.

After that, an array (veight[] > 308) that holds the synaptic weights of the neural network is written. The synaptic weights are held in this array in the order convenient for calculation. In this example, the synapses 31 to 36 are Retained.

Next, an array of external variables (wfro
m[] and WtO[] ) 309 and 310 are described. These external variables wfrom and wt.

holds numbers indicating the unit serving as the starting point and the unit to which it is connected. The order in which they are held follows the order of the external variable weight[l described above.

Furthermore, an array (Bias[l) 311 of external variables that holds the bias of the intermediate unit and output unit is described. Each bias is stored in this array in order of convenience for calculation. In this example units 22 to 24
Bias is preserved in the order of .

Following the definition of such variables, a processing routine for the neural network is described starting with a comment 312 indicating that it is a processing routine for the neural network.

This processing routine consists of a processing routine that sequentially activates each unit in the middle layer and the final layer, and a processing routine that calculates the output of each unit. is used.

That is, before writing the processing routine that activates the unit, a function declaration (coutput□) 313 related to the processing routine that calculates the unit output is defined, and then a declaration 313 indicating that the processing routine is to be activated is defined. A comment 314 is written, and a function (activate) 315, which is expressed in the format void activate□, is written as the processing routine to be activated last.

This processing routine 315 is responsible for the function of causing the neural network to execute calculations.

In addition, the input to the two-reciprocal network is the external variable unit[o]...unit applied to the input unit.
[NINPUT-11 (In this example, hole crab knit is 2
This processing function ac
This is a prerequisite for calling tvate. That is, when another processing module uses the source file 42, the external variable unit[o] = unit[NIN
PUT-1]. Here, the constant NINPUT indicates the number of input nits as described above, and its value is defined in the header file 41.

According to this processing routine 315, the output of the intermediate unit and output unit is saved in the external variable unit[], so if other processing modules want to refer to the output of the unit, they can refer to those variables. .

Following the description of the processing routine that activates such a unit, there is a description of the processing routine that calculates the unit output. The processing routine that calculates this unit output has a comment 316 indicating that it is such a processing routine, and a flo that actually performs the calculation.
at output (net) and float ne
It consists of a description 317 of the processing routine itself (function output) specified in the format t.

As mentioned above, this processing routine includes the intermediate unit,
This determines the output of the output (see Figure 7).
. Further, this processing routine is called by the processing routine (function activate) that activates the units in order to calculate the output of each unit, and is not called directly by other processing modules.

Note that the source file 42 shown in FIG. 12 is given the filter name net, c.

As mentioned above, the neural network constructed by the neural network simulator has been converted into the source file 42 and header file 4■ written in a high-level language, so you can construct and initialize the neural network by reading the data file as in the past. Users can use neural networks built with other processing modules without having to create their own modules (corresponding to header files) or modules (corresponding to source files) that cause the neural network to perform calculations.

FIG. 13 is an explanatory diagram showing an example of a processing module using the constructed neural network, that is, a processing module (file name xor, c) using a header file 41 and source file 42 written in a high-level language , FIG. 14 is an explanatory diagram showing the output result of executing the processing module of FIG.

The processing module shown in FIG. 13 uses the neural network shown in FIG. It is to be displayed.

Following the comment 350 regarding the processing module, the file 5tdio, h prepared in the system for executing the C language program, and the above-mentioned header file net, h (Fig. 11) are included in the file xor, c. Compiler control rods 351 and 352 are written to instruct the program to be executed.The compiler control rods 352 control various constants necessary for executing the program, external variables defined in the source file 42 shown in FIG. Functions can be referenced and called.

The function (m
ain) 354 displays the item names of the truth table and sequentially calls a function XOr, which will be described later.

The function described next with a comment 355 (xor>356 sets the external variables unit[o] and unit[1] depending on whether the input cover turns bo and bl are 0 or
] to -0, 9 or 0.9, and the above function act
ivate and causes the neural network to perform calculations. Then, the re-input value unit[0]
, Unit[1] and the obtained output value unit[4] are displayed at predetermined positions.

The processing module file X shown in FIG.
Or, C and the source file net shown in Figure 12,
By compiling, linking, and executing c, the processing results shown in FIG. 14 are obtained.

In Figure 14, the string promptx is a prompt from the computer, and the command line starting from this prompt (prompt%cc -oxpor, c-1m
) 400 is an instruction to compile and link files net, c and xor, c. The option -oxor in this line instructs that the name of the linked executable file be xor.

Option -In instructs to link a library of mathematical functions. This is written because the function aCtiVate in the source file 42 calls the mathematical function tanh.

Three lines 401 following this line indicate that the computer has finished compiling and linking.

The next command line (prompt%xor) 402 instructs to execute the executable file xor, and the result 403 of that execution appears subsequently.

As described above, according to the embodiment described above, the characteristics and other data of the neural network constructed by the neural network simulator are converted and saved together with the necessary functions into a high-level language source program. can be linked to other programs as a module, and the following effects can be expected.

The characteristics of the neural network are already incorporated into the program, and the effects of eliminating the need for a data file that retains the characteristics of the neural network are as follows.

(1) Only the executable file can be managed, eliminating the need to manage two files, an executable file and a data file, as in the past.

(2) There is no need for the user to create a function for reading a data file and initializing the neural network, and a function for causing the neural network to perform calculations.

(3) Since it is not necessary to initialize the neural network by reading a data file, the time required to initialize the program is reduced.

(4) Less space is used in auxiliary storage (disk, etc.).

Furthermore, the effects of converting a neural network into a high-level language source program with equivalent functionality are as follows.

(1) It is easy to port the neural network simulator not only to the processing system in which it operates, but also to other processing systems.

(2) It is easy for the user to modify and improve the program.

Note that in the above-described embodiments, a neural network description language was used to construct the neural network, but any other method that allows constructing a neural network may be used.

In addition, although the neural network was constructed on the main memory of the computer system, this construction is not limited to the main memory, and can be placed in any suitable area such as the main memory of a coprocessor card (auxiliary memory such as a disk). Including equipment〉
It can be done on top of.

Furthermore, the language of the source file does not necessarily need to be C language, and any high-level language such as FORTRAN can be used.

Further, in the above description, the high-level language conversion/storage function is provided as a function of the neural network simulator, but it may be provided as a function of the computer system rather than as a function of the neural network simulator. Note that in this case, since data files are also saved, the effect in terms of file management is less than that of the above embodiment.

[Effects of the Invention] As described above, according to the present invention, the constructed neural network is converted into a high-level language source program and saved, making it easier for the user to use it in other processing programs. This further improves usability.

[Brief explanation of drawings]

Fig. 1 is a processing flowchart of an embodiment of the neural network simulator according to the present invention, Fig. 2 is a block diagram showing an example of an implementation configuration of the software simulator, and Fig. 3 is a block diagram showing an example of an implementation configuration using a Cobrocessor card. , FIG. 4 and FIG. 5 are respectively explanatory diagrams of how other processing programs utilize the neural network constructed by the conventional neural network simulator, FIG. 6 is an explanatory diagram of the processing of the neural network unit, and FIG. The figure is a characteristic diagram showing the input/output characteristics of functions used in unit processing, Figure 8 is an explanatory diagram of the input/output characteristics of the neural network to be constructed, and Figure 9 is the neural network constructed according to the above example. Figure 1O is a detailed flowchart of the process of converting and saving a high-level language program, and Figure 11 is an explanation showing the header file 4 in C language created for the neural network shown in Figure 9. figure,
Fig. 2 is an explanatory diagram showing a source file 42 in C language created for the neural network shown in Fig. 9, and Fig. 13 shows an example of another processing program using the constructed neural network. Explanatory diagram, FIG. 14 is an explanatory diagram showing the output result of executing the processing program of FIG. 13. 41... Header file, 42... Source file, 156... Conversion and storage processing step into a high-level language source program.

Claims (2)

[Claims] (1) A neural network simulator that builds a neural network and saves it as a file is characterized by converting the built neural network into a high-level language source program with equivalent functionality and saving it. Neural network simulator. (2) A computer system characterized in that a neural network constructed by a neural network simulator is converted into a high-level language source program having equivalent functions and stored.