CN112270406B - Nerve information visualization method of brain-like computer operating system - Google Patents

Abstract

The invention discloses a neural information visualization method for a brain-inspired computer operating system, comprising the following steps: obtaining a connection description file of a pulse neural network, and performing global coding on neuron nodes of each layer according to the connection description file, and each neuron node obtains A globally unique identifier, each layer of neuron nodes is aggregated and represented by different colors, and the connection relationship of neuron nodes between adjacent layers is represented by edges and distinguished by different colors; according to the number of neuron nodes and the number of edges The weight calculates the importance of each neuron node, and selects the neuron node for visualization according to the importance. In order to solve the problems of many connection description files, a large number of neuron nodes, and many connections between neuron nodes that are difficult to distinguish. Realize the reasonable visualization of the neural information of the brain-like computer operating system, and facilitate the understanding of the working process of the spiking neural network.

Description

A Neural Information Visualization Method for Brain-like Computer Operating System

technical field

The invention belongs to the technical field of new computers, and in particular relates to a neural information visualization method for a brain-inspired computer operating system.

Background technique

Since its inception in 2006, deep learning has been highly concerned by scientific research institutions and the industry. It has made great progress in many fields such as image and voice, and has surpassed traditional algorithms in many fields. Spiking Neuron Networks (SNN-Spiking Neuron Networks) are often hailed as the third generation of artificial neural networks. The first generation of neural networks was the perceptron, which was a simple model of neurons and could only process binary data. The second-generation neural network includes a wide range, including the BP neural network with more applications. The spiking neural network is closer to the actual simulation of the connection relationship and behavior of brain neurons.

At present, there are many tools for visualization of convolutional neural network structure publicly available on the Internet, mainly including: 1) Netscope: It uses the model definition file as input to obtain the visual structure diagram of the neural network. It is a web-based visual neural network topology tool , only supports the caffe deep learning framework of the University of California, Berkeley. 2) ConvNetDraw: Use visualization commands as input to visualize the output neural network model structure, and it is also a web-based tool. It is displayed in the model of the structural block, and the proportion of the structural block can be adjusted in three dimensions, which is very visual and intuitive. 3) Netron: The model definition file and model weight file (can be defaulted) are used as input to obtain the visual structure diagram of the neural network. It is also based on the webpage and uses js and python. It can support ONNX, Keras, CoreML, TensorFlow, Mainstream deep learning frameworks such as caffe and MXNET.

Cao Lihong from Communication University of China disclosed a method for visualizing large-scale neural networks in 3D in a patent application with publication number CN106372721A, showing the structure of neural networks in 3D; Qiu Chunfang from Shanghai Institute of Precision Metrology and Testing et al. The patent application with the publication number CN107392085A discloses a method for visualizing convolutional neural networks, which can be well demonstrated to help understand convolutional neural networks and to explore the superiority of convolutional neural networks; Beijing Computer Technology The patent application of CN110782031A disclosed by the Research Institute of Technology and Application discloses a multi-frame convolutional neural network model structure visualization and network reconstruction method, which can intuitively modify different layers to achieve network reconstruction, and at the same time can change the properties of the neural network and The display is updated in real time.

Due to the complexity of the spiking neural network structure of the brain-like computer operating system, when visualizing the spiking neural network structure of the brain-like computer operating system, there will be many connection description files, a large number of neuron nodes, and connections between neuron nodes. There are many lines that are difficult to distinguish. Therefore, all the visualization methods mentioned above are not suitable for the visualization of the spiking neural network structure of the brain-like computer operating system.

Contents of the invention

The purpose of the present invention is to provide a neural information visualization method for a brain-inspired computer operating system to solve the problems of many connection description files, a large number of neuron nodes, and many connections between neuron nodes that are difficult to distinguish.

In order to realize the above-mentioned purpose of the invention, the present invention provides the following technical solutions:

A neural information visualization method for a brain-inspired computer operating system, comprising the following steps:

Obtain the connection description file of the spiking neural network, and globally encode the neuron nodes of each layer according to the connection description file. Each neuron node obtains a globally unique identifier. The connection relationship of neuron nodes between layers is represented by edges and differentiated by different colors;

The importance of each neuron node is calculated according to the number of connected edges of the neuron node and the weight of the connected edge, and the neuron node is selected for visualization according to the importance.

Preferably, all neuron nodes in the same layer of the spiking neural network are aggregated and expressed in the same color, and the aggregated representations of neuron nodes in two adjacent layers are also in adjacent positions, which is convenient for neuron nodes in adjacent two layers. connection relationship between.

Preferably, neuron nodes in each layer are aggregated into geometric shapes for representation. The geometric shape may be a smooth geometric shape without corners such as a circle or an ellipse, or may be a geometric shape with corners such as a matrix or a triangle.

Preferably, all connections between neuron nodes in the two layers are represented by edges of the same color. To improve visualization clarity, neuron nodes are colored differently from their corresponding edges.

Preferably, when performing neural information visualization, the neuron information layout is stored in a data structure, which includes data units, connection relationship units, and layer units, wherein the data units are used to store the global unique identification of neuron nodes, Visualize the position coordinates and the level; the connection relationship unit is used to store two neuron nodes and the visual color of the edge between two neuron nodes; the layer unit is used to store the level name.

Due to the large number of neuron nodes, choose whether to display a certain neuron according to the number of connections between the neuron node and other neurons and the weight of the connections. If the neuron nodes are sampled too much, the neuron nodes will be too dense after visualization, and the connection relationship between each neuron node cannot be seen; if the neuron nodes are sampled too little, the main information will be lost. The main structure of the spiking neural network cannot be observed. In order to perform a reasonable sparse sampling of neurons, the importance of neuron nodes needs to be calculated and sorted. Preferably, the following formula is used to calculate the importance of each neuron node:

表示第i个神经元节点node_i的正则化以前神经元的绝对重要性，InNum代表神经元节点的入度，w₁是神经元节点的入度系数，OutNum代表神经元节点的出度，w₂是神经元节点的出度系数，Weight代表神经元节点连边权重的绝对值之和，w₃是神经元节点的权重系数，/>

是最终的神经元节点的重要性，/>

表示第j个神经元节点node_j的正则化以前神经元的绝对重要性，n表示神经元节点的总个数。其中，将接入当前神经元节点的连边个数作为当前神经元节点的入度，将当前神经元节点接出的连边个数作为当前神经元节点的出度。in,

Indicates the absolute importance of neurons before the regularization of the i-th neuron node node _i , InNum represents the in-degree of the neuron node, w ₁ is the in-degree coefficient of the neuron node, OutNum represents the out-degree of the neuron node, w ₂ is the out-degree coefficient of the neuron node, Weight represents the sum of the absolute value of the neuron node’s edge weight, w ₃ is the weight coefficient of the neuron node, />

is the importance of the final neuron node, />

Indicates the absolute importance of the neuron before the regularization of the jth neuron node node _j , and n indicates the total number of neuron nodes. Wherein, the number of connected edges connected to the current neuron node is taken as the in-degree of the current neuron node, and the number of connected edges connected to the current neuron node is taken as the out-degree of the current neuron node.

Preferably, when selecting neuron nodes for visualization based on importance, an importance threshold is set, and neuron nodes with importance greater than the importance threshold are selected for visualization. In this way, the more edges connected to the neuron node and the greater the absolute value of the weight of the edge will be preferentially reserved.

Compared with the prior art, the beneficial effects of the present invention at least include:

The neural information visualization method of the brain-like computer operating system provided by the present invention solves the problem of too many connection description files by globally encoding neurons, and screens and visualizes neurons according to the importance of neuron nodes. The chemical processing method solves the problem that it is difficult to display too many neuron nodes. At the same time, the connection relationship is expressed in the form of edges and differentiated by different colors, which solves the problem that the connections between neuron nodes are too many and difficult to distinguish. Realize the reasonable visualization of the neural information of the brain-like computer operating system, and facilitate the understanding of the working process of the spiking neural network.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a flowchart of a neural information visualization method for a brain-inspired computer operating system provided by an embodiment of the present invention;

Fig. 2 is the internal data structure of the connection description file of the spiking neural network provided by the embodiment of the present invention;

Fig. 3 is the internal data structure of the intermediate json file used for neural information visualization provided by the embodiment of the present invention;

Fig. 4 is the quaternion inside the data field of the json file that the embodiment of the present invention provides;

Fig. 5 is the triplet inside the links field of the json file provided by the embodiment of the present invention;

Fig. 6 is a neural information visualization diagram of an EEG simulation pulse neural network provided by an embodiment of the present invention;

Fig. 7 is a visualization diagram of neural information of the memory model spiking neural network provided by the embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, and do not limit the protection scope of the present invention.

In the process of visualizing the neural information of the spiking neural network of the brain-like computer operating system of the Darwin 2 neuromorphic chip of Zhejiang University, it was difficult to encounter many connection description files, a large number of neuron nodes, and many connections between neuron nodes. problem of distinction. In order to solve the problem of difficulty in visualizing neural information due to these problems, an embodiment of the present invention provides a neural information visualization method for a brain-inspired computer operating system. Description files for neural information visualization.

Fig. 1 is a flowchart of a neural information visualization method for a brain-inspired computer operating system provided by an embodiment of the present invention. As shown in Figure 1, the neural information visualization method includes the following steps:

Step 1, according to the connection relationship between the layers of the spiking neural network, read the connection description file of the spiking neural network in the order from the input layer to the output layer.

The connection description file of the spiking neural network describes the connection relationship between neurons and neurons in the spiking neural network. There are multiple connection description files of the spiking neural network, and each file describes the connection relationship between two layers of neurons. Figure 2 is the internal data structure of the connection description file of the spiking neural network. As shown in Figure 2, each line of the connection description file is a quaternion, and the first field of the quaternion is the number in the starting neuron layer, that is, the number of the starting neuron in the starting layer. The second field is the number in the target neuron layer, that is, the number of the target neuron in the target layer, and the third field is the weight, that is, the weight of the neuron’s processing input value, which is the inherent attribute of the neuron’s edge. The fourth field is the spike delay, that is, the delay of the spike signal from the originating neuron to the target neuron.

Step 2. According to the read spiking neural network connection description file, re-number the neuron nodes of each layer globally, convert the intra-layer number into a globally unique identifier, and determine the color of each layer of neurons to form the properties of the neurons themselves.

Step 3, according to the connection description file and the neuron node's own attributes, generate a triplet describing the connection relationship of the node (starting neuron number, target neuron number, edge color), and the color of the edge is based on the location of the edge connection neuron. The layer is determined.

In step 4, neuron nodes are laid out according to the connection relationship between neurons in the spiking neural network. Neuron nodes of the same layer are aggregated and placed in the same area, and neurons of adjacent layers are aggregated and placed in adjacent positions.

Step 5, choose whether to display a certain neuron according to the number of connections between the neuron and other neurons and the weight of the connections.

Because there are too many neuron nodes and connections in the original spiking neural network, if all of them are displayed, all the nodes and connections will be crowded together, and the key structure of the spiking neural network cannot be displayed, so the spiking neural network needs to be sparse displayed. If the sampling of neurons is too much, the neurons after visualization will be too dense, and the connection relationship between each neuron cannot be seen; if the sampling of neurons is too small, the main information will be lost, and the pulse cannot be observed. The main structure of the neural network. The specific method is to calculate the importance of the neuron according to the number of connections between the neuron and other neurons and the weight of the connection, and sort the neurons according to the importance, and the neurons with greater importance will be reserved first.

The formula for the importance of a neuron is as follows:

After calculating the importance of neurons, set the importance threshold ∈(∈∈(0,1)) to control which neurons are displayed. Neurons whose importance is greater than the threshold will be displayed, and neurons less than the threshold will be hidden. .

Step 6, export the json file describing the neuron layout and edge color, and use echarts to visualize the exported json file.

In the embodiment, the python code is used to generate the json file, and the graph type of echarts is used to visualize the json file. Figure 3 is the data structure of the json file, which contains three fields: data unit data, connection relationship unit links and layer unit layers. The data field is an array, and each element inside is a quadruple, which contains the key value name, position coordinates (x, y) and layer information of the neuron node, as shown in Figure 4. The links field is also an array, and each element inside is a triplet, which contains the key value source of the starting neuron node, the key value target of the target neuron node, and the color lineStyle of the connection, as shown in the figure 5.

Using the neural information visualization method of the above-mentioned brain-like computer operating system, the neural information visualization results of the EEG simulated spiking neural network and the memory model spiking neural network are shown in Figures 6 and 7. The spiking nerves are clearly shown in the figure. The structure of the network, that is, it shows the neuron nodes of the spiking neural network and the connection relationship between each neuron node.

The above-mentioned specific embodiments have described the technical solutions and beneficial effects of the present invention in detail. It should be understood that the above-mentioned are only the most preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, supplements and equivalent replacements made within the scope shall be included in the protection scope of the present invention.

Claims (3)

1. A neural information visualization method of a brain-like computer operating system, characterized in that, comprising the following steps: Step 1, obtain the connection description file of the spiking neural network, where each line of the connection description file is a quaternion, the first field of the quaternion is the number in the starting neuron layer, and the second field is the target The number in the neuron layer, the third field is the weight, which is the inherent attribute of the neuron edge, and the fourth field is the pulse delay; Step 2. Globally code the neuron nodes of each layer according to the connection description file, each neuron node obtains a globally unique identifier, and at the same time determine the color of each layer of neurons to form the properties of the neuron itself; Step 3, according to the connection description file and the neuron node's own attributes, generate a triplet describing the connection relationship of the node (starting neuron number, target neuron number, edge color), and the color of the edge is based on the location of the edge connection neuron. layer determination; Step 4. The neuron nodes are laid out according to the connection relationship between neurons in the spiking neural network. The neuron nodes of each layer are aggregated and displayed with different colors. The neuron nodes of the same layer are aggregated and placed in the same area, and aggregated into geometric The neurons of adjacent layers are aggregated and placed in adjacent positions, which facilitates the representation of the connection relationship between neuron nodes of two adjacent layers. The connection relationship of neuron nodes between adjacent layers is represented by edges and adopts Different color distinction; Step 5, calculate the importance of each neuron node according to the number of connected edges and the weight of the connected edges of the neuron node, set the importance threshold, and screen the neuron nodes whose importance is greater than the importance threshold for visualization, where the following formula is used Compute the importance of each neuron node:

in,

Indicates the absolute importance of neurons before the regularization of the i-th neuron node node _i , InNum represents the in-degree of the neuron node, w ₁ is the in-degree coefficient of the neuron node, OutNum represents the out-degree of the neuron node, w ₂ is the out-degree coefficient of the neuron node, Weight represents the sum of the absolute value of the neuron node’s edge weight, w ₃ is the weight coefficient of the neuron node, />

is the importance of the final neuron node, />

Indicates the absolute importance of the neuron before the regularization of the jth neuron node node _j , and n indicates the total number of neuron nodes; Among them, the neuron information layout is stored in a data structure, which includes data units, connection relationship units, and layer units, where the data units are used to store the globally unique identifiers, visual position coordinates, and levels of neuron nodes; the connection The relationship unit is used to store two neuron nodes and the visual color of the edge between two neuron nodes; the layer unit is used to store the layer name. 2. The neural information visualization method of a brain-inspired computer operating system as claimed in claim 1, wherein all connections between neuron nodes of the two layers are represented by edges of the same color. 3. The neural information visualization method of a brain-inspired computer operating system as claimed in claim 1, wherein the color of a neuron node is different from the color of its corresponding edge.

Images