CN103488662A - Clustering method and system of parallelized self-organizing mapping neural network based on graphic processing unit - Google Patents

Abstract

The present invention relates to a parallel self-organizing map neural network clustering method and system based on a graphics processing unit. Compared with the traditional serial clustering method, the present invention adopts the parallelization of the algorithm and the parallel acceleration system based on the graphics processing unit , can achieve clustering of large-scale data faster. The present invention mainly relates to the contents of two aspects: (1) at first, aiming at the characteristics of the high parallel computing capability of the graphic processing unit, a kind of clustering method of parallelized self-organizing map neural network is designed, and this method is through parallelized statistics document The word frequency matrix is obtained from the keyword frequency, the feature matrix of the data set is generated by parallelizing the feature vector of the text, and the cluster structure of massive data objects is obtained through the parallel self-organizing map neural network clustering; (2) secondly, using the graphics processing unit Based on the complementarity of computing power between (GPU) and central processing unit (CPU), a set of parallel text clustering system based on CPU/GPU cooperation framework is designed.

Description

Self-organizing map neural network clustering method and system based on graphics processing unit

technical field

The invention relates to a parallel self-organizing map neural network clustering method and system, in particular to a graphic processing unit-based parallel self-organizing map neural network clustering method and system.

Background technique

At present, with the popularization of computers, the number of Internet users continues to increase, and Internet users generate a large amount of information on the Internet every day. At the same time, in some social media systems with a large number of users, a large amount of new data is added every day. Data mining and machine learning algorithms provide feasible methods for us to extract valuable information from these data, but most of the algorithms have complex learning processes, require iterative learning, and take a long time to process massive data. Although useful information is extracted, the information may no longer be time-sensitive, which requires the development of faster algorithms or the use of higher-performance computing devices. The use of high-performance machines or CPU clusters can certainly speed up the calculation process of algorithms, but enterprises need to bear huge capital investment. At present, multi-core technology has developed relatively maturely. The numerical calculation performance of graphics processing unit (GPU) far exceeds the performance of CPU. Using the multi-core characteristics of GPU to fully explore the parallel ability of algorithms has become a research hotspot in computer science today.

In the field of data mining, some data mining algorithms have been improved to run on graphics processing unit devices, and achieved at least 5-6 times of acceleration, and some even achieved 20-30 times of acceleration. An important research direction in the field of data mining is the mining of text data, and text clustering plays an important role in the field of text mining. Clustering is based on the characteristics of the data, according to the degree of similarity between the data, clustered into different text clusters. According to statistics, 80% of the information in human society exists in the form of text. Text clustering technology can effectively organize, summarize and navigate text data.

The SOM network is an artificial neural network designed by simulating the characteristics of the human brain's processing of external information. It is an unsupervised learning method and is very suitable for clustering problems of high-dimensional text data. SOM (Self-Organizing Mapping, referred to as "SOM") network does not require the user to specify the number of clusters, the network will adaptively cluster during the training process, it is not sensitive to outlier noise data, and has a strong anti-noise ability . SOM clusters according to the sample distribution law in the training samples, and is not sensitive to the shape of the data. However, the existing SOM algorithm has the characteristics of slow network convergence and long clustering time when dealing with high-dimensional data.

Text clustering is a kind of data mining technology, which divides text document resources into several clusters according to the specified similarity standard, so that the interior of each cluster is as similar as possible, and the similarity between different clusters is as small as possible. Text clustering is mainly based on the well-known clustering assumption: documents of the same class have a greater similarity, while documents of different classes have a smaller similarity. As an unsupervised machine learning method, clustering has certain flexibility and high automatic processing ability because it does not require a pre-training process and does not need to manually label categories in advance. An important means of effectively organizing, summarizing, and navigating has attracted the attention of more and more researchers.

Contents of the invention

The technical problem that the present invention solves is: build a kind of parallelized self-organizing mapping neural network clustering method and system based on graphics processing unit ((Graphic Processing Unit, graphics processing unit, referred to as "GPU")), overcome existing technology in the text During the clustering process, due to the large amount of data, the calculation speed is slow.

The technical scheme of the present invention is: provide a kind of parallel self-organizing map neural network clustering method based on graphics processing unit, comprise the steps:

Parallel keyword frequency statistics: segment the text content into words and get a set of keywords, and count the frequency of keywords in the document in parallel to get a word frequency matrix;

Parallel eigenvector calculation: convert the keyword frequency matrix into a corresponding eigenvector matrix, and each eigenvector represents a document.

Parallel SOM clustering: Design the SOM network structure according to the eigenvector matrix, initialize the SOM network, calculate the distance between the input sample and all output neuron weight vectors in parallel, compare the size of each distance, and obtain the best neuron J with the smallest distance. The best neuron, the neuron weight vector value in its neighborhood, the learning rate and the neighborhood size of the best neuron, and then calculate the network error rate E _t in parallel through the graphics processing unit, if the network error rate E _t <= target error ε Or the number of iterations t>=the maximum number of training iterations T, then the SOM network training ends, otherwise a new round of training is performed; the result of each learning makes the neighborhood area of the best matching neuron approach the input data vector value, and the distance Similar input feature vectors are aggregated into the same cluster, and the cluster set formed is the final clustering result.

A further technical solution of the present invention is: the process of counting the keyword frequency of each document is independent of each other, and the present invention designs a thread for each document to count the word frequency, and then uses the multi-threaded parallel statistics of the graphics processing unit.

A further technical solution of the present invention is: the feature vector calculation process of each document is independent of each other, the present invention designs a thread for each document to calculate the feature vector, and then executes concurrently through the multi-thread of the graphics processing unit. Its eigenvectors are calculated using the formula

x _ij = log ₂ (tf _ij +1.0)*log(m/m _j ),

and normalized to $x_{\mathrm{ij}}=\frac{x_{\mathrm{ij}}}{\sqrt{\underset{p=1}{\overset{\mathrm{<figure-callout\; id="2"\; label="no\; x\; ip"\; filenames="DEST\_PATH\_131024142412.png"\; state="\{\{state\}\}">no</figure-callout>}}{\mathrm{\&Sigma;}}}{x}_{\mathrm{ip}}^{}{}_2^{}}}.$

In the formula, x _ij is the value of the feature vector of the jth feature word in the document d _i , tf _ij is the number of occurrences of the jth feature word in the document d _i , m/m _j is the inverse of the jth feature word Document frequency, m is the total number of documents, and m _j is the number of documents containing the jth feature word.

A further technical solution of the present invention is: in the parallel feature vector calculation step, the feature vector of each document is calculated in parallel using multi-threads based on graphics processing.

A further technical solution of the present invention is: the calculation process of the distance between the input feature vector and the weight vector of each output neuron is independent of each other, and the distance between the input feature vector and each output neuron vector is calculated in parallel by multiple threads based on graphics processing, and the system Open a thread for each neuron, and use multi-threaded parallel computing.

A further technical solution of the present invention is: the calculation process of the weight vector error of two adjacent iterations of each neuron is independent of each other, and multiple threads based on graphics processing are used to calculate the weight vector error of each neuron in parallel. The neuron starts a thread and uses multi-threaded parallel computing.

The technical solution of the present invention is: build a kind of self-organizing map neural network clustering system based on graphics processing unit, including hardware part and software part, hardware part: adopt CPU/GPU cooperative frame design, serial execution code runs on CPU , the parallel execution code runs on the GPU, and the data between the video memory and the memory is exchanged through the data transmission method provided by the GPU; the software part is divided into three modules, including the parallel keyword frequency statistics module, the parallel feature vector calculation module, Parallelized SOM clustering module, unit, eigenvector calculation unit for calculating eigenvectors, and text clustering unit for text clustering, the parallelized keyword frequency statistics module divides text content into words and obtains a set of keywords, parallel The frequency of the keyword in the statistics document obtains the term frequency matrix; the parallelized feature vector calculation module converts the keyword term frequency matrix into a corresponding feature vector matrix, and each feature vector represents a document; the parallelized SOM clustering module according to Design the SOM network structure by eigenvector matrix, initialize the SOM network, calculate the distance between the input sample and all output neuron weight vectors in parallel, compare the size of each distance, obtain the best neuron J with the smallest distance, and update the best neuron and its neighbors The neuron weight vector value, learning rate and the neighborhood size of the best neuron in the domain, and then calculate the network error rate E _t in parallel through the graphics processing unit, if the network error rate E _t <= target error ε or iteration number t >= If the maximum number of iterations of training is T, the SOM network training ends, otherwise a new round of training is performed; the result of each learning makes the neighborhood area of the best matching neuron close to the input data vector value, and the input feature vectors with similar distances are gathered into the same cluster, the formed cluster set is the final clustering result.

A further technical solution of the present invention is: several kernel functions are designed in the parallel keyword frequency statistics module, the parallel feature vector calculation module and the parallel SOM clustering module to accelerate the operation of the algorithm in parallel.

The further technical scheme of the present invention is: in the parallel keyword word frequency statistics module, designed a kernel function that is used for keyword word frequency statistics; In the parallel feature vector calculation module, designed two kernel functions that are used for feature vector calculation and two kernel functions for eigenvector normalization.

The further technical solution of the present invention is: in the parallel SOM clustering module, a kernel function used to calculate the distance between the input feature vector and the output neuron is designed, and a network used to calculate the adjacent two iterations of each neuron A kernel function for the error of the weight vector and a kernel function for reducing the error of the network weight vector.

The technical effect of the present invention is: the present invention is a set of parallel self-organizing map neural network clustering method and system based on graphics processing unit. The invention designs a parallelized text clustering algorithm based on the CPU/GPU collaboration framework by designing a parallelized text clustering algorithm and utilizing the complementarity of computing power between the graphics processing unit (GPU) and the central processing unit (CPU). clustering system. Specifically, the present invention includes two parts: 1. A clustering method based on a parallel self-organizing neural network of a graphics processing unit is designed. In this method, three aspects of document keyword frequency statistics, document feature vector calculation and SOM clustering algorithm are parallelized. 2. Developed a set of parallel text clustering system based on CPU/GPU cooperation framework. In the system, the present invention designs three calculation modules: a parallel keyword frequency statistics module, a parallel feature vector calculation module and a parallel SOM clustering module. At the same time, several kernel functions are designed on each module to speed up the operation of the algorithm. The invention can achieve clustering of large-scale data faster through the parallelization of algorithms and the parallel acceleration system based on graphics processing units, and is very suitable for processing clustering problems similar to high-dimensional text data.

Description of drawings

Fig. 1 is a flowchart of the present invention.

FIG. 2 is a schematic diagram of multi-threaded word frequency statistics in the present invention.

Figure 3 is a schematic diagram of the statistics of the number of documents where the serial keywords are located.

Fig. 4 is a diagram of the parallel characteristic matrix calculation process of the present invention.

Fig. 5 is a schematic diagram of statistics of the number of documents where parallel keywords are located in the present invention.

FIG. 6 is a schematic diagram of a multi-thread computing characteristic matrix of the present invention.

Fig. 7 is a schematic diagram of the multi-thread calculation vector model of the present invention.

FIG. 8 is a schematic diagram of multi-thread normalization in the present invention.

Fig. 9 is the topology structure of the SOM network of the present invention.

Fig. 10 is a CPU/GPU hardware architecture diagram of the present invention

Fig. 11 is a schematic diagram of the parallel SOM algorithm CPU/GPU cooperation framework of the present invention

Fig. 12 is the flow chart of the parallel statistical document word frequency matrix kernel function of the present invention

Fig. 13 is the kernel function flow chart of the present invention's parallel statistical keyword place document number

Fig. 14 is a schematic diagram of distance calculation between an input feature vector and a neuron in the present invention

Fig. 15 is the distance kernel function flow chart of calculating input feature vector and nerve of the present invention

Fig. 16 is a schematic diagram of data difference operation in the present invention.

FIG. 17 is a schematic diagram of the summation operation of the error matrix by row or column according to the present invention.

Fig. 18 is the kernel function flowchart of error matrix summation by row or column of the present invention

Detailed ways

The technical solutions of the present invention will be further described below in conjunction with specific embodiments.

As shown in Figure 1, the specific embodiment of the present invention is: provide a kind of parallel self-organizing map neural network clustering method based on graphics processing unit, comprise the steps:

Step 1: Parallel keyword frequency statistics, namely: segment the text content into words and obtain a collection of keywords; for large-scale text data, the large-scale computing unit of the graphics processing device can provide each text document with a thread parallel statistical document The frequency of keywords is obtained as a word frequency matrix.

The specific implementation process is as follows: Computers do not have human intelligence. After reading an article, people can have a vague understanding of the content of the article according to their own understanding ability, but computers cannot easily "understand" the article. Fundamentally speaking, It only understands 0 and 1, so the text must be converted into a format that the computer can understand. Currently, in the direction of information processing, the representation of text mainly adopts the vector space model (Vectorspace model, referred to as "VSM"). The basic idea of the vector space model is to represent the text as a vector: (X ₁ ,X ₂ ,...X _n ), where X _j is the weight of the jth feature item, so what should be selected as the feature item? Generally, words or words can be selected. According to the experimental results, it is generally believed that selecting words as feature items is better than words and phrases. Therefore, to represent the text as a vector space model with words as units, the text must first be segmented, and the word after segmentation is used as the dimension in the vector space to represent the text. After word segmentation processing is performed on the document content, denoising processing is performed to obtain the keyword set corresponding to the text.

Parallel keyword frequency statistics is to segment each document, after denoising, count the frequency of keywords in the document, and then form the word frequency matrix of the entire data set. Since the process of counting the keyword frequency of each document is independent of each other, a thread can be opened on the graphics processing unit for each document, so as to achieve a high degree of parallelism in calculation, as shown in Figure 2. A row of the term frequency matrix represents a document, a column of the matrix represents a keyword, and the intersection value of the row and column represents the frequency of occurrence of a keyword in a document. If a keyword does not appear in a document, the value is set to zero in the term frequency matrix.

Step 2: Parallel eigenvector calculation, that is, converting the keyword frequency matrix into a corresponding eigenvector matrix through parallel calculation by a graphics processing unit. Each feature vector represents a text document.

The specific implementation process is as follows: According to the keyword frequency matrix, the feature vector corresponding to the document is calculated in parallel to generate a feature vector matrix. One row of the feature vector represents a document, one column of the feature vector represents a feature, and the intersection value of the row and column is a certain value of the document. eigenvalues of a feature.

The present invention uses a vector space model to describe the document, and it only pays attention to the words appearing in the document, and does not pay attention to the relationship between words and the structure of the document. In the vector space model, the document space is regarded as a vector space composed of feature vectors. A document is a feature vector in the vector space, and the feature words in the document can be regarded as dimensions in the vector space model. The feature vector is recorded as d _i =(x _i1 ,x _i2 ,...,x _in ), where x _ij represents the weight of word j in document d _i . A rough way to describe the weight is to use Boolean value 0 or 1 to indicate whether the feature word has appeared in a certain document. tf*idf (term frequency*inverse document frequency) is a commonly used document feature weight representation method. This weight calculation method mainly considers the term frequency tf, inverse document frequency idf and normalization factor of feature words. In order to ensure the clustering effect, the present invention uses the LTC weight as the weight of the feature word, as described in the following formula

x _ij =log ₂ (tf _ij +1.0)*log(m/m _j ).

In the formula, x _ij is the value of the feature vector of the jth feature word in the document d _i , tf _ij is the number of occurrences of the jth feature word in the document d _i , m/m _j is the inverse of the jth feature word Document frequency, m is the total number of documents, and m _j is the number of documents containing the jth feature word.

The LTC weight formula takes the logarithm of the word frequency tf value on the basis of the tf*idf formula, which again reduces the influence of the word frequency tf on the feature vector. This formula is more reasonable in practical applications. At the same time, since different document lengths will affect the weight value of the vector, it is necessary to normalize the weight value calculated by the formula, namely:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="x\; ip"\; filenames="DEST\_PATH\_131024142412.png"\; state="\{\{state\}\}">x</figure-callout></span><figure-callout\; id="2"\; label="x\; ip"\; filenames="DEST\_PATH\_131024142412.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}^{\mathrm{<span\; class="notranslate">ip</span>}}}}<span\; class="notranslate">\; .</span>$

For high-dimensional text data, it is very time-consuming to calculate the weight value of each dimension of the text feature vector, so the present invention designs a parallel calculation method to speed up the weight calculation process. When calculating the LTC weight, the value of m _j is missing, that is, the number of times a certain keyword appears in the document collection. Under high-dimensional large-scale data sets, traditional serial statistics are very time-consuming, as shown in Figure 3. After the value of m _j is obtained, the weight is calculated for each feature word of each document. Since the dimension of text data is very high, if the traditional serial algorithm is used, the time complexity is also very high. After obtaining the weight vector, the process of normalizing the weight vector needs to calculate the length of each vector, and then normalize each weight. The whole process also takes a lot of time. Therefore, in the present invention, the three time-consuming parts in the above-mentioned eigenvector weight calculation process are designed in parallel in the graphics processing unit environment, and the parallel eigenvector calculation process is shown in FIG. 4 .

(1) Multi-threaded parallel statistics of the document frequency m _j of each keyword appearing in the document collection

For each feature word, the value of the corresponding m _j needs to be counted for the calculation of the subsequent weight value. We convert this problem into counting statistics of non-zero values for each column of the word frequency matrix, and the calculation method is as follows

$m_j=\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}{x}_{\mathrm{ij}},$ in $x_{\mathrm{ij}}=\left\{\begin{array}{cc}0,& {\mathrm{tf}}_{\mathrm{ij}=0}\\ 1,& {\mathrm{tf}}_{\mathrm{ij}}>0,\end{array}\right.$

where m is the number of documents in the dataset. The above formula is the calculation formula of statistical m _j . When the word frequency is zero, it means that the feature word does not appear in the corresponding document, and it does not participate in the counting; when the word frequency of the feature word is greater than zero, it means that the word appears in the document. Here, the m _j count is incremented by one. Since the m _j value of each feature word is counted independently, for the data in the form of a matrix, a multi-threaded implementation scheme can be adopted, which can make full use of the parallel processing capability of the graphics processing unit to accelerate the process. Fig. 5 is a schematic diagram of statistics on the number of documents where parallel keywords are located.

(2) Multi-thread parallel computing feature matrix

In the calculation process of this stage, the input is the word frequency matrix and the value of _mj , and the output is the feature matrix as shown in Figure 6. The calculation formula of the characteristic matrix is as follows

x _ij =log ₂ (tf _ij +1.0)*log(m/m _j ).

Fig. 6 is a schematic diagram of multi-threaded parallel calculation of feature matrix. If the number of documents is m and the dimension of feature words is n-dimensional, the execution frequency of the calculation formula is m*n times. For application scenarios with large-scale document collections and high text dimensions, the calculation of the feature matrix is very large, so this paper designs a parallel multi-threaded execution method to accelerate this operation. Since the calculation process of each eigenvector is independent of each other, each thread is responsible for the calculation process of one eigenvector, and multiple threads are executed concurrently to improve the calculation speed of the eigenmatrix.

(3) Multi-thread feature matrix normalization

After the above feature matrix calculation process is completed, the feature value of each keyword of the document will be obtained. Due to the different lengths of the documents, the document feature vector with a longer length may have a significant inhibitory effect on the feature vectors of other documents, so the feature vector is used. Vector normalization methods to equalize the feature vectors. The formula for eigenvector normalization is as follows

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="x\; ip"\; filenames="DEST\_PATH\_131024142412.png"\; state="\{\{state\}\}">x</figure-callout></span><figure-callout\; id="2"\; label="x\; ip"\; filenames="DEST\_PATH\_131024142412.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}^{\mathrm{<span\; class="notranslate">ip</span>}}}}<span\; class="notranslate">\; .</span>$

This formula represents dividing the weight value of each feature word in the document by the square of the modulus of the feature vector corresponding to the document to equalize the vector. For the operation of the normalized vector, the present invention adopts two kernel functions on the graphics processing unit to realize, one kernel function is responsible for calculating the value of the modulus of the vector, and the other kernel function is responsible for the normalization calculation of the weight.

Figure 7 is a schematic diagram of multi-threaded calculation of vector modulus. A dotted line in the figure represents a thread. The function of the thread is to sum the square values of elements in a row of the feature matrix to obtain the square of the modulus value of a document feature vector. After all the threads are calculated, we will get the square of the modulus of each document feature vector for subsequent normalization operations. If the graphics processing unit is not used for multi-threaded parallel acceleration, the time complexity of the modulo operation is O(m*n), where m represents the number of documents, and n represents the dimension of the document feature vector. The time complexity of the improved algorithm running in parallel on GPU is O(n).

After obtaining the modulus of the feature vector of each document, it is necessary to perform weight normalization on the weight matrix. For each element in the matrix, it needs to be divided by the square of the modulus of the corresponding document feature vector. For this type of matrix problem, it is also suitable to use the multi-threading of the graphics processing unit to accelerate the efficiency of the algorithm operation. Fig. 8 is a schematic diagram of multi-thread normalization. A dotted line in Fig. 8 represents a thread. After all threads are calculated, the final normalized document feature vector is obtained for subsequent clustering operations. If the graphics processing unit is not used for multi-threaded parallel acceleration, the time complexity of the process is O(m*n), where m represents the number of documents, and n represents the dimension of the document feature vector. The time complexity of the improved algorithm running in parallel on the graphics processing unit is O(1).

Step 3: Parallel SOM text clustering, that is: design the SOM network structure according to the eigenvector matrix, initialize the SOM network, calculate the distance between the input sample and all output neuron weight vectors in parallel through the graphics processing unit, compare the size of each distance, and obtain the minimum distance The optimal neuron of , by updating the optimal neuron, the neuron weight vector value in the neighborhood, the learning rate and the neighborhood size of the optimal neuron J, and then calculate the network error rate E _t in parallel through the graphics processing unit, if the network Error rate E _t <= target error ε or number of iterations t>= training maximum number of iterations T, then the SOM network training ends, otherwise a new round of training is performed; the result of each learning makes the best matching neuron neighborhood area Close to the input data vector value, gather the input feature vectors with similar distances into the same cluster, and the formed cluster set is the final clustering result.

The specific implementation process is as follows:

The SOM network is a neural network model that simulates the human brain. One of its most important features is self-organization, that is, the neurons in the external input network will automatically adjust the connection weights, and the corresponding neurons will gather to form a response to the outside. The SOM network is to simulate the self-organization characteristics of brain cells to achieve clustering, identification, sorting, topological invariance mapping, etc. The neuron nodes in the SOM network can accept input data from other neurons, and can also output data to other neurons. The trained SOM network forms its own conceptual patterns for external input patterns. SOM is suitable for dealing with nonlinear and uncertain data.

The SOM network is an unsupervised learning network consisting of two layers: an input layer and an output layer. The input layer calculates the distance between the input vector and the weight vector, which reflects the degree of matching, so the input layer is also called the matching layer. The output layer is also called the competition layer. Each neuron competes according to the matching degree. The neuron with a large matching degree is called the winning neuron. The weight vector of the winning neuron and the neurons in its neighborhood will move closer to the input vector. Update, after repeated iterative competition updates, a stable network is finally formed, and the neurons save the corresponding weight vectors. Subsequently, the trained network can be used for operations such as clustering and spatial mapping. The training process of the SOM network is a self-organizing learning process, and the training is divided into two parts: the selection of the best matching neuron and the update of the network weight vector. In a common SOM network, neurons in the input layer are arranged in one dimension, and neurons in the output layer are arranged in two dimensions. The topology is shown in Figure 9.

The SOM network uses the self-organizing map learning method, which belongs to unsupervised learning. The result of each learning makes the neighborhood area of the best matching neuron close to the input data vector value, so that the input vectors with similar distances can be gathered. together to form a cluster. After large-scale training of the SOM network, the connection weights between neurons can represent the characteristics of the input pattern, and the input vectors with similar patterns are merged into one class to complete the automatic clustering process of the SOM network. The core process of the SOM algorithm is the screening of the best matching neurons and the updating of the weights in the neuron neighborhood. Screening of the best matching neuron is based on the distance between all neurons and the input vector, and the one with the smallest distance is the best matching neuron; the self-organization adjustment of the connection weights between neurons is to adjust each neuron in its neighborhood according to the best matching neuron. The weight value of the neuron. The SOM network conducts a self-organizing adaptation process on the input vector every time it learns, strengthening the mapping of new matching patterns and weakening the mapping of old patterns. In the traditional serial SOM clustering algorithm, there are two steps that spend 80% of the algorithm running time: (1) Calculate the distance between the input sample and all the output neural weight vectors, compare the size of each distance, and obtain the best algorithm with the smallest distance. neuron; (2) Calculate the network error rate E _t of two adjacent iterations. Therefore, aiming at the above two features, the present invention designs a parallel SOM clustering algorithm based on graphics processing unit. The logic flow of the parallel SOM algorithm is described as follows:

Step1: Suppose there are m input samples, and the dimension of each input sample is n-dimensional. Design the SOM network structure, determine the number of neurons in the input layer as n, and the number of neurons in the output layer as k, train the maximum number of iterations T and the target error ε.

Step2: Initialize the SOM network, including initializing the connection weight vector W ₀ =(W ₁₀ ,W ₂₀ ,...,W _k0 ), the learning rate α _i (0)∈(0,1), and the size of the neighborhood N _i (0), i∈{1,2,...,n}, iteration counter t=1.

Step3: Calculate the distance d _j between the input sample _Xi and all output neuron weight vectors W _j in parallel, the formula is as follows

${\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\sqrt{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ip</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; jp</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Step4: Compare the size of each distance, and the neuron with the smallest distance is the best neuron J.

Step5: Update the weight vector value of the best neuron J and the neurons in the neighborhood N _j (t-1)

W _j,t =W _j,t-1 +α _i (t-1)(X _i -W _j,t-1 ). (2-2)

Step6: Update the learning rate α _j (t) and the neighborhood size N _j (t) of the best neuron J. The correction formula for the learning rate is as follows

α _i (t)=α _i (0)(1−t/T). (2-3)

Assuming that the coordinate value of neuron g in the competition layer in the two-dimensional column is (X _g , Y _g ), the range of the neighborhood is a square area, and the coordinates of the upper right corner of the square are (X _g +N _j (t), Y _g +N _j (t)), the coordinates of the lower left corner of the square are (X _g -N _j (t),Y _g -N _j (t)), the correction formula of the neighborhood is as follows

N _j (t)=INT[N _j (0)·(1-t/T)], (2-4)

Among them, INT[X] represents the rounding operation on X.

Step7: Calculate the error of two adjacent iterations of the neuron in parallel, as the formula is as follows

${\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Step8: If E _t <=ε or t>=T, that is, the SOM network converges to the expected error rate or reaches the maximum number of iterations, then the SOM network training ends, otherwise turn to Step3 for a new round of training. The result of each learning makes the neuron weights in the best matching neuron neighborhood close to the input data vector value, and the input feature vectors with similar distances are gathered together to form text clusters.

As shown in Fig. 10 and Fig. 11, the specific embodiment of the present invention is: build a kind of self-organizing map neural network clustering system based on graphics processing unit, it is characterized in that, comprises hardware part and software part, and hardware part: adopts CPU /GPU collaborative framework design, the serial execution code runs on the CPU, the parallel execution code runs on the GPU, and the data between the video memory and the memory is exchanged through the data transmission method provided by the GPU; the software part is divided into three modules, including parallel A keyword frequency statistics module, a parallelized feature vector calculation module, a parallelized SOM clustering module, a unit, a feature vector calculation unit for calculating feature vectors, and a text clustering unit for text clustering, the parallelized keyword frequency statistics The module divides the text content into words and obtains a collection of keywords, and calculates the frequency of keywords in the document in parallel to obtain a word frequency matrix; the parallelized feature vector calculation module converts the keyword frequency matrix into a corresponding feature vector matrix, and each feature The vector represents a document; the parallelized SOM clustering module designs the SOM network structure according to the feature vector matrix, initializes the SOM network, calculates the distance between the input sample and all output neuron weight vectors in parallel, compares the size of each distance, and obtains the maximum value of the minimum distance. The best neuron J, by updating the best neuron, the neuron weight vector value in its neighborhood, the learning rate and the neighborhood size of the best neuron, and then calculate the network error rate E _t in parallel through the graphics processing unit, if the network error Rate E _t <= target error ε or number of iterations t>= maximum number of training iterations T, then the SOM network training ends, otherwise a new round of training is carried out; the result of each learning makes the neighborhood area of the best matching neuron to The input data vector values are close, and the input feature vectors with similar distances are gathered into the same cluster, and the formed cluster set is the final clustering result.

The specific clustering process is as follows: the present invention is based on the design of the CPU/GPU framework for the parallel self-organizing mapping neural network clustering system based on the graphics processing unit, as shown in Figure 10 for the hardware framework of the system, the scheduling of the CPU control system, and the allocation of task, prepare the running space for the graphics processing unit, etc., and the graphics processing unit executes computing tasks in parallel under the environment prepared by the CPU. Figure 11 shows the software collaboration framework of SOM clustering. The system uses the Compute Unified Device Architecture (CUDA for short) programming platform to accelerate the application of the SOM algorithm to the text data clustering process.

In the design based on the CPU/GPU cooperation framework, through the reasonable allocation and framework design of the cooperation tasks of CPU and GPU, the respective advantages of CPU and GPU are fully utilized to accelerate the algorithm. The system divides its tasks into two parts for allocation, one part is the task with obvious running advantage on the CPU, and the other is the task with obvious running advantage on the graphics processing unit. The tasks suitable for running on the CPU mainly include: SOM network initialization, data I/O operations, algorithm logic flow control, and kernel function calls. The tasks suitable for running on the graphics processing unit are mainly data computing tasks, including: parallel word frequency statistics, parallel feature vector calculation, distance calculation between input samples and neurons, and error calculation of network weights.

In terms of system software, the accelerated operation of the algorithm is realized mainly by designing kernel functions for each module. In the parallel word frequency statistics module, the system designs a kernel function, which allocates a thread on the graphics processing unit for each document, opens m threads in total, m is the number of documents, and counts the appearance of each keyword in the document frequency, the calculation process of its kernel function is shown in Figure 12.

In the parallel eigenvector calculation module, the system designed three kernel functions for this module to deal with the three time-consuming parts of the algorithm: (1) Calculating the number of documents where keywords are located. Kernel function, as shown in Figure 4, is the kernel function Open n threads, n is the number of keywords, the calculation process of its kernel function is shown in Figure 13; (2) Calculate the document feature vector kernel function, as shown in Figure 6, open m times n threads for the kernel function, its calculation formula for

x _ij =log ₂ (tf _ij +1.0)*log(m/m _j ).

(3) Since the length of each document may vary greatly in practical applications, in order to overcome this problem, two kernel functions are designed in this module to normalize document feature vectors, respectively: Calculate each feature vector The kernel function of the modulus, as shown in Figure 7, opens m threads for the kernel function; the kernel function of the normalized eigenvector, as shown in Figure 8, opens m times n threads for the kernel function.

The specific flow of the parallel SOM clustering module is shown in Figure 11. After analyzing the execution logic flow of the serial SOM algorithm, it is found that the large-scale operation module of the serial SOM algorithm has two parts: one is that after the SOM network receives a new input vector, it calculates the input vector to each output neuron. Distance, from which the sudden neuron with the shortest distance is selected as the module of the optimal neuron; part of it is a module that calculates the network error of two adjacent iterations of the neuron and compares it with the algorithm's preset acceptable error rate. These two parts are large-scale floating-point calculations that the CPU is not good at. If the calculation process is executed serially, the time spent by the algorithm on these two parts accounts for more than 80% of the algorithm's running time, so module 3 is for these two parts. In the first part, three kernel functions are designed to speed up the execution speed of the algorithm and improve the execution efficiency of the algorithm. For the above two parts, the system designs two sub-modules that run on the graphics processing unit.

(1) Parallel operation to find the best neuron sub-module. The best neuron is the neuron closest to the input pattern vector, which requires the algorithm to calculate the distance between the vector and each neuron in the network when receiving an input pattern vector. The calculation formula corresponding to the distance between the neuron and the input vector is as follows

${\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\sqrt{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ip</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; jp</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; .</span>$

In view of the fact that all distances need to be square rooted in the calculation process, the formula can be simplified as

${\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ip</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; jp</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; .</span>$

The calculation steps of the distance formula are simple, but when the number of neurons in the SOM network is large, the dimension of the input samples is large, and the number of input samples is large, the amount of calculation is very large. When the sample dimension is n, the number of neurons is k, and the number of iterations of the algorithm is T times, the frequency of the difference operation in this formula is k*n*T, so the improvement of parallel execution of the algorithm on the formula will improve the performance of the program. The performance boost is very effective. In this sub-module, the system designs a kernel function to calculate the distance between the input feature vector and each neuron in the network, which can make full use of the parallel computing performance of the graphics processing unit and speed up the execution speed of the algorithm.

As shown in Figure 14, in order to facilitate the parallelization improvement design of the serial program, the following describes the distance calculation problem in the sense of a matrix. Define the distance operation between the neuron weight matrix and the input pattern vector as , the neuron weight matrix W is a k*n matrix, where k is the number of output neurons, n is the dimension of the input pattern vector, the input pattern vector x _k is a vector of dimension n, and the distance operation result d _j is the dimension of vector of k. In order to speed up the operation of the algorithm, the system can simultaneously start k threads to calculate in parallel, and the calculation process of its kernel function is shown in Figure 15.

(2) Calculate the error sub-modules of the network weights of two adjacent rounds in parallel. The meaning represented by the SOM network index E _t is the difference between the current weight value of the network and the weight value at the end of the last round of training. If the value of E _t is less than the error rate defined before the algorithm initialization, it means that the weight value of each neuron has no large-scale changes during the iterative training of the SOM network. It is considered that the network has converged to a certain state and the network training is complete. The formula for judging whether the current SOM network has been trained to the allowable range of error is as follows

${\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; .</span>$

By observing and analyzing the above calculation formula, it is found that this formula is a deformation of the difference operation between two matrices. The serial execution speed of this large-scale matrix operation on the CPU is far behind the parallel execution speed on the graphics processing unit. Therefore, in this sub-module, the error calculation of the network is designed as a kernel function running on the graphics processing unit. to speed up the algorithm. At the same time, when calculating the sum of network errors, in order to speed up the operation of the algorithm, the system uses row or column parallel calculation of network errors. Therefore, in this sub-module, we also design a kernel function to calculate network errors in parallel.

As shown in Figure 16, for the calculation sub-module for calculating the error rate of the SOM network, it is divided into two steps of parallel operation on the graphics processing unit, and the first part is the difference operation between the two matrices, that is, W _t -W _t-1 =E _t Operation, the second part is the reduction part of the absolute value sum of each element in the matrix. Both parts are accelerated using graphics processing units. In FIG. 16, W _t -W _t-1 =E _t . A kernel function can be designed to calculate the difference of the matrix, and the system starts m times n threads for the kernel function, and each thread calculates the Euclidean distance between two neurons.

Usually the above-mentioned difference operation is not the final result we want. We need to continue to add the absolute value of the elements of each row of the C matrix to obtain an n-dimensional vector. Finally, through the serial operation on the CPU, the value in the vector Accumulate and compare the actual change amount of the final SOM network with the error rate. If it is greater than the error rate, continue to the next round of training. If it is less than the error rate, stop the training, which proves that the network has converged to a certain extent. The addition of the matrix requires two steps, summing by row or column, and then summing the results of the first step. The operation process is shown in Figure 17. The system can design a kernel function to calculate the first step in parallel, and open n threads for the kernel function at the same time. The calculation process of the kernel function is shown in Figure 18.

Note that the operation of summing the matrix element values in Figure 17 to generate an n*1-dimensional vector is not necessarily reduced by row, but also by column, which depends on the relationship between the number of rows and the number of columns in practical applications. If the number of rows of the matrix is much greater than the number of columns, reduce by row, because the reduction by row can open more parallel threads at a time. Conversely, if the number of columns of the matrix is much greater than the number of rows, it needs to be reduced by column.

Several corresponding data structures need to be maintained for the above operation process: the two-dimensional matrix W that stores the weight values of SOM network neurons, the training sample matrix X, and the distance vector D. Since the E _t value needs to be calculated, it is necessary to save the two-dimensional matrix of the weight values of the SOM network neurons in the last round.

The parallel self-organizing map neural network clustering method and system based on the graphic processing unit of the present invention designs a parallel SOM text clustering algorithm. Simultaneously, the present invention designs a set of parallelized text clustering system based on CPU/GPU cooperation framework by utilizing the complementarity of computing power between graphics processing unit (GPU) and central processing unit (CPU). The hardware part of the system is designed as a CPU/GPU collaborative framework, and the software part is designed into three modules: a parallel word frequency statistics module, a parallel feature vector calculation module and a parallel SOM algorithm clustering module. The text clustering based on the self-organizing map neural network of the graphics processing unit of the present invention can make full use of the high parallelism of the graphics processing equipment, effectively improve the clustering speed of the algorithm, and is very suitable for processing the clustering problem of high-dimensional text data .

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (9)

1. A parallel self-organizing map neural network clustering method based on graphics processing unit, comprising the steps: Parallel keyword frequency statistics: segment the text content into words and get a set of keywords, and count the frequency of keywords in the document in parallel to get a word frequency matrix; Parallel eigenvector calculation: convert the keyword frequency matrix into a corresponding eigenvector matrix, each eigenvector represents a document; Parallel SOM clustering: Design the SOM network structure according to the eigenvector matrix, initialize the SOM network, calculate the distance between the input sample and all output neuron weight vectors in parallel, compare the size of each distance, and obtain the best neuron J with the smallest distance. The best neuron, the neuron weight vector value in its neighborhood, the learning rate and the neighborhood size of the best neuron, and then calculate the network error rate E _t in parallel through the graphics processing unit, if the network error rate E _t <= target error ε Or the number of iterations t>=the maximum number of training iterations T, then the SOM network training ends, otherwise a new round of training is performed; the result of each learning makes the neighborhood area of the best matching neuron approach the input data vector value, and the distance Similar input feature vectors are aggregated into the same cluster, and the cluster set formed is the final clustering result. 2. the self-organizing map neural network clustering method based on graphics processing unit according to claim 1, is characterized in that, in the keyword word frequency step of obtaining document, adopts the multi-thread parallel statistical word frequency based on graphics processing unit. 3. the self-organizing map neural network clustering method based on graphics processing unit according to claim 1, characterized in that, in the parallel feature vector calculation step, the feature vector of each document is calculated based on the multi-threaded parallel calculation of graphics processing . 4. according to the self-organizing map neural network clustering method based on graphics processing unit described in claim 1, it is characterized in that, the calculation process of input feature vector and each output neuron weight vector distance is independent of each other, adopts the multiple based on graphics processing Each thread calculates the distance between the input feature vector and each output neuron vector in parallel, and the system opens a thread for each neuron, using multi-thread parallel calculation. 5. according to the self-organizing map neural network clustering method based on graphics processing unit described in claim 1, it is characterized in that, the calculation process of the weight vector error of adjacent two iterations of each neuron is independent of each other, adopts based on graphics processing Multiple threads calculate the weight vector error of each neuron in parallel, and the system starts a thread for each neuron, using multi-threaded parallel calculation. 6. A self-organizing map neural network clustering system based on a graphics processing unit, characterized in that it includes a hardware part and a software part, and the hardware part: adopts CPU/GPU collaborative framework design, serial execution code runs on the CPU, parallel The execution code runs on the GPU, and the data between the video memory and the memory is exchanged through the data transmission method provided by the GPU; the software part is divided into three modules, including the parallelized keyword frequency statistics module, the parallelized feature vector calculation module, and the parallelized SOM clustering module, unit, feature vector calculation unit for calculating feature vectors, text clustering unit for text clustering, the parallelized keyword frequency statistics module divides text content into words and obtains a set of keywords, and performs parallel statistics on documents The frequency of keywords in the middle, obtains term frequency matrix; Described parallelization feature vector calculation module converts keyword term frequency matrix into corresponding feature vector matrix, and each feature vector represents a document; Described parallelization SOM clustering module according to feature vector Matrix design SOM network structure, initialize SOM network, calculate the distance between the input sample and all output neuron weight vectors in parallel, compare the size of each distance, obtain the best neuron J with the smallest distance, and update the best neuron and its neighborhood Neuron weight vector value, learning rate and the neighborhood size of the optimal neuron, and then calculate the network error rate E _t in parallel through the graphics processing unit, if the network error rate E _t <= target error ε or number of iterations t > = training maximum If the number of iterations is T, the training of the SOM network is over, otherwise a new round of training is carried out again; the result of each learning makes the neighborhood area of the best matching neuron close to the value of the input data vector, and the input feature vectors with similar distances are aggregated into the same A cluster, the formed cluster set is the final clustering result. 7. according to the clustering system of the parallelized self-organizing map neural network based on graphics processing unit according to claim 6, it is characterized in that, described parallelized keyword frequency statistics module, described parallelized feature vector calculation module and described Several kernel functions are designed in the parallel SOM clustering module to accelerate the operation of the algorithm in parallel. 8. The clustering system based on the parallelized self-organizing map neural network of the graphics processing unit according to claim 6, it is characterized in that, in the parallel keyword frequency statistics module, a kernel function for keyword frequency statistics is designed ; In the parallel eigenvector calculation module, two kernel functions for eigenvector calculation and two kernel functions for eigenvector normalization are designed. 9. The clustering system based on the parallel self-organizing map neural network of the graphics processing unit according to claim 6, it is characterized in that, in the parallel SOM clustering module, a neuron for calculating the input feature vector and the output neuron is designed The kernel function of the distance, a kernel function used to calculate the error of the network weight vector of two adjacent iterations of each neuron and a kernel function used to reduce the error of the network weight vector.

Images