JP6851634B2 - Feature conversion module, pattern identification device, pattern identification method, and program - Google Patents

Description

The present invention relates to an arbitrary multidimensional vector feature conversion module, the above-mentioned pattern identification learning device for a multidimensional vector, a pattern identification device, an optical calculation data generation device, an optical storage device, a pattern identification learning processing method and a pattern identification processing. It relates to a method, a data generation method for optical calculation, a pattern recognition learning program, and a storage device that records them. The present application claims priority based on Japanese Patent Application No. 2016-025892 filed in Japan on February 15, 2016, the contents of which are incorporated herein by reference.

In recent years, research and development aimed at obtaining new knowledge and knowledge from large-scale data groups has attracted attention. In machine learning in computer vision, object recognition, matching, and detection from two-dimensional images are realized by constructing a learning device. In order to improve the accuracy in object recognition / matching / detection, high-quality and enormous data groups for learning and high-speed arithmetic processing technology for learning are required, and improvement of the operating frequency of general-purpose processors is required. However, it is difficult to further improve the operating frequency of a general-purpose processor due to problems such as heat generation of a circuit due to miniaturization of a semiconductor manufacturing process. Therefore, a fusion system that efficiently combines existing technology and new computing technology, such as technology to improve the performance of learners by parallelizing processor cores and arithmetic units, and having a dedicated processor take charge of the strong points. Research and development is being carried out.

For example, as a process for realizing the inner product operation of a multidimensional vector at high speed, an optical operation process can be mentioned (see Non-Patent Document 1). Optical arithmetic processing is not very versatile, but in a specific algorithm, it excels in parallel high-speed arithmetic based on analog characteristics and large-capacity storage of information. For example, Patent Document 1 and Non-Patent Document 1 disclose an optical correlation system using a holographic optical disk capable of achieving both parallel high-speed calculation and large-capacity storage of information as described above.

Japanese Patent Application Laid-Open No. 2008-282374

K. Ikeda and E. Watanabe, “High-speed optical correlator with coaxial holographic system” Jpn. J. Appl. Phys. 54, 09ME02 (2015).

As described above, in order to realize pattern identification of multidimensional vectors such as a large amount of images and images at high speed, with high accuracy and energy saving, the inner product calculation between multidimensional vectors should be speeded up and the inner product between multidimensional vectors should be realized. It is required to improve the efficiency of the storage device that stores the calculation result. In the pattern identification based on the optical arithmetic processing disclosed in Patent Document 1 and Non-Patent Document 1, the internal product operation between the multidimensional vectors is performed at high speed, but the applicable range of the pattern identification based on these optical arithmetic processing is specified. There is a problem that it is difficult to apply to general-purpose object recognition / matching / detection because it is limited to image matching. In addition, a module that can be applied to general-purpose object recognition, collation, and detection, and can smoothly create a learning model suitable for optical arithmetic processing from arbitrary learning data has been desired.

In view of the above circumstances, the present invention smoothly creates a learning model suitable for optical arithmetic processing from arbitrary learning data, and uses optical arithmetic processing without being limited to specific image matching from ultra-large-scale data. The purpose is to realize the identification of the multidimensional pattern of the above at ultra-high speed.

The present invention employs the following means in order to solve the above problems.

The feature amount conversion module according to the present invention is a feature amount conversion module that converts the feature amount of training data by learning in a neural network, and learning data having an arbitrary first dimension number and an arbitrary feature amount is input. The training data input unit is input to the training data input unit, which is composed of a function consisting of a training data input unit and a function having a weight and a bias having a second dimension number satisfying a predetermined optical calculation constraint condition as factors. An intermediate layer setting unit that inputs the learning data and sets an intermediate layer having the second dimension number, and an output layer having the first dimension number from the intermediate layer set by the intermediate layer setting unit. The output calculation unit that calculates the output of the neural network, the error calculation unit that calculates the error between the output of the neural network calculated by the output calculation unit and the training data, and the error calculation unit. A determination unit that determines whether or not the calculated error satisfies a predetermined condition, and a weight bias that changes the weight and the bias based on the error to obtain the changed weight and the bias. When the change unit, the learning model output unit that outputs a learning model composed of a function consisting of the changed weight and the bias as factors, and the determination unit satisfy a predetermined error. When it is determined, the output unit outputs a learning model composed of a function composed of an equation having the changed weight and the bias as factors, and the error determines a predetermined condition in the determination unit. When it is determined that the conditions are not satisfied, the output calculation unit applies the weight and the bias included in the equation of the learning model to the changed weight and the bias obtained by the weight bias changing unit, respectively. A control unit that changes and calculates the output of the neural network is provided, and the optical calculation constraint condition is a predetermined number of quantizations in an optical calculation performed using the output of the neural network, and light used in the optical calculation. It is characterized by being determined by the optical parameters of the correlator and a predetermined number of energies.
In the present specification, an error with the training data is calculated for the output of the neural network obtained by the output calculation unit, and the learning model is updated by the weight / bias changing unit until the error satisfies a predetermined condition. The process is called learning.

The pattern identification device according to the present invention uses the above-mentioned feature amount conversion module and the learning data to which the class label is added as input, and maximizes the identification accuracy of the learning data provided with the constraint condition for optical calculation. A plurality of learning means for learning such a feature extractor, an optical calculation data conversion unit that converts the feature amount extracted from the feature extractor into a feature extractor, and an optical storage unit that stores optical calculation data. The optical calculation means by the similarity calculation unit that provides the optical correlation function by calculating the similarity in the optical calculation data listening of the above, and the registration data that becomes the dictionary of the identification target data are input and obtained by the learning means. The feature amount extracted by the feature extractor is converted into an optical calculation feature amount for optical calculation by the optical calculation data conversion unit, and is identified with a registration means registered in the optical storage unit of the optical calculation means. Using the target data as input, the similarity calculation unit of the optical calculation means calculates the similarity between the feature amount extracted from the feature extractor generated by the learning means and the feature amount registered in the optical storage unit of the registration means. However, it is characterized in that it includes an identification means for outputting the identification result from the output unit.

The pattern identification device according to the present invention uses the learning data to which the class label is attached as input, and learns a feature extractor that maximizes the identification accuracy of the learning data provided with the constraint condition for optical calculation. Similarity between the learning means, the optical calculation data conversion unit that converts the feature amount extracted from the feature extractor into the feature extractor, the optical storage unit that stores the optical calculation data, and a plurality of optical calculation data listening. Using the optical calculation means by the similarity calculation unit that provides the optical fighting function by calculating the degree, and the registration data that becomes the dictionary of the identification target data as input, and the feature extractor obtained by the learning means. The extracted feature amount is converted as an optical calculation feature amount for optical calculation by the optical calculation data conversion unit, and the registration means registered in the optical storage unit of the optical calculation means and the identification target data are input as described above. The similarity between the feature amount extracted from the feature extractor generated by the learning means and the feature amount registered in the optical storage unit in the registration means is calculated by the similarity calculation unit in the optical calculation means, and the identification result is obtained from the output unit. It is characterized by including an identification means for outputting.

According to the present invention, a learning model suitable for optical arithmetic processing can be smoothly created from the learning data using ultra-large-scale data as training data, and identification of an arbitrary multidimensional pattern from the ultra-large-scale data can be realized at ultra-high speed. Can be done. In addition, the efficiency of optical calculation can be improved by high-speed identification and energy saving.

It is a block diagram which shows the structural example of the pattern identification apparatus which concerns on this invention. It is a flowchart of each process of learning process, registration process, and identification process of the pattern identification method which concerns on this invention. This is an implementation example of the learning means 100 according to the first embodiment of the present invention. It is a value distribution of the feature amount obtained by the feedforward processing of the learned learning means shown in FIG. It is a figure which shows the identification accuracy using the feature quantity obtained by the feedforward processing of the learned learning means shown in FIG. It is a figure which shows the similarity calculation result by the optical calculation in Example 1 of this invention when the k-nearest neighbor classifier by the optical calculation process of FIG. 1 is applied to the sketch-based three-dimensional collation. It is a figure which shows the multi-class identification result by the optical calculation in Example 1 of this invention when the k-nearest neighbor classifier by the optical calculation process of FIG. 1 is applied to the sketch-based three-dimensional collation. It is a figure which shows the discrimination result by the k-nearest neighbor method by the optical correlation calculation in Example 1 of this invention when the k-nearest neighbor classifier by the optical calculation process of FIG. 1 is applied to sketch-based three-dimensional collation. It is a flowchart of the optical calculation means 300 shown in FIG. It is a schematic diagram which shows an example of conversion in the data conversion unit 310 for optical calculation shown in FIG. It is a schematic diagram which shows an example of the optical means used for the feature vector light recording unit 320 for optical calculation and the similarity calculation 330 shown in FIG. It is a flowchart of the linear classifier by the optical arithmetic processing which concerns on Example 2 of this invention. It is a figure which shows the implementation example of the feature extraction learning part 121 which concerns on Example 2 of this invention. It is a figure which shows the implementation example of the data conversion unit 122 for optical calculation which concerns on Example 2 of this invention. It is a figure which shows the implementation example of the data conversion unit 310 for optical calculation which concerns on Example 2 of this invention. It is an output result of the discrimination function when the B-HOG feature quantity is used in the linear classifier which concerns on Example 2 of this invention. It is a block diagram which shows the structural example of the feature amount conversion module which concerns on this invention, and the pattern identification apparatus provided with the feature amount conversion module. It is a flowchart of each process of the learning process, the feature amount conversion process, the registration process, and the identification process of the pattern identification method which concerns on this invention. It is a figure which shows an example of the network structure in Example 3 of this invention. It is a figure which shows the value distribution of the optical calculation data output from the optical calculation feature conversion part in Example 3 of this invention. It is a figure which shows the identification accuracy in Example 3 of this invention. It is a graph which shows the example of the inner product calculation result which concerns on Example 3 of this invention. It is a graph which shows an example of the structure of the optical correlator which concerns on Example 3 of this invention.

Hereinafter, embodiments and examples of the present invention will be described with reference to the drawings, but the present invention is not limited to the following examples.

FIG. 1 is a block diagram showing a pattern identification device of the present invention.
FIG. 2 is a flowchart schematically showing the flow of pattern recognition.

The pattern recognition device 1 according to the present invention includes a learning means 100 that performs learning processing, a feature extraction unit in an optical storage unit, a registration means 200 that performs data registration processing via an optical calculation data generation unit, and an optical calculation data conversion unit. Via the 310, the optical calculation means 300 that holds the feature vector optical storage unit 320 for optical calculation and performs the similarity calculation 330, the feature extraction unit 220 that extracts the feature amount from the identification target data, and the data conversion unit 310 for optical calculation. It is composed of identification means 400 that performs identification processing that performs similarity calculation 330.

In the learning means 100, the learning data to which the class label is added prepared in advance is input from the learning data input unit 110, transmitted to the feature extractor generation unit 120, and the feature extraction unit 121 and the optical calculation constraint addition unit 122 are combined. Through this, learning is performed so as to maximize the identification accuracy of the training data. The feature extraction unit 121 generated by the feature extractor generation unit 120 is incorporated into the registration means 200, the identification means 400, and the optical calculation means 300.

In the registration means 200, registration data that serves as a dictionary of identification target data is input from the input unit 210. Further, in the registration means 200, the feature amount extracted by the feature extractor obtained by the learning means 100 is converted as a feature amount for optical calculation for optical calculation, and the feature vector optical storage for optical calculation is performed in the optical calculation means 300. It is registered in the unit 320.

The optical calculation means 300 is composed of a data conversion unit 310 for optical calculation, an optical storage unit 320 for storing data for optical calculation, and a similarity calculation unit 330 for calculating the similarity between a plurality of data corresponding to optical calculation. It has a function to output the similarity calculation result by correlation calculation.

The optical calculation data conversion unit 310 performs a process of converting electronic data into optical data. When converting to optical calculation data, the data is displayed on a spatial optical modulation device, and the data is electronically and optically converted by irradiating with laser light or the like. Therefore, data conversion that depends on the device specifications of the spatial light modulator is performed. As the spatial light modulator, a liquid crystal spatial light modulator, a digital micromirror device, or the like is used. Since the spatial light modulator corresponding to the high-speed response is often binary, binary conversion is performed.

In the identification means 400, the identification target data is input by the identification target data input unit 410, the feature amount is extracted by the feature extraction unit 220 generated by the learning means 100, and the extracted feature amount is the optical calculation data conversion unit 310. Is transmitted to. In the registration means 200, the similarity between the feature quantities registered in the optical storage unit 320 is calculated by the similarity calculation unit 330 in the optical calculation means 300. After that, the identification result is output by the output unit 420.

An embodiment of the k-nearest neighbor identification method by optical arithmetic processing will be described with reference to the block diagram of FIG.

The k-nearest neighbor identification method by optical arithmetic processing in the embodiment is composed of learning means 100, registration means 200, optical arithmetic means 300, and identification means 400.

The learning means 100 is a means for generating a feature extraction unit used in the registration means 200 and the identification means 400.

Specifically, the learning means 100 includes a learning data input unit 110, a feature amount obtained from a feature extractor generation unit 120 that extracts a feature amount from data acquired from the learning data input unit 110, and a feature amount obtained from the feature extractor generation unit 120. Optical calculation that converts data into feature quantities applicable for optical correlation in the identification unit 130 that performs identification determination based on the above, the learning system evaluation unit 140 that evaluates the identification result by the identification unit 130, and the feature extractor generation unit 120. It is composed of a constraint addition unit 122 and a verification data input unit (not shown) for inputting verification data for evaluating the progress of learning.

In the learning data input unit 110 in the learning means 100, the learning data and the information serving as a teacher in identification are input as a set.

The feature amount extraction unit 120 in the learning means 100 is composed of a plurality of feature amount extraction layers. In each feature extraction layer, convolution, dimension reduction, and total coupling processing are performed on the training data.

In the identification unit 130 in the learning means 100, it is preferable that the obtained feature dimension is a fully connected layer that is the dimension of the identification target class.

The learning system evaluation unit 140 in the learning means 100 determines whether the output of the identification result obtained by the identification unit 130 and the information serving as a teacher are correct or incorrect. If the identification result is incorrect, the error from the correct answer is calculated.

The feature extraction unit 121 in the learning means 100 updates the weights in the feature extraction unit so as to minimize the error obtained by the learning system evaluation unit 140.

Here, the data handled by the optical calculation in the optical calculation means 300 is binary data. The feature amount extracted from the multi-valued data is multi-valued, and it is generally known that the amount of information is impaired in the conversion for binarizing the feature amount. This lack of information amount is likely to affect the discrimination performance by optical calculation. Therefore, the feature extraction unit 121 and the optical calculation constraint addition unit 122 in the learning means 100 maintain an infinite amount of information even if the feature amount that can be finally output from the feature extraction unit 121 is subjected to binary data conversion processing in advance. Design to learn as much as possible.

Specifically, the activation function in at least one of the final layers of each feature extraction layer obtained by the feature extraction unit 121, which is closer to the input side, or a plurality of layers closer to the input side is approximated by a sigmoid function. By using a step function, learning is promoted so that the value of each dimension of the feature amount finally output from the feature amount extractor generator 120 becomes a value as close to 0 or 1, that is, a binary approximate feature amount. be able to.

Here, a method for advancing learning so as to obtain a binary approximate feature will be described in detail.

FIG. 3 is an implementation example of the learning means 100 in the optical arithmetic processing of FIG.

In the implementation of the feature extraction unit 121 and the constraint addition unit 122 for optical calculation in the learning means 100, the convolution layer is laminated with three layers sandwiching the max pooling, and then four fully connected layers are laminated to classify the final output layer. It was set to 10 units, which is the same as the number.

The characteristic point of the implementation is that the step function is used as the activation function in the layer fc (n-l) immediately before the final output layer fc (n), and the number of units is set to 20,000, and fc ( The point is that the number is overwhelmingly larger than that of the n-2) layer.

The feature of the learner implementation is that the activation function is a step function for the purpose of incorporating it into binary data for optical correlation. The step function, which digitizes data as an activation function as much as possible, is concerned that its expressiveness will be lower than that of the Relu (Rectified linear unit) function, which reflects multi-valued data uniformly, and compensates for the above concern. For the purpose, the number of units in the fc (n-1) layer was increased compared to fc (n-2).

The details of applying the step function to the activation function in the above implementation will be described. The step function was replaced by multiplying the exponent k of the number of Napiers in the sigmoid function by a real number to approximate the step function. In this experiment, k = 100 was set from the observation of the degree of convergence of learning (see equation (2) shown below).

On the other hand, the derivative of the approximate step function used in the activation function in the above feedforward can be expressed by Eq. (3).

In other words, the weight update formula in error back propagation is equation (4), and since the real number multiple part is directly involved in the update weight, the movement of the weight vector due to learning becomes extremely large, and learning does not proceed. confirmed.

Therefore, in the actual error back propagation, the weight update was performed by removing this real number multiple part. Further, as described above, the number of units is set to 20,000 in the layer fc (n-1) immediately before the final output layer fc (n) in the above-mentioned mounting.

This is because the implementation of the number of units cannot extract a sufficient binary approximate feature amount as the amount of information only by applying the step function approximated by the sigmoid function as the activation function.

Specifically, in the multi-valued data, the number of expressions of the feature quantity in each dimension is infinite, whereas the number of representations of the binary feature quantity in each dimension is only 0 and 1.

Therefore, it was decided to drastically increase the number of units in at least the final layer of each feature extraction layer or in other multiple layers to which the step function approximated by the sigmoid function is applied.

The implementation of the increase in the number of units will increase the dimension of the binary feature amount, and as a result, the increase in the number of dimensions will increase the expression amount of the binary feature amount.

Generally, in digital processing, increasing the number of dimensions of a feature is avoided because it leads to an increase in the amount of calculation, but in optical calculation, each dimension data can be taken out as direct light and processed in parallel, so the number of dimensions of the feature. The increase in is not a problem.

The registration means 200 is a means for converting registration data into optical calculation data and storing it in the optical calculation feature vector optical storage unit 320. The feature extraction unit 220 in the registration means 200 and the identification means 400 takes out the feature amount learned by the learning means 100 and transfers and uses it as a feature amount extractor. The data input to the registration input data unit 210 is input to the feature extraction unit 220 provided with the feature amount extractor described above, and the output obtained from one layer of the final output layer of the learning means by feed-forward processing. Is extracted as a feature amount.

FIG. 4 shows the value distribution of the feature amount obtained by the feedforward process for a specific sketch input using the learned learning means of FIG. 3 as a feature extractor.

In FIG. 4, the horizontal axis shows the size of each dimension of the feature amount, and the vertical axis shows the size of the feature amount in each dimension. As can be seen from FIG. 4, it was confirmed that the magnitudes of the features were distributed in the range of 0 to 1, and that nearly 90% of all dimensions were infinitely close to 0 or 1. In this implementation, the feature quantity shown in FIG. 4 was subjected to competitive value processing at an intermediate point of 0.5 and converted into a binary.

Next, in the optical calculation data conversion unit 310 of the optical calculation means 300, the binary feature amount obtained by the feature extraction unit 220 is color-converted with 0 being black and 1 being white, and then the data is imaged. Was done.

FIG. 10A shows an example of coordinate conversion after performing color conversion when the number of elements is 12, FIG. 10B shows the registration data, and FIG. 10C actually converts the identification target data. This is an example. Data for optical calculation is generated by arranging in a spiral shape from the center to the outside.

The optical calculation feature vector optical storage unit 320 in the optical calculation means 300 stores the optical calculation data generated by the optical calculation data conversion unit 310 as a hologram.

FIG. 11 is a diagram showing an example of schematic optical means used for the feature vector optical recording unit 320 for optical calculation and the similarity calculation unit 330 shown in FIG.

The light emitted from the hologram recording light source 341 is formed into parallel light sufficiently spread by the pem forming optical means 342. For example, a beam expander and the like are included. The parallel light is given a phase difference by the wave plate 343, folded back by the mirror 344, given a phase difference by the wavelength plate 346, folded back by the polarizing beam splitter 347, and then incident on the spatial light modulation element 348. At this time, data obtained by adding an image serving as reference light for creating a hologram to the data to be recorded generated by the optical calculation data conversion unit 310 is input to the spatial light modulation element.

The incident light is given a phase difference only in white pixels by the spatial light modulator, only the light of white pixels passes through the polarizing beam splitter 347, the image is relayed by the lens 349 and the lens 353, and the objective mounted on the actuator 355. It is Fourier transformed by the lens and data is recorded on the hologram recording medium 356. Here, depending on the spatial light modulation element used, only white pixels may be passed by combining a 1/4 wave plate (not shown).

Here, when the mechanism for rotating the disk is used, the servo light source 364 having a wavelength different from that at the time of recording or collation is held, folded back by the mirror 366, and the phase difference is given by the wave plate 367, and the dichroic plum splitter 354. The light emitted to the hologram recording medium by the actuator 355 returns from the reflecting surface of the recording medium, is folded back by the polarizing beam splitter 365, is detected by the light receiving element 368, and is detected by the light receiving element 368, and the position of the objective lens on the actuator is determined by its intensity. Adds focus and tracking servo functions to move.

Here, the medium for storing the optical calculation data as a hologram is not limited to the holographic optical disk. For example, the above-mentioned medium includes a polygonal medium, and the material of the medium also includes a photopolymer, a crystal, a liquid crystal, and the like.

Next, the identification means 400 performs identification by correlation matching between the data input from the identification target data input unit 410 and the data stored in the optical calculation feature vector optical storage unit 320 in the registration means 200.

Also in the feature extraction by the identification means 400, the feature amount of the data input from the identification target data input unit 410 is extracted by using the same feature extraction unit 220 as that used in the registration means 200.

The optical calculation data conversion unit 310 in the identification means 400 also uses the same optical calculation data conversion unit 310 used in the registration means 200 to convert the feature amount extracted from the feature extraction unit 220 into optical calculation data. Convert.

The similarity calculation unit 330 in the identification means 400 registers the feature amount obtained by the feature extraction unit 220 from the data input by the identification target data input unit 410 with the data converted into optical calculation data by the optical calculation data conversion unit 310. The similarity with the data stored in the optical calculation feature vector optical storage unit 320 that stores the registration data in the means 200 is calculated.

In the example of the optical means shown in FIG. 11, the light emitted from the matching light source 357 is formed into parallel light sufficiently spread by the pem forming optical means 358. The parallel light is given a phase difference by the wave plate 359, folded back by the polarizing beam splitter 345, given a phase difference by the wavelength plate 346, folded back by the polarizing beam splitter 347, and then incident on the spatial light modulator 348.

At this time, the data generated by the optical calculation data conversion unit 310 from the data input from the identification target data input unit 410 is input to the spatial light modulation element. Only white pixel light passes through the polarizing beam splitter 347, the image is relayed by the lens 349 and the lens 353, Fourier transformed by the objective lens mounted on the actuator 355, and irradiated to the hologram recording medium 356 on which the data is recorded. The lens. The light reflected by the mirror surface of the disk is folded back by the polarizing beam splitter 350, passes through the lens 360, the mirror 361, and the lens 362, and the light intensity is detected by the light receiving element 363.

The similarity obtained by the similarity calculation unit 330 is output from the output unit 420 of the identification means 400 as a higher light intensity as the similarity is higher and a lower light intensity as the similarity is lower.

When the result obtained by the majority vote of the class belonging to k stored data from the one with the larger light intensity output from the output unit 420 of the identification means 400 is used as the identification result, this method implements the k-nearest neighbor method digitally. It is synonymous with what you did.

An experiment was conducted by applying the k-nearest neighbor classifier obtained by the optical arithmetic processing of FIG. 1 to sketch-based three-dimensional collation.

The specific experimental method is to use the learning data as mixed data of the outline data of the three-dimensional model and the sketch data at a fixed ratio, generate an identification learning device with 10 classes based on the learning means 100, and sketch. Learning proceeded using the data as verification data, and the feature extraction unit of the obtained learned learning means was used as a feature extractor. The outline data of the three-dimensional model was input to the above-mentioned feature extractor, and the extracted feature vector for optical calculation was stored in the optical calculation feature vector optical storage unit 320 of the optical calculation means 300.

Here, the sketch data to be the identification target data is input from the identification target data input unit 410 of the identification means 400, and the obtained optical storage feature vector is stored in the optical calculation feature vector optical storage unit 320. The similarity was calculated in response to the above, and the output result was obtained from the output unit 420. As described above, the obtained output has a higher light intensity as the similarity is higher, and a lower light intensity as the output has a lower similarity.

Therefore, the result obtained by the majority vote of the class to which the k stored data belongs is identified from the output light intensity, and the k three-dimensional model data has a similar shape to the input sketch data. The result was a three-dimensional model collation.

FIG. 5 shows the identification accuracy of the obtained feature amount. From FIG. 5, the identification accuracy in the learning means was about 86% in the test data. The identification accuracy of the k-nearest neighbor method based on the binaryized features obtained by using the learned feature extraction unit of the learning means as a feature extractor is about 87% in the test data, and this learning means It was confirmed that the obtained feature amount is a feature amount that can maintain its expressive power even if it is pinaried.

Further, in the collation of the sketch-based 3D model data in the vicinity of the actual digital k, when two types of tree class sketches (sketch data A and sketch data B) are input, the 3D has a similar shape to the sketch data A. The collation result of the model was a three-dimensional model having a shape similar to the sketch data A.

On the other hand, the collation result of the three-dimensional model having a similar shape to the sketch data B was a three-dimensional model belonging to the tree class, although it was not very similar to the sketch data B. At this time, the distance between the sketch data A and the vicinity of k and the distance between the sketch data B and the vicinity of k were smaller in the former and larger in the latter. That is, it was confirmed that the feature quantity obtained by this learning means is a binary feature quantity having an expression quantity of a similar shape.

FIG. 6 shows a case where a sketch of a chair, which is positive data, is input as identification target data in a storage unit of a chair class when the k-near classifier obtained by optical arithmetic processing of FIG. It is a figure which showed the similarity calculation result by the optical calculation when the sketch other than the chair is input as the identification target data by the light intensity of the vertical axis.

As can be seen from FIG. 6, it can be confirmed that the light intensity increases when the positive class identification target data is input, while the light intensity decreases when the negative class identification target data is input. It shows that the similarity can be calculated by optical calculation.

FIG. 7 is a diagram showing a multi-class identification result by optical calculation when the k-nearest neighbor classifier obtained by optical calculation processing of FIG. 1 is applied to sketch-based three-dimensional collation.

In FIGS. 7A and 7B, the left side is the part that stores the feature vector for optical calculation extracted from the 3D model outline data of the tire class, and the right side is the optical calculation extracted from the 3D model outline data of the guitar class. This is the part that stores the feature vector.

As can be seen from FIGS. 7 (a) and 7 (b), when the sketch data of the guitar is input as the identification target data, the similarity is high with the right side of the storage unit, but the similarity is small with the left side. Further, when the tire sketch data is input as the identification target data, it can be seen that the degree of similarity is high with that of the left side of the storage unit, but the degree of similarity is small with that of the right side. From this, it was confirmed that the similarity calculation by optical calculation can be applied to multi-discrimination.

FIG. 8 is a diagram showing a three-dimensional model collation result by the k-nearest neighbor method by the optical correlation calculation when the k-nearest neighbor classifier obtained by the optical arithmetic processing of FIG. 1 is applied to the sketch-based three-dimensional collation.

As described above, in the optical correlation calculation, the higher the similarity, the higher the light intensity, and the lower the similarity, the lower the light intensity, which is output from the output unit 420 of the identification means 400. When the result obtained by the majority vote of the class to which k stored data belongs from the one with the larger light intensity output from the output unit 420 of the identification means 400 is used as the identification result, this method implements the k-nearest neighbor method digitally. It is synonymous with that. That is, FIG. 8 is a diagram showing the result of three-dimensional model data collation having a similar shape based on a sketch by the k-nearest neighbor method by this optical calculation.

FIG. 8 shows the result of performing 5 neighborhood identification with k = 5.
From FIG. 8, it was confirmed that even when the optical calculation is applied to the collation portion of the sketch-based three-dimensional model data, the three-dimensional model collation having a similar shape can be accurately performed with respect to the input sketch data.

Next, a flowchart of the k-nearest neighbor classifier by optical arithmetic processing according to this embodiment will be described with reference to FIG. The description and each step are abbreviated as "S".

The flowchart of the k-nearest neighbor classifier by the optical calculation process described above includes a learning process, a registration process, and an identification process.

In the learning process, first, the teacher label and the learning data are input as a set in S101. Next, in S102, preprocessing is performed on the data input in S101. Specifically, the values are normalized by dividing the array values of all the training data by the maximum value among the array values of the training data.

Feature extraction is performed in S103 using the data normalized by the preprocessing of S102. The feature amount obtained in S103 is multiplied by the discrimination function in S104 to obtain the discrimination result, and the weight vector is optimized based on the discrimination result. S101, S102, S103 and S104 are repeated a certain number of times until it is determined that the learning is completed.

In the registration process, first, the registration data is input as a set with the teacher label in S201. Then, in S202, the same pretreatment as in S102 is performed.

In S203, the feature amount is extracted from the data obtained in the preprocessing of S202 by extracting the learned feature extraction unit whose weight vector is optimized in the learning process and using it as a feature extractor. The extracted features are subjected to competitive value processing in S204, and the features are binarized to obtain a binary feature. The obtained binary feature amount is converted into optical calculation data with 0 as a black pixel value and 1 as a white pixel value, and stored in a hologram in S205.

In the identification process, first, the data to be identified to be identified without a label in S301 is input. Next, in S302, the same pretreatment as in S102 is performed. In S303 as well, as in S203, the feature amount is extracted from the data obtained in the preprocessing of S302 by extracting the learned feature extraction unit whose weight vector is optimized by the learning process and using it as a feature extractor. ..

A binary feature amount is obtained by performing a fighting value process in S304 on the extracted feature amount and converting the feature amount into data for optical calculation. Using the obtained binary features, in S305, the similarity with the binary features stored in the hologram in S205 in the registration means 200 is calculated by optical calculation. The identification determination is performed in S306 based on the similarity obtained by the optical calculation.

Next, a flowchart of the optical calculation means 300 will be described with reference to FIG.

FIG. 9A is a flowchart of the optical calculation means 300 in the registration means 200. The feature amount obtained by the feature extraction unit is input in S401, and binary conversion is performed in S402. In S403, the obtained features are relocated according to the spatial light modulator and the optical system.

Reference light is added to the pattern obtained in S403 in S404, displayed on the spatial light modulation element in S405, and irradiated with light in S406. The steps from S401 to S406 are the registration means 200 and the optical calculation data conversion process 310.

After S406, the feature amount storage process 320 for optical calculation is performed, the light irradiated by S406 reaches the recording medium, and the feature amount generated from the registration data is registered. S401, S402, S403, S404, S405 and S406 are repeated until it is determined that the storage process has been completed for all the data for registration.

FIG. 9B is a flowchart of the optical calculation means 300 in the identification means 400. The feature amount obtained by the feature extraction unit in S501 is input, and binary conversion is performed in S502. In S503, the obtained features are relocated according to the spatial light modulator and the optical system.

The pattern obtained in S503 is displayed on the spatial light modulation element in S504, and light is irradiated in S505. The steps from S501 to S505 are the processing of the optical calculation data conversion unit 310 in the identification means 400.

After S505, processing is performed by the similarity calculation unit 330, the light irradiated by S505 reaches the recording medium in which the feature amount of the registration data is stored in S407, collation is performed, and the collation result is obtained in S506. The light intensity is received and the similarity is determined in S507.

Example 2 of the identification method using a linear classifier by optical arithmetic processing will be described with reference to the configuration diagram of FIG.

The linear classifier by the optical calculation process in the first embodiment is composed of the learning means 100, the registration means 200, the optical calculation means 300, and the identification means 400, and FIG. 12 is a flowchart of each means.

The learning means 100 is a means for constructing the feature extraction unit 121 and the optical calculation constraint addition unit 122 used in the registration means 200 and the identification means 400.

Specifically, the learning means 100 includes a learning data input unit 110, a feature extractor generation unit 120 that generates an extractor that extracts a feature amount from the data acquired from the learning data input unit 110, and a feature extractor generation unit 120. Based on the evaluation by the identification unit 130 that performs the identification judgment based on the feature amount extracted from the feature extractor obtained, the learning system evaluation unit 140 that evaluates the identification result by the identification unit 130, and the learning system evaluation unit 140. In the feature extraction unit 121 that updates the weight of the feature extraction unit, the feature extraction unit 120, the optical calculation constraint addition unit 122 that converts data into the feature amount applicable to the optical calculation, and the learning progress state are evaluated. It is a means including a verification data input unit 150 for inputting verification data.

The learning data input unit 110 in the learning means 100 sets and inputs the learning data and the information that is the correct answer in the identification. Further, the feature extractor generation unit 120 is composed of a plurality of feature amount extraction layers. Each feature extraction layer performs convolution, dimension reduction, and full coupling processing on the training data.

In the identification unit 130 in the learning means 100, it is preferable that the feature quantity dimension obtained is a fully connected layer having dimensions equal to the number of classes to be identified. Further, the learning system evaluation unit 140 determines whether the identification result obtained by the identification unit 130 is correct or incorrect. If the identification is incorrect, find the error from the correct answer. Further, the feature extraction unit 121 in the learning means 100 updates the weight in the feature extraction unit so as to minimize the error obtained by the learning system evaluation unit 140.

In the optical calculation constraint addition unit 122 in the learning means 100, the data obtained by applying the feature extraction unit 121 to the learning data input from the learning data input unit 110 and the information that is the correct answer in the identification are set and input, and the data is input. Find the optimal weight vector to classify the data into the appropriate class. Further, the above-mentioned weight vector obtained is decomposed into a plurality of binary vectors and real number coefficients.

Here, the data handled by the optical calculation in the optical calculation means 300 is pinary data in the second embodiment. It is known that the feature amount extracted from the multi-valued data is multi-valued, and in the conversion to binarize the feature amount, the data expressive power is lowered and the amount of information is impaired. This lack of information amount is likely to affect the discrimination performance by optical calculation. Therefore, the number of data dimensions can be increased from that of the multi-valued data by designing the feature amount that holds the information amount in the feature extraction unit 121 of the learning means 100, or by setting a plurality of relational values and performing binarization. The expressive power has been improved so that the amount of information can be maintained even if binary data conversion processing is performed.

Specifically, the feature amount extraction unit 121 advances learning so that the feature amount based on the multivalued data can be obtained, and the feature amount based on the multivalued data obtained by the feature amount extraction learning unit is binarized by the dark value and converted into a binary. A highly expressive binary feature is obtained by repeating for the number of fighting values.

Here, a method for obtaining a feature amount of multi-valued data and a method for binarizing the obtained feature amount will be described.

13 and 14 are diagrams showing an implementation example of the feature amount extraction unit 121 of the learning means 100 of FIG.

In the implementation of the feature amount extraction unit 121 in the learning means 100, the convolution layer is one layer with the max pooling process interposed therebetween, and the fully connected layer is two layers, and the final output layer is two units having the same number of classes.

The ReLU (Rectified Linear Unit) function was used as the activation function in the above implementation.

The weight of each layer of the above-mentioned implementation can be optimized, and the feature quantity by multiple values can be calculated by the output of the convolution layer. However, when the features were extracted, convolution was performed by optical calculation. This is because the convolution process can be replaced by optical calculation.

The feature amount calculated as described above is converted into data for optical calculation by the binary processing shown in FIG. Among the multi-valued data, the one that is larger than the dark value is set to 1 and the one that is not is set to 0 to binarize, and the above-mentioned binarization is performed by a plurality of fighting values.

In the optical calculation constraint addition unit 122 in the learning means 100 for calculating the optimum weight vector of the discrimination function from the binarized features calculated as described above, a linear SVM (Support Vector Machine) was used as the optimization algorithm.

The discrimination function obtained by calculating the weight vector is given by Eq. (5). Here, ω is the weight vector and x is the data to be identified.

Next, the optical calculation data conversion unit 320 in the optical calculation means 300 decomposes the weight vector generated by the optical calculation constraint addition unit 122 into a binary vector and a real number coefficient by the decomposition algorithm shown in FIG. 15, and obtains the binary vector. Input to the data conversion unit for optical calculation. The optical calculation data is stored in the hologram optical memory by performing color conversion with 0 being black and 1 being white and arranging them in a spiral shape from the center to the outside.

In the decomposition algorithm shown in FIG. 15, the positive number of the multi-valued weight vector corresponds to 1 and the negative number corresponds to 0, and the original multi-valued weight vector and the binarized vector have the same magnitude. The real number coefficient is obtained so that, and the product of the obtained real number coefficient and the binary vector is binarized again by positive and negative. By repeating the above, the multi-valued weight vector is decomposed into a binary vector and a real number coefficient.

When decomposed into a binary vector and a real number coefficient, the discriminant function is transformed as shown in Eq. (6).

The optical calculation feature vector optical storage unit 320 in the optical calculation means 300 stores the optical calculation data generated by the optical calculation data conversion unit 310 in the hologram optical memory.

Next, the identification means 400 identifies by similarity matching between the data input from the identification target data input unit 410 and the data stored in the optical calculation feature vector optical storage unit 320 in the registration means 200.

In the feature extraction in the identification means 400, the feature extraction unit 220 is used to extract the feature amount of the data input from the identification target data input unit 410.

The feature extraction unit 220 in the second embodiment uses the feature extraction unit 121 of the feature extractor generation unit 120 of the learning means 100.

In the optical calculation data conversion unit 310 of the identification means 400, the binary vector obtained by the feature extraction unit 220 is color-converted with 0 being black and 1 being white, and arranged in a spiral shape from the center to the outside. Convert to optical calculation data with.

In the similarity calculation unit 330 of the identification means 400, the feature amount obtained by the feature extraction unit 220 from the data input by the identification target data input unit 410 described above is converted into data for optical calculation by the optical calculation data conversion unit 330. , The degree of similarity is calculated by listening to the data stored in the optical calculation feature vector optical storage unit 320 in which the registration data is stored in the registration means 200.

The similarity obtained by the similarity calculation unit 330 is output to the output unit 420 of the identification means 400 as a higher light intensity as the degree of similarity is higher and a lower light intensity as the degree of similarity is lower.

The output of the identification function is obtained by adding the real number coefficient obtained by the optical calculation data conversion unit to the obtained output and performing the calculation of the equation (7). Here, β is a real number coefficient, b is a binary vector stored in the optical storage feature vector optical storage unit 320, and x is the processing of the feature extraction unit 220 with respect to the identification target data input from the identification target data input unit 410. It is a vector with.

Next, the flowchart shown in FIG. 12 will be described. The description and each step are abbreviated as "S".

The identification method by the linear classifier in the above-mentioned optical arithmetic processing includes a learning process, an optical arithmetic constraint addition part in the learning process, a registration process, and an identification process.

In the learning process, first, the learning data is input as a set with the teacher label in S101. Next, in S102, preprocessing is performed on the data input in S101. Specifically, the value is normalized by dividing the array values of all the training data by the maximum value among the array values of the training data, and grayscale is performed.

Feature extraction is performed in S103 using the data normalized by the preprocessing of S102. The feature amount obtained in S103 is multiplied by the discrimination function in S104 to obtain the discrimination result, and the weight vector is optimized based on the discrimination result. S101, 102, S103 and S104 are repeated a certain number of times until it is determined that the learning is completed.

In the optical calculation constraint addition unit, learning data is input as a set with the teacher label in S201. Next, in S202, preprocessing is performed on the data input in S201. Specifically, the value is normalized by dividing the array values of all the training data by the maximum value among the array values of the training data, and grayscale is performed.

Using the data normalized by the preprocessing of S202, binarization is performed in S204, and the weight vector of the identification function is optimized by linear SVM in S205 so that the input data can be classified for each class.

The weight vector obtained by the learning of S205 is decomposed into a plurality of binary vectors and a plurality of real number coefficients by the pinary decomposition method.

In the registration process, first, the binary vector obtained in S206 of the optical calculation constraint addition unit in S301 is input as registration data.

In S302, S303, and S204, the binary vector vector obtained in S206 of the optical calculation constraint addition unit is converted into data for optical calculation with 0 as a black pixel value and 1 as a white pixel value, and stored in a hologram in S205.

In the identification process, first, in S401, the data to be identified to be identified without a label is input. Next, in S402, the same pretreatment as in S102 is performed. In S403 as well, as in S203, the feature amount is extracted by taking out the learned feature extraction unit whose weight vector is optimized by the learning process and applying it to the input data.

In S404, a binary feature amount is obtained by binarizing the extracted feature amount by the same processing as in S204, and the obtained pinary feature amount is subjected to the same processing as in S304 for optical calculation. Convert to data. Using the obtained binary features, in S405, the similarity with the binary features stored in the hologram in S305 in the registration process is calculated by optical calculation. The identification determination is performed in S406 based on the similarity obtained by the optical calculation.

In the feature extraction unit 121 of the learning means 100, the feature quantity obtained by learning of CNN (Convolutional Neural Network) was not binarized, but the B-HOG feature quantity was used for the human detection experiment. The B-HOG feature is obtained by binarizing the gradient histogram obtained from the HOG feature with positive and negative values according to the relational value, and adding a co-occurrence expression of XOR in each block of the HOG feature to prevent information loss. ..

A specific experimental method is to calculate a B-HOG feature amount based on 100 based on a learning means from learning data consisting of a human image and other images, and generate an identification function from the feature amount. The binary vector obtained by the optical calculation constraint addition unit 122 of the learning means 100 was input to the optical calculation data conversion unit of the optical calculation means 300, and stored in the optical calculation feature vector optical storage unit 320. Here, the image to be the identification target data is input from the identification target data input unit 410 of the identification means 400, and the obtained optical storage feature vector is combined with the binary vector stored in the optical calculation feature vector optical storage unit 320. Calculate the similarity with the question. The output result was obtained from the output unit 420. As described above, the obtained output has a higher light intensity as the similarity is higher, and a lower light intensity as the output has a lower similarity. The output of the discriminant function is calculated by adding the real number coefficient obtained by the decomposition method of the constraint addition unit 122 for optical calculation to the obtained output. The identification result can be obtained by determining the fighting value of the obtained output.

In the learning means 110, the identification accuracy of the identification function using the B-HOG feature amount was about 92%.

The output of the identification function calculated from the output of the output unit 420 of the identification means 400 is shown in FIG. By determining the darkness value from FIG. 16, it can be confirmed that it is possible to distinguish between a person and other image data.

Example 3 according to the present invention (that is, one embodiment of the pattern recognition method by optical arithmetic processing) will be described with reference to the configuration diagram of FIG.

As shown in FIG. 17, the pattern recognition device 2 by the optical calculation processing in the third embodiment is provided by the learning means 100, the feature amount conversion means (feature amount conversion module) 500, the registration means 200, the optical calculation means 300, and the identification means 400. FIG. 18 is a flowchart of each of the above-mentioned means.

The learning means 100 is a means for determining a conversion function of the optical correlation data conversion unit. Specifically, the learning means 100 also includes the learning data input unit 110, the feature extractor generation unit 120 that converts the data acquired from the learning data input unit 110, and the converted data obtained from the feature extractor generation unit 120. It is composed of an identification unit 130 for making a determination, a learning system evaluation unit 140 for evaluating the determination result by the identification unit 130, and a verification data input unit 150 for inputting verification data for evaluating the progress state of learning. In addition, the software for executing deep learning distributed by various academic institutions on a computer is equipped with a function that can output, read, and use learned weights and biases. It is also possible to use the trained feature amount extractor without going through the learning means 100.

The feature amount conversion means 500 is a feature amount conversion module that converts the feature amount of the training data by training in a neural network (sometimes simply referred to as a “network”), and has an arbitrary first dimension number and an arbitrary feature. A non-linear function consisting of a learning data input unit 510 in which training data having a quantity is input, and a polynomial (expression) having a weight and a bias having a second dimension number satisfying a predetermined optical calculation constraint condition as factors. The learning data input to the training data input unit 510 is input to the learning model composed of the above, and the intermediate layer setting unit 520 for setting the intermediate layer having the second dimension number and the intermediate layer setting unit 520 are set. The output layer having the first number of dimensions is calculated from the intermediate layer, and the output calculation unit 530 that calculates the output of the neural network and the error between the neural network output and the training data calculated by the output calculation unit 530 are calculated. The error calculation unit 540 to be performed, the determination unit 550 to determine whether or not the error calculated by the error calculation unit 540 satisfies a predetermined condition, and the weight and the bias are changed based on the error, and the weight after the change is changed. It includes a weight bias changing unit 560 that obtains the weight and the bias, and a learning model output unit 570 that outputs a learning model composed of a non-linear function composed of a neural network having the changed weight and the bias as factors. Further, when the determination unit 550 determines that the error satisfies a predetermined condition, the feature amount conversion means 500 uses a function composed of an equation having the changed weight and bias as factors from the learning model output unit 570. When the configured learning model is output and the determination unit 550 determines that the error does not satisfy a predetermined condition, the output calculation unit 530 sets the weights and biases included in the polynomial of the learning model to the weight bias change unit 560. It is provided with a control unit 580 that changes the weight and bias after the change obtained in the above and calculates the output of the neural network.

The above-mentioned optical calculation constraint condition is determined by a predetermined quantization number in the optical calculation performed using the output of the neural network, an optical parameter of the optical correlator used in the optical calculation, and a predetermined energy number.

Specifically, as an optical calculation constraint condition,
(1) The number of quantizations is limited (2) The number of dimensions satisfies the optimum value for the optical correlation system (optical correlator) (3) The amount of energy is constant (4) Information on the feature amount I will not lose it. In this embodiment, the quantum number of (1) is limited to 2 by the binary data. Further, (3) is equivalent to a constant white rate.

Regarding the above-mentioned constraint condition (2), as shown in FIG. 23, the light source 602, the beam diameter expansion lens system 604, the mirror 606, the spatial light modulator (display element) 608, the polarization beam splitter 612, the reduction optical system 614, and the like. The optical correlator 600 in which the objective lens (lens) 616 is arranged preferably satisfies the equation (1). However, the reduction optical system 614 can be omitted.

In equation (1), z is the second dimension number, a is the pixel pitch [mm] of the spatial light modulator 608, r is the radius [mm] of the lens objective lens 616, and b is the reduction of the reduction optical system 614. Indicates the magnification.

In the learning data input unit 510, the control unit inputs data having an arbitrary number of dimensions (first dimension number) n having feature information as teacher information. The neural network may have one layer or may be composed of a plurality of layers. Each layer performs convolution, dimension reduction, and full coupling processing on the training data.

The intermediate layer setting unit 520 receives the teacher information and makes the following settings for the neural network stored in the initial network storage unit. First, the same number of dimensions n as the teacher information is set in the input layer and the output layer. In addition, in order to satisfy the above-mentioned constraint condition (2), a dimension number (second dimension number, that is, z in the equation (1)) m suitable for the optical correlation calculation is set for the intermediate layer. In this embodiment, a network consisting of three layers, an input layer, an intermediate layer, and an output layer, as shown in FIG. 19 was used.

In the network initial setting unit, arbitrary values are set as initial values for the weights w and w'and the biases b and b'stored in the initial network storage unit. Here, the activation function y for the optical correlation system is set as the activation function of the intermediate layer, and the activation function having the same range as the range of the teacher information as the output is set as the activation function g of the output layer.

The activation function for photocorrelation is an exponent of the number of Napiers in the sigmoid function k in order to satisfy constraint 1.
Is multiplied by a real number to approximate the step function (see equation (8)). Approximate binarization of input data is realized by approximating the step function. In this study, k = 1000 from the observation of the convergence of learning.
It is said.

Further, the constraint condition (3) is applied by the input value offset c. In the sigmoid function, the output value gradually changes to 0 or 1 with reference to the input value of 0, but in order to satisfy the constraint condition (3), in this activation function, an offset c for subtracting the input value is set. The reference value was changed by dynamically setting according to the input value, and the white rate was set to 20%.

Here, the white rate to be set may be changed according to the optical correlator used. It is also easy to make the white rate completely uniform by sorting the input data group in descending order and then dynamically setting it according to the input.

The backpropagation method used for learning calculates the derivative of the activation function. When the above activation function is differentiated, it becomes Eq. (9), and the weight update equation in the backpropagation method is Eq. (10). In equation (10), the real multiple part is directly involved in the update weight. This leads to the renewal of weights due to learning becoming extremely large, and learning does not proceed. Therefore, the equation (11) in which the real number multiple portion is removed is used as the weight update equation in the activation function f.

In the network output calculation unit that receives the network initialized by the network initialization unit, the teacher information is passed through the input layer and propagated to the intermediate layer and the output layer. When the output of the unit of the input layer propagates to the unit of the intermediate layer, the output of the unit is subjected to the first weight w, the first bias b is added, and this result is further multiplied by the activation function f. And the output of the activation function f is the output of the unit in the intermediate layer. That is, the first weight w = [w ₁₁ , w ₁₂ , ..., w _mn-1 , w _mn ], the first bias b = [b ₁ , b ₂ , ..., b _m-1 , b _m ]. Then, the output of the intermediate layer is represented by the equation (12).

When the output of the unit in the intermediate layer propagates to the output layer, a second weight w'is applied to the output of the unit in the intermediate layer, a bias b'is added, and this result is added to the range of the input value. It is input to the activation function g having the same range as, and the output of this activation function g becomes the output of the unit of the output layer, that is, the output of the network.

The error calculation unit 540 uses the network output from the output calculation unit 530 and the teacher information received from the learning data input unit 510 to calculate the error between the two so that the network output and the teacher information have the same value. To do.

The determination unit 550 receives an error from the error calculation unit 540, and determines whether the change in the error has almost disappeared or whether the process has been repeated a specified number of times. If none of the above is satisfied, the network and error are passed to the weight bias changing unit 560. When either of them is satisfied, the output layer, the second weight w'and the bias b'are removed from the network, and only the input layer, the first weight w, the bias b and the intermediate layer are left, and the data conversion for optical correlation calculation is performed. Output as a vessel.

The weight bias changing unit 560 receives a network and an error from the determination unit 550, changes the weights w, w'and bias b, b'based on the error, and passes the changed network to the output calculation unit 530 again.

Actually, based on the determination in the determination unit 550, the control unit 580 communicates between the weight bias changing unit 560 and the output calculation unit 530.

By adding optical calculation constraints to the number of dimensions and activation function of the intermediate layer and then training so that the teacher information and the network output are the same, the above-mentioned constraints (1)-(3) ), But the data that satisfies the condition to prevent information loss of constraint 4 can be output from the intermediate layer. A converter is incorporated between the feature extraction unit and the optical correlation data conversion unit.

The registration means 200 first extracts the feature amount by inputting the registration data to the feature extraction unit, then inputs this feature amount to the feature amount conversion means, and then inputs the feature amount to the optical calculation data conversion unit to perform optical calculation. It is a means for converting into data for use and recording data for optical calculation in an optical storage unit.

The identification means 400 first extracts the feature amount by inputting the identification target data into the feature extraction unit, converts the feature amount into the feature amount conversion module, and then inputs the feature amount to the optical calculation data conversion unit to perform the optical calculation data. It is a means for performing identification by performing correlation matching with the data stored in the feature vector storage unit for optical calculation in the registration means by the similarity calculation unit.

The identification means 400 uses the same feature extraction unit, feature amount conversion module, and optical calculation data conversion unit as those used in the registration means 200 to extract the feature amount of the data input from the identification target data input unit. Then, it was converted into data for optical calculation. Further, as for the similarity obtained by the similarity calculation unit, the higher the similarity, the higher the light intensity, and conversely, the lower the similarity, the lower the light intensity is output to the output unit.

Next, the flowchart shown in FIG. 18 will be described. The explanation and writing steps are abbreviated as "S".

Identification in the pattern recognition method by optical arithmetic processing includes learning processing for better feature extraction, feature conversion means for converting features with optical correlation constraint conditions added, registration processing, and identification processing. As mentioned above, the software for executing deep learning distributed by various academic institutions on a computer is equipped with a function that can output, read, and use learned weights and biases. It is also possible to use the learned feature amount extractor without going through the above-mentioned learning means. Therefore, the learning process may or may not be performed.

In the learning process 100, the feature amount extraction is performed in S103 using the data normalized by the preprocessing of S102. The feature amount obtained in S103 is multiplied by the identification function in S104 to obtain the identification result, and the feature extractor is optimized based on the identification result. S101, 102, S103 and S104 are repeated a certain number of times until it is determined that the learning is completed.

In the feature amount conversion means, learning is performed to add a light constraint. In S201, the data whose features have been extracted by the feature amount extractor is input. Next, in S202, the data input in S201 is encoded in S202. After that, the feature amount converter is optimized in S204 by combining in S203 and using the difference between the data input in S201 and the data combined in S203.

In the registration process, first, the registration data is input in S301, and the same preprocessing as in S102 is performed. After the feature amount is extracted using the learning process 1 or the feature extractor prepared in advance in S303, an optical calculation constraint is added using the feature amount converter created by the feature amount conversion means in S305, and the result is obtained in S304. Convert the binary vector as registration data. The obtained registration data is stored in the hologram in S306.

In the identification process, first, the data to be identified in S401 is input. Next, in S402, the same pretreatment as in S102 is performed. In S403 as well, as in S203, the feature amount is extracted by taking out the learned feature amount extractor optimized by the learning process 1 or the feature amount extractor prepared in advance and applying it to the input data.

In S404, a binary feature amount is obtained by binarizing the extracted feature amount by the same processing as in S305, and the obtained feature amount is converted into optical calculation data in the same manner as in S304. To do. Using the obtained binary features, in S406, the similarity with the binary features stored in the hologram in S306 in the registration process is calculated by optical calculation. The identification determination is performed in S407 based on the similarity obtained by the optical calculation.

In this example, an experiment was conducted by applying pattern recognition by optical arithmetic processing to a sketch-based product image search system.

As a concrete experimental method, a trained CNN network that searches for product images by sketching is used for the feature extraction unit, and the number of units in the intermediate fully connected layer in the coding unit of the feature conversion unit for optical calculation is calculated in 2560 dimensions. By making 40,000 units for the input data that can be expressed, it has more than 15 times more expressive power than binarizing the 2560 dimensions.

As a data set, 97 pairs of chair product images and sketch images were used. The term "pair" is used here because the sketch image is drawn based on the product image, and there is a one-to-one correspondence between the product image and the sketch.

Here, in the registration means, the feature amount is extracted by inputting the product image of the chair into the feature extraction unit, and after passing through the feature amount conversion module, the above-mentioned feature amount is optically calculated by the feature conversion unit for optical calculation. The data was converted into data for optical calculation, and the data for optical calculation was stored in the feature vector storage unit for optical calculation of the optical calculation means.

In the identification means in the experiment, the feature amount is extracted by inputting the sketch of the chair into the feature extraction unit, and this feature amount is converted into the data for optical calculation by the feature conversion unit for optical calculation. A similarity calculation was performed between the optical calculation data and the optical calculation data of the product image of the chair stored in the optical calculation feature vector storage unit, and the output result was obtained from the output unit.

FIG. 20 shows the value distribution of the multi-valued features extracted by the feature extraction unit and the value distribution of the optical calculation data output from the optical calculation feature conversion unit when a sketch of a specific chair is input. Shown.

FIG. 20 (a) shows the value distribution of the multi-valued feature amount extracted from the feature extraction unit, and FIG. 20 (b) shows the value distribution of the optical calculation data output from the optical calculation feature conversion unit. In both cases, the horizontal axis indicates the magnitude of the feature amount, and the vertical axis indicates the number of occurrences of the magnitude of the feature amount. The feature quantity with a wide range of -2.5 to 2 extracted by the feature extraction unit is limited to 0 to 1 by the feature conversion unit for optical calculation, and is further approximated to 0 or 1 infinitely. It was confirmed.

Here, by utilizing the fact that there is a one-to-one correspondence between the product image and the sketch in the data set, what percentage of the output of the input of the sketch image corresponds to the product image within the upper K number in the entire data set. We introduced acc @ K, which indicates whether it exists, as an evaluation index. Based on this evaluation index, the feature conversion for optical calculation was evaluated.

FIG. 21 shows the identification accuracy. From the results shown in FIG. 21, it can be seen that the feature quantity can maintain the expressive power in the pattern recognition by the optical arithmetic processing.

In FIG. 22, the horizontal axis is the index of the database, and the vertical axis is the size of the internal product value. From the results shown in FIG. 22, it was confirmed that the degree of correlation of the product image corresponding to the sketch image was large, and that the other parts were suppressed.

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various aspects of the present invention are described within the scope of the claims. It can be transformed and changed.

100 Learning means 110 Learning data input unit 120 Feature extractor generator 121 Feature extraction unit 122 Optical calculation constraint addition unit 130 Identification unit 140 Learning system evaluation unit 150 Verification data input unit 200 Registration means 210 Registration data input unit 300 Optical calculation Means 310 Optical calculation data conversion unit 320 Optical calculation feature vector optical storage unit 330 Similarity calculation unit 500 Feature amount conversion means (feature amount conversion module)
510 Learning data input unit 520 Intermediate layer setting unit 530 Output calculation unit 540 Error calculation unit 550 Judgment unit 560 Weight bias change unit 570 Learning model output unit 580 Control unit

Claims (16)

A feature conversion module that transforms the features of training data by learning in a neural network.
A learning data input unit in which learning data having an arbitrary first dimension number and an arbitrary feature quantity is input, and
The training data input to the training data input unit is input to a training model composed of a function consisting of an equation having a weight and a bias having a second dimension number satisfying a predetermined optical calculation constraint condition as factors. , The intermediate layer setting unit for setting the intermediate layer having the second dimension number, and
An output calculation unit that calculates an output layer having the first dimension number from the intermediate layer set by the intermediate layer setting unit and calculates the output of the neural network.
An error calculation unit that calculates an error between the output of the neural network calculated by the output calculation unit and the learning data, and an error calculation unit.
A determination unit that determines whether or not the error calculated by the error calculation unit satisfies a predetermined condition,
A weight bias changing unit that changes the weight and the bias based on the error to obtain the changed weight and the bias, respectively.
A learning model output unit that outputs a learning model composed of a function composed of an equation having the changed weight and the bias as factors, and a learning model output unit.
When the determination unit determines that the error satisfies a predetermined condition, the output unit outputs a learning model composed of a function composed of an equation having the changed weight and the bias as factors. When the determination unit determines that the error does not satisfy a predetermined condition, the output calculation unit obtains the weight and the bias included in the equation of the learning model in the weight bias changing unit. A control unit that changes the weight and the bias after the change and calculates the output of the neural network.
With
The optical calculation constraint condition is characterized by being determined by a predetermined quantization number in an optical calculation performed using the output of the neural network, an optical parameter of an optical correlator used in the optical calculation, and a predetermined energy number. Feature conversion module. The optical correlator includes a display element that displays optical calculation data based on the output of the neural network, and a lens that collects light that reflects the calculation data.
The feature amount conversion module according to claim 1, wherein the optical parameters include the pixel pitch of the display element and the radius of the lens. The optical correlator reduces the beam diameter of light on which the optical calculation data displayed on the display element is reflected, and emits light having the reduced beam diameter toward the lens. Further prepare
The feature amount conversion module according to claim 2, wherein the optical parameter includes a reduction magnification of the reduction optical system. The feature amount conversion module according to claim 3, wherein the second dimension number satisfies the following equation (1).

In the above equation (1), z is the second dimension number, a is the pixel pitch [mm] of the display element, r is the radius [mm] of the lens, and b is the reduction magnification of the reduction optical system. Is shown. The feature amount conversion module according to any one of claims 1 to 4,
A learning means for learning a feature extractor that maximizes the identification accuracy of the learning data having a constraint condition for optical calculation by using the learning data with a class label as an input.
An optical calculation data conversion unit that converts the feature amount extracted from the feature extractor into the feature extractor, and
An optical storage unit that stores optical calculation data, an optical calculation means by a similarity calculation unit that provides an optical correlation function by calculating similarity in a plurality of optical calculation data hearings, and an optical calculation means.
The feature amount extracted by using the feature extractor obtained by the learning means with the registration data as a dictionary of the identification target data as an input is used in the optical calculation data conversion unit for the optical calculation feature amount. And the registration means registered in the optical storage unit in the optical calculation means.
With the identification target data as input, the similarity between the feature amount extracted from the feature extractor generated by the learning means and the feature amount registered in the optical storage unit in the registration means is set in the similarity calculation unit in the optical calculation means. An identification means that calculates and outputs the identification result from the output unit,
A pattern identification device comprising. A learning means for learning a feature extractor that maximizes the identification accuracy of the learning data having a constraint condition for optical calculation by using the learning data with a class label as an input.
To calculate the similarity between a data conversion unit for optical calculation that converts the feature amount extracted from the feature extractor into a feature extractor, an optical storage unit that stores data for optical calculation, and a plurality of data for optical calculation. Optical calculation means by the similarity calculation unit that provides the optical correlation function by
The feature amount extracted by using the feature extractor obtained by the learning means with the registration data as a dictionary of the identification target data as an input is used in the optical calculation data conversion unit for the optical calculation feature amount. And the registration means registered in the optical storage unit in the optical calculation means.
With the identification target data as input, the similarity between the feature amount extracted from the feature extractor generated by the learning means and the feature amount registered in the optical storage unit in the registration means is set in the similarity calculation unit in the optical calculation means. An identification means that calculates and outputs the identification result from the output unit,
A pattern identification device comprising. In the learning means, in the course of learning in the generation of feature extractor, so that the output by the feature extractor generated after learning becomes feature amounts binary approximation, the activation function definitive to the feature extractor, step The pattern according to claim 5 or 6, wherein a function that distributes the values of each dimension of the feature amount, which is a step function approximated by a function or a sigmoid function, in a value range of 0 to 1 is applied. Identification device. In the learning means, in the course of learning in the generation of feature extractor, output by the feature extractor to be generated becomes feature amounts binary approximation after learning, and, if expressive with the feature quantity is a multi-value and as a feature quantity not inferior as compared, the number of parameters portions updating is performed by the learning in the generation of the feature extractor, compared with the case of generating a feature extractor for extracting a feature value of the multi-level The pattern identification device according to any one of claims 5 to 7, wherein the number is large. The binary-approximate feature amount in which the feature amount of each dimension is distributed in the range of 0 to 1 obtained from the feature extractor obtained as a result of the learning of the feature extractor generation unit in the learning means is converted into the data for optical calculation. The pattern identification device according to any one of claims 5 to 8, characterized in that the unit is binarized with a specific dark value. In the optical calculation constraint addition unit of the feature extractor generation unit in the learning means, the feature amount extracted by giving input data to the feature amount extractor obtained by the learning means was optimized by a linear classifier. The pattern identification device according to claim 5 or 6, further comprising a method for binarizing a weight vector. In the optical calculation constraint addition unit of the feature extractor generation unit in the learning means, the feature amount extracted by giving input data to the feature amount extractor obtained by the learning means is subjected to the darkness of each of the multiple sword values. The pattern identification device according to claim 10, further comprising a method of binarizing using a magnitude relationship between a value and a feature amount value of each dimension. The pattern identification device according to any one of claims 5 to 11, wherein the optical storage unit is a holographic recording medium. The hologram Fitted click recording medium is formed in a disk shape, while rotating the holographic recording medium, the pattern identification apparatus according to claim 12, characterized in that the identification target data irradiation. The pattern identification device according to any one of claims 5 to 13, wherein in the optical calculation means, feature extraction of a data conversion unit for optical calculation is performed by optical calculation. In the pattern identification method of the pattern identification device that identifies the input data into some class, the learning data to which the class label is added by the pattern identification device is input, and the learning data having the constraint condition for optical calculation is provided. Learning steps to learn feature extractors that maximize identification accuracy,
To calculate the similarity between a data conversion unit for optical calculation that converts the feature amount extracted from the feature extractor into a feature extractor, an optical storage unit that stores data for optical calculation, and a plurality of data for optical calculation. The feature amount extracted by using the feature extractor obtained by the learning step with the optical calculation step by the similarity calculation unit that provides the optical correlation function and the registration data to be the dictionary of the identification target data as input. A registration step in which the data conversion unit for optical calculation converts the feature amount for optical calculation for optical calculation and registers it in the optical storage unit in the optical calculation step.
With the identification target data as input, the similarity between the feature amount extracted from the feature extractor generated in the learning step and the feature amount question registered in the optical storage unit in the registration step is set in the similarity calculation unit in the optical calculation step. An identification step that calculates and outputs the recognition result from the output unit,
Pattern identification method including. A learning step of learning a feature extractor that uses a computer as an input of learning data to which a class label is attached and that maximizes the identification accuracy of the training data having constraints for optical calculation.
To calculate the similarity between a data conversion unit for optical calculation that converts the feature amount extracted from the feature extractor into a feature extractor, an optical storage unit that stores data for optical calculation, and a plurality of data for optical calculation. An optical calculation step by a similarity calculator that provides an optical correlation function by
The feature amount extracted by using the feature extractor obtained in the learning step by inputting the registration data to be the dictionary of the identification target data is used in the optical calculation data conversion unit for the optical calculation feature amount. And the registration step of converting as and registering in the optical storage unit in the optical calculation step,
With the identification target data as input, the similarity between the feature amount extracted from the feature extractor generated in the learning step and the feature amount registered in the optical storage unit in the registration step is set in the similarity calculation unit in the optical calculation step. A program that calculates and functions as an identification step that outputs the identification result from the output section.

Images