CN116029350A - Two-dimensional photonic coherent convolution acceleration chip and its application system based on time interleaving - Google Patents

Abstract

The invention discloses a two-dimensional photon coherent convolution acceleration chip based on time interleaving and an application system thereof, belonging to the technical field of photoelectric integration and being applicable to all artificial intelligent neural networks comprising convolution operation. The invention integrates the intensity modulator, M delay weight phase shift units, M-1 second-stage delay waveguides, a second-stage beam combiner and a photoelectric detector which finish convolution acceleration operation through a photon integration technology. The convolution kernel matrix coefficient weighting, the primary time interleaving and the coherent summation of the signals to be convolved are realized by using a delay weighting phase shifting unit comprising an adjustable coupler, a delay waveguide, a phase shifter and a primary beam combiner, the secondary time interleaving of the weighted modulated optical signals is realized by using a secondary delay waveguide, and the weighted summation operation is realized by using a secondary beam combiner and a photoelectric detector. The invention takes light as an information carrier, and can greatly improve the speed and the energy efficiency ratio of convolution operation.

Description

Two-dimensional photonic coherent convolution acceleration chip and its application system based on time interleaving

technical field

The invention relates to a deep learning-oriented photonic neural network convolution acceleration chip, specifically a two-dimensional photon coherent convolution acceleration chip based on time interleaving and an application system thereof, belonging to the field of photon integration technology.

Background technique

Artificial intelligence is now widely used in fields such as machine vision, natural language processing, and autonomous driving. Among them, artificial neural networks, one of the important models of artificial intelligence technology, are widely used because of their excellent generalization ability and stability. The artificial neural network essentially establishes a similar neuron network interconnection mode by imitating the structure of the biological nervous system. Based on the mature development of electronic technology, the training and testing of mainstream neural network models are mainly based on electronic integrated chips, such as CPU, GPU, FPGA and application-specific integrated circuits. Since the current electronic chip adopts a classic computer structure that separates the program space from the data space, the data load between the storage unit and the computing unit is unstable and the power consumption is high, which limits the efficiency of network model training. Although the computing efficiency can be improved by improving the integration of electronic chips or through in-memory calculations, but limited by the microscopic quantum characteristics and macroscopic high-frequency response characteristics of electronic chips, these technical directions are also facing great challenges (see [Chen Hongwei, Yu Zhenming, Zhang Tian, et al. Development and Challenges of Photonic Neural Networks. China Laser, 2020, 47(5): 0500004.]). Photon technology, which uses photons as information carriers, has the characteristics of large bandwidth, low loss, and parallelism, and is widely used in radar, communication, and imaging fields (see [J. Capmany, D. Novak, "Microwavephotonics combines two worlds," Nature photonics , vol. 1, no. 6, pp. 319-330,2007.]), the combination of photon technology and traditional neural network is expected to give full play to the advantages of the two technologies, breaking through the high power consumption and long delay of traditional electronic neural network Technology development bottleneck with limited time and speed (see [Shen Y, Harris N C, Skirlo S, et al. "Deep learning with coherent nanophotonic circuits," Nature Photonics, vol. 11, no. 7, pp. 441-446, 2017 .]). First of all, the photonic neural network adopts an analog computing architecture, and the storage and calculation are performed at the same time, which can reduce the computing delay while increasing the computing speed; second, based on the essential characteristics of the optical transmission medium, the optical link has low loss characteristics, which can indirectly reduce the system power. Finally, compared with electronic devices, the effective working bandwidth of photonic devices has increased by several orders of magnitude, which is more suitable for high-speed real-time computing of neural networks. At present, there are a variety of photon convolution calculation methods based on wavelength division multiplexing technology released, but the realization basis of wavelength division multiplexing technology is multi-wavelength light source, and the realization method of multi-wavelength light source is often more complicated and costly.

Contents of the invention

The purpose of the present invention is to overcome the disadvantages of the prior art and provide a two-dimensional photon coherent convolution acceleration chip based on time interleaving and its application system. Based on the photon integration technology, the present invention utilizes a delay weighted phase-shifting unit including an adjustable coupler, a delay waveguide, a phase shifter and a first-order beam combiner to realize the convolution kernel matrix coefficient weighting and first-order time interleaving of the signal to be convoluted And coherent summation, using the second-level delay waveguide to realize the second-level time interleaving of the weighted modulated optical signal, through the second-level beam combiner and photodetector to realize the weighted summation operation, without wavelength division multiplexing, and the delay waveguide can be Multiplexing, suitable for multi-dimensional data convolution operations.

The present invention specifically adopts the following technical solutions to solve the above technical problems:

A two-dimensional photonic coherent convolution acceleration chip based on time interleaving, the chip is composed of an intensity modulator, M delay weighted phase shifting units, M-1 secondary delay waveguides, secondary beam combiner and photoelectric detection device integration; where:

The intensity modulator has 1 electrical input terminal, 1 optical input terminal and 1 optical output terminal, the optical input terminal is the optical input terminal of the whole chip, and is used to receive external optical signals, and the optical output terminal is connected to the first extension The optical input end of the time-weighted phase-shifting unit; the electrical input end is used to receive an external signal to be convoluted, and the signal to be convolved is modulated by the intensity modulator to intensity-modulate the optical signal input to the modulator to obtain an intensity-modulated optical signal;

The M delay-weighted phase-shifting units have the same structural design, and each delay-weighted phase-shifting unit is composed of N adjustable couplers, N phase shifters, and a first-level beam combiner; the adjustable The adjustable coupler includes at least one optical input terminal and two optical output terminals, wherein the first optical output terminals of the N adjustable couplers are connected in series with the input terminals sequentially, that is: the first optical output terminal of the first adjustable coupler One optical output end is connected with the optical input end of the second adjustable coupler, the first optical output end of the second adjustable coupler is connected with the optical input end of the third adjustable coupler, and so on; The optical input end of the first adjustable coupler is the optical input end of the delay weighted phase shift unit, and the first optical output end of the Nth adjustable coupler is the first optical output end of the delay weighted phase shift unit , the second optical output terminals of the N adjustable couplers are respectively connected to the optical input terminals of a phase shifter, and the optical output terminals of the N phase shifters are connected to the N optical input terminals of the first-level beam combiner, and the first-level The optical output end of the beam combiner is the second optical output end of the delay weighted phase shifting unit. M delay-weighted phase-shift units are connected in series through a two-stage delay waveguide, wherein the first optical output end of the previous delay-weighted phase-shift unit passes through the second-stage delay waveguide and the light of the latter delay-weighted phase-shift unit The input terminals are connected; the M second optical output terminals of the M delay weighted phase-shifting units are connected to the optical input terminals of the secondary beam combiner; the secondary beam combiner is composed of M optical input terminals and 1 optical output terminal Composition; by controlling the coupling coefficient of the adjustable coupler and the phase of the phase shifter to realize the weighting of the coefficient of the convolution kernel matrix, the delay weighted optical signal can be obtained at the optical output end of the secondary beam combiner;

The optical output end of the secondary beam combiner is connected to the optical input end of the photodetector, and the photodetector has an optical input end and an electrical output end, which are used to obtain the optical output end of the secondary beam combiner. The time-delayed weighted optical signal undergoes photoelectric conversion to obtain an electrical output signal, which is collected, processed and reconstructed to obtain the characteristic signal after the two-dimensional convolution operation of the signal to be convoluted.

Preferably, the adjustable coupler can be realized based on Mach-Zehnder interference structure and microring structure.

Further, the adjacent adjustable coupler in the delay-weighted phase-shifting unit has a length of The delay waveguide connection of , where c is the speed of light in vacuum, n _w is the effective refractive index of the waveguide delay line, is the duration of a single symbol of the signal to be convolved, and _SM is the symbol rate of the signal to be convolved.

Further, the length of the two-stage delay waveguide connecting adjacent delay-weighted phase-shifting units is , where P is the number of columns of the two-dimensional data matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix.

Further, the signal to be convoluted is a one-dimensional time series obtained after flattening the two-dimensional data to be convoluted, and the two-dimensional data to be convoluted is obtained by matrix transformation of the original two-dimensional data, and the specific transformation process is as follows:

The original two-dimensional data A _Q×O is slidingly divided into H sub-two-dimensional data B _Q×P in the column direction by stepping P-N+1, and each sub-two-dimensional data is a two-dimensional data to be convoluted, where Q is The number of rows of the original two-dimensional data, O is the number of columns of the original two-dimensional data, P is the number of columns of the two-dimensional data matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix.

Further, the convolution kernel matrix coefficient weighting is realized by controlling the coupling coefficient of the adjustable coupler and the phase of the phase shifter, specifically:

The coupling coefficient of the adjustable coupler and the phase of the phase shifter are respectively determined according to the size and sign of the convolution kernel matrix coefficient, and the coupling coefficient of the adjustable coupler is changed by the thermo-optic effect or the electro-optical effect, wherein according to the coefficient of the kernel matrix The size determines the coupling coefficient, and the phase is determined according to the positive and negative signs of the convolution kernel matrix coefficients. N adjustable couplers and N phase shifters in each delay weighted phase shifter correspond to a row of coefficients in the convolution kernel matrix, M The M×N adjustable couplers and the M×N phase shifters in the delay weighted phase shifting units correspond to two-dimensional convolution kernel matrix coefficients with a size of M×N.

Preferably, the chip is integrated based on III-V material integration process, or silicon-based integration process.

Further, the original two-dimensional data can be decomposed into three-dimensional or multi-dimensional original data.

Based on the same principle, the present invention also provides a convolution operation application system based on a time-interleaved two-dimensional photon coherent convolution acceleration chip, including:

The above-mentioned two-dimensional photon coherent convolution acceleration chip based on time interleaving, a coherent light source, a signal source to be convoluted, a convolution kernel control unit, and an acquisition and processing unit; wherein, the output terminal of the signal source to be convolved is connected to the two-dimensional photon The electrical input end of the coherent convolution acceleration chip is connected, the optical output end of the coherent light source is connected with the optical input end of the two-dimensional photon coherent convolution acceleration chip, and the convolution kernel control unit is used to control the convolution kernel according to the size of the convolution kernel matrix coefficient and The positive and negative signs respectively determine and control the coupling coefficient of the adjustable coupler and the phase of the phase shifter; the acquisition processing unit is connected to the electrical output terminal of the two-dimensional photon coherent convolution acceleration chip, and is used to reconstruct the electrical output signal to obtain The characteristic signal after the two-dimensional convolution operation is completed on the signal to be convolved.

Further, the signal to be convoluted outputted by the signal source to be convolved is a one-dimensional time series obtained after flattening the two-dimensional data to be convoluted, and the two-dimensional data to be convoluted is obtained by matrix transformation of the original two-dimensional data, The specific conversion process is:

The original two-dimensional data A _Q×O is slidingly divided into H sub-two-dimensional data B _Q×P in the column direction by stepping P-N+1, and each sub-two-dimensional data is a two-dimensional data to be convoluted, where Q is The number of rows of the original two-dimensional data, O is the number of columns of the original two-dimensional data, P is the number of columns of the two-dimensional data matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix.

The original two-dimensional data is obtained by decomposing three-dimensional or multi-dimensional original data.

Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

1) The present invention is based on a single wavelength carrying the signal to be convoluted, and uses coherent technology to realize the weighting of the convolution kernel matrix coefficients in the real number domain. Compared with wavelength division multiplexing technology, it does not require multi-wavelength optical signals, and the solution is simple and compact.

2) The present invention realizes two-stage time interleaving of single-wavelength weighted modulated optical signals through two-stage delay waveguides, and can realize two-dimensional convolution kernel convolution acceleration operation of two-dimensional data in a single signal period, and solve data redundancy in traditional methods problem, the solution is simple and efficient.

3) The present invention realizes the two-level time interleaving of the weighted modulation signal by connecting the delay-weighted phase-shifting unit in series with two-stage delay waveguides, which can realize the delay multiplexing of waveguides, and the relative parallel connection can reduce the chip size while reducing the delay waveguide band The incoming signal transmission loss, thereby improving the energy utilization efficiency of the chip.

Description of drawings

FIG. 1 is a schematic structural diagram of a two-dimensional photonic coherent convolution acceleration chip based on time interleaving in the present invention.

FIG. 2 is a schematic structural diagram of a specific embodiment of a two-dimensional photonic coherent convolution acceleration chip based on time interleaving according to the present invention.

FIG. 3 is a schematic structural diagram of a delay-weighted phase-shifting unit in a specific embodiment of a two-dimensional photonic coherent convolution acceleration chip based on time interleaving in the present invention.

FIG. 4 is a schematic diagram of a matrix transformation process from original two-dimensional data to two-dimensional data to be convoluted in a specific embodiment of a time-interleaving-based two-dimensional photonic coherent convolution acceleration chip of the present invention.

Fig. 5 is a schematic diagram of flattening processing of a two-dimensional data matrix to be convoluted in a specific embodiment of a two-dimensional photon coherent convolution acceleration chip based on time interleaving according to the present invention, wherein A in Fig. 5 is two-dimensional data to be convolved Matrix and convolution kernel matrix, B in Figure 5 is a schematic diagram of a one-dimensional flattening processing method for a two-dimensional data matrix to be convoluted, and C in Figure 5 is a two-dimensional feature data matrix recombined with one-dimensional feature data.

FIG. 6 is a graph showing the relationship between the time sequence and the wavelength of the intensity modulator outputting the intensity modulated optical signal in a specific embodiment of a two-dimensional photonic coherent convolution acceleration chip based on time interleaving according to the present invention.

Fig. 7 is a schematic diagram of the spectrum of each working node in a specific embodiment of a two-dimensional photonic coherent convolution acceleration chip based on time interleaving according to the present invention, wherein A in Fig. 7 is the first N delay-weighted phase-shifting units The time sequence of the weighted modulated optical signal output by the second optical output end of the adjustable coupler and the relationship diagram of each adjustable coupler, B in Figure 7 is the delay obtained by the first optical output end of the first delay weighted phase shifting unit The relationship between the time series and the wavelength of the intensity modulated optical signal, C in Figure 7 is the time series of the time series of the delay weighted modulated optical signal output by the second optical output end of the first delay weighted phase shift unit and the relationship diagram between each phase shifter , D in FIG. 7 is a diagram showing the relationship between the time sequence of the delay-weighted modulated optical signal output from the second optical output end of the last delay-weighted phase-shifting unit and each phase shifter.

FIG. 8 is a diagram showing the relationship between the time sequence of the delay-weighted optical signal output by the secondary beam combiner and each phase shifter in a specific embodiment of a two-dimensional photonic coherent convolution acceleration chip based on time interleaving according to the present invention.

FIG. 9 is a schematic diagram of a matrix transformation process from two-dimensional feature data to original two-dimensional feature data in a specific embodiment of a two-dimensional photon convolution acceleration chip of the present invention.

Detailed ways

Aiming at the deficiencies of the prior art, the idea of the present invention is based on the photonic integration technology, and realizes the convolution of the signal to be convoluted by using a delay weighted phase shift unit including an adjustable coupler, a delay waveguide, a phase shifter and a first-level beam combiner. Product matrix coefficient weighting, first-level time interleaving and coherent summation, using second-level delay waveguides to realize second-level time interleaving of weighted modulated optical signals, and realizing weighted summation operations through second-level beam combiners and photodetectors, reducing Traditional methods require multi-wavelength signals.

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

The terminology used in this application is for the purpose of describing particular embodiments only, and is not intended to limit the application.

As used in this application and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present application, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

Fig. 1 is a schematic structural diagram of a two-dimensional photon coherent convolution acceleration chip based on time interleaving in the present invention. As shown in Fig. 1, the photon components integrated on the two-dimensional photon coherent convolution acceleration chip include: intensity modulator, M delays Time-weighted phase-shifting unit, M-1 secondary delay waveguides, secondary beam combiner and photodetector; the photonic components are connected through optical waveguides; the intensity modulator has 1 electrical input terminal, 1 An optical input end and an optical output end, the optical input end of the intensity modulator is the optical input end of the entire chip, which is used to receive an external optical signal, and the electrical input end is used to receive an external signal to be convolved, and the intensity modulator The optical output end of the device is connected to the optical input end of the first delay weighted phase shifting unit;

The M delay-weighted phase-shifting units are connected in series through M-1 secondary delay waveguides, and the M delay-weighted phase-shifting units have the same structural design, and each delay-weighted phase-shifting unit consists of N adjustable Coupler, N phase shifters and a first-level beam combiner; take the first delay-weighted phase-shift unit as an example, where the optical input end of the first adjustable coupler is delay-weighted phase-shift The optical input end of the unit, the first optical output end of the first adjustable coupler is connected to the optical input end of the next adjustable coupler, the second optical output end is connected to the optical input end of the phase shifter, and so on. N adjustable couplers are connected in series, the second optical output ends of the N adjustable couplers are respectively connected to the optical input end of a phase shifter, and the first optical output end of the last adjustable coupler is a delay The first optical output terminal of the time-weighted phase-shifting unit; the optical output terminals of the N phase shifters are connected to the N optical input terminals of the primary beam combiner, and the optical output terminals of the primary beam combiner are delay-weighted phase-shifting units the second optical output terminal. M delay-weighted phase-shifting units are connected in series through two-stage delay waveguides, wherein the first optical output end of the previous delay-weighted phase-shifting unit is connected to the optical input end of the subsequent delay-weighted phase-shifting unit; M The M second optical output terminals of the delay weighted phase-shifting unit are connected to the optical input terminals of the secondary beam combiner; the secondary beam combiner is composed of M optical input terminals and 1 optical output terminal; by controlling the The coupling coefficient of the coupler and the phase of the phase shifter can be adjusted to realize the weighting of the coefficient of the convolution kernel matrix, and the delay weighted optical signal can be obtained at the optical output end of the secondary beam combiner.

The adjustable coupler can be realized based on the Mach-Zehnder interference structure and the microring structure, and at least includes one optical input port and two optical output ports to realize the corresponding functions. This embodiment is preferably a 1×2 adjustable coupler ; The chip can be integrated based on III-V material integration process, or silicon-based integration process.

A specific embodiment of a convolution operation application system based on a time-interleaved two-dimensional photonic coherent convolution acceleration chip is shown in Figure 2, which includes: the above-mentioned photonic neural network convolution acceleration chip, a coherent light source, a signal source to be convolved, Convolution kernel control unit and acquisition processing unit.

First, the coherent light source generates an optical signal and sends it to the intensity modulator through the optical input terminal of the photonic neural network convolution acceleration chip, and the signal to be convoluted is sent to the intensity modulator through the electrical input terminal of the photonic neural network convolution acceleration chip. The product signal is loaded onto the optical signal through an intensity modulator to obtain an intensity modulated optical signal, wherein the signal to be convoluted is a one-dimensional time series obtained after matrix flattening of two-dimensional data to be convoluted; the intensity modulated optical signal is sent to Into the optical input end of the first delay-weighted phase-shift unit, in the first delay-weighted phase-shift unit, the convolution kernel matrix control unit controls N 1×2 adjustable By adjusting the coupling coefficient of the coupler, the intensity-modulated optical signal is divided into N intensity-modulated optical signals, and the convolution kernel coefficient weighting of the N intensity-modulated optical signals is realized respectively, at the first optical output end of the first delay weighted phase-shifting unit The first time-delayed intensity modulated optical signal is obtained, and the corresponding N first weighted modulated optical signals are obtained at the second optical output ends of the N 1×2 tunable couplers. Then, the phase quantities of the N phase shifters are controlled by the convolution kernel matrix control unit according to the positive and negative signs of the convolution kernel matrix coefficients, and the corresponding N first weighted phase shift modulated optical signals are obtained at the output terminals of the N phase shifters , N first weighted phase-shift modulated optical signals are coupled through a first-stage beam combiner, and the first time-delayed weighted modulated optical signal is obtained at the second optical output end of the first time-delayed weighted phase-shifted unit; the first time-delayed intensity modulated optical signal After the signal is delayed by the first two-stage delay waveguide, it is sent to the optical input end of the second delay-weighted phase-shift unit to perform operations similar to the first delay-weighted phase-shift unit; and so on, at the Mth The second optical output terminal of the delay-weighted phase-shifting unit obtains the Mth delay-weighted modulated optical signal; the M delay-weighted modulated optical signals are coupled through a secondary beam combiner to obtain a delay-weighted optical signal; the delay-weighted optical signal passes through the photoelectric After the detector completes the photoelectric detection, the electrical output signal can be obtained. After the signal is collected, processed and reconstructed, the characteristic signal after the two-dimensional convolution operation of the signal to be convolved can be obtained.

In order to facilitate the public's understanding, the technical solution of the present invention will be further described in detail through a specific embodiment below:

First, the coherent light source outputs an optical signal with a wavelength of λ, and the signal strength of the optical signal is A. The optical signal is sent to the optical input end of the intensity modulator of the photonic chip through the fiber-waveguide coupling technology, and the output signal to be convoluted is The signal is modulated by the intensity modulator to the optical signal, and the signal to be convolved is respectively loaded on the optical signal. The signal sequence to be convolved can be expressed as , where i represents the discretized time sequence number, R=QP is the length of the signal to be convoluted, the signal to be convolved is a one-dimensional signal obtained after the two-dimensional signal to be convolved is processed by matrix flattening, and the two-dimensional signal to be convolved The original two-dimensional data is obtained through matrix transformation, and the transformation process is shown in Figure 4. The original two-dimensional data A _Q×O is divided into H sub-two-dimensional data B _Q×P in the column direction by sliding in steps of P-N+1, Each sub-two-dimensional data is a two-dimensional signal to be convoluted, where Q is the number of rows of the original two-dimensional data, O is the number of columns of the original two-dimensional data, P is the number of columns of the two-dimensional signal matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix. The two-dimensional signal matrix to be convolved is shown as A in FIG. 5 , which is a matrix of Q rows and P columns. The specific operation of matrix flattening is to convert a two-dimensional or multi-dimensional matrix into a one-dimensional matrix, and the process is shown in B in Figure 5. Each intensity-modulated carrier corresponds to a signal to be convoluted to obtain an intensity-modulated optical signal. The intensity-modulated optical signal S _Mod can be expressed as a matrix:

(1)

The corresponding time series and wavelength relationship diagram of the intensity modulated optical signal is shown in FIG. 6 . The intensity-modulated optical signal output by the intensity modulator is sequentially sent to M delay-weighted phase-shifting units connected in series through two-stage delay waveguides. The structural diagram of the delay-weighted phase-shift unit is shown in Figure 3. Each delay-weighted phase-shift unit consists of N 1×2 tunable couplers (TOC), N phase shifters (PS) and a first-stage The beam combiner is composed of adjustable couplers with a length of The delay waveguide (DW) connection, where n _w is the effective refractive index of the waveguide delay line, is the single symbol duration of the signal to be convolved, _SM is the symbol rate of the signal to be convolved, and c is the speed of light in vacuum. The coupling coefficient of the adjustable coupler is determined according to the size of the convolution kernel matrix coefficient, and the coupling coefficient of the adjustable coupler is changed by the thermo-optical effect or the electro-optical effect. The size of a row of elements in the product matrix. The convolution kernel matrix control unit first controls the coupling coefficient of the first adjustable coupler, so that the intensity modulated optical signal output by the intensity modulator is coupled to the corresponding phase shifter according to a specific coupling coefficient, and the convolution kernel matrix coefficient is realized. Size weighting. The intensity modulated optical signal enters the delay waveguide after passing through the first adjustable coupler Time delay, the delayed intensity modulated optical signal is weighted by the coefficient size through the second adjustable coupler, and so on, M delay weighted phase shifting units complete MN signal weighting in total. MN weighted modulated optical signals are obtained at the second optical output ports of MN adjustable couplers, and the convolution kernel matrix M _con can be expressed as:

(2)

w represents the size of the convolution kernel matrix elements; then the first weighted modulated optical signal S _{Modcon_1} output by the second optical output terminals of the N adjustable couplers of the first delay weighted phase shift unit can be expressed as:

(3)

The time sequence of the first weighted modulated optical signal output by the second optical output terminals of the N adjustable couplers of the first delay weighted phase shifting unit and the relationship between each adjustable coupler is shown in A in Figure 7. The relationship between the time sequence and the wavelength of the first delayed intensity modulated optical signal obtained at the first optical output terminal of the first delay weighted phase shifting unit is shown in B in FIG. 7 . The convolution kernel matrix control unit controls the phase shift amount of MN phase shifters in the M delay weighted phase shifting units, so that the weighted modulated optical signal input to the phase shifter undergoes a phase shift, and the positive and negative signs of the convolution kernel matrix coefficients are realized. Weighted, the phase shift M _pm can be expressed as:

(4)

Indicates the amount of phase shift, its value is 0 or ; The first weighted phase-shift modulated optical signal output by the N phase shifters of the first time-delay weighted phase-shift unit is coupled through a first-stage beam combiner, and the second light output terminal of the first time-delay weighted phase-shift unit obtains the first A delay weighted modulated optical signal, the first delay weighted modulated optical signal S _{Mpm_1} can be expressed as:

(5)

The time sequence of the first delay-weighted modulated optical signal output from the second optical output end of the first delay-weighted phase-shifting unit and each phase shifter is shown in C in FIG. 7 . The length of the first delayed intensity modulated optical signal output by the first optical output terminal of the first delay weighted phase shifting unit is sent to The second-level delay waveguide realizes the second-level delay, and the optical signal after the second-level delay is sent to the second delay-weighted phase-shift unit for the same operation as in the first delay-weighted phase-shift unit. The same operation is also performed in the delay-weighted phase-shifting unit and the second-stage delay waveguide until the last delay-weighted phase-shifting unit. The m-th delay-weighted modulated optical signal output by the second optical output end of each delay-weighted phase-shifting unit is expressed as:

( m=1,2,..,M) (6)

Wherein, the time sequence of the Mth delay-weighted modulated optical signal output from the second optical output terminal of the last delay-weighted phase-shifting unit and the relationship diagram of each phase shifter is shown in D in FIG. 7 .

The delay-weighted modulated optical signals output by the second optical output terminals of the M delay-weighted phase-shifting units are sent to the M optical input terminals of the secondary beam combiner to be coupled into delay-weighted optical signals, and the time of the delay-weighted optical signals is The relationship between the sequence and each phase shifter is shown in Figure 8. The delay-weighted optical signal is input to the photodetector through the waveguide to realize photoelectric conversion, and the electrical output signal of the two-dimensional photon convolution acceleration chip is obtained. The signal in the effective time sequence of the electrical output signal can be expressed as:

Among them, S _ca (r) is the result of the rth convolution operation, w _mn is the convolution kernel matrix coefficient, is the amount of phase shift. After the acquisition and processing unit collects the signal, the two-dimensional reconstruction of the signal can be realized in the digital domain in the opposite way to the matrix flattening process for the effective timing signal. The two-dimensional reconstruction data is shown in C in Figure 5, where the last The N-1 columns are redundant data. After removing the redundant data, the two-dimensional characteristic signal after the convolution operation of the two-dimensional signal to be convoluted can be obtained. The above process is a description of a specific embodiment performed under the condition that the original data is not filled with zeros. When zero-padded to the original data, the zero-padded data can be used as the original two-dimensional data to perform the same operation as above.

Finally, the H two-dimensional feature signals are combined into one feature signal corresponding to the original two-dimensional data through the method shown in FIG. 9 , and then the convolution operation of the original two-dimensional data is completed.

Finally, it should be noted that what is listed above are only specific embodiments of the present invention. The present invention is not limited to the above embodiments, and many modifications are possible. All deformations that can be directly derived or associated by those skilled in the art from the content disclosed in the present invention should be considered as the protection scope of the present invention.

Claims (10)

1. A two-dimensional photonic coherent convolution acceleration chip based on time interleaving, characterized in that the chip consists of an intensity modulator, M delay weighted phase-shifting units, M-1 secondary delay waveguides, secondary Integrated integration of beam combiner and photodetector; where: The intensity modulator has 1 electrical input terminal, 1 optical input terminal and 1 optical output terminal, the optical input terminal is the optical input terminal of the whole chip, and is used to receive external optical signals, and the optical output terminal is connected to the first extension The optical input end of the time-weighted phase-shifting unit; the electrical input end is used to receive an external signal to be convoluted, and the signal to be convolved is modulated by the intensity modulator to intensity-modulate the optical signal input to the modulator to obtain an intensity-modulated optical signal; The delay-weighted phase-shifting unit is composed of N adjustable couplers, N phase shifters and a first-level beam combiner; the adjustable coupler includes at least one optical input terminal and two optical output terminals , wherein the first optical output ends and input ends of the N adjustable couplers are sequentially connected in series, the optical input end of the first adjustable coupler is the optical input end of the delay-weighted phase-shifting unit, and the Nth adjustable coupler can be The first optical output end of the adjustable coupler is the first optical output end of the delay weighted phase shift unit, and the second optical output ends of the N adjustable couplers are respectively connected to the optical input end of a phase shifter, N The optical output terminal of the phase shifter is connected to the N optical input terminals of the primary beam combiner, and the optical output terminal of the primary beam combiner is the second optical output terminal of the delay weighted phase shift unit; M delay weighted phase shift units The units are connected in series through two-stage delay waveguides; the two-stage beam combiner is composed of M optical input ports and one optical output port; M second optical output ports of the M delay-weighted phase-shifting units are connected to the two-stage The optical input end of the beam combiner; realize the convolution kernel matrix coefficient weighting by controlling the coupling coefficient of the adjustable coupler and the phase of the phase shifter; The optical output end of the secondary beam combiner is connected to the optical input end of the photodetector, and the photodetector has an optical input end and an electrical output end for the optical output of the secondary beam combiner The terminal obtains the time-delayed weighted optical signal for photoelectric conversion to obtain the electrical output signal, and the electrical output signal is collected, processed and reconstructed to obtain the characteristic signal after the two-dimensional convolution operation of the signal to be convoluted. 2. A two-dimensional photonic coherent convolution acceleration chip based on time interleaving according to claim 1, wherein the structure of the adjustable coupler is a Mach-Zehnder interference structure or a microring structure. 3. A kind of two-dimensional photonic coherent convolution acceleration chip based on time interleaving as claimed in claim 1, wherein the adjacent adjustable coupler in the time-delay weighted phase-shifting unit has a length of The delay waveguide connection of , where c is the speed of light in vacuum, n _w is the effective refractive index of the waveguide delay line, is the duration of a single symbol of the signal to be convolved, and _SM is the symbol rate of the signal to be convolved. 4. A kind of two-dimensional photonic coherent convolution acceleration chip based on time interleaving as claimed in claim 1, wherein the length of the secondary delay waveguide connecting the adjacent delay weighted phase shifting units is , where P is the number of columns of the two-dimensional data matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix. 5. A two-dimensional photonic coherent convolution acceleration chip based on time interleaving as claimed in claim 1, wherein the signal to be convoluted is the one-dimensional time obtained after the two-dimensional data to be convolved is flattened. Sequence, the two-dimensional data to be convolved is the original two-dimensional data obtained through matrix transformation, the specific transformation process is: The original two-dimensional data A _Q×O is slidingly divided into H sub-two-dimensional data B _Q×P in the column direction by stepping P-N+1, and each sub-two-dimensional data is a two-dimensional data to be convoluted, where Q is The number of rows of the original two-dimensional data, O is the number of columns of the original two-dimensional data, P is the number of columns of the two-dimensional data matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix. 6. A two-dimensional photonic coherent convolution acceleration chip based on time interleaving as claimed in claim 5, wherein the original two-dimensional data is decomposed into three-dimensional or multi-dimensional original data. 7. A kind of two-dimensional photonic coherent convolution acceleration chip based on time interleaving as claimed in claim 1, wherein the convolution is realized by controlling the coupling coefficient of the adjustable coupler and the phase of the phase shifter Kernel matrix coefficient weighting, specifically: The coupling coefficient of the adjustable coupler and the phase of the phase shifter are respectively determined according to the size and sign of the coefficient of the convolution kernel matrix, and the coupling coefficient of the adjustable coupler is changed by the thermo-optic effect or the electro-optical effect, and each delay weighted shift The N adjustable couplers and N phase shifters in the phase unit correspond to a row of coefficients in the convolution kernel matrix, and the M×N adjustable couplers and M×N phase shifters in the M delay weighted phase shift units The device corresponds to a two-dimensional convolution kernel matrix with a size of M×N. 8. A two-dimensional photonic coherent convolution acceleration chip based on time interleaving as claimed in claim 1, characterized in that the chip is integrated based on the integration process of III-V materials or silicon-based integration process. 9. A convolution operation application system based on a time-interleaved two-dimensional photon coherent convolution acceleration chip, characterized in that it includes: The two-dimensional photonic coherent convolution acceleration chip based on time interleaving, the coherent light source, the signal source to be convoluted, the convolution kernel control unit, and the acquisition and processing unit according to any one of claims 1-8; wherein, the signal source to be convoluted The output end of the coherent light source is connected to the electrical input end of the two-dimensional photon coherent convolution acceleration chip, the optical output end of the coherent light source is connected to the optical input end of the two-dimensional photon coherent convolution acceleration chip, and the convolution kernel control unit is used for Determine and control the coupling coefficient of the adjustable coupler and the phase of the phase shifter according to the size and the positive and negative signs of the convolution kernel matrix coefficients; The method is to reconstruct the electrical output signal to obtain the characteristic signal after the two-dimensional convolution operation of the signal to be convolved is completed. 10. The convolution operation application system according to claim 9, wherein the signal to be convoluted outputted by the signal source to be convoluted is a one-dimensional time series obtained after two-dimensional data to be convolved is flattened, The two-dimensional data to be convolved is the original two-dimensional data obtained through matrix transformation, and the specific transformation process is as follows: The original two-dimensional data A _Q×O is slidingly divided into H sub-two-dimensional data B _Q×P in the column direction by stepping P-N+1, and each sub-two-dimensional data is a two-dimensional data to be convoluted, where Q is The number of rows of the original two-dimensional data, O is the number of columns of the original two-dimensional data, P is the number of columns of the two-dimensional data matrix to be convoluted, and N is the number of columns of the two-dimensional convolution kernel matrix.

Images