Photonic

Neuromorphic computing

Presenter: Rafatul Faria

PhD student, ECE, Purdue University

Major: Micro and Nanoelectronics (MN)
 Outline
What is neuromorphic computing and why is it important?
General Overview
of Neuromorphic Basics of neurons and synapses, learning
computing
Different technologies targeting neuromorphic computing

Different photonic devices for neuromorphic computing

Photonic Devices
for Neuromorphic Pros and cons
computing
Future directions
 Outline
What is neuromorphic computing and why is it important?
General Overview
of Neuromorphic Basics of neurons and synapses, learning
computing
Different technologies targeting neuromorphic computing

Different photonic devices for neuromorphic computing

Photonic Devices
for Neuromorphic Pros and cons
computing
Future directions
 What is neuromorphic computing?
Neuromorphic computing

Mimicking human brain function

for low energy, high speed
cognitive computing and learning

• human brain contains over 100 billion

neurons and 100 trillion to 150 trillion
smart phones, sensor networks, self-
synapses. driving automobiles, robots, public safety,
medical imaging, real-time video analysis,
• Power consumption: roughly 20 Watts!!! signal processing, olfactory detection, and
digital pathology and so on …

https://www.openpr.com/news/371546
Why is neuromorphic computing important?
one million programmable “neurons” and
IBM trueNorth (2014) 256 million “synapses”
http://www.ibtimes.com/ibm-creates-cognitive-chip-mimics-
(DARPA funded custom hardware) human-brain-1652858

GPU vendors
Nvidia, AMD Google:
Tensor Processing
unit (TPU)

Building an efficient
Apple neuromorphic chip

Facebook
Mircosoft

Intel
Loihi
(2017) https://singularityhub.com/2017/09/29/intels-
new-brain-like-chip-will-learn-on-the-fly/
 Why is neuromorphic computing important?
one million programmable “neurons” and
IBM trueNorth (2014) 256 million “synapses”
http://www.ibtimes.com/ibm-creates-cognitive-chip-mimics-
(DARPA funded custom hardware) human-brain-1652858

GPU vendors
Nvidia, AMD Google:
Tensor Processing
unit (TPU)

Building an efficient
Apple neuromorphic chip

Facebook
Mircosoft

Google DeepMind AI program AlphaGo (March 2016)

Intel
Loihi
https://www.engadget.com/2016/03/14/the-final-lee-sedol-vs- (2017) https://singularityhub.com/2017/09/29/intels-
alphago-match-is-about-to-start/ new-brain-like-chip-will-learn-on-the-fly/
Biological neuron Neural Network: neurons and synapses
Diep et al.,
APL, 2014

Mathematical operations:
ü Multiplication
ü summation
Modeling a simple Perceptron neuron
Biological neuron Neural Network: neurons and synapses
Diep et al.,
APL, 2014

Mathematical operations:
ü Multiplication
ü summation
Modeling a simple Perceptron neuron
Biological neuron Neural Network: neurons and synapses
Diep et al.,
APL, 2014

Mathematical operations:
ü Multiplication
ü summation
Modeling a simple Perceptron neuron

Artificial neural network (ANN)

 Neural Network: neurons and synapses
Stochastic spiking neuron

Membrane potential

Biological neurons
are stochastic in
nature

Burkitt, Anthony N. "A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input."
Biological cybernetics 95.1 (2006): 1-19.
Training a Neural Network

Most widely used

learning
mechanism:
Back propagation

Nature, vol. 323, 1986

 Beyond CMOS devices to mimic neuron
Moores law ending
 Beyond CMOS devices to mimic neuron
Moores law ending

CMOS based implementation of neural

network: Von neuman bottleneck

• Low bandwidth
between CPU and
memory
• Majority power loss
in data transfer
process
 Beyond CMOS devices to mimic neuron
Moores law ending
Neuromorphic
computing

CMOS based implementation of neural

network: Von neuman bottleneck

• Low bandwidth
between CPU and
memory
• Majority power loss
in data transfer
process

2682 references!!!
 Outline
What is neuromorphic computing and why is it important?
General Overview
of Neuromorphic Basics of neurons and synapses, learning
computing
Different technologies targeting neuromorphic computing

Different photonic devices for neuromorphic computing

Photonic Devices
for Neuromorphic Pros and cons
computing
Future directions
Why Photonic neuromorphic computing?

Lecture 8
ECE 695
Nanophotonics and
Metamaterials
 Why Photonic neuromorphic computing?

Lecture 8
ECE 695
Nanophotonics and
Metamaterials

Required properties of devices for neumorphic computing:

• High connectivity for parallel operation
• Low power Photonic devices can potentially meet all these criteria
• Faster computing
• Collocating memory and processing
• Lower footprint area, scalable
 Photonic Spiking neuron
Schematic and operation of a LIF neuron
 Photonic Spiking neuron
Schematic and operation of a LIF neuron Spiking neuron properties
 Photonic Spiking neuron
Schematic and operation of a LIF neuron Spiking neuron properties

Potential photonic elements for fabricating LIF neuron

 Spiking neuron (continued)
First bench-top model for photonic neuron (LIF neuron)

nonlinear optical
Carrier modulation
loop mirror (NOLM)

G: Variable attenuator
T: Tunable delay line
l1 : low power pulse train
SOA: Semiconductor Optical Amplifier
 Spiking neuron (continued)
First bench-top model for photonic neuron (LIF neuron)

nonlinear optical
Carrier modulation
loop mirror (NOLM)

G: Variable attenuator Drawbacks:

T: Tunable delay line • Performs integration and thresholding (2 required properties
l1 : low power pulse train out of 5). Lacks reset condition, ability to generate pulses and
truly asynchronous behavior.
SOA: Semiconductor Optical Amplifier
• Fiber based neuron, larger footprint area, not scalable
 Spiking neuron (continued)
Simple application using previous bench-top fiber based neuron model

Barn Owl Auditory Localization

 Spiking neuron (continued)
Simple application using previous bench-top fiber based neuron model

Barn Owl Auditory Localization

Input signals far apart: NO output spike

Input signals close: output spikes

 Excitable Laser Neuron
Generalized model

G (t ) : Gain
Q (t ) : Absorption
Rate equations: I (t ) : Laser intensity
A : Bias current of the gain
B : Level of absorption
a : differential absorption relative to differential gain
g G : Relaxation rate of gain
g Q : Relaxation rate of absorber
g I : inverse photon lifetime
e f (G ) : small contribution to intensity due to noise
 Excitable Laser Neuron
Generalized model

Threshold condition:
G (t ) > Q (t ) + 1

G (t ) : Gain
Q (t ) : Absorption
Rate equations: I (t ) : Laser intensity
A : Bias current of the gain
B : Level of absorption Smaller footprint area,
a : differential absorption relative to differential gain
g G : Relaxation rate of gain scalable J
g Q : Relaxation rate of absorber
g I : inverse photon lifetime
e f (G ) : small contribution to intensity due to noise
 VCSEL Neuron
VCSEL: Vertical Cavity Surface Emitting Laser
 VCSEL Neuron
VCSEL: Vertical Cavity Surface Emitting Laser

• Scalable
• Low power
 Silicon photonic weight bank
MRR: Microring resonator
BPD: Balanced photo diode
LD: Laser diode
MZM: Mach-Zehnder modulator (neuron)
WDM: Wavelength-division-multiplexer
AWG: Arrayed waveguide grating

Neuron 1 Microring resonator as weight bank Experimental set up

Neuron 4
Neuron 4

Tait, Alexander N., et al. "Neuromorphic photonic networks using silicon photonic weight banks." Scientific Reports 7.1 (2017): 7430.
 Silicon photonic weight bank
MRR: Microring resonator
• Weight logic implemented by tunable BPD: Balanced photo diode
microring resonator (MRR) LD: Laser diode
MZM: Mach-Zehnder modulator (neuron)
• Hybrid approach: optical+electrical WDM: Wavelength-division-multiplexer
AWG: Arrayed waveguide grating

Neuron 1 Microring resonator as weight bank Experimental set up

Neuron 4
Neuron 4

Tait, Alexander N., et al. "Neuromorphic photonic networks using silicon photonic weight banks." Scientific Reports 7.1 (2017): 7430.
 Fully optical neural network
• Fully optical neural network (ONN)

• ONN composed of Optical

Interference unit (OIU) and Optical
nonlinearity unit (ONU).

• OIU implements any real-valued

matrix multiplication by using
optical beam-splitters, phase
shifters and attenuators.

• ONU can be implemented using

common optical non-linearity such
as saturable absorption e.g.
Grphene saturable absorber.

• This scheme is experimentally

demonstrated within a subset of a
programable nanophotonic
processor (PNP)- a silicon photonic
integrated circuit fabricated in the
OPSIS foundry.

Shen, Yichen, et al. "Deep learning with coherent nanophotonic

Fully optical circuits." Nature Photonics (2017)
 Fully optical neural network (continued)
Two layer neural network for vowel recognition

Correlation matrices
2000 node coherent Ising machine with all-to-all connections
All to all connection by FPGA module

Max-Cut problem

https://en.wikipedia.org/wiki/Maximum_cut

Inagaki, Takahiro, et al. "A coherent Ising machine for 2000-node optimization problems."
Science 354.6312 (2016): 603-606.
 Summary
• A very blooming field which is already shaping our daily life.

• Many different areas are trying to come up with the best Ultimate goal: low power,
implementation using many different physics. high speed brain like
• Photonic application is very promising for low power high speed computing
computing and transfer of data and may eventually be the winner.

Neuromorphic
computing

electronics Photonics/plasmonics
spintronics
Thank you
J