Sr.

Categor Model Propose Architec Param Disadvantag

N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

CNN- Simple and

Yann Limited to Not scalable
Based effective for Handwriting Image
1 LeNet-5 1998 LeCun et CNN ~60K small-scale for large
Models digit recognition Classification
al. images datasets
recognition

High
Alex High
accuracy, Requires ImageNet Image
2 AlexNet 2012 Krizhevsk CNN ~60M computational
introduced powerful GPUs Classification Classification
y et al. cost
ReLU

K.
Deep Large memory
Simonyan
architecture and ImageNet Image
3 VGG-16 2014 , A. CNN ~138M Slow training
with small computation Classification Classification
Zisserma
filters required
n

GoogLeNet Efficient with

Szegedy Complex Implementatio ImageNet Image
4 (Inception 2014 CNN ~6.8M fewer
et al. structure n complexity Classification Classification
v1) parameters

5 ResNet-50 2015 K. He et CNN ~25.6M Solves Requires a High ImageNet Image

al. vanishing large amount computation Classification Classification
gradient of data cost
problem
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

Reduced
Dense
vanishing-
DenseNet- G. Huang Memory- connectivity ImageNet Image
6 2017 CNN ~8M gradient,
121 et al. intensive can be Classification Classification
fewer
redundant
parameters

Efficient
Requires Less effective Image
François depthwise Image
7 Xception 2017 CNN ~22.9M substantial on smaller Classification,
Chollet separable Classification
computation datasets Segmentation
convolutions

Lightweight, Trade-off Mobile and

A. G. Lower
MobileNetV fast for between embedded Image
8 2017 Howard et CNN ~4.2M accuracy than
1 mobile speed and vision Classification
al. larger models
devices accuracy applications

Complex
Mingxing Balances Limited by
EfficientNet scaling ImageNet Image
9 2019 Tan, Quoc CNN ~5.3M performance scaling
-B0 requires Classification Classification
V. Le and efficiency assumptions
tuning

High
Barret Automated
computation High training Image Image
10 NASNet-A 2018 Zoph et CNN ~88.9M architecture
cost for cost Classification Classification
al. search
search

Object
Joseph Real-time Lower Struggles with
detection in Object
11 YOLOv1 2016 Redmon CNN ~62M object accuracy for overlapping
video Detection
et al. detection small objects objects
streams

12 Faster R- 2015 Shaoqing CNN ~60M High Slower than Requires a Object Object
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

accuracy in
large amount
CNN Ren et al. object YOLO Detection Detection
of data
detection

Combines Requires
Mask R- Kaiming Slower than Instance Object
13 2017 CNN ~44M detection and significant
CNN He et al. YOLO Segmentation Detection
segmentation computation

Effective for
Olaf Medical
biomedical Requires large Sensitive to Image
14 U-Net 2015 Ronneber CNN ~31M Image
image memory initialization Segmentation
ger et al. Segmentation
segmentation

Liang-
High High
DeepLabV3 Chieh Complex Semantic Image
15 2018 CNN ~43M segmentation computational
+ Chen et architecture Segmentation Segmentation
accuracy cost
al.

Transfor High
VIT (Vision
mer- Dosovitski Transfor performance Requires large Computationa Image Image
16 Transformer 2020 ~86M
Based y et al. mer with less datasets lly expensive Classification Classification
)
Models inductive bias

Hierarchical
Object
Swin Ze Liu et Transfor structure for Complex Requires
17 2021 ~29M Detection, Vision Tasks
Transformer al. mer dense architecture careful tuning
Segmentation
prediction

18 DeiT 2021 Hugo Transfor ~22M Data-efficient Requires large Lacks Image Image
Touvron mer with smaller datasets for inductive bias Classification Classification
et al. models training of CNNs
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

Efficient High
Transfor token-to- computation Requires fine- Image Image
19 T2T-ViT 2021 Li et al. ~21M
mer token for large tuning Classification Classification
interaction images

GNN- GCN (Graph Captures Limited to Not suitable

Kipf and Node Graph-Based
20 Based Convolution 2016 GNN Varies graph fixed-size for dynamic
Welling classification Tasks
Models al Network) structure well graphs graphs

Efficient Approximation
Hamilton Scaling Link Graph-Based
21 GraphSAGE 2017 GNN Varies sampling of introduces
et al. challenges prediction Tasks
neighbors errors

Uses
GAT (Graph attention High
Veličković Overfitting on Node Graph-Based
22 Attention 2018 GNN Varies mechanisms computational
et al. small datasets classification Tasks
Network) for node cost
features

High
GIN (Graph expressivene Molecular
Overfitting on Requires large Graph-Based
23 Isomorphis 2019 Xu et al. GNN Varies ss, capable of graph
small datasets datasets Tasks
m Network) distinguishing classification
graphs

R-GCN Handles High Knowledge

Schlichtkr Limited Graph-Based
24 (Relational 2017 GNN Varies relational computational graph
ull et al. scalability Tasks
GCN) data well cost embeddings

25 DGI (Deep 2019 Veličković GNN Varies Unsupervised Sensitive to Overfitting on Node Graph-Based
Graph et al. learning of graph small graphs classification, Tasks
Infomax) graph structure link
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

representatio
changes prediction
ns

Hochreite Prone to
LSTM (Long Handles long- High
r and vanishing Sequence Sequence
26 Short-Term 1997 RNN Varies term computational
Schmidhu gradient prediction Tasks
Memory) dependencies cost
ber problem

Trade-off
Simpler than
between
GRU (Gated LSTM, Less
complexity Sequence Sequence
27 Recurrent 2014 Cho et al. RNN Varies effective for expressive
RNN- and prediction Tasks
Unit) shorter than LSTM
Based expressivenes
sequences
Models s

BiLSTM Considers Doubles

Graves et Limited Sequence Sequence
28 (Bidirection 2005 RNN Varies future and computational
al. scalability prediction Tasks
al LSTM) past context cost

Focuses on Machine
High
Attention- Bahdanau important Sensitive to translation, Sequence
29 2014 RNN Varies computational
based RNN et al. parts of the input length speech Tasks
cost
input recognition

Autoenc Hinton May not Data

Vanilla Dimensionalit Limited to
oder- and Autoenco capture compression, Dimensionalit
30 Autoencode 2006 Varies y reduction, reconstruction
Based Salakhutd der complex noise y Reduction
r denoising tasks
Models inov features reduction

31 Variational 2013 Kingma Autoenco Varies Generates Can be less Training can Generative Generative
Autoencode and der new data sharp in be unstable modeling, Tasks
r (VAE) Welling samples generated anomaly
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

images detection

Generates
Requires large Conditional
Conditional Sohn et Autoenco data Complex Generative
32 2015 Varies datasets for data
VAE (CVAE) al. der conditioned model training Tasks
training generation
on inputs

Enhanced
Disentangled
Higgins et Autoenco disentangled Requires Limited Generative
33 Beta-VAE 2018 Varies representatio
al. der representatio careful tuning scalability Tasks
ns
n

Noise
Denoising Limited to Not suited for
Vincent et Autoenco Robust to reduction, Dimensionalit
34 Autoencode 2008 Varies reconstruction complex
al. der noise feature y Reduction
r tasks structures
learning

Hybrid GAN Image

Ian Training Requires
Models (Generative Generates generation, Generative
35 2014 Goodfello GAN Varies instability, careful
Adversarial realistic data data Tasks
w et al. mode collapse balancing
Network) augmentation

Limited to
Unpaired Can struggle Image style
specific types
image-to- with high- transfer, Image
36 CycleGAN 2017 Zhu et al. GAN Varies of
image resolution domain Translation
transformatio
translation images adaptation
ns

37 Pix2Pix 2016 Isola et al. GAN Varies Paired image- Requires Limited to Image-to- Image
to-image paired specific image Translation
translation datasets domains translation
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

Multi-domain
Can struggle Requires Multi-domain
image-to- Image
38 StarGAN 2018 Choi et al. GAN Varies with complex extensive image
image Translation
domains training translation
translation

High-
High-quality Requires
Karras et Computationa resolution Generative
39 StyleGAN2 2019 GAN Varies image substantial
al. lly intensive image Tasks
generation resources
generation

Liang-
Combines High Limited to
Chieh Semantic Image
40 DeepLab 2016 Hybrid Varies deep learning computational specific image
Chen et segmentation Segmentation
with CRFs cost sizes
al.

Combines
Mask R- Kaiming detection Computationa Limited Instance Object
41 2017 Hybrid Varies
CNN He et al. with lly demanding scalability segmentation Detection
segmentation
Hybrid
Models Efficient Object
Can be
Mingxing object Requires detection, Object
42 EfficientDet 2020 Hybrid Varies complex to
Tan et al. detection and careful tuning instance Detection
implement
segmentation segmentation

Improves
TTA (Test
model Increased Limited to Classification, Classification,
Time Brock et
43 2016 Hybrid Varies performance computational inference detection Object
Augmentati al.
at inference cost scenarios enhancement Detection
on)
time

44 Siamese Siamese 1993 Bromley Siamese Varies Effective for Requires Limited Face Similarity
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

careful design verification,

similarity
Network et al. for distance scalability one-shot Learning
learning
metrics learning

Improves Face
Can be Requires a lot
Triplet Schroff et discriminativ recognition, Similarity
45 2015 Siamese Varies sensitive to of training
Network al. e power of metric Learning
margin choice data
embeddings learning
Network Object
s Enhanced Limited to
Bertinetto Computationa tracking in
46 DeepSiam 2017 Siamese Varies tracking tracking Tracking
et al. lly intensive video
performance scenarios
streams

Combines
Siamese Requires large Object Object
Siamese R- Zhang et Computationa
47 2016 Siamese Varies networks amounts of detection and Detection,
CNN al. lly demanding
with data verification Verification
detection

Meta- MAML
Requires Few-shot
Learnin (Model- Effective for
Meta- careful tuning Training can learning, Few-Shot
48 g Agnostic 2017 Finn et al. Varies few-shot
Learning of meta- be slow transfer Learning
Models Meta- learning
parameters learning
Learning)

Learns
Limited to
Snell et Meta- prototypes Requires large Few-shot Few-Shot
49 ProtoNet 2018 Varies few-shot
al. Learning for few-shot support sets classification Learning
scenarios
classification

50 Relation 2018 Sung et Meta- Varies Learns High Not suitable Few-shot Few-Shot
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

relational learning,
computational for large-scale
Network al. Learning representatio relation Learning
cost datasets
ns modeling

Requires Few-shot
Effective for
Matching Vinyals et Meta- Limited to careful classification, Few-Shot
51 2016 Varies few-shot
Networks al. Learning small datasets distance few-shot Learning
classification
metrics learning

Others GANs with Generative

Captures High
Latent Kingma et Training can tasks, Generative
52 2017 GAN Varies complex data computational
Variable al. be unstable complex data Tasks
distributions cost
Models modeling

Self- Limited to Difficult to Clustering,

Effective for
53 Organizing 1990 Kohonen SOM Varies unsupervised interpret pattern Clustering
clustering
Maps (SOM) learning results recognition

Memory
Excellent for Limited to Not suitable
Hopfield John models, Memory
54 1982 Hopfield Varies associative small-scale for large-scale
Networks Hopfield associative Models
memory problems data
memory

55 Bayesian 1980 Judea Bayesian Varies Handles Requires High Probabilistic Probabilistic
Networks Pearl uncertainty extensive complexity reasoning, Reasoning
well computation decision
support
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

Restricted Feature
Effective for
Boltzmann Hinton et Training can Requires large extraction, Dimensionalit
56 2006 RBM Varies dimensionalit
Machines al. be slow datasets dimensionalit y Reduction
y reduction
(RBM) y reduction

Sequence
Models Limited to
Markov Andrey Requires prediction, Sequence
57 1906 Markov Varies sequential discrete
Chains Markov discrete data state Modeling
data well states
modeling

Quantum Leverages Quantum

Lloyd et Computationa Still Quantum
58 Neural 2014 Quantum Varies quantum computing
al. lly intensive experimental Computing
Networks computing applications

Image
Geoffrey Preserves Requires
Capsule Limited recognition, Image
59 2017 Hinton et Capsule Varies spatial significant
Networks adoption object Classification
al. relationships computation
detection

Fast training
Extreme Not suitable
and simple Limited to Classification, Classification,
60 Learning 2006 Gao et al. ELM Varies for complex
implementati linear models regression Regression
Machines datasets
on

Learning Good for Requires

Classification,
Vector classification tuning of Limited
61 1988 Kohonen LVQ Varies pattern Classification
Quantizatio with noisy distance scalability
recognition
n data metrics

62 Neural 2014 Graves et Neural Varies Learns to High Still Algorithmic Algorithmic
Turing al. Turing perform computational experimental tasks, Tasks
Machines algorithmic complex
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

tasks cost reasoning

Effective for
Deep Feature
Salakhutd complex Requires large High training Feature
63 Boltzmann 2009 DBM Varies extraction,
inov et al. feature datasets complexity Extraction
Machines deep learning
extraction

Focuses on
Attention- Image
Bahdanau Attention relevant Computationa Requires large Image
64 based 2015 Varies classification,
et al. -CNN parts of input lly intensive datasets Classification
CNNs detection
images

Alleviates Image
Deep High
Deep vanishing Requires deep classification, Image
65 Residual 2016 He et al. Varies computational
Residual gradient architectures object Classification
Networks cost
problem detection

Neural Automated Model design,

Zoph et Computationa Requires fine-
66 Architectur 2018 NAS Varies architecture architecture Model Design
al. lly expensive tuning
e Search search optimization

Neural Models Time series

Ordinary Chen et continuous- Computationa Still analysis, Time Series
67 2018 NODE Varies
Differential al. time lly intensive experimental continuous Analysis
Equations processes modeling

Meta-
Deep Flexible High learning,
Limited to
68 HyperNetw 2016 Ha et al. HyperNet Varies architecture computational flexible Meta-Learning
specific tasks
orks generation cost model
architectures
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

Transfers
Neural Limited to Artistic image
Gatys et styles Computationa
69 Style 2015 NST Varies artistic transformatio Style Transfer
al. between lly intensive
Transfer applications ns
images

Shared
Multi-Task representatio Can suffer Multi-task
Ruder et Multi- Requires task- Multi-Task
70 Learning 2018 Varies ns for from negative learning, joint
al. Task specific tuning Learning
Models multiple transfer learning
tasks

Combines Increased
Ensemble multiple complexity Limited Classification,
Ensembl Classification,
71 Learning 1996 Dietterich Varies models for and interpretabilit regression,
e Regression
Models improved computational y prediction
performance cost

Utilizes Domain
Transfer
Pan and knowledge Requires task Limited to adaptation, Transfer
72 Learning 2010 Transfer Varies
Yang from related similarity related tasks knowledge Learning
Models
tasks transfer

Recognizes Requires good High reliance Object

Zero-Shot
Zero- classes not semantic on classification Zero-Shot
73 Learning 2017 Xian et al. Varies
Shot seen during representation representation without Learning
Models
training s s labeled data

Few-shot
Few-Shot Learns from Requires
Limited to classification, Few-Shot
74 Learning 2018 Finn et al. Few-Shot Varies very few careful fine-
small datasets few-shot Learning
Models examples tuning
learning
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

Few-shot
Meta- Adapts High
Meta- Requires learning,
75 Learning 2017 Finn et al. Varies quickly to computational Meta-Learning
Learning meta-training transfer
Models new tasks cost
learning

Self- Utilizes
Self- High Unsupervised Self-
Supervised Doersch unlabeled Requires large
76 2018 Supervis Varies computational representatio Supervised
Learning et al. data datasets
ed cost n learning Learning
Models effectively

Neural Reduces Model

Can impact
Network Compres model size Requires deployment, Model
77 2016 Han et al. Varies model
Compressio sion for trade-offs edge Compression
performance
n Models deployment computing

Improved Training can

Robust Requires Security-
Madry et robustness to be Adversarial
78 Neural 2019 Robust Varies specialized sensitive
al. adversarial computational Robustness
Networks techniques applications
attacks ly intensive

Combines
Versatile
Hybrid different Complexity in Limited to
Zhang et applications Hybrid
79 Neural 2020 Hybrid Varies neural design and specific
al. across Learning
Networks network training applications
domains
types

80 Neural 2019 Molchano Pruning Varies Reduces Can affect Requires Model Model Pruning
Network v et al. model performance retraining efficiency,
Pruning complexity deployment
Models without
losing
Sr.
Categor Model Propose Architec Param Disadvantag
N Year Advantages Limitations Applications Tasks
y Name d By ture eters es
o

accuracy

Generative
Enhances Data
Adversarial
training data Requires large augmentation
Networks Antoniou Data quality Data
81 2018 GAN Varies through computational , synthetic
for Data et al. can vary Augmentation
augmentatio resources data
Augmentati
n generation
on

Neural Automates Automated

High
Architectur the search for Requires model
82 2019 Liu et al. NAS Varies computational Model Design
e Search optimal expert tuning design,
cost
Models architectures optimization

Optimizes
Hyperpara model Model
Requires Hyperparamet
meter Hyperpar performance Computationa optimization,
83 2020 Li et al. Varies extensive er
Optimizatio ameter through lly expensive hyperparame
resources Optimization
n Models parameter ter tuning
tuning