CSCI 5204/EE5364

Advanced Computer Architecture

Introduction

Pen-Chung Yew
Department Computer Science and Engineering
University of Minnesota

With slides from: Profs. Zhai, Mowry, Falsafi, Hill, Hoe, Lipasti, Shen,
Smith, Sohi, Vijaykumar, Patterson, Culler
Teaching Staffs and Course Resources
Instructor: Prof. Pen-Chung Yew
• Office: 6-225C Keller Hall
• E-mail: yew@umn.edu
• Office phone: 612-625-7387
• Office hour: Tuesday 14:30 pm -15:30 pm
TA: Kartik Ramkrishnan
• Office: 6-244 Keller Hall
• Email: ramkr004@umn.edu
• Office hour:
Web Page: https://canvas.umn.edu/courses/460175

2
Defining Computer Architecture
“Traditional” view of computer architecture
(e.g. CS4203/EE4363)
• Instruction Set Architecture (ISA) design, i.e. regarding
system design decisions such as:
• No. of registers, memory addressing, addressing modes,
instruction operands, available operations, control flow
instructions, instruction encoding, support of high-level
language structures …
“Broader” view of computer architecture
(e.g. CS5204/EE5364)
• Specific system requirements of a target machine
• Design to maximize performance within constraints of cost,
technologies/power, availability/reliability, and security
• Includes ISA, microarchitecture/organization, and
hardware design/implementation/optimization.
Traditional View of Computer Architecture

“The term architecture is used here to describe the

attributes of a system as seen by the programmer,
i.e., the conceptual structure and functional behavior
as distinct from the organization of the dataflow and
controls, the logic design, and the physical
implementation.”

Gene Amdahl, IBM Journal of R&D, April 1964

4
Traditional View of Computer Architecture
Instruction Set Architecture (ISA) is a “contract”
between SW programmers and HW designers

Programmers

Software (SW)

Instruction Set Architecture

Hardware (HW)

Silicon
 System Stack in a Computer System
Computer architecture: Cover
Users many levels of abstractions
Software Layers including ISA, organization and
Applications hardware implementation &
optimization for power &
Compilers Operating System performance
Firmware
Instruction Set Architecture
Processor I/O System
Datapath & Control
Digital Design
Circuit Design
Hardware Layers Layout & Fab

Silicon
What This Course Is About
• We assume you have some exposure to basic computer
architectural concepts and programming
• Instruction set architecture (ISA) – assembly languages
• Memory hierarchy – cache memory, virtual memory
• Familiar with some programming language(s)
• Know compilation process and basic OS regarding running
your programs.
• Do ask questions if any subject/concept not clear to you
during lectures !!
• This course is about what, why and how various
factors/requirements impact computer architectural designs,
and what tradeoffs need to be made in real implementation
to meet various design targets (i.e., performance, power….)
Factors/Requirements Impact Computer Design
• Trend in Technologies
• Trend in Power and Energy in Integrated Circuits
• Trend in Costs
• Trend in Applications – Multimedia/AI/Machine Learning
• Dependability
• Security – most recent concerns
=== > Different considerations of above factors in different
commercial markets lead to different classes of computers, which
may have the same ISA, but very different hardware designs and
implementation.
=== > It may even require ISA extensions!!
Impact of Those Factors - Classes of Computers

Personal Mobile Device (PMD)

• Smart phones, tablet computers, smart watches, etc.
• Emphasis on energy efficiency and real-time response

Desktop Computing
• Emphasis on price-performance
Servers
• Emphasis on availability, scalability, and throughput

Clusters / Warehouse-Scale Computers

• Used for “Software as a Service (SaaS)”
• Emphasis on availability and price-performance
• Sub-class: Supercomputers, emphases on floating-point
performance and fast internal networks
Embedded Computers
• Emphasis on price and real-time response 10
Classed of Computers and Their Characteristics

11
Embedded Processors Market Is Huge

Time Magazine
July 5, 2021

12
Main Topics
• Instruction set architecture (ISA)
• Execution pipeline design (out-of-order execution)
• Instruction-level parallelism (dynamic scheduling)

• Memory hierarchy design (optimization techniques)

• Multiprocessors and thread-level parallelism
• Architectural issues related to computer Security
• Side/Covert-channel attacks and defenses

• Domain/Application-specific computers
• Warehouse-scale computers
Textbooks
Required Readings:
• Computer Architecture: A Quantitative Approach, 6th
Edition, John Hennessey and David Patterson. Morgan
Kauffman Publishers, 2018. (Available at the University
Bookstore.)

Suggested Readings
• Various research papers: Available online from the course
webpage.

15
More Information and Tech Papers
Major Architecture Conferences:
• International Symposium on Computer Architecture (ISCA)
• International Symposium on Microarchitecture (Micro)
• High-Performance Computer Architecture (HPCA)
• Architectural Support for Programming Languages and Operating
Systems (ASPLOS)
Major Compiler Conferences:
• Programming Language Design and Implementation (PLDI)
• Code Generation and Optimization (CGO)

Technical Weekly Reports

https://www.hpcwire.com/

16
Tentative Course Organization
• 2 Mid-term Exams (50%)
• 75-minute in-class exam each;
• Schedule is on Canvas course web site;
• Each covers its respective portion of course materials;
• No final exam
• No make-up exams.

• 2 Homework Assignments (20%)

• Project (30%)
• Team projects - a team of 2-3 students.
• A scaled-down version of a real research project (8-week);
• Preferably aligned with your research interests, e.g., MS/PhD
work, or future job prospects.
• Novel ideas are encouraged, but not necessary
• Could be 1-person paper surveys
• Discuss with Professor Yew about your project proposal
• Project proposals due on Thursday 9/26/2024
 Key to High Performance and Low Power:
Exploiting Various Kinds of Parallelism
• Classes of parallelism in applications:
• Data-Level Parallelism (DLP)
• Task-Level Parallelism (TLP)
• Request-Level Parallelism (RLP) - serverless computing

• Classes of architectural parallelism:

• Instruction-Level Parallelism (ILP)
• Vector architectures/Graphic Processor Units (GPUs)
• Thread-Level Parallelism (multicores)
• Request-Level Parallelism (warehouse-scale
computers) 18
 Flynn’s Taxonomy
• Single instruction stream, single data stream (SISD)
• Single instruction stream, multiple data streams (SIMD)
• Vector architectures
• Multimedia extensions (Intel AVX-512)
• Graphics processor units (GPUs)

• Multiple instruction streams, multiple data streams (MIMD)

• Tightly-coupled MIMD - scientific computing (latency)
• Loosely-coupled MIMD - mutliprogramming (throughput)

• Multiple instruction streams, single data stream (MISD)

• No commercial implementation
Factors/Requirements Impact Computer Design
• Trend in Technologies
• Trend in Power and Energy in Integrated Circuits
• Trend in Costs
• Trend in Applications – Multimedia/AI/Machine Learning
• Dependability
• Security – most recent concerns
=== > Different considerations of above factors in different
commercial markets lead to different classes of computers, which
may have the same ISA, but very different hardware designs and
implementations
=== > It may even require ISA extensions!!
Trends in Technology
• Integrated circuit technology (~1985 - ~2005, Moore’s Law)
• Transistor density: ↑ ~35%/year
• Die size: ↑ ~10-20%/year
• Integration overall: ↑ ~40-55%/year – No longer holds!!

• DRAM capacity: 25-40%/year (slowing)

• Flash memory capacity: 50-60%/year

• Magnetic disk capacity: recently slowed to 5%/year

• Network technologies (covered in other courses)

Processor Performance Over 40 Years
Move to multi-processor

RISC

Gordon Moore’s prediction in 1965 (“Moore’s Law”):

Number of transistors doubled every 18 months to 2 years

22
Growth in Clock Rate over 40 Years
Transistors and Wires – Feature Sizes
• Minimum size of transistor or wire in x and y dimension
• 10 microns (10 x 10-6 meters) in 1971 to .011 microns (i.e. 11 nano-
meters, 11 x 10-9 meters) in 2017, and 5nm in 2021 (TSMC), going
forward to 2nm
https://www.anandtech.com/show/16639/tsmc-update-2nm-in-development-
3nm-4nm-on-track-for-2022
• As a comparison: COVID SARS-CoV-2 virus has a size of about 50 –
140 nm. https://www.news-medical.net/health/The-Size-of-SARS-CoV-2-
Compared-to-Other-Things.aspx. N95 masks should remove 95% of
particles of size less than 300nm
• A water molecule is 0.27nm in diameter (~7.5X smaller than 2nm)
https://www.nanooze.org/sizing-up-a-molecule/
• Transistor performance scales linearly
• But, wire delay does not improve with feature size due to
resistance and capacitance!
• Integration density scales quadratically
Domain-Specific Processors - Energy Efficiency

Figure 1.13 Comparison of the energy and die area of arithmetic operations and
energy cost of accesses to SRAM and DRAM. Area is for TSMC 45 nm technology
1 joule = 1 ampere through 1 ohm for 1 second; 1 watt = 1 joule/second
A mosquito’s wing beat takes ~240 pJ
Intel IC Technology Road Map (~2020)

• Intel 7 – 10nm (equivalent to TSMC 7nm technology)

• Intel 4, 3 – Equivalent to TSMC 5nm
• Intel 20A, 18A – 2nm and 1.8nm (seem on track in 2024)
 Memory Technology Trend
• DRAM capacity: ↑ ~25-40%/year (slowing)
• 8 Gbits (2014), 16 Gbits (2019)

• HBM – High-Bandwidth Memory

• HBM 3 (~2021) – 24GB (819 GB/s)
• HBM 4 (~2026) – 32 GB (1434 GB/s)
Wikipedia
• Flash memory capacity: ↑ ~50-60%/year
• 8-10X cheaper/bit than DRAM
• Non-volatile memory (NVM) – “solid-state disks” (SSD)
• Samsung is to release 32TB/64TB/128TB SSD in 2024 By Thomas Coughlin
in Forbes, Aug 2024
• Magnetic disk capacity:
• 32TB by Seagate in 2024
• Density increases may no longer be
possible, more platters maybe needed
• 8-10X cheaper/bit than SSD 29
• 200-300X cheaper/bit than DRAM
Factors/Requirements Impact Computer Design
• Trend in Technologies
• Trend in Power and Energy in Integrated Circuits
• Trend in Costs
• Trend in Applications – Multimedia/AI/Machine Learning
• Dependability
• Security – most recent concerns
=== > Different considerations of above factors in different
commercial markets lead to different classes of computers, which
may have the same ISA, but very different hardware designs and
implementation.
=== > It may even require ISA extensions!!
ISA Extensions for Various New Applications
Instruction set extensions
•Alpha (MVI)
•ARM (NEON, SVE, SME, AMX)
•MIPS (MDMX, MIPS-3D, MXU, MIPS SIMD
SIMD (RISC)
•PA-RISC (MAX)
•Power ISA (VMX)
•SPARC (VIS)
•MMX (1996)
•3DNow! (1998)
•SSE (1999)
•SSE2 (2001)
•SSE3 (2004)
•SSSE3 (2006)
•SSE4 (2006)
•SSE5 (2007)
SIMD (x86)
•AVX (2008)
•F16C (2009)
•XOP (2009)
•FMA (FMA4: 2011, FMA3: 2012)
•AVX2 (2013)
•AVX-512 (2015)
•AMX (2022)
•AVX10 (2023)
From Wikipedia
ISA Extensions for Various New Applications
Instruction Set Extensions
•BMI (ABM: 2007, BMI1: 2012, BMI2: 2013, TBM:
Bit manipulation 2012)
•ADX (2014)
•Thumb
Compressed instructions •MIPS16e ASE
•RVC
•PadLock (2003)
•AES-NI (2008); ARMv8 also has AES instructions
•CLMUL (2010)
Security •RDRAND (2012)
and cryptography •SHA (2013)
•MPX (2015)
•SGX (2015)
•TDX (2021)
•TSX (2013)
Transactional memory
•ASF
•VT-x (2005)
Virtualization •AMD-V (2006)
•VT-d (AMD-Vi)
Suspended extensions' dates are struck through.
From Wikipedia