CS391R: Robot Learning

Perception and Decision Making: Architectures, Algorithms, and Applications

Prof. Yuke Zhu

Fall 2021

CS391R: Robot Learning (Fall 2021) 1

CS391R: Robot Learning (Fall 2021)
CS391R: Robot Learning (Fall 2021)
Traditional form of automation

CS391R: Robot Learning (Fall 2021)

Credit: WIRED
General-purpose robot autonomy

CS391R: Robot Learning (Fall 2021) 5

 Today’s Agenda

● What is Robot Learning?

● Why studying Robot Learning now?

● Course content overview

● Logistics

● Student introduction

CS391R: Robot Learning (Fall 2021) 6

 Special-Purpose Robot Automation

custom-built human expert special-purpose

robots programming behaviors

General-Purpose Robot Autonomy

?
general-purpose general-purpose
robots behaviors

CS391R: Robot Learning (Fall 2021) 7

 Special-Purpose Robot Automation

custom-built human expert special-purpose

robots programming behaviors

General-Purpose Robot Autonomy

Robot Learning

general-purpose general-purpose
robots behaviors

CS391R: Robot Learning (Fall 2021) 8

 General-Purpose Robot Autonomy: Imaginations

Unimate - The First Industrial Robot

British TV (1968)

CS391R: Robot Learning (Fall 2021) 9

 General-Purpose Robot Autonomy: Challenges

DARPA Robotics Challenge

(2015)

“The Moravec's paradox”

CS391R: Robot Learning (Fall 2021) 10

 General-Purpose Robot Autonomy: Progress
We will learn the algorithms and techniques behind the latest progress.

Grasping (DexNet 4.0; 2019) Locomotion (ANYmal; 2020) Manipulation (OpenAI; 2019)

CS391R: Robot Learning (Fall 2021) 11

 What is Robot Learning?

Definition #1
The study of machine learning algorithms and principles
with their applications to robotics problems

Definition #2
The study of methods and principles that make robots
learn from data

Definition #3
The research field at the intersection of machine learning
and robotics (copied from Wikipedia)

CS391R: Robot Learning (Fall 2021) 12

 When NOT to Make Robots Learn?
Learning is not a solution to every problem in robotics.

Harnessing the priors and structures of a Learning is most effective when used in
problem goes a long way… conjunction with modeling.

CS391R: Robot Learning (Fall 2021) 13

 When to Make Robots Learn?
Learning is critical for taking robots to the real world.

object variation environment uncertainty adaptation

CS391R: Robot Learning (Fall 2021) 14

 Now is the best time to study and work on Robot Learning.

Artificial Intelligence Computing Power Robot Hardware

Recent breakthroughs in machine Your smartphone is millions of times More reliable and affordable
learning and computer vision, e.g., deep more powerful than all of NASA’s cobot hardware that costs around
learning (Turing awards 2018) combined computing in 1969. annual salary of American
workers

CS391R: Robot Learning (Fall 2021) 17

 Now is the best time to study and work on Robot Learning.

Positive and negative societal impacts of

robot learning research is an important
part of our in-class discussions.

https://www.therobotreport.com/tag/coronavirus/

CS391R: Robot Learning (Fall 2021) 18

 Robot Learning as a Growing Research Community

60

Number of Papers (k)

50

40

30 6x
20

10

2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
Year

Conference on Robot Learning is 4 years old. Growth of “Robot Learning” Publications

[Source: Google Scholar]

CS391R: Robot Learning (Fall 2021) 19

 Course Content We review the Robot Learning literature in these topics.

Part I: Robot Perception Part II: Robot Decision Making

Topic 1-10 Topic 11-20

seeing and understanding planning and control of
the physical world robot behaviors

Prerequisite: coursework / experience in AI and Machine Learning

CS391R: Robot Learning (Fall 2021) 21

 Course Content We review the Robot Learning literature in these topics.

Part I: Robot Perception Part II: Robot Decision Making

Topic 1-10 Topic 11-20

seeing and understanding planning and control of
the physical world robot behaviors

Prerequisite: coursework / experience in AI and Machine Learning

CS391R: Robot Learning (Fall 2021) 22

 Robot Perception

synthetic data for robot perception self-supervised visual learning

2D object detection
implicit neural representations
3D data processing

multimodal understanding
recursive state estimation attention architectures interactive perception
CS391R: Robot Learning (Fall 2021) 23
 Course Content We review the Robot Learning literature in these topics.

Part I: Perception Part II: Decision Making

Topic 1-10 Topic 11-20

seeing and understanding planning and control of
the physical world robot behaviors

Prerequisite: coursework / experience in AI and Machine Learning

CS391R: Robot Learning (Fall 2021) 24

 Robot Decision Making

offline RL

model-free RL inverse RL model-based RL

task and motion planning

adversarial IL imitation as supervised learning

hierarchical policy & neural programming
CS391R: Robot Learning (Fall 2021) 25
 Learning Objectives

● understand the potential and societal impact of general-purpose robot autonomy in the
real world, the technical challenges arising from building it, and the role of machine
learning and AI in addressing these challenges;

● get familiar with a variety of model-driven and data-driven principles and algorithms on
robot perception and decision making;

● be able to evaluate, communicate, and apply advanced AI-based techniques to

robotics problems.

… through literature reviews, research presentations, and course projects

CS391R: Robot Learning (Fall 2021) 26

 Learning Objectives

Get a taste of Robot Learning research in the full circle

CS391R: Robot Learning (Fall 2021) 27

 Logistics

Lectures

Time: 9:30-11:00am CT, Tuesdays and Thursdays

Location: Online or in-person (Zoom links on Canvas)

Office Hours

Instructor: 3-4pm Mondays (GDC 3.422) or by appointment

TA: 4-5pm Wednesdays (GDC 3.516)

CS391R: Robot Learning (Fall 2021) 28

 Instruction Modality

Now to September 17

Online lectures + in-person office hours

After September 17

Adjusting plans based on university policy

In-Person Experiences

Office hours, instructor/TA meetings by appointment, GDC 4.302

CS391R: Robot Learning (Fall 2021) 29

 https://www.cs.utexas.edu/~yukez/cs391r_fall2021/

Logistics

Instructor Lectures
overview of research topics

Student Presentations
presentation of research papers

Final Project Spotlights

spotlight talks of course projects

CS391R: Robot Learning (Fall 2021) 30

 https://www.cs.utexas.edu/~yukez/cs391r_fall2021/

Logistics

Required Readings (No Review)

overview or survey papers with lectures

Required Readings
key papers that will be discussed in class

Optional Readings
recommended papers for in-depth reviews

CS391R: Robot Learning (Fall 2021) 31

 Logistics
20% each

Grading Policy • At least one presentation for each student (chances to do more)

• Length: 20min (± 2min) + 3min Q&A

Student presentation (20%)
• Format: problem formulation, technical approach, results, … (see
Paper reviews (30%) slide template for more details)

Course project (40%) • Followed by 5-10min in-class discussions

In-class participation (10%) • Email the slides to the TA and the instructor seven days (EOD)
prior to the presentation date

• Presentation recordings posted in Canvas (protected under

FERPA)

• Breakout rooms and in-class discussions will NOT be recorded.

CS391R: Robot Learning (Fall 2021) 32

 Logistics
2% each x 15 reviews

Grading Policy • Due by 9:59pm the previous night of each student presentation

• Write a review for one paper from the required readings (2 choices
Student presentation (20%)
for each class)
Paper reviews (30%) • Online review form in R:SS format

Course project (40%)

In-class participation (10%)

• No late date - but more than 15 presentation classes (feel free to
skip some)

• Have energy to do more? Top-scored 15 for grading

• Class attendance and participation is required for review grades

CS391R: Robot Learning (Fall 2021) 33

 Logistics
2% each x 15 reviews

• Due by 9:59pm the previous night of each student presentation

• Write a review for one paper from the required readings (2 choices
for each class)

• Online review form in R:SS format

• No late date - but more than 15 presentation classes (feel free to

skip some)

• Have energy to do more? Top-scored 15 for grading

• Class attendance and participation is required for review grades

CS391R: Robot Learning (Fall 2021) 34

 Logistics
40%

Grading Policy • Project Proposal (5%). Due Thu Sept 16.

• Project Milestone (5%). Due Thu Oct 21.
Student presentation (20%) • Final Report (25%). Due Fri Dec 10.
• Spotlight Talk (5%). Week 15.
Paper reviews (30%)

Course project (40%)

In-class participation (10%)

Hands-on experience of
robot learning research
+

CS391R: Robot Learning (Fall 2021) 35

 Logistics

Grading Policy

Student presentation (20%)

Paper reviews (30%)

Course project (40%)

In-class participation (10%)

project platform: robosuite (robosuite.ai)

Tutorials, computing resources,

Alternative projects require instructor approval.
project instructions, …

CS391R: Robot Learning (Fall 2021) 36

 Logistics

Grading Policy

Student presentation (20%)

Paper reviews (30%)

Course project (40%)

In-class participation (10%)

CS391R: Robot Learning (Fall 2021) 37

 Tell Us About Yourself

CS391R: Robot Learning (Fall 2021) 38

 Robotics beyond CS391R
Be part of the Robotics + AI revolution.

of

UT Robot Perception & Learning Lab

Mission: Building General-Purpose Robot Autonomy in the Wild

https://robotics.utexas.edu/

CS391R: Robot Learning (Fall 2021) 39