Generative AI

Nanodegree Program

Embark on a transformative journey into Generative AI! We'll start by diving into the essentials with an introductory course, progress to mastering text generation with Large Language Models, unravel the complexities of image creation in computer vision and cap it off by bringing AI to life in real-world applications. From foundational theories to building sophisticated chatbots and AI agents, this program will empower you with job-ready skills in the exciting field of Generative AI.

Intermediate

4 months

Real-world Projects

Completion Certificate

Last Updated July 12, 2024

Skills you'll learn:

Vectors • Retrieval-Augmented Generation • OpenAI API • LangChain

Prerequisites:

Database fundamentals • Intermediate Python

Courses In This Program

Course 1 • 45 minutes

Welcome to the Nanodegree Program!

Welcome to Udacity! We're excited to share more about your Nanodegree program and start this journey with you!

Lesson 1

Welcome!

Welcome to Udacity. Takes 5 minutes to get familiar with Udacity courses and gain some tips to succeed in courses.

Lesson 2

Getting Help

You are starting a challenging but rewarding journey! Take 5 minutes to read how to get help with projects and content.

Lesson 1

Welcome!

Welcome to Udacity. Takes 5 minutes to get familiar with Udacity courses and gain some tips to succeed in courses.

Lesson 2

Getting Help

You are starting a challenging but rewarding journey! Take 5 minutes to read how to get help with projects and content.

Course 2 • 4 weeks

Generative AI Fundamentals

Dive into generative AI with this course, which explores its fundamental principles and relationship to prior artificial intelligence innovations. We will walk through popular generative models and how they work, how deep learning models are developed using tools like PyTorch and Hugging Face, and finally, how to customize pre-trained open-source models for a specific use case. In the project, you will apply a cutting-edge technique called parameter-efficient fine-tuning (PEFT), which allows for the adaptation of massive foundation models with minimal usage of computational resources.

Lesson 1

Introduction to Generative AI Fundamentals

This lesson provides the foundational knowledge needed about generative AI: what it is, how it's applied, and explanations of some popular algorithms and architectures for text and image generation.

Lesson 2

Deep Learning Fundamentals

This lesson covers the essentials of deep learning for the generative AI practitioner. From perceptrons to transfer learning including an introduction to the PyTorch and Hugging Face Python libraries.

Lesson 3

Foundation Models

This lesson explores foundation models in AI, how they differ from traditional models, how you can apply them to various tasks and evaluate their performance, and the ethical implication of their use.

Lesson 4

Adapting Foundation Models

This lesson covers a range of techniques for adapting foundation models, including prompt tuning, in-context learning, full fine-tuning, and parameter-efficient fine-tuning (PEFT).

Lesson 5 • Project

Apply Lightweight Fine-Tuning to a Foundation Model

Load and customize a Hugging Face foundation model using parameter-efficient fine-tuning. This technique allows you to harness the power of a pre-trained model for your custom task.

Lesson 1

Introduction to Generative AI Fundamentals

This lesson provides the foundational knowledge needed about generative AI: what it is, how it's applied, and explanations of some popular algorithms and architectures for text and image generation.

Lesson 2

Deep Learning Fundamentals

Lesson 3

Foundation Models

Lesson 4

Adapting Foundation Models

This lesson covers a range of techniques for adapting foundation models, including prompt tuning, in-context learning, full fine-tuning, and parameter-efficient fine-tuning (PEFT).

Lesson 5 • Project

Apply Lightweight Fine-Tuning to a Foundation Model

Load and customize a Hugging Face foundation model using parameter-efficient fine-tuning. This technique allows you to harness the power of a pre-trained model for your custom task.

Course 3 • 4 weeks

Large Language Models (LLMs) & Text Generation

Dive deeper into how computers understand and create language, and learn how to build a custom chatbot using unsupervised machine learning, prompt engineering, and retrieval augmented generation. We'll start with a high-level overview of the types of LLMs, the differences between them, and how best to account for their strengths and weaknesses. Then we'll get into the internal details, including natural language processing (NLP) techniques like tokenization, as well as modern transformer architectures and attention mechanisms. Finally, we'll build a practical LLM application that combines an LLM with a custom dataset.

Lesson 1

Introduction to LLMs

This lesson covers the types of LLMs, an intuitive understanding of their limitations and capabilities, inference and decoding hyperparameters, and strategies for effective prompt engineering.

Lesson 2

NLP Fundamentals

This lesson covers the essential Natural Language Processing topics needed to use the latest LLM technology. You will learn the basics of NLP and then dive into text encoding and text generation.

Lesson 3

Transformers and Attention Mechanism

In this lesson, you will open up the black box of transformer architectures and learn about the attention mechanisms and other components that make these powerful models possible.

Lesson 4

Retrieval Augmented Generation

In this lesson, we will learn how to create a custom Q&A bot powered by OpenAI! Along the way, you'll learn how OpenAI works and how to leverage its powerful language processing capabilities.

Lesson 5

Build Custom Datasets for LLMs

In this lesson, you will learn how to construct a relevant, quality dataset for fine-tuning large language models and performing retrieval augmented generation.

Lesson 6 • Project

Project: Build Your Own Custom Chatbot

For this project, you will use everything you learned in this course to create a custom chatbot using a dataset of your choice.

Lesson 1

Introduction to LLMs

This lesson covers the types of LLMs, an intuitive understanding of their limitations and capabilities, inference and decoding hyperparameters, and strategies for effective prompt engineering.

Lesson 2

NLP Fundamentals

This lesson covers the essential Natural Language Processing topics needed to use the latest LLM technology. You will learn the basics of NLP and then dive into text encoding and text generation.

Lesson 3

Transformers and Attention Mechanism

In this lesson, you will open up the black box of transformer architectures and learn about the attention mechanisms and other components that make these powerful models possible.

Lesson 4

Retrieval Augmented Generation

In this lesson, we will learn how to create a custom Q&A bot powered by OpenAI! Along the way, you'll learn how OpenAI works and how to leverage its powerful language processing capabilities.

Lesson 5

Build Custom Datasets for LLMs

In this lesson, you will learn how to construct a relevant, quality dataset for fine-tuning large language models and performing retrieval augmented generation.

Lesson 6 • Project

Project: Build Your Own Custom Chatbot

For this project, you will use everything you learned in this course to create a custom chatbot using a dataset of your choice.

Course 4 • 4 weeks

Computer Vision and Generative AI

Learn how computers process and understand image data, then harness the power of the latest Generative AI models to create new images.

Lesson 1

Introduction to Image Generation

In this lesson, you will define image generation and understand its relevance in AI and machine learning.

Lesson 2

Computer Vision Fundamentals

Learn how computers see images and perform key image processing techniques using classic image processing techniques such as image transformation, noise reduction, and more.

Lesson 3

Image Generation and GANs

Explore the landscape of Gen AI tools for Computer Vision and learn how they are evaluated. Learn what a generative adversarial network is and how it is utilized to generate images.

Lesson 4

Transformer-Based Computer Vision Models

In this lesson, we will be exploring Vision Transformers and the architecture that makes them work. Along the way we will explore Vision Transformers like DALL-E, DINO, and SAM.

Lesson 5

Diffusion Models

Learn the fundamentals of transformers. Then, get hands-on with the creation of a diffusion algorithm and work with Huggingface Diffusers to generate and work with images.

Lesson 6 • Project

AI Photo Editing with Inpainting

In this project, you will utilize Generative AI to take a famous painting and swap out the background with an image generated by Stable Diffusion.

Lesson 1

Introduction to Image Generation

In this lesson, you will define image generation and understand its relevance in AI and machine learning.

Lesson 2

Computer Vision Fundamentals

Learn how computers see images and perform key image processing techniques using classic image processing techniques such as image transformation, noise reduction, and more.

Lesson 3

Image Generation and GANs

Explore the landscape of Gen AI tools for Computer Vision and learn how they are evaluated. Learn what a generative adversarial network is and how it is utilized to generate images.

Lesson 4

Transformer-Based Computer Vision Models

In this lesson, we will be exploring Vision Transformers and the architecture that makes them work. Along the way we will explore Vision Transformers like DALL-E, DINO, and SAM.

Lesson 5

Diffusion Models

Learn the fundamentals of transformers. Then, get hands-on with the creation of a diffusion algorithm and work with Huggingface Diffusers to generate and work with images.

Lesson 6 • Project

AI Photo Editing with Inpainting

In this project, you will utilize Generative AI to take a famous painting and swap out the background with an image generated by Stable Diffusion.

Taught By The Best

Brian Cruz

Head of AI Engineering

Brian Cruz leads AI Engineering at Advocate, where Generative AI is used to help disabled individuals apply for benefits. He previously worked at Samba TV, contributing significantly to advancements in AI for edge devices, as well as at Salesforce, where he led ML engineering for sales revenue forecasting.

Emily McMilin

Research Scientist

Emily McMilin is a Senior Research Scientist and Independent Researcher working at the intersection of NLP and Causal Inference. She obtained her Ph.D. in Electrical Engineering from Stanford University and prior to that an M.Sc. from University of Victoria, and a B.Sc. from Stanford in Symbolic Systems.

Victor Geislinger

Machine Learning Engineer

Victor Geislinger is a machine learning engineer and is dedicated to sharing his knowledge with others. Victor recently joined Google as a software engineer focused on AI/ML but has been programming and educating others for over a decade since studying physics and math at the University of California, Santa Cruz.

Jason Lin

Chief Scientist, Reasonly AI

Jason has developed deep learning algorithms and AI applications at Lyft self-driving, Spotify and Google DeepMind. Formerly a Stanford Online and UN keynote speaker, he's earned a M.S. in Machine Learning from Georgia Tech and coauthored NLP and computer vision papers with MIT.

Erick Galinkin

Principal AI Researcher

Erick Galinkin is a hacker and computer scientist, leading research at the intersection of security and artificial intelligence at Rapid7. He has spoken at numerous industry and academic conferences on topics ranging from malware development to game theory in security.

Giacomo Vianello

Principal Data Scientist

Giacomo Vianello is an end-to-end data scientist with a passion for state-of-the-art but practical technical solutions. He is Principal Data Scientist at Cape Analytics, where he develops AI systems to extract intelligence from geospatial imagery bringing, cutting-edge AI solutions to the insurance and real estate industries.

Chuyi Shang

UC Berkeley ML Researcher

Chuyi Shang is a machine learning researcher at Berkeley and a member of the Machine Learning @ Berkeley organization. He conducts research in video understanding and multimodal learning at Berkeley's AI Research Lab (BAIR), and has also conducted ML research at Berkeley's Haas School of Business.

Annabel Ng

UC Berkeley ML Researcher

Annabel Ng is an EECS undergrad at UC Berkeley, where she's researching brain-inspired vision models to improve image encodings in a Berkeley AI Research lab. She also leads the workshop division at Machine Learning @ Berkeley where she delivers lectures and develops interactive ML content for students.

Derek Xu

Derek Xu, VP of Education of Machine Learning @ Berkeley

Derek Xu is the Vice President of Education of Machine Learning @ Berkeley. He studies EECS and Business Administration through the M.E.T. Program and teaches a modern computer vision course at UC Berkeley. Derek was previously a machine learning engineering intern at TikTok working on the ML Platforms team, and an undergraduate researcher at UC Berkeley Sky Computing Lab. He also has experience as a software engineering intern at Salesforce and Ramp.

Nathaniel Haynam

ML Researcher at BAIR

Nathaniel Haynam is an ML Researcher at BAIR, where they push the edge of inverse reinforcement learning for multi-agent simulations. They are a ML Engineer and Lecturer in Machine Learning at Berkeley, teaching a modern computer vision course at UC Berkeley. They are a computer science major at UC Berkeley.

Valerie Scarlata

Senior Technical Content Developer at Udacity

Valerie is a Sr. Technical Content Developer at Udacity who has developed and taught a broad range of computing curricula for multiple colleges and universities. She is a former professor and software engineer for over 10 years specializing in web, mobile, voice assistant, and full-stack application development.

Chang She

CEO and Co-founder of LanceDB

Chang has nearly two decades of experience building and teaching data / ML tooling. He was the second major contributor to pandas, an adjunct at Columbia for introduction to data science, and ran engineering at TubiTV focusing on recommender systems. Most recently, Chang co-founded LanceDB to build the next generation database for AI.

Sergei Kozyrenko

Senior Staff Engineer

Sergei Kozyrenko is a technology leader with over 20 years of diverse industry experience - he’s built trading engines, banking software, learning management systems, co-founded an AI startup that accurately predicted street parking availability and even automated shooting of high-powered lasers at blocks of chocolate.

The Udacity Difference

Combine technology training for employees with industry experts, mentors, and projects, for critical thinking that pushes innovation. Our proven upskilling system goes after success—relentlessly.

Demonstrate proficiency with practical projects

Projects are based on real-world scenarios and challenges, allowing you to apply the skills you learn to practical situations, while giving you real hands-on experience.

Gain proven experience
Retain knowledge longer
Apply new skills immediately

Top-tier services to ensure learner success

Reviewers provide timely and constructive feedback on your project submissions, highlighting areas of improvement and offering practical tips to enhance your work.