This repo contains all the code used in the Python for Data Engineering Course
Code is free to use, if you want in-depth understanding of concepts then check out my python course designed to teach you how to think programmatically and perform data engineering tasks using python
Module 1: iPhone Data Analysis Project
Get right into it, don't waste time learning basics, and execute the complete project to boost your confidence.
W 68DC ork on Python Data Analysis project on iPhone Apple Dataset, do some basic transformation and analysis.
Module 2: Python Basics
Build strong programming fundamentals by learning the basics of python
- Working with Strings
- Working with Numbers
- Understanding DataTypes
- Python Data Structures (List, Dictionary, Sets, Tuples)
- Working with Loops (For & While)
- Conditional Statements (IF-ELSE)
Module 3: Python Intermediate
Let's learn some intermediate concepts to build a strong foundation.
- List Compherention
- Understand Python Packages
- Exception Handling
- Working with a text file
- Working with Functions
- Lambda functions basics
Module 4: Python Advance
Let's learn some advanced topics and packages used by data engineer on a daily basis
- Py Package - Numpy
- Py Package - Pandas (Basics and Understanding Transformation with Project)
Module 5: Python for Data Engineering
Python for data engineering, learn how to work with different file formats and handle DateTime columns
- Working with DateTime Formats
- Working with different file formats (JSON, CSV, Excel, AVRO)
- Scrape Amazon Data Using Python
Implement Complete Data Pipeline Data Engineering Project using Spotify
- Integrating with Spotify API and extracting Data
- Deploying code on AWS Lambda for Data Extraction
- Adding trigger to run the extraction automatically
- Writing transformation function
- Building automated trigger on transformation function
- Store files on S3 properly
- Building Analytics Tables on data files using Glue and Athena