Python for Data Engineering

This repo contains all the code used in the Python for Data Engineering Course

Code is free to use, if you want in-depth understanding of concepts then check out my python course designed to teach you how to think programmatically and perform data engineering tasks using python

Module 1: iPhone Data Analysis Project

Get right into it, don't waste time learning basics, and execute the complete project to boost your confidence.

Work on Python Data Analysis project on iPhone Apple Dataset, do some basic transformation and analysis.

Module 2: Python Basics

Build strong programming fundamentals by learning the basics of python

Working with Strings
Working with Numbers
Understanding DataTypes
Python Data Structures (List, Dictionary, Sets, Tuples)
Working with Loops (For & While)
Conditional Statements (IF-ELSE)

Module 3: Python Intermediate

Let's learn some intermediate concepts to build a strong foundation.

List Compherention
Understand Python Packages
Exception Handling
Working with a text file
Working with Functions
Lambda functions basics

Module 4: Python Advance

Let's learn some advanced topics and packages used by data engineer on a daily basis

Py Package - Numpy
Py Package - Pandas (Basics and Understanding Transformation with Project)

Module 5: Python for Data Engineering

Python for data engineering, learn how to work with different file formats and handle DateTime columns

Working with DateTime Formats
Working with different file formats (JSON, CSV, Excel, AVRO)
Scrape Amazon Data Using Python

Module 6: Spotfiy Data Pipeline End-To-End Python Data Engineering Project

Implement Complete Data Pipeline Data Engineering Project using Spotify

Integrating with Spotify API and extracting Data
Deploying code on AWS Lambda for Data Extraction
Adding trigger to run the extraction automatically
Writing transformation function
Building automated trigger on transformation function
Store files on S3 properly
Building Analytics Tables on data files using Glue and Athena

Provide feedback

Saved searches

Use saved searches to filter your results more quickly