awsglue
Here are 24 public repositories matching this topic...
This project focuses on real-time data streaming with Kinesis, using Flink for advanced processing and OpenSearch for analytics. This architecture has succinctly handled the complete lifecycle of data from ingestion to actionable insights, making it a comprehensive solution.
-
Updated
Aug 4, 2024 - Java
This projects uses ETL (Extract, Transform and Load) pipeline to extract data from Spotify using its API and loads the data to a data source(AWS Athena). The entire pipeline will be built using Amazon Web Services (AWS).
-
Updated
Jul 8, 2023 - Jupyter Notebook
In this project I have used the Trending YouTube Video Statistics data from Kaggle to analyze and prepare it for usage.
-
Updated
Nov 7, 2022
Big data and Cloud Deployment
-
Updated
Jan 15, 2024 - Jupyter Notebook
This project demonstrates how you can build downstream data pipeline using dbt in athena
-
Updated
Dec 24, 2022 - Python
AWS S3 & Sentiment Analysis, Basic Plotting with Matplotlib, & Supervised Learning & Machine Learning with Sklearn.
-
Updated
Jul 6, 2024 - Jupyter Notebook
Data Engineering Specialization offered by Joe Reis in partnership with DeepLearning.AI through Coursera...
-
Updated
Sep 29, 2024 - Jupyter Notebook
This project showcases a data transformation pipeline utilizing AWS Glue and Amazon Athena to process Spotify data from CSV files. It involves loading, transforming, and storing data in an S3 datawarehouse, enabling seamless querying through Amazon Athena.
-
Updated
Mar 28, 2024 - Python
This project demonstrates the use of Amazon Web Services (AWS) to analyze superstore sales data. The analysis was performed using AWS S3 for data storage, AWS Glue for data cataloging, AWS Athena for SQL-based serverless data querying, and AWS Quick Sight for visualization. The project’s objective was to provide actionable insights into sales trend
-
Updated
Jan 2, 2025
This project sets up a real-time data pipeline to fetch data from Reddit, transform it using AWS Glue, and store it in Amazon S3. This involves data streaming, cloud storage, ETL (Extract, Transform, Load) processes, and orchestration using Apache Airflow.
-
Updated
Sep 18, 2024 - Python
Projects on Big Data Using Pyspark and AWS
-
Updated
Apr 28, 2023 - Jupyter Notebook
This project offers a robust data pipeline solution designed to efficiently extract, transform, and load (ETL) Reddit data into a Redshift data warehouse. Leveraging a blend of industry-standard tools and services, the pipeline ensures seamless data processing and integration.
-
Updated
Jun 19, 2024 - Jupyter Notebook
In this project we can run an ETL in AWS Glue by Orchestrating it with Airflow. This project we create a Docker Compose to raise the services as Airflow, Redis and PostgreSQL. PostgreSQL was use in this project to save metadata get of Airflow
-
Updated
Sep 26, 2024 - Python
This project aims to securely manage, process, and analyze structured and semi-structured YouTube data based on video categories and trending metrics. The architecture leverages AWS services to ingest, store, transform, analyze, and visualize data efficiently and at scale.
-
Updated
Dec 28, 2024 - Python
Incremental Data Load from S3 Bucket to Amazon Redshift Using AWS Glue
-
Updated
Aug 15, 2024 - Python 9518
This project builds a pipeline to analyze Superstore sales data using the power of AWS. It transforms the data to make it ready for exploration. Querying the transformed data using SQL queries to uncover trends and patterns. Analyzing results and creates easy-to-understand visualizations, providing clear insights into Superstore sales performance.
-
Updated
May 27, 2024
Repositório para demonstração da ferramenta AWS GLUE - COmputação em Nuvem - DCOMP/UFS - 2024.1
-
Updated
Sep 26, 2024 - Python
Improve this page
Add a description, image, and links to the awsglue topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the awsglue topic, visit your repo's landing page and select "manage topics."