📌 Web Scraping, REST API & Stock Data Operations

📜 Overview

This repository contains multiple projects related to web scraping, REST API data extraction, and stock market data operations. It includes Jupyter Notebooks demonstrating how to collect, clean, analyze, and visualize data from various sources such as stock market APIs, census data, movie databases, and more.

The project leverages Python libraries like BeautifulSoup, Requests, Pandas, Matplotlib, Seaborn, SQLite, and Plotly to extract insights and build interactive visualizations.

📂 Repository Structure

📁 Stock Market Data Analysis

Analyzing_Historical_Stock_Revenue Data and Building a Dashboard.ipynb – Extracts and visualizes historical stock data using APIs and Pandas.
Extracting Stock Data Using a Python Library.ipynb – Uses a Python library to retrieve stock data for analysis.
Extracting Stock Data Using a Web Scraping.ipynb – Demonstrates web scraping techniques for gathering stock market information.
Golden_State.pkl – Pickle file storing processed stock data for further analysis.
apple.csv, apple.json, amd.json – Stock data files for Apple and AMD.

📁 Web Scraping & REST API

Web Scraping Top 2019 Movies at the Worldwide Box Office.ipynb – Extracts movie revenue data using web scraping techniques.
Content Web Scraping_Web-Scraping-Lab.ipynb – Covers different methods of extracting data from websites.
Extracting Data Via API.ipynb – Demonstrates how to retrieve data using REST APIs.
REST API.ipynb – Hands-on implementation of working with RESTful APIs.

📁 Database & SQL Operations

CVD_Database_Create_Tables_Script.sql – SQL script for creating structured databases.
SQLITE DATABASES with PYTHON.ipynb – Shows how to interact with SQLite databases using Python.
FinalDB.db – SQLite database storing structured data.

📁 Census & Public Data Analysis

Census_Data_-_Selected_socioeconomic_indicators_in_Chicago__2008___2012_20240227.csv – Census dataset containing socioeconomic indicators.
ChicagoCensusData.csv, ChicagoCrimeData.csv, ChicagoPublicSchools.csv, Chicago_Public_Schools_-_Progress_Report_Cards__2011-2012__20240227.csv – Chicago’s census, crime, and education datasets for exploratory analysis.
games.csv – Data related to games industry analysis.

📁 Miscellaneous

file_list.txt – List of files in the repository.

🚀 Key Features

✅ Web Scraping: Extracts structured and unstructured data from web sources.
✅ REST API Integration: Retrieves real-time stock and public data via APIs.
✅ Stock Market Analysis: Uses Python libraries to analyze and visualize stock trends.
✅ Database Management: Demonstrates SQL and Python-based database interactions.
✅ Data Cleaning & Preprocessing: Applies EDA techniques to real-world datasets.
✅ Interactive Dashboards: Uses Plotly and Matplotlib to create visual reports.

📊 Technologies Used

🔹 Python – Pandas, Matplotlib, Seaborn, Plotly, BeautifulSoup, Requests
🔹 Web Scraping – BeautifulSoup, Requests, Selenium
🔹 REST APIs – API calls with Python
🔹 Stock Market Analysis – Yahoo Finance, Alpha Vantage APIs
🔹 SQL – SQLite database operations
🔹 Data Visualization – Matplotlib, Seaborn, Dash

📌 Next Steps

📌 Expand stock analysis with real-time data feeds
📌 Automate data extraction for daily stock updates
📌 Build a Flask API to serve stock insights
📌 Enhance web scraping with Selenium for dynamic content

📢 Contributing

Feel free to contribute by submitting issues, feature requests, or pull requests!