longdocassist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "930e3b88b11fa84b",
   "metadata": {},
   "source": [
    "<div style=\"text-align: center;\">\n",
    "    <h1>Long Doc Assist</h1>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcba45e362001c4",
   "metadata": {},
   "source": [
    "## Problem Statement"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab8cd7322d2309b1",
   "metadata": {},
   "source": [
    "### Project Goal\n",
    "The goal of this project is to create a system that can help users understand and extract information from long documents, such as insurance policies, by leveraging the capabilities of language models and vector databases.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1035337610619cb6",
   "metadata": {},
   "source": [
    "### Data Sources\n",
    "\n",
    "**Long Documents**: The system will be designed to work with long documents, such as insurance policies, legal contracts, or technical manuals. These documents can be in PDF format and may contain complex language and structure."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a487f21a0c9bdd9b",
   "metadata": {},
   "source": [
    "### Challenges faced\n",
    "\n",
    "- Going through the various options available in the langchain documentation and understanding the apis.\n",
    "- Not all pdfs work out of the box, they need to be in a certain format with lots of text."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f886c61fff6c0c",
   "metadata": {},
   "source": [
    "### Next Steps\n",
    "\n",
    "- This application is able to understand most pdfs, however there needs to be support built for many different pdf formats.\n",
    "- The chunking strategy should be available in a dropdown format for the user to choose from, may be an option to choose the chunking strategy.\n",
    "- Application's AI Evaluations should be conducted to get a measurable value that would represent how well is the application performing.\n",
    "- Overall, this is a general pdf upload and query application that works"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38de0ea404bf6988",
   "metadata": {},
   "source": [
    "<div style=\"page-break-after: always;\"></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f338b7f4fa792ab",
   "metadata": {},
   "source": [
    "## Solution Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15cf89d08f78db64",
   "metadata": {},
   "source": [
    "### Design Choices\n",
    "\n",
    "- **Document Ingestion**: The system will use a document loader to read and process long documents, splitting them into smaller, manageable chunks for better retrieval and understanding.\n",
    "- **Vector Store**: A vector database will be used to store the document chunks, allowing for efficient retrieval based on semantic similarity. Qdrant will be used as the vector store.\n",
    "- **Language Model**: The system will use a language model to generate responses based on user queries and the retrieved document chunks. OpenAI's GPT-3.5-turbo will be used for this purpose.\n",
    "- **User Interface**: A web application will be built using Streamlit to provide an interactive interface for users to upload documents, ask questions, and receive answers based on the content of the documents.\n",
    "- **Retrieval-Augmented Generation (RAG)**: The system will implement a RAG approach, where the language model retrieves relevant information from the vector store before generating a response to the user's query. This allows for more accurate and context-aware answers."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d96f955a28fb6f45",
   "metadata": {},
   "source": [
    "### Why Langchain?\n",
    "- Langchain provides a modular and flexible framework for building applications with language models, making it easier to integrate various components such as document loaders, text splitters, vector stores, and language models.\n",
    "- It supports various document loaders and vector stores, allowing for easy integration with different data sources and storage solutions.\n",
    "- Langchain has a strong community and extensive documentation, making it easier to find support and resources for building applications.\n",
    "- Langchain's integration with popular cloud providers and deployment platforms makes it easy to deploy applications at scale, ensuring that the solution can handle large volumes of data and user queries efficiently."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dfd6def3613a81d",
   "metadata": {},
   "source": [
    "<div style=\"page-break-after: always;\"></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76967f02c8355dca",
   "metadata": {},
   "source": [
    "### Architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78bfa55766ba39d3",
   "metadata": {},
   "source": [
    "![Architecture Diagram](./langchain-RAG.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d088a85e-b1ae-4c6d-8f9e-4bbd14de3840",
   "metadata": {},
   "source": [
    "<div style=\"page-break-after: always;\"></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a981bb95c38076e",
   "metadata": {},
   "source": [
    "### Architecture Description\n",
    "\n",
    "This diagram represents the architecture of a **Long Document Assistant** system that allows users to interact with large documents using a chatbot interface.\n",
    "\n",
    "#### 🧑‍💻 User Interaction\n",
    "- The **user** uploads a document.\n",
    "- Then interacts with the **Long Doc Bot** (built using Streamlit) to ask questions about the document.\n",
    "\n",
    "#### 🧠 Document Ingestion\n",
    "- The **Document Ingestion Engine** handles:\n",
    "  - Reading the document\n",
    "  - Chunking it into manageable parts\n",
    "  - Embedding each chunk using **OpenAI** embeddings\n",
    "- The embedded chunks are stored in **Qdrant**, a vector database.\n",
    "\n",
    "#### 🔍 Query Processing\n",
    "- The **Query Engine**:\n",
    "  - Retrieves the most relevant chunks from **Qdrant**\n",
    "  - Combines them with the user’s question\n",
    "  - Sends the combined context to **OpenAI** to generate a response\n",
    "\n",
    "#### 🤖 Response Delivery\n",
    "- The **Long Doc Bot** presents the generated answer back to the user, enabling conversational querying of large documents."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "345be82e03db34ec",
   "metadata": {},
   "source": [
    "<div style=\"page-break-after: always;\"></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5de1fc573e8667db",
   "metadata": {},
   "source": [
    "### Solution Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8752ef0fe0d71e57",
   "metadata": {},
   "source": [
    "#### Technology Stack\n",
    "- **Langchain**: A framework for building applications with language models.\n",
    "- **Qdrant**: A vector database for storing and retrieving document chunks.\n",
    "- **OpenAI**: A language model for generating responses based on user queries and creating embeddings for document chunks.\n",
    "- **Streamlit**: A web application framework for building interactive applications."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3644e1d07cd345b0",
   "metadata": {},
   "source": [
    "#### Development Environment\n",
    "\n",
    "- Install poetry for dependency management\n",
    "- Install pyenv for python version management\n",
    "- Install docker and docker-compose to start qdrant db locally\n",
    "- Place an OPENAI_API_KEY.txt file with your open ai api key at the project root.\n",
    "- run `docker-compose up -d` to start qdrant db, then view the qdrant dashboard at http://localhost:6333:dashboard\n",
    "- run `poetry install` to install the dependencies\n",
    "- run `poetry shell` to activate the virtual environment\n",
    "- run `jupyter lab` to start the jupyter notebook server\n",
    "- run `sh build.sh` to update the `README.md` & `output_longdocassist.html`\n",
    "- run `sh run.sh` to start the streamlit application"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1728b770e5de5860",
   "metadata": {},
   "source": [
    "#### OpenAI components\n",
    "\n",
    "- **get_openai_api_key**: Reads the OpenAI API key from a file.\n",
    "- **get_embeddings_model**: Returns an instance of OpenAIEmbeddings for generating embeddings.\n",
    "- **get_llm**: Returns an instance of ChatOpenAI for generating responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "21b104b4b898b93d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-01T07:40:42.955148Z",
     "start_time": "2025-06-01T07:40:42.952763Z"
    }
   },
   "outputs": [],
   "source": [
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "def get_openai_api_key():\n",
    "    \"\"\"Reads the OpenAI API key from a file.\n",
    "    Returns:\n",
    "        str: The OpenAI API key.\n",
    "    \"\"\"\n",
    "    with open(\"./OPENAI_API_KEY.txt\", \"r\") as file:\n",
    "        return file.read().strip()\n",
    "\n",
    "def get_embeddings_model():\n",
    "    \"\"\"Returns an instance of OpenAIEmbeddings for generating embeddings.\n",
    "    Returns:\n",
    "        OpenAIEmbeddings: An instance of OpenAIEmbeddings.\n",
    "    \"\"\"\n",
    "    return OpenAIEmbeddings(model=\"text-embedding-ada-002\", openai_api_key = get_openai_api_key())\n",
    "\n",
    "def get_llm():\n",
    "    \"\"\"Returns an instance of ChatOpenAI for generating responses.\n",
    "    Returns:\n",
    "        ChatOpenAI: An instance of ChatOpenAI.\n",
    "    \"\"\"\n",
    "    return ChatOpenAI(model_name=\"gpt-3.5-turbo\", openai_api_key=get_openai_api_key())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2023f8e05499154",
   "metadata": {},
   "source": [
    "#### Qdrant components\n",
    "\n",
    "- **re_create_collection**: Deletes and recreates a Qdrant collection with the specified name and vector parameters.\n",
    "- **get_vector_store**: Creates a QdrantVectorStore instance for storing and retrieving document chunks, using the specified collection name and embedding model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "adec2ae76d2549ec",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-01T07:40:45.512924Z",
     "start_time": "2025-06-01T07:40:45.489232Z"
    }
   },
   "outputs": [],
   "source": [
    "from langchain_qdrant import QdrantVectorStore\n",
    "from qdrant_client import QdrantClient\n",
    "from qdrant_client.http.models import Distance, VectorParams\n",
    "\n",
    "client = QdrantClient(\"http://localhost:6333\")  # URL of your Qdrant instance\n",
    "\n",
    "def re_create_collection(collection_name):\n",
    "    \"\"\"Deletes and recreates a Qdrant collection with the specified name and vector parameters.\n",
    "    Args:\n",
    "        collection_name (str): The name of the collection to be created.\n",
    "    \"\"\"\n",
    "    client.delete_collection(collection_name)\n",
    "    client.create_collection(\n",
    "            collection_name=collection_name,\n",
    "            vectors_config=VectorParams(size=1536, distance=Distance.COSINE))\n",
    "\n",
    "def get_vector_store(collection_name):\n",
    "    \"\"\"Creates a QdrantVectorStore instance for storing and retrieving document chunks.\n",
    "    Args:\n",
    "        collection_name (str): The name of the collection to be used for the vector store.\n",
    "    Returns:\n",
    "        QdrantVectorStore: An instance of QdrantVectorStore.\n",
    "    \"\"\"\n",
    "    re_create_collection(collection_name)\n",
    "    return QdrantVectorStore(\n",
    "        client=client,\n",
    "        collection_name=collection_name,\n",
    "        embedding=get_embeddings_model()\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e146a494b2bf68c7",
   "metadata": {},
   "source": [
    "#### Ingestion Engine components\n",
    "\n",
    "- **get_documents**: Loads documents from a PDF file using PyPDFLoader.\n",
    "- **get_chunks**: Splits the loaded documents into smaller chunks using RecursiveCharacterTextSplitter.\n",
    "- **ingest_document**: Ingests the document chunks into a Qdrant vector store, using the file name as the collection name.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "8f2f275ccf780df0",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-01T07:58:37.990488Z",
     "start_time": "2025-06-01T07:58:37.987547Z"
    }
   },
   "outputs": [],
   "source": [
    "# Document Ingestion Engine\n",
    "from langchain_community.document_loaders import PyPDFLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "def get_documents(file_path):\n",
    "    \"\"\"Loads documents from a PDF file using PyPDFLoader.\n",
    "    Args:\n",
    "        file_path (str): The path to the PDF file.\n",
    "    Returns:\n",
    "        list: A list of documents loaded from the PDF file.\n",
    "    \"\"\"\n",
    "    loader = PyPDFLoader(file_path)\n",
    "    documents = loader.load()\n",
    "    return documents\n",
    "\n",
    "def get_chunks(documents):\n",
    "    \"\"\"Splits the loaded documents into smaller chunks using RecursiveCharacterTextSplitter.\n",
    "    Args:\n",
    "        documents (list): A list of documents to be split into chunks.\n",
    "    Returns:\n",
    "        list: A list of document chunks.\n",
    "    \"\"\"\n",
    "    text_splitter = RecursiveCharacterTextSplitter(\n",
    "        chunk_size=1000,  # Size of each chunk\n",
    "        chunk_overlap=200  # Overlap between chunks\n",
    "    )\n",
    "    return text_splitter.split_documents(documents)\n",
    "\n",
    "def ingest_document(file_path):\n",
    "    \"\"\"Ingests the document chunks into a Qdrant vector store, using the file name as the collection name.\n",
    "    Args:\n",
    "        file_path (str): The path to the PDF file to be ingested.\n",
    "    Returns:\n",
    "        QdrantVectorStore: An instance of QdrantVectorStore containing the ingested document chunks.\n",
    "    \"\"\"\n",
    "    documents = get_documents(file_path)\n",
    "    chunks = get_chunks(documents)\n",
    "    collection_name = file_path.split(\"/\")[-1].split(\".\")[0].replace(\" \", \"_\")  # Use file name as collection name\n",
    "    vector_store = get_vector_store(collection_name)  # Use file name as collection name\n",
    "    vector_store.from_documents(documents = chunks, embedding=get_embeddings_model(), collection_name=collection_name)\n",
    "    print(f\"ingested {vector_store.client.count(collection_name=collection_name).count} documents into collection {collection_name}\")\n",
    "    return vector_store\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97dc4de6b8f06cf4",
   "metadata": {},
   "source": [
    "#### Query Engine components\n",
    "\n",
    "- **get_user_query_response**: Takes a user query and a vector store, retrieves relevant documents, and generates a response using the language model. It uses RetrievalQA to handle the retrieval and response generation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "8971f2f977343242",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-01T07:41:00.680228Z",
     "start_time": "2025-06-01T07:41:00.678004Z"
    }
   },
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "def get_user_query_response(query, vector_store):\n",
    "    \"\"\"Takes a user query and a vector store, retrieves relevant documents, and generates a response using the language model.\n",
    "    Args:\n",
    "        query (str): The user's query.\n",
    "        vector_store (QdrantVectorStore): The vector store containing the document chunks.\n",
    "    Returns:\n",
    "        dict: A dictionary containing the generated response and the source documents.\n",
    "    \"\"\"\n",
    "    llm = get_llm()\n",
    "    retrieval_chain = RetrievalQA.from_chain_type(\n",
    "        llm=llm,\n",
    "        chain_type=\"stuff\",\n",
    "        retriever=vector_store.as_retriever(),\n",
    "        return_source_documents=True\n",
    "    )\n",
    "    response = retrieval_chain.invoke(query)\n",
    "    return response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77ff3ea108beb590",
   "metadata": {},
   "source": [
    "#### Streamlit Application (User Interface)\n",
    "\n",
    "- The Streamlit application allows users to upload a PDF document, which is then processed and stored in the Qdrant vector store.\n",
    "- Users can ask questions about the document, and the application retrieves relevant information and generates responses using the language model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c6961a41d847154",
   "metadata": {},
   "outputs": [],
   "source": [
    "import streamlit as st\n",
    "import tempfile\n",
    "\n",
    "st.title(\"Long Doc Assist\")\n",
    "uploaded_file = st.file_uploader(\"Upload your long document\", type=[\"pdf\"])\n",
    "\n",
    "if uploaded_file is not None:\n",
    "    with tempfile.NamedTemporaryFile(delete=False, suffix=\".pdf\") as tmp:\n",
    "        tmp.write(uploaded_file.read())\n",
    "        tmp_path = tmp.name\n",
    "\n",
    "    vector_store = ingest_document(tmp_path)\n",
    "    st.success(\"PDF file loaded and processed successfully!\")\n",
    "\n",
    "    if 'chat_history' not in st.session_state:\n",
    "        st.session_state.chat_history = []\n",
    "\n",
    "    # Chat input\n",
    "    query = st.chat_input(\"Ask a question about the document...\")\n",
    "\n",
    "    if query:\n",
    "        st.session_state.chat_history.append((\"user\", query))\n",
    "        with st.spinner(\"Thinking...\"):\n",
    "            answer = get_user_query_response(query, vector_store)\n",
    "        st.session_state.chat_history.append((\"ai\", answer['result']))\n",
    "\n",
    "    # Display chat messages\n",
    "    for role, msg in st.session_state.chat_history:\n",
    "        st.chat_message(role).markdown(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21e40d9b2356b3ab",
   "metadata": {},
   "source": [
    "<div style=\"page-break-after: always;\"></div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}