Part 8: Creating an End-to-End Document Q&A Chatbot Using Google Gamma and Groq

Introduction

In this blog post, we will guide you through the creation of an end-to-end document Q&A chatbot using Google's open-source model Gamma and Groq, a powerful inferencing engine. Gamma is a family of lightweight, state-of-the-art models built from the same research and technology used to create the Gemini models. We will also leverage Groq for fast inferencing. Let's dive into the project step-by-step.

Agenda

Here's what we will cover:

Introduction to Gamma and Groq
Setting Up Your Environment
Creating the Requirements File
Writing the Code Step by Step
Running the Application
Testing the Chatbot
Conclusion

Introduction to Gamma and Groq

Gamma is an open-source model developed by Google, similar to other open-source models like Lambda. Gamma has several variants, including:

Gamma: The base model.
Code Gamma: Optimized for code assistance.
Gamma Vision: A vision-language model.
Recurrent Gamma: For recurrent tasks.

Groq is an inferencing engine designed to provide faster inferencing for AI applications. It uses an LPU (Language Processing Unit) that is more efficient than traditional GPUs for language processing tasks.

Setting Up Your Environment

Before we get started, ensure you have the following prerequisites:

Python Version: Python 3.12 or higher.
API Keys: Make sure you have your Groq and Google API keys ready.

Step 1: Create a Virtual Environment

Let's start by setting up a virtual environment to manage our dependencies and keep our project organized.

conda create -p venv python=3.12
conda activate venv/

Step 2: Install Required Packages

Create a requirements.txt file with the following content:

faiss-cpu
groq
langchain-groq
PyPDF2
langchain_google_genai
langchain
streamlit
langchain_community
python-dotenv
pypdf
google-cloud-aiplatform>=1.38

Install the packages by running:

pip install -r requirements.txt

Writing the Code

Create a Python Script (main.py):

import streamlit as st
import os
from langchain_groq import ChatGroq
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains import create_retrieval_chain
from langchain_community.vectorstores import FAISS
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain_google_genai import GoogleGenerativeAIEmbeddings
from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()
groq_api_key = os.getenv('GROQ_API_KEY')
os.environ["GOOGLE_API_KEY"] = os.getenv("GOOGLE_API_KEY")

st.title("Gamma Model Document Q&A")

# Initialize the Language Model (LLM)
llm = ChatGroq(groq_api_key=groq_api_key,
               model_name="gamma-7b")

# Set up the prompt template
prompt = ChatPromptTemplate.from_template(
    """
    Answer the questions based on the provided context only.
    Please provide the most accurate response based on the question.
    <context>
    {context}
    <context>
    Questions:{input}
    """
)

# Function to create vector embeddings
def vector_embedding():
    if "vectors" not in st.session_state:
        st.session_state.embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001")
        st.session_state.loader = PyPDFDirectoryLoader("./us_census")  # Data Ingestion
        st.session_state.docs = st.session_state.loader.load()  # Document Loading
        st.session_state.text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)  # Chunk Creation
        st.session_state.final_documents = st.session_state.text_splitter.split_documents(st.session_state.docs[:20])  # Splitting
        st.session_state.vectors = FAISS.from_documents(st.session_state.final_documents, st.session_state.embeddings)  # Vector Embeddings

# Text input for the user's question
prompt1 = st.text_input("Enter Your Question From Documents")

# Button to create vector embeddings
if st.button("Documents Embedding"):
    vector_embedding()
    st.write("Vector Store DB Is Ready")

import time

# If a question is entered, process it
if prompt1:
    document_chain = create_stuff_documents_chain(llm, prompt)
    retriever = st.session_state.vectors.as_retriever()
    retrieval_chain = create_retrieval_chain(retriever, document_chain)
    start = time.process_time()
    response = retrieval_chain.invoke({'input': prompt1})
    print("Response time :", time.process_time() - start)
    st.write(response['answer'])

    # Display the document similarity search results
    with st.expander("Document Similarity Search"):
        for i, doc in enumerate(response["context"]):
            st.write(doc.page_content)
            st.write("--------------------------------")

Running the Application

To run your Streamlit app, open your terminal and execute:

streamlit run main.py

Testing the Chatbot

Once the app is running, open the Streamlit interface in your web browser. Upload multiple PDF files and type a question in the input box. For example:

User: What is health insurance coverage? Bot: Health insurance coverage refers to an annual average of current comprehensive health insurance coverage status.
User: What is the difference in the uninsured rate by state in 2022? Bot: In 2022, the uninsured rate was as low as 2.4%.

Conclusion

Congratulations! You've successfully built a document Q&A chatbot using Google's Gamma model and Groq's inferencing engine. This project showcases the power and efficiency of combining open-source models with advanced inferencing engines to create practical AI applications. Keep experimenting with different models and embeddings to explore the vast potential of AI technologies.

Happy coding!