RAG Powered Doc QnA & Semantic Caching with Gemini AI

Subsequent, let’s construct our semantic cache service for storing LLM responses and utilizing them as responses for comparable queries. Beneath is the code for the CustomGPTCache instrument:

from typing import Record
import config as cfg

from langchain.schema import Doc
from langchain.vectorstores.deeplake import DeepLake
from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings


class CustomGPTCache:
    def __init__(self) -> None:
        # Initialize the embeddings mannequin and cache vector retailer
        self.embeddings = SentenceTransformerEmbeddings(
            model_name="all-MiniLM-L12-v2"
        )
        self.response_cache_store = DeepLake(
            dataset_path="database/cache_vectorstore",
            embedding=self.embeddings,
            read_only=False,
            num_workers=4,
            verbose=False,
        )

    def cache_query_response(self, question: str, response: str):
        # Create a Doc object utilizing question because the content material and it is 
        # response as metadata
        doc = Doc(
            page_content=question,
            metadata={"response": response},
        )

        # Insert the Doc object into cache vectorstore
        _ = self.response_cache_store.add_documents(paperwork=[doc])

    def find_similar_query_response(self, question: str, threshold: int):
        strive:
            # Discover comparable question based mostly on the enter question
            sim_response = self.response_cache_store.similarity_search_with_score(
                question=question, okay=1
            )

            # Return the response from the fetched entry if it is rating is extra 
            # than threshold
            return [
                {
                    "response": res[0].metadata["response"],
                }
                for res in sim_response
                if res[1] > threshold
            ]
        besides Exception as e:
            elevate Exception(e)

Let’s stroll by how we’re constructing the cache system. We’re utilizing the all-MiniLM-L12-v2 mannequin from Sentence Transformers for creating embeddings for the cache. The motivation for this method was that: for embeddings of upper dimension, we get a better vary of similarity rating between dissimilar queries which ends up in mis-triggering of the cache.

From intensive analysis, I’ve noticed that utilizing embeddings of decrease dimensions, say 256, we get a wider vary of similarity scores between 0.3 to 1.0. Coming again to the code, we initialize the embeddings utilizing Langchain’s SentenceTransformerEmbeddings by specifying the mannequin title. Subsequent, we initialize the Deeplake vector retailer the place the question and responses will likely be saved.

Word that when initializing cache vector retailer we don’t set the overwrite parameter to True as that will be counter-intuitive. We’ll add two strategies to the CustomGPTCache class: cache_query_response and find_similar_query_response. Let’s focus on about them.

• cache_query_response: This methodology is known as within the QAChain when an LLM response is generated. A Doc object is created with the question within the page_content and the response within the metadata subject. Then, that Doc object is added to the vector retailer.
• find_similar_query_response: This methodology is known as within the QAChain for each run. It returns a listing of dictionaries containing responses. The consumer question is handed into this methodology for each run. It makes use of the question to do a similarity search on the cache vector retailer to search out probably the most comparable question that’s cached. We solely fetch one comparable entry from the vector retailer. Then we create a listing of these responses and put a threshold on the similarity rating to forestall returning irrelevant responses.

Step 4: Doc Questions and Answering Utilizing QAChain

Lastly, we’ll have a look at how QAChain makes use of the cache vector retailer and docstore to generate responses to consumer queries. We initialize Gemini embeddings and Gemini mannequin within the class constructor utilizing the required parameters like mannequin title, API key, and so forth. We additionally initialize the cache right here which will likely be used within the different strategies. Then, we’ll add two strategies to the QAChain class for dealing with consumer queries: generate_response and ask_question.

• generate_response: This methodology handles the LLM response a part of the QAChain. It merely takes a consumer question and generates a response for the question utilizing Langchain’s RetrievalQA Chain. After the response is generated, it’s cached utilizing the cache_query_response methodology of CustomGPTCache, and the response is returned.
• 2. ask_question: This methodology is known as for each consumer response. First, the find_similar_query_response methodology of CustomGPTCache is known as utilizing the consumer question. If it returns a listing of responses, the response is returned, in any other case, the generate_response methodology is known as utilizing the consumer question. Moreover, the generate_response methodology is known as if any exception is raised from the cache aspect. Beneath is the codebase for the QAChain pipeline for reference.

import config as cfg

from src.cache import CustomGPTCache
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate
from langchain.vectorstores.deeplake import DeepLake
from langchain_google_genai import (
    GoogleGenerativeAIEmbeddings, 
    ChatGoogleGenerativeAI
)


class QAChain:
    def __init__(self) -> None:
        # Initialize Gemini Embeddings
        self.embeddings = GoogleGenerativeAIEmbeddings(
            mannequin="fashions/embedding-001",
            google_api_key=cfg.GOOGLE_API_KEY,
            task_type="retrieval_query",
        )

        # Initialize Gemini Chat mannequin
        self.mannequin = ChatGoogleGenerativeAI(
            mannequin="gemini-pro",
            temperature=0.3,
            google_api_key=cfg.GOOGLE_API_KEY,
            convert_system_message_to_human=True,
        )

        # Initialize GPT Cache
        self.cache = CustomGPTCache()
        self.text_vectorstore = None
        self.text_retriever = None

    def ask_question(self, question):
        strive:
            # Seek for comparable question response in cache
            cached_response = self.cache.find_similar_query_response(
                question=question, threshold=cfg.CACHE_THRESHOLD
            )

            # If comparable question response is current,vreturn it
            if len(cached_response) > 0:
                print("Utilizing cache")
                outcome = cached_response[0]["response"]
            # Else generate response for the question
            else:
                print("Producing response")
                outcome = self.generate_response(question=question)
        besides Exception as _:
            print("Exception raised. Producing response.")
            outcome = self.generate_response(question=question)

        return outcome

    def generate_response(self, question: str):
        # Initialize the vectorstore and retriever object
        vstore = DeepLake(
            dataset_path="database/text_vectorstore",
            embedding=self.embeddings,
            read_only=True,
            num_workers=4,
            verbose=False,
        )
        retriever = vstore.as_retriever(search_type="similarity")
        retriever.search_kwargs["distance_metric"] = "cos"
        retriever.search_kwargs["fetch_k"] = 20
        retriever.search_kwargs["k"] = 15

        # Write immediate to information the LLM to generate response
        prompt_template = """
        <YOUR PROMPT HERE>
        Context: {context}
        Query: {query}

        Reply:
        """

        PROMPT = PromptTemplate(
            template=prompt_template, input_variables=["context", "question"]
        )

        chain_type_kwargs = {"immediate": PROMPT}

        # Create Retrieval QA chain
        qa = RetrievalQA.from_chain_type(
            llm=self.mannequin,
            retriever=retriever,
            verbose=False,
            chain_type_kwargs=chain_type_kwargs,
        )

        # Run the QA chain and retailer the response in cache
        outcome = qa({"question": question})["result"]
        self.cache.cache_query_response(question=question, response=outcome)
        
        return outcome

Now that we have now your entire pipeline prepared, let’s use them to check our utility. Step one is to ingest the doc. We initialize the Ingestion object and name the ingest_document methodology utilizing the file path for the doc that’s to be saved.

from src.ingestion import Ingestion

ingestion = Ingestion()

file = "Apple 10k.pdf"

ingestion.ingest_documents(
    file=file
)

After the doc is ingested, we’ll initialize the QAChain object. Then we name the ask_question methodology utilizing the consumer question to generate a response.

from src.qachain import QAChain

qna = QAChain()

%%time

question = "What have been the highlights for 2nd quarter of FY2023?"

outcomes = qna.ask_question(
    question=question
)

print(outcomes)

# OUTPUT:
# Producing response
#         The highlights for the second quarter of FY2023 have been:
#         •	MacBook Professional 14”, MacBook Professional 16” and Mac mini; and
#         •	Second-generation HomePod.
# CPU instances: whole: 2.94 s
# Wall time: 11.4 s

%%time

question = "Second quarter highlights, FY2023."

outcomes = qna.ask_question(
    question=question
)

print(outcomes)

# OUTPUT:
# Utilizing cache
#         The highlights for the second quarter of FY2023 have been:
#         •	MacBook Professional 14”, MacBook Professional 16” and Mac mini; and
#         •	Second-generation HomePod.
# CPU instances: whole: 46.9 ms
# Wall time: 32.8 ms

Software Efficiency and Limitations

There are a number of RAG analysis libraries that can be utilized to guage the efficiency of the applying. The 2 hottest ones are RAGAS and Tonic Validate Metrics. Each supply comparable metrics like Reply similarity rating, Retrieval precision, Augmentation precision, Augmentation accuracy/relevance, and Retrieval k-recall, which assist consider the applying efficiency. Though few exceptions are dealt with within the above codebases, there are a number of circumstances the place the applying may crash. The PDFReader presently has no verify on the enter file kind which must be dealt with.

This utility is extensively reusable in a variety of enterprise use circumstances like medical, monetary, industrial, e-commerce, and so forth. In medical industries, RAG pipelines might help in shortly troubleshooting instrument failures, deciding the proper drugs for a medical situation, and so forth. RAG pipelines might help reply queries from a big corpus of economic paperwork within the blink of an eye fixed. Their fast retrieval and answering capabilities make them superb for query answering over massive datasets. RAG pipelines may also allow fast evaluation of earlier analysis developments of a subject and recommend additional analysis scope.

Whereas our present pipeline is prepared for use for any use case, there are particular limitations. Gemini professional mannequin presently has a context size of 32k which could not be appropriate for giant data bases. An alternate suggestion can be to make use of the Gemini Professional 1.5 mannequin which helps a context size of as much as 1 million. The present pipeline has no chat reminiscence, so, the mannequin is not going to have the context to earlier conversations. Langchain offers a number of reminiscence choices (Reminiscence Docs) that may be built-in into the pipeline effortlessly.

Code Reusability

• The Ingestion pipeline code could be reused to construct ingestion programs for different purposes that require doc storage and retrieval.
• The PDF Reader instrument could be repurposed for different purposes that require processing and metadata extraction from PDF paperwork.
• The CustomGPTCache code can be utilized in different tasks to implement environment friendly caching and retrieval of LLM responses or comparable use circumstances.
• The QAChain class could be tailored for different question-answering programs that use doc shops and caching mechanisms.

Conclusion

So, people, that’s how one can talk together with your paperwork by RAG-powered query answering and semantic caching utilizing Gemini Professional and Langchain!

On this article, we’ve developed a pipeline that showcases a complete method to dealing with paperwork for clever QnA. By integrating Google’s Gemini embeddings, a customized PDF reader, and semantic caching, this RAG system maintains effectivity and accuracy in offering solutions to consumer queries. Furthermore, the codebase is very adaptable scalable, and dependable for a wide range of QnA use circumstances with minimal code modifications!

Key Takeaways

• The RAG system entails constructing an Ingestion pipeline to add and retailer paperwork for question-answering.
• The Ingestion pipeline makes use of Gemini embeddings for doc embeddings and a customized PDF reader for metadata extraction.
• A customized semantic cache service is applied to retailer and retrieve LLM responses for comparable queries effectively.
• The QAChain class is answerable for producing responses to consumer queries and using the cache and vector retailer.

The media proven on this article shouldn’t be owned by Analytics Vidhya and is used on the Creator’s discretion.