Tips on how to Implement Hugging Face Fashions utilizing Langchain?- Analytics Vidhya

Introduction

Massive Language Fashions have been the spine of development within the AI area. With the discharge of varied Open supply LLMs, the necessity for ChatBot-specific use instances has grown in demand. HuggingFace is the first supplier of Open Supply LLMs, the place the mannequin parameters can be found to the general public, and anybody can use them for inference. However, Langchain is a sturdy, massive language mannequin framework that helps combine AI seamlessly into your utility with the assistance of a language mannequin. By combining Langchain and HuggingFace, one can simply incorporate domain-specific ChatBots.

Studying Aims

• Perceive the necessity for open-source massive language fashions and the way HuggingFace is without doubt one of the most necessary suppliers.
• Discover three strategies to implement Massive Language Fashions with the assistance of the Langchain framework and HuggingFace open-source fashions.
• Learn to implement the HuggingFace activity pipeline with Langchain utilizing T4 GPU free of charge.
• Learn to implement fashions from HuggingFace Hub utilizing Inference API on the CPU with out downloading the mannequin parameters.
• Implementation of LlamaCPP utilizing gguf format Massive language fashions format.

This text was printed as part of the Knowledge Science Blogathon.

HuggingFace and Open Supply Massive Language fashions

HuggingFace is the cornerstone for growing AI and deep studying fashions. The in depth assortment of open-source fashions within the Transformers repository by HuggingFace makes it a go-to alternative for a lot of practitioners. Publicly accessible studying parameters characterize open-source massive language fashions, corresponding to LLaMA, Falcon, Mistral, and so forth. In distinction, closed-source massive language fashions have personal studying parameters. Using such fashions might necessitate interacting with API endpoints, as seen with GPT-4 and GPT -3.5, for example.

That is the place HuggingFace is useful. HuggingFace supplied HuggingFace Hub, a platform with over 120k fashions, 20k datasets, and 50k areas (demo AI purposes).

What’s Langchain?

With the development of Massive Language Fashions in AI, the necessity for informative ChatBots is in excessive demand. Let’s say you based a brand new Gaming firm with many person manuals and shortcut documentation. It’s essential to combine a ChatBot like ChatGPT for this firm’s knowledge. How can we obtain this?

That is the place Langchain is available in. Langchain is a sturdy Massive Language mannequin framework that integrates numerous elements corresponding to embedding, Vector Databases, LLMs, and so forth. Utilizing these elements, we are able to present exterior paperwork to the numerous language fashions and construct AI purposes seamlessly.

Set up

We have to set up the required libraries to get began with alternative ways to make use of HuggingFace on Langchain.

To make use of Langchain elements, we are able to instantly set up Langchain with the next command:

!pip set up langchain

To make use of HuggingFace Fashions and embeddings, we have to set up transformers and sentence transformers. Within the newest replace of Google Colab, you don’t want to put in transformers.

!pip set up transformers
!pip set up sentence-transformers
!pip set up bitsandbytes speed up

To run the GenAI purposes on edge, Georgi Gerganov developed LLamaCPP. LLamaCPP implements the Meta’s LLaMa structure in environment friendly C/C++.

!pip set up llama-cpp-python

Method 1: HuggingFace Pipeline

The pipelines are an ideal and simple means to make use of fashions for inference. HuggingFace offers a pipeline wrapper class that may simply combine duties like textual content era and summarization in only one line of code. This code line accommodates the calling pipeline attribute by instantiating the mannequin, tokenizer, and activity identify.

We should load the Massive Langauge mannequin and related tokenizer to implement this. Since not everybody can entry A100 or V100 GPUs, we should proceed with the Free T4 GPU. To run the massive language mannequin for inference utilizing pipeline, we are going to use orca-mini 3 billion parameter LLM with quantization configuration to cut back the mannequin dimension.

from langchain.llms.huggingface_pipeline import HuggingFacePipeline
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, 
from transformers import BitsAndBytesConfig

nf4_config = BitsAndBytesConfig(
   load_in_4bit=True,
   bnb_4bit_quant_type="nf4",
   bnb_4bit_use_double_quant=True,
   bnb_4bit_compute_dtype=torch.bfloat16
)

Within the supplied code snippet, we make the most of AutoModelForCausalLM to load the mannequin and AutoTokenizer to load the tokenizer. As soon as the mannequin and tokenizer are loaded, assign the mannequin and tokenizer to the pipeline and point out the duty to be textual content era. The pipeline additionally permits adjustment of the output sequence size by modifying max_new_tokens.

model_id = "pankajmathur/orca_mini_3b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
mannequin = AutoModelForCausalLM.from_pretrained(
                 model_id,
                 quantization_config=nf4_config
                 )
pipe = pipeline("text-generation", 
               mannequin=mannequin, 
               tokenizer=tokenizer, 
               max_new_tokens=512
               )

Good job on working the pipeline efficiently. HuggingFacePipeline wrapper class helps to combine the Transformers mannequin and Langchain. The code snippet under defines the immediate template for the orca mannequin.

hf = HuggingFacePipeline(pipeline=pipe)

question = "Who's Shah Rukh Khan?"

immediate = f"""
### System:
You might be an AI assistant that follows instruction extraordinarily nicely. 
Assist as a lot as you possibly can. Please be truthful and provides direct solutions

### Person:
{question}

### Response:
"""

response = hf.predict(immediate)
print(response)

Method 2: HuggingFace Hub utilizing Inference API

In strategy one, you might need observed that whereas utilizing the pipeline, the mannequin and tokenization obtain and cargo the weights. This strategy is likely to be time-consuming if the size of the mannequin is gigantic. Thus, the HuggingFace Hub Inference API is useful. To combine HuggingFace Hub with Langchain, one requires a HuggingFace Entry Token.

Steps to get HuggingFace Entry Token

• Log in to HuggingFace.co.
• Click on in your profile icon on the top-right nook, then select “Settings.”
• Within the left sidebar, navigate to “Entry Token.”
• Generate a brand new entry token, assigning it the “write” position.

from langchain.llms import HuggingFaceHub
import os
from getpass import getpass

os.environ["HUGGINGFACEHUB_API_TOKEN"] = getpass("HF Token:")

When you get your Entry token, use HuggingFaceHub to combine the Transformers mannequin with Langchain. On this case, we use the Zephyr, a fined-tuned mannequin on Mistral 7B.

llm = HuggingFaceHub(
    repo_id="huggingfaceh4/zephyr-7b-alpha", 
    model_kwargs={"temperature": 0.5, "max_length": 64,"max_new_tokens":512}
)

question = "What's capital of India and UAE?"

immediate = f"""
 <|system|>
You might be an AI assistant that follows instruction extraordinarily nicely.
Please be truthful and provides direct solutions
</s>
 <|person|>
 {question}
 </s>
 <|assistant|>
"""

response = llm.predict(immediate)
print(response)

Since we’re utilizing Free Inference API, there are a number of limitations on utilizing the bigger language fashions with 13B, 34B, and 70B fashions.

Method 3: LlamaCPP

LLamaCPP permits using fashions packaged as. gguf recordsdata format that runs effectively in CPU-only and combined CPU/GPU environments utilizing the llama.

To make use of LlamaCPP, we particularly want fashions whose model_path ends with gguf. You can obtain the mannequin from right here: zephyr-7b-beta.This autumn.gguf. As soon as this mannequin is downloaded, you possibly can instantly add it to your drive or every other native storage.

from langchain.llms import LlamaCpp

from google.colab import drive
drive.mount('/content material/drive')

llm_cpp = LlamaCpp(
            streaming = True,
            model_path="/content material/drive/MyDrive/LLM_Model/zephyr-7b-beta.Q4_K_M.gguf",
            n_gpu_layers=2,
            n_batch=512,
            temperature=0.75,
            top_p=1,
            verbose=True,
            n_ctx=4096
            )

The immediate template stays the identical since we’re utilizing the Zephyr mannequin.

question = "Who's Elon Musk?"

immediate = f"""
 <|system|>
You might be an AI assistant that follows instruction extraordinarily nicely.
Please be truthful and provides direct solutions
</s>
 <|person|>
 {question}
 </s>
 <|assistant|>
"""

response = llm_cpp.predict(immediate)
print(response)

Conclusion

To conclude, we efficiently carried out HuggingFace open-source fashions with Langchain. Utilizing these approaches, one can simply keep away from paying OpenAI API credit. This information primarily centered on utilizing the Open Supply LLMs, one main RAG pipeline element.

Key Takeaways

• Utilizing HuggingFace’s Transformers pipeline, one can simply decide any top-performing Massive Language fashions, Llama2 70B, Falcon 180 B, or Mistral 7B. The inference script is lower than 5 traces of code.
• As not all can afford to make use of A100 or V100 GPUs, HuggingFace offers Free Inference API (Entry Token) to implement a number of fashions from HuggingFace Hub. Probably the most most well-liked mannequin on this case is the 7B mannequin.
• LLamaCPP is used when it’s worthwhile to run Massive Language fashions on the CPU. At the moment, LlamaCPP is just supported with gguf mannequin recordsdata.
• It is strongly recommended to comply with the immediate template to run the predict() technique on the person question.

Reference

Often Requested Questions

The media proven on this article just isn’t owned by Analytics Vidhya and is used on the Writer’s discretion.

Associated