Finetuning Phi-Medium to Generate Cypher Question from Textual content

Introduction

The rise of Retrieval-Augmented Era (RAG) and Data Graphs has revolutionized how we work together with complicated knowledge units by offering a structured, interconnected illustration of data. Data Graphs, corresponding to these utilized in Neo4j, facilitate the querying and visualization of relationships inside knowledge. Nonetheless, translating pure language into structured question languages like Cypher stays a difficult process. This information goals to bridge this hole by detailing the fine-tuning of the Phi-3 Medium mannequin to generate Cypher queries from pure language inputs. By leveraging the compact but highly effective capabilities of the Phi-3 Medium mannequin, even small-scale builders can effectively convert textual content to Cypher queries, enhancing the accessibility and usefulness of Data Graphs.

Studying Goals

• Perceive the significance of Cypher Question technology from pure language for developer effectivity.
• Find out about Microsoft’s Phi 3 Medium and its function in reworking English queries into code.
• Discover Unsloth’s effectivity enhancements and reminiscence administration for Giant Language Fashions.
• Arrange the atmosphere for fine-tuning Phi 3 Medium with Unsloth effectively.
• Put together datasets suitable with Phi 3 Medium and Unsloth for efficient fine-tuning.
• Grasp fine-tuning Phi 3 Medium with particular coaching arguments utilizing SFTTrainer.

This text was printed as part of the Knowledge Science Blogathon.

What’s Phi 3 Medium?

The Phi household of Giant Language Fashions is launched by Microsoft to characterize that even small language fashions can carry out higher and could also be on par with the larger fashions. Microsoft has educated this small household of fashions with several types of datasets, thus making these fashions good at completely different duties together with entity extraction, summarization, chatbots, roleplay, and extra.

Microsoft has launched these fashions conserving in thoughts that their small measurement may help even small builders work with them, and prepare them on their very personal datasets, thus mentioning many alternative purposes. Just lately, Microsoft has introduced the third technology of the phi household known as the Phi 3 collection of Giant Language Fashions.

Within the Phi 3 collection, the context size was purchased from 4k tokens to now 128k tokens, thus permitting extra context to slot in. The Phi 3 household of fashions comes with completely different sizes ranging from the smallest 3.8 billion parameter mannequin known as the Phi 3 Mini, adopted by the Phi 3 Small which is a 7B parameter mannequin, and at last the Phi 3 Medium which is a 14 billion parameter mannequin, the one we’ll prepare on this Information. All of those fashions have a protracted context model extending the context size to 128k tokens.

Who’s Unsloth?

Developed by Daniel and Michael Han, Unsloth emerged to be one the most effective Optimized Frameworks designed to enhance the fine-tuning course of for giant language fashions (LLMs). Identified for its blazing velocity and reminiscence effectivity, Unsloth can enhance coaching speeds by as much as 30 occasions whereas lowering reminiscence utilization by a powerful 60%. All these capabilities make it the appropriate framework for builders aiming to fine-tune LLMs with accuracy and velocity.

Unsloth helps several types of {Hardware} Configs, from NVIDIA GPUs just like the Tesla T4 and H100 to AMD and Intel GPUs. It even employs complicated methodologies like clever weight upcasting, which minimizes the necessity for weight upscaling throughout QLoRA, thereby optimizing reminiscence use.

As an open-source instrument below the Apache 2.0 license, Unsloth integrates seamlessly into the fine-tuning of distinguished LLMs like Mistral 7B, Llama, and Gemma, reaching as much as a 5x enhance in fine-tuning velocity whereas concurrently lowering reminiscence utilization by 60%. Moreover, it’s suitable with various fine-tuning strategies like Flash-Consideration 2, which not solely quickens inference however even the fine-tuning course of.

Setting Creation

We’ll first create the environment. For this we’ll obtain Unsloth for Google Colab.

!pip set up "unsloth[colab] @ git+https://github.com/unslothai/unsloth.git"

Then we’ll create some default Unsloth values for coaching. These are:

from unsloth import FastLanguageModel
import torch

sequence_length_maximum = 2048
weights_data_type = None
quantize_to_4bit = True

We begin by importing the FastLanguageModel class from the Unsloth library. Then we outline some variables to be labored with all through the information:

• sequence_length_maximum: It’s the max sequence size {that a} mannequin can deal with. We give it a worth of 4096.
• weights_data_type: Right here we inform what knowledge sort the mannequin weights ought to be. We gave it None, which can auto-select the information sort.
• quantize_to_4bit: Right here, we give it a worth of True. This then tells the mannequin to load in 4 bits, in order that it may possibly simply match within the Colab GPU.

Downloading Mannequin and Creating LoRA Adaptors

Right here, we’ll begin downloading the Phi 3 Medium Mannequin. We’ll do that with the Unsloth’s FastLanguageModel class.

mannequin, tokenizer = FastLanguageModel.from_pretrained(
  model_name = "unsloth/Phi-3-medium-4k-instruct",
  max_seq_length = sequence_length_maximum,
  dtype = weights_data_type,
  load_in_4bit = quantize_to_4bit,
  token = "YOUR_HF_TOKEN"
)

Once we run the code, the output generated will be seen within the pic above. Each the Phi 3 Medium mannequin and its tokenizer will likely be downloaded to the Colab atmosphere by fetching it from the HuggingFace Repository.

We can not finetune the entire Phi 3 Medium mannequin. So we simply prepare a couple of weights of the Phi 3 Mannequin. For this, we work with LoRA (Low-Rank Adaptation), which works by coaching solely a subset of parameters. So for this, we have to create a LoRA config and get the Parameter Environment friendly Finetuned Mannequin (peft mannequin) from this LoRA config. The code for this will likely be:

mannequin = FastLanguageModel.get_peft_model(
  mannequin,
  r = 16,
  target_modules = ["q_proj", "k_proj", "down_proj", "v_proj", "o_proj", 
  "up_proj", "gate_proj"],
  lora_alpha = 16,
  bias = "none",
  lora_dropout = 0,
  random_state = 3407,
  use_gradient_checkpointing = "True",
)

• Right here “r” is the Rank of the LoRA Matrix. If now we have the next rank, then we have to prepare extra parameters, and if decrease rank, then a decrease variety of parameters. We set this to a worth of 16.
• Right here the lora_alpha is the scaling issue of the weights current within the LoRA Matrix. It’s often stored the identical as rank to get optimum outcomes.
• Dropout will randomly shut down a few of the weights within the LoRA weight matrix. We now have stored it to 0, in order that we will get a rise within the coaching velocity and it has little affect on the efficiency.
• We are able to have a bias parameter for the weights within the LoRA matrix. However setting to None will additional enhance the reminiscence effectivity and reduce the coaching time,

After working this code, the LoRA Adapters for the Phi 3 Medium will likely be created. Now we will work with this peft mannequin and finetune it with a dataset of our alternative.

Making ready the Dataset for High quality-tuning

Right here, we will likely be coaching the Phi 3 Medium Giant Language Mannequin with a dataset that may permit the mannequin to generate Cypher Queries that are crucial for querying the Data Graph Databases just like the neo4j. So for this, we’ll obtain the dataset supplied from a GitHub Repository. The command for this will likely be:

!wget https://uncooked.githubusercontent.com/neo4j-labs/text2cypher
/foremost/datasets/synthetic_gpt4turbo_demodbs/text2cypher_gpt4turbo.csv

The above command will obtain a CSV file. This CSV file incorporates the dataset that we are going to be working with to coach the Phi 3 Medium LLM. Earlier than that, we have to do some preprocessing. We’re solely taking a sure half i.e. a subset of the dataset. The code for this will likely be:

import pandas as pd
df = pd.read_csv('/content material/text2cypher_gpt4turbo.csv')
df = df[(df['database'] == 'suggestions') & 
(df['syntax_error'] == False) & (df['timeout'] == False)]
df = df[['question','cypher']]
df.rename(columns={'query': 'enter','cypher':'output'}, inplace=True)
df.reset_index(drop=True, inplace=True)

Right here, we filter the information. We want the information coming from the suggestions database. We want solely these columns which don’t have any syntax error and the place there isn’t a timeout. That is crucial as a result of we’d like the Phi 3 to offer us a syntax error-free Cypher Queries when requested.

The dataset incorporates many columns, however solely the query and the cypher column are those we’d like. And we even renamed these columns to enter and output, the place the query column is the enter and the cypher column is the output that must be generated by the Giant Language Mannequin.

Within the output pic, we will see the primary 5 rows of the dataset. It incorporates solely two columns, enter and output. The database we’re working with, for the coaching knowledge, has a schema to it.

Schema for this Database

graph_schema = """
Node properties:
- **Film**
  - `url`: STRING Instance: "https://themoviedb.org/film/862"
  - `runtime`: INTEGER Min:1, Max: 915
  - `income`: INTEGER Min: 1, Max: 2787965087
  - `price range`: INTEGER Min: 1, Max: 380000000
  - `imdbRating`: FLOAT Min: 1.6, Max: 9.6
  - `launched`: STRING Instance: "1995-11-22"
  - `international locations`: LIST Min Measurement: 1, Max Measurement: 16
  - `languages`: LIST Min Measurement: 1, Max Measurement: 19
  - `imdbVotes`: INTEGER Min: 13, Max: 1626900
  - `imdbId`: STRING Instance: "0114709"
  - `yr`: INTEGER Min: 1902, Max: 2016
  - `poster`: STRING Instance: "https://picture.tmdb.org/t/p/w440_and_h660_face/uXDf"
  - `movieId`: STRING Instance: "1"
  - `tmdbId`: STRING Instance: "862"
  - `title`: STRING Instance: "Toy Story"
- **Style**
  - `identify`: STRING Instance: "Journey"
- **Person**
  - `userId`: STRING Instance: "1"
  - `identify`: STRING Instance: "Omar Huffman"
- **Actor**
  - `url`: STRING Instance: "https://themoviedb.org/individual/1271225"
  - `bornIn`: STRING Instance: "France"
  - `bio`: STRING Instance: "​From Wikipedia, the free encyclopedia  Lillian Di"
  - `died`: DATE Instance: "1954-01-01"
  - `born`: DATE Instance: "1877-02-04"
  - `imdbId`: STRING Instance: "2083046"
  - `identify`: STRING Instance: "François Lallement"
  - `poster`: STRING Instance: "https://picture.tmdb.org/t/p/w440_and_h660_face/6DCW"
  - `tmdbId`: STRING Instance: "1271225"
- **Director**
  - `url`: STRING Instance: "https://themoviedb.org/individual/88953"
  - `bornIn`: STRING Instance: "Burchard, Nebraska, USA"
  - `bio`: STRING Instance: "Harold Lloyd has been known as the cinema’s “first m"
  - `died`: DATE Min: 1930-08-26, Max: 2976-09-29
  - `born`: DATE Min: 1861-12-08, Max: 2018-05-01
  - `imdbId`: STRING Instance: "0516001"
  - `identify`: STRING Instance: "Harold Lloyd"
  - `poster`: STRING Instance: "https://picture.tmdb.org/t/p/w440_and_h660_face/er4Z"
  - `tmdbId`: STRING Instance: "88953"
- **Particular person**
  - `url`: STRING Instance: "https://themoviedb.org/individual/1271225"
  - `bornIn`: STRING Instance: "France"
  - `bio`: STRING Instance: "​From Wikipedia, the free encyclopedia  Lillian Di"
  - `died`: DATE Instance: "1954-01-01"
  - `born`: DATE Instance: "1877-02-04"
  - `imdbId`: STRING Instance: "2083046"
  - `identify`: STRING Instance: "François Lallement"
  - `poster`: STRING Instance: "https://picture.tmdb.org/t/p/w440_and_h660_face/6DCW"
  - `tmdbId`: STRING Instance: "1271225"
Relationship properties:
- **RATED**
  - `ranking: FLOAT` Instance: "2.0"
  - `timestamp: INTEGER` Instance: "1260759108"
- **ACTED_IN**
  - `function: STRING` Instance: "Officer of the Marines (uncredited)"
- **DIRECTED**
  - `function: STRING`
The relationships:
(:Film)-[:IN_GENRE]->(:Style)
(:Person)-[:RATED]->(:Film)
(:Actor)-[:ACTED_IN]->(:Film)
(:Actor)-[:DIRECTED]->(:Film)
(:Director)-[:DIRECTED]->(:Film)
(:Director)-[:ACTED_IN]->(:Film)
(:Particular person)-[:ACTED_IN]->(:Film)
(:Particular person)-[:DIRECTED]->(:Film)
"""

The schema incorporates all of the Node properties and the Relationships between the nodes which can be offered within the suggestions graph database. Now, we’ll convert these to an instruction format, so the mannequin will solely output a cypher question solely when it has been instructed to take action. The operate for this will likely be.

immediate = """Given are the instruction under, having an enter 
that gives additional context.
### Instruction:
{}
### Enter:
{}
### Response:
{}"""
token_eos = tokenizer.eos_token
def format_prompt(columns):
    directions = f"Use the under textual content to generate a cypher question. 
    The schema is given under:n{graph_schema}"
    inps       = columns["input"]
    outs      = columns["output"]
    text_list = []
    for enter, output in zip(inps, outs):
        textual content = immediate.format(directions, enter, output) + token_eos
        text_list.append(textual content)
    return { "textual content" : texts, }

• Right here we first outline our Immediate Template. On this template, we begin by defining the instruction then adopted by the enter, and at last the output.
• Then we create a operate known as format_prompt(). This takes within the knowledge after which extracts the enter and output columns from the information. 
• Then we iterate by means of every row within the enter and output column and match them to the Immediate Template.
• Together with that, we even added the end-of-sentence token known as token_eos to the Immediate, which can inform the mannequin that the technology must be stopped.
• We lastly return the listing containing all these Prompts in a dictionary format.

This operate above will likely be handed to our dataset to create the ultimate column. The code for this will likely be:

from datasets import Dataset
dataset = Dataset.from_pandas(df)
dataset = dataset.map(format_prompt, batched = True)

• Right here, we begin by importing the Dataset class from the datasets library.
• Then we convert our dataset, which is of sort DataFrame to the Dataset sort by calling the .from_pandas() technique and passing it to the DataFrame.
• Now, we’ll map the operate that now we have created to create our last dataset for coaching.

Operating the code will create a brand new column known as “textual content”, which can include the prompts that now we have outlined within the format_prompt() operate. From the pic above, we will see that there are a complete of 700+ rows of knowledge in our dataset and there are three columns, which can be textual content, enter, and output. With this, now we have our knowledge prepared for fine-tuning.

High quality-tuning Phi 3 Medium for Text2Cypher Question

We at the moment are able to fine-tune the Phi 3 Medium on the Cypher Question dataset. On this part, we begin by creating our Coach and the corresponding Coaching Arguments that we have to prepare our mannequin on this dataset that now we have ready. The code for this  will likely be:

from trl import SFTTrainer
from transformers import TrainingArguments

coach = SFTTrainer(
    mannequin = mannequin,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "textual content",
    max_seq_length = sequence_length_maximum,
    dataset_num_proc = 2,
    packing = False,
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        num_train_epochs=1,
        learning_rate = 2e-4,
        fp16 = True,
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.02,
        lr_scheduler_type = "linear",
        output_dir = "outputs",
    ),
)

• We begin by importing the SFTTrainer from the trl library which we’ll work with to carry out the Supervised High quality Tuning.
• We even import the TrainingArguments class from the transformers library to set the coaching config for coaching the mannequin.
• Then we create an occasion of SFTTrainer with varied parameters and retailer it within the coach variable mannequin = mannequin: Tells the pre-trained mannequin to be fine-tuned.
• tokenizer = tokenizer: Tells the tokenizer related to the modeltrain_dataset = dataset: Units the dataset that now we have ready for coaching the mannequin.
• dataset_text_field = “textual content”: Signifies the sphere within the dataset that incorporates the textual content knowledge.
• max_seq_length = sequence_max_length: Right here, we offer the utmost sequence size for the mannequin.
• dataset_num_proc = 2: Variety of processes to make use of for knowledge loading.
• packing = False: Disables packing of sequences, which might velocity up coaching for brief sequences.

Whereas coaching a Giant Language Mannequin or a Deep Studying mannequin, we should set many alternative hyperparameters, which carry out the best-performing mannequin. These embody completely different parameters.

Totally different Parameters

• At a time we ship two examples to the processor, so we choose a batch measurement of two.
• We want 4 accumulation steps earlier than updating the gradients within the backward cross. So now we have set it to 4.
• We now have set the warmup steps to three, so the educational fee is not going to be in impact till three steps are accomplished.
• We need to run the coaching for the entire dataset, so gave one epoch for the coaching.
• We have to print out the metrics after each step, so we’ll log the coaching metrics just like the accuracy and the coaching loss for every step.
• The optimizer will deal with the gradients in order that they are going to attain a world minimal in order that the accuracy loss is decreased. Right here for the optimizer, we’ll go along with the Adam optimizer.
• Weight decay is required so the weights don’t go to excessive values. So gave it a decay worth of 0.02.
• The educational fee scheduler will change the educational fee whereas the coaching is going on. Right here we wish it to vary linearly so we gave it the choice known as “linear”.

We at the moment are achieved with defining our Coach and the TrainingArguments for coaching our quantized Phi 3 Medium 14Billion Giant Language Mannequin. Operating the coach.prepare() will begin the coaching. 

trainer_stats = coach.prepare()

Operating the above will begin the coaching course of. In Google Colab, working with the free T4 GPU, it takes round 1 hour and 40 minutes to undergo 1 epoch on the coaching knowledge. It has taken round 95 epochs to finish one epoch. Lastly, the coaching is accomplished.

Producing Cypher Question with Phi 3 Medium

We now have now completed coaching the mannequin. Now we’ll check the mannequin to test how nicely it generates cypher queries given a textual content.

FastLanguageModel.for_inference(mannequin)
inputs = tokenizer(
[
   prompt.format(
       f"Convert text to cypher query based on this schema: n{graph_schema}",
      "What are the top 5 movies with a runtime greater than 120 minutes"
       "",
   )
], return_tensors = "pt").to("cuda")
outputs = mannequin.generate(**inputs, max_new_tokens = 128)
print(tokenizer.decode(outputs[0], skip_special_tokens = True))

• We begin by loading the educated mannequin for inference by passing it to the for_inference() technique of the FastLanguageModel class.
• Then we name the tokenizer and provides it the enter Immediate. We work with the identical Immediate Template that now we have outlined and provides the questions “What are the highest 5 motion pictures?.
• These are then given to the mannequin to offer out the output tokens and now we have set the max new tokens to 128 and retailer the generated outcome within the output variable.
• Lastly, we decode the output tokens and print it.

We are able to see the outcomes of working this code within the above pic. We see that the Cypher Question generated by the mannequin matches the bottom fact, Cypher Question. Allow us to check with some extra examples to see the efficiency of the fine-tuned Phi 3 Medium for Cypher Question technology.

inputs = tokenizer(
[
   prompt.format(

       f"Convert text to cypher query based on this schema: n{graph_schema}",
       "Which 3 directors have the longest bios in the database?"
       "",
   )
], return_tensors = "pt").to("cuda")
outputs = mannequin.generate(**inputs, max_new_tokens = 128)
print(tokenizer.decode(outputs[0], skip_special_tokens = True))

inputs = tokenizer(
[
   prompt.format(
       f"Convert text to cypher query based on this schema: n{graph_schema}",
       "List the genres that have movies with an imdbRating less than 4.0.",
       "",
   )
], return_tensors = "pt").to("cuda")
outputs = mannequin.generate(**inputs, max_new_tokens = 128)
print(tokenizer.decode(outputs[0], skip_special_tokens = True))

We are able to see that in each the examples above, the fine-tuned Phi 3 Medium mannequin has generated the proper Cypher Question for the supplied query. Within the first instance, the Phi 3 Medium did present the appropriate reply however took barely a distinct strategy. With this, we will say that finetuning Phi 3 Medium on the Cypher Dataset has made its technology barely extra correct whereas producing Cypher Queries given a textual content.

Conclusion

This information has detailed the fine-tuning strategy of the Phi 3 Medium mannequin for producing Cypher queries from pure language inputs, aimed toward enhancing accessibility to Data Graphs like Neo4j. By way of leveraging instruments like Unsloth for environment friendly mannequin coaching and deploying strategies corresponding to LoRA adapters to optimize parameter utilization, builders can successfully translate complicated knowledge queries into structured Cypher instructions.

Key Takeaways

• Phi 3 Household of fashions developed by Microsoft offers small builders to coach these fashions on their personalised datasets for various situations.
• Unsloth, a Python library is a superb instrument for fine-tuning small language fashions which enhance the coaching speeds and reminiscence effectivity.
• Creating the atmosphere includes putting in crucial libraries and configuring parameters just like the sequence size and knowledge sort.
• Lora is a technique that permits us to coach solely a subset of the entire parameters of the Giant Language Mannequin thus permitting us to coach them on a shopper {hardware}.
• Textual content to Cypher question technology will permit builders to let Giant Language Fashions entry Graph Databases to supply extra correct responses.

Regularly Requested Questions

The media proven on this article is just not owned by Analytics Vidhya and is used on the Creator’s discretion.