Using Fine-Tuned Models with Azure AI Inference in Frends

How to use models tuned specifically for your use case.

The Frends Intelligent AI Connector allows you to integrate powerful Large Language Models (LLMs) directly into your business processes. While Frends offers a curated catalog of models, you can also connect to your own custom models hosted on Microsoft Azure.

This guide walks you through the process of fine-tuning a model in Azure AI Studio, deploying it, and connecting it to Frends using the Azure Inference option in the AI Connector.

Prerequisites

Before you begin, ensure you have the following:

• An active Microsoft Azure subscription with access to Azure AI Studio.

• Sufficient permissions to create projects, fine-tune models, and deploy endpoints in Azure.

• A Frends environment (version 6.1 or later).

Overview

The end-to-end process involves three main stages:

1. Fine-Tune a Model in Azure AI Studio: Customize a base model (like Llama or GPT) with your own data to improve its performance for a specific task.

2. Deploy the Model to an Endpoint: Make your fine-tuned model available for use by deploying it to a serverless API endpoint in Azure. This will provide you with a URL and an API key.

3. Configure the Frends AI Connector: Use the endpoint URL and API key to configure the AI Connector in your Frends Process to send prompts to your custom model.

Fine-Tune a Model in Azure AI Studio

Fine-tuning adapts a pre-trained model to your specific dataset, which can lead to significantly better results than prompt engineering alone. It allows the model to learn your domain's specific terminology, tone, and formats.

Fine-tuning your AI model will have the following benefits:

• Higher Quality Results: Achieve greater accuracy for your specific use case.

• Train on More Data: Overcome the context window limitations of prompting.

• Lower Latency & Cost: Fine-tuned models often require shorter prompts, reducing token usage and speeding up responses.

How to perform fine-tuning

The process of fine-tuning is managed entirely within Azure AI Studio. You can fine-tune a wide range of open-source and Microsoft models, such as those from Meta (Llama) and Microsoft (Phi).

The general steps include:

1. Prepare Your Dataset: Create a quality dataset with examples of the inputs and desired outputs. For initial testing, Microsoft recommends starting with 50-100 examples.

2. Select a Base Model: In the Azure AI Studio model catalog, choose a model that supports fine-tuning.

3. Run the Fine-Tuning Job: Configure and launch the fine-tuning job using your dataset. Azure offers serverless options that are fully managed and billed based on consumption.

The specific steps, supported models, and data formats for fine-tuning are subject to change. For the most current and detailed instructions, always refer to the official Microsoft Azure documentation.

Deploying the Model to an Endpoint

Once your model is fine-tuned, you must deploy it to make it accessible for inference (i.e., for making predictions). The deployment creates a secure API endpoint that your applications, including Frends, can call.

Follow the steps here to deploy your model in Azure:

1. Navigate to your project in Azure AI Studio.

2. Find your fine-tuned model under "Models".

3. Select the model and choose to deploy it. Azure provides several options, with Serverless API endpoints being a common choice for workloads with variable traffic, as they are fully managed by Azure and offer pay-as-you-go billing.

4. During the deployment process, Azure will generate the critical pieces of information you'll need for Frends:

   • Endpoint URL: The target URL for the API.

   • API Key: The secret key for authentication.

For the most up-to-date instructions on deploying models and managing endpoints and keys, please consult the official Microsoft documentation.

Configure the Frends AI Connector

With your Azure endpoint ready, you can now configure the Frends AI Connector to use it.

1. Drag an AI Connector shape onto your Frends Process canvas.

2. In the shape's parameters, navigate to the Configuration tab.

3. Set the Service type to Azure Inference. This will display the fields required for connecting to your custom model.

4. Enter your service connection details and the name of the model you have fine-tuned.

5. Configure the shape otherwise, adding a prompt for the AI and configuring the options for AI.