How to implement a simple RAG in Frends

Retrieval-augmented generation at its simplest is a standard Frends Process with AI.

Retrieval-Augmented Generation, or RAG, is a method in AI space where a pre-trained large language model is provided with additional context and data at runtime to extend its capabilities at reasoning for the user prompt, without the need to modify or re-train the LLM each time related information changes.

At its simplest, a RAG can be created using queried data from standard databases, production systems in use, websites or other data sources, that is then provided to the LLM as part of the prompt. Even at this level this allows the AI to work on up-to-date data that relates directly to the task at hand, that the LLM was not necessary pre-trained upon to know by default.

More advanced methods of creating a RAG includes vectorizing the queried data or the whole available dataset into embeddings for the LLM to use in its native format and to allow it to find by its own the data that matches the prompt, but these require additional capabilities from the AI backend, as well as additional database for storing the embeddings.

Prerequisites

As we focus on using standard Frends Process capabilities, you will only need the Editor role in Frends to create and edit Processes. You will also need some Frends Credits available to you in order to use the AI Connector.

Creating a RAG in Frends

The basis for this guide and example is to build a Frends Process that fetches data from an external data source and answers a question presented by the input values based on the content that was queried.

Input parameters

To start with, let's create a new Process and have it use a Manual Trigger with two parameters: WebsiteUrl and Question. Website url is used to load a webpage using HTTP Request, while the Question will be used as part of the AI prompt.

As this is a generic example for what is possible, we stick to mostly user-specified functionality for this Process, with the Process providing only the framework.

The manual parameters could easily be replaced by a standard function for this Process, such as processing customer or order data, and run it on schedule or as part of an API and receive the input from the caller. We could receive for example a customer ID, that we then use to query data from a database, CRM or ERP system to receive more data for us to use in the AI prompt.

Data Retrieval

Let's start the Process by making the HTTP Request to the website. It's assumed here the webpage is publicly available, standard HTML webpage and does not require authentication, and that we can use GET method for retrieving the contents of it.

Alternatively, if we knew the target website publishes a llms.txt or llms-full.txt files as part of the proposed standard for allowing LLMs to more easily use the content, we could query for that in addition to the provided URL.

Instead of HTTP Request for a website, we also could perform here database queries, request for data from external systems like CRM or ERP, or any combination of data retrieval from other systems.

Preprocessing

With the page loaded, we should trim the HTML content to only keep the text content. This is a good idea to save some tokens from the AI processing, as the extra HTML will be included in the prompt but have no effect on the data we are interested in. If you know more specifically what content you need for your answer from the website, it's possible to filter down even further, for example to only using the largest <div> element or the <main> or <article> elements from the result.

This is the example code to filter out HTML tags and extra content:

{
    // Assume you already have the HTML content as a string
    System.String htmlContent = #result[HTTP Request].Body;

    // Step 1: Remove <script> and <style> blocks
    htmlContent = System.Text.RegularExpressions.Regex.Replace(
        htmlContent,
        "<script.*?</script>",
        "",
        System.Text.RegularExpressions.RegexOptions.Singleline | System.Text.RegularExpressions.RegexOptions.IgnoreCase
    );

    htmlContent = System.Text.RegularExpressions.Regex.Replace(
        htmlContent,
        "<style.*?</style>",
        "",
        System.Text.RegularExpressions.RegexOptions.Singleline | System.Text.RegularExpressions.RegexOptions.IgnoreCase
    );

    // Step 2: Remove all HTML tags
    System.String textOnly = System.Text.RegularExpressions.Regex.Replace(
        htmlContent,
        "<.*?>",
        ""
    );

    // Step 3: Decode HTML entities
    textOnly = System.Net.WebUtility.HtmlDecode(textOnly);

    // Step 4: Normalize whitespace
    textOnly = System.Text.RegularExpressions.Regex.Replace(textOnly, @"\s+", " ").Trim();

    return textOnly;
}

Other preprocessing steps could include data classification or extraction by AI, or mapping or reformatting the data using different techniques to make it more ingestable by the AI.

Generating answer with AI

With the additional data retrieved and preprocessed for our use, we are ready to generate an answer to the user's question. For this we use the AI Connector with a prompt that includes both the question as well as the filtered data, in addition to some framework to instruct the AI to answer in correct format.

Returning the Answer

Finally to return the answer as result from the Process, we'll use Return shape to return the Response field from AI. This finalizes the Process and we can save and run it.