CRC Open WebUI GenAI Platform

1. CRC GenAI Platform

1.1. Joining the CRC GenAI Open WebUI Platform

The Center for Research Computing (CRC) launched a GenAI platform in December 2024 for faculty, students and staff of the ND community.

This platform supports many large language models (LLMs), utilizing systems resources, such as, servers, memory, GPUs; software resources such as, stable OS, frameworks, libraries; network resources including wide bandwidth, firewalls) and other resources.

You can join the CRC Hosted Open WebUI GenAI platform at this link: CRC Hosted Open WebUI Platform (on campus or VPN, Notre Dame Google login required).

1.2. About the CRC GenAI Platform

The CRC GenAI integrated platform leverages the Open WebUI application. Open WebUI (OWUI) is an extensible, feature-rich, and user-friendly AI platform that supports various Large Language Model (LLM) runners like Ollama and OpenAI-compatible APIs, with a built-in inference engine for Retrieval-Augmented Generation (RAG). Refer to the Open WebUI documentation for more information.

Users can have certain features enabled for them individually, or preferrably as groups, on the platform, if they request.

1.3. About the CRC GenAI User guide

The CRC OWUI-based GenAI platform comprises a comprehensive guide on using your Open WebUI API key to access models on the CRC hosted Open WebUI GenAI platform, from the commmand prompt.

The CLI guide is intended for users that need more control in using the models.

2. Accessing Open WebUI with an API

2.1. API Key Access and Storage

The steps in obtaining your API key from the GUI are as follows:

Step 1: Obtain your API key from Settings > Account on the CRC Hosted Open WebUI application

Generate / fetch your Open WebUI API

Step 2: Create an environmental file on your main project directory with the following content, and append your Open WebUI API key. The filename must start with a dot.

Filename: .env

# .env
 MY_OPEN_WEBUI_API_KEY="sk-…”

Step 3: Include the environmental file as follows from the command line:

source .env

Step 4: Curl at the url path and list available models as follows from the command line:

curl -s -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/models | jq '.data[].name'

You may have to install the tool – ‘jq’ if required. You can do a brew install on mac.

Note

brew install jq

3. API Use in Shell

3.1. Querying a model

The following is a code block for chatting with a model

curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "codellama:70b",
    "messages": [
    {
        "role": "user",
        "content": "write python codes for calculating the mean squared error of two vector inputs of numbers"
    }
    ]
}'

You could also use variables as follows:

chosen_model="phi4:latest"
my_query="Narrate a brief history of Canada"
curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "'"${chosen_model}"'",
    "messages": [
    {
        "role": "user",
        "content": "'"${my_query}"'"
    }
    ]
}'

To avoid errors, the model must exist in your models’ list, and the name must be correct.

4. Document Uploads and File Ids Retrieval from the Command Line (CLI)

4.1. Directory for RAG documents

It makes sense to create a folder for your documents for RAG.

Step 1: Create a directory named for e.g. ‘myragdocuments’

mkdir myragdocuments

Step 2: Place your files intended for RAG in the folder - ‘myragdocuments’.

Step 3: Include your API key

source .env

4.2. Single document upload

To upload one document to a given URL do the following:

curl -X POST -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" -H "Accept: application/json" \ -F "file=@myragdocuments/firstfile.docx" https://openwebui.crc.nd.edu/api/v1/files/

4.3. Multiple documents upload

To upload multiple documents to a given URL do the following:

Step 1: Save the following to a file named ‘upload_files.sh’

File: ‘upload_files.sh’

#!/bin/bash

# API key
source .env

# Directory containing the files to upload
DIRECTORY="myragdocuments"

# API endpoint
API_URL="https://openwebui.crc.nd.edu/api/v1/files/"

# Loop through each file in the directory and upload it
for file in "$DIRECTORY"/*; do
  if [ -f "$file" ]; then
    echo "Uploading $file..."
    curl -X POST -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" -H "Accept: application/json" \
    -F "file=@$file" $API_URL
  fi
done

Step 2: Make the file executable:

chmod +x upload_files.sh

Step 3: Run the file (must be preceded by a dot as shown)

./upload_files.sh

4.4. File ids Retrieval from the CLI

The files you upload have file ids. You can retrieve the file ids and print them to the screen.

Step 1: Retrieve your file ids and store them in a variable

myfile_ids=$(curl -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/v1/files/ | jq -r '.[].id')

Step 2: Convert the file IDs to an array and print them to the screen

myfile_ids_array=($myfile_ids)
echo "Fetched file IDs: ${myfile_ids_array[@]}"

5. Retrieval Augmented Generation (RAG) from Shell

5.1. RAG using file uploads

An LLM, your query and your files are required for RAG. In these examples, you will retrieve your uploaded files from a given URL.

5.1.1. Choosing a model

Step 1: Curl at the url path and list available models as follows:

curl -s -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/models | jq '.data[].name'

Step 2:

You can either define your model and query right in the terminal or save them to a file. In this example, we are saving the model and query definitions to a file. If you choose to define your model and query right in the terminal, please remember to edit the file ‘do_rag.sh’ in Section 5.1.2.

If you prefer to save your model and query definitions to a file, then copy a model of your choice and overwrite “your_chosen_model”. Overwrite “your_query” with your preferred query in the following:

# chosen_model_and_query.sh
chosen_model="your_chosen_model"
query="Explain the concepts in this document."

Step 3: Create a file named ‘model_and_query.sh’ with the modified content above

Step 4 Make the file executable:

chmod +x model_and_query.sh

5.1.2. RAG with all documents IDs

The following shell block of codes loads your API key, your model and query definitions, fetches all your file ids, and does RAG on your file contents using your query and chosen model. It saves the output to a file named ‘myragoutput.txt’.

You can create a shell file and save the codes to the file, then run the file.

Step 1 Save the shell codes to a file named ‘do_rag.sh’

File: do_rag.sh

#!/bin/bash

# API key
source .env

# model choice and query
source model_and_query.sh

myfile_ids=$(curl -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/v1/files/ | jq -r '.[].id')
myfile_ids_array=($myfile_ids)
chosen_model="phi3:latest"
output_file="myragoutput.txt"

files_json=""
for file_id in "${myfile_ids_array[@]}"
do
    files_json+="{\"type\": \"file\", \"id\": \"${file_id}\"},"
done

# Remove trailing comma
files_json=${files_json%,}

response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "'"${chosen_model}"'",
    "messages": [
        {"role": "user", "content": "'"${your_query}"'"}
        ],
    "files": [
        '"${files_json}"'
        ]
    }')

echo "$response" > "$output_file"

Step 2 Make the file executable:

chmod +x do_rag.sh

Step 3 Run the file (must be preceded by a dot as shown)

./do_rag.sh

Step 4 Open the file ‘myragoutput.txt’ for your results.

cat myragoutput.txt

5.1.3. RAG with select document IDs

The following shell codes loads your API key, your model and query definitions, and does RAG on your specific file contents using your query and chosen model. It saves the output to a file named ‘myoutput.txt’.

File: select_ids_rag.sh

#!/bin/bash

your_query="Explain the concepts in these documents."
selected_file_ids=("file_id_1" "file_id_2" "file_id_3") # Add as many file IDs as needed

# API key
source .env

# model choice and query
source model_and_query.sh

output_file="myoutput.txt"

files_json=""
for file_id in "${selected_file_ids[@]}"
do
    files_json+="{\"type\": \"file\", \"id\": \"${file_id}\"},"
done

# Remove trailing comma
files_json=${files_json%,}

response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${my_api_key}" \
-H "Content-Type: application/json" \
-d '{
"model": "'"${chosen_model}"'",
"messages": [
    {"role": "user", "content": "'"${your_query}"'"}
    ],
"files": [
    '"${files_json}"'
    ]
}')

echo "$response" > "$output_file"

6. API Use in Python

6.1. Required Libraries and Imports

Certain libraries are required for the API use in Python environment. The following guide prepares the user to make API calls from the python environment.

Step 1 Ensure that your API key is already stored in your environmental file. Refer to the Section `API Key Access and Storage`_ on this if you have not done so.

Ensure that the dot_env and requests libraries are installed. You can install them using pip if required:

pip install python-dotenv

pip install requests

Step 2 Import your libraries

import os
import requests
import json
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

Filename: .env

# .env
 MY_OPEN_WEBUI_API_KEY="sk-…”

6.2. Models’ Listing in Python

It is helpful to list models available to you before using them. Models can be listed in python by doing the following:

Step 1

Create a new file named listmodels.py and copy the following python codes to it.

File: listmodels.py

# List available models
import os
import requests
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Check API key availability and validity
def check_api_key(api_key):
    if not api_key:
        print("API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable.")
        return False
   return True

# List available models
def list_models():
    if not check_api_key(api_key):
        return

    url = "https://openwebui.crc.nd.edu/api/models"
    headers = {"Authorization": f"Bearer {api_key}"}
    response = requests.get(url, headers=headers)

    if response.status_code == 200:
        models = response.json().get('data', [])
       for model in models:
            print(model['name'])
    elif response.status_code == 401:
       print("Invalid API key. Please check your API key and try again.")
   else:
        print("Failed to retrieve models. Status code:", response.status_code)

if __name__ == "__main__":
    list_models()

Step 2

Call the function from python

list_models()

Or run the file from the terminal

python listmodels.py

Or run from Jupyter notebook, Google Colab

!python listmodels.py

7. Chatting with Models in Python

7.1. Function definition

The following comprise a basic python code block for querying a model.

# Interact with the model
def interact_with_model(chosen_model, my_query):
    url = "https://openwebui.crc.nd.edu/api/chat/completions"
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    payload = {
        "model": chosen_model,
        "messages": [{"role": "user", "content": my_query}],
    }
    response = requests.post(url, headers=headers, data=json.dumps(payload))
    if response.status_code == 200:
        print(response.json())
    else:
        print("Failed to interact with the model")

7.2. Function calling

The function can be called as follows:

chosen_model = "phi4:latest"
my_query = "Narrate a brief history of America"
interact_with_model(chosen_model, my_query)

To avoid errors, the model must exist in your models’ list, and the name must be correct.

8. Document Uploads in Python

8.1. Directory for RAG documents

It is good practise to create a folder for your documents for RAG.

Step 1: Create a directory named for e.g. ‘myragdocuments’

Step 2: Place your files intended for RAG in the folder - ‘myragdocuments’.

8.2. Multiple documents upload

The following python code block uploads documents from your folder named ‘‘myragdocuments’’ to api_url “https://openwebui.crc.nd.edu/api/v1/files/” provided in the codes.

Step 1: Save the following to a file named ‘uploadfiles.py’

File: ‘uploadfiles.py’

import os
import requests
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Directory containing the files to upload
directory = "myragdocuments" # default directory

# API endpoint
api_url = "https://openwebui.crc.nd.edu/api/v1/files/"


# Loop through each file in the directory and upload it
def upload_files(directory):
    if not os.path.exists(directory):
        print(f"Directory '{directory}' does not exist.")
        return

    files_found = False
    for file_name in os.listdir(directory):
        file_path = os.path.join(directory, file_name)
        if os.path.isfile(file_path):
            files_found = True
            print(f"Uploading {file_path}...")
            with open(file_path, "rb") as file:
                response = requests.post(
                    api_url,
                    headers={
                        "Authorization": f"Bearer {api_key}",
                        "Accept": "application/json",
                    },
                    files={"file": file},
                )
                if response.status_code == 200:
                    print(f"Uploaded {file_path} successfully!")
                else:
                    print(
                        f"Failed to upload {file_path}. Status code: {response.status_code}"
                    )

    if not files_found:
        print(f"No files found in the directory '{directory}'.")


if __name__ == "__main__":
    upload_files(directory)

Step 2

Call the function from python

directory = "myragdocuments"
upload_files(directory)

Or run the file from the terminal

python uploadfiles.py

Or run from Jupyter notebook, Google Colab

!python uploadfiles.py

9. RAG using file uploads in Python

An LLM, your query and your files are required for RAG. In these examples, you will retrieve your uploaded files from a given URL.

The files you uploaded have file ids. During RAG the file Ids will be retrieved in the python codes in ‘do_rag.py’.

9.1. Choosing a model

list your models as shown in the Section 6.2. here.

9.2. RAG using all file Ids

The following python block of codes loads your API key, your model and query definitions, fetches all your file ids, and does RAG on your file contents using your query and chosen model. It saves the output to a file named ‘myragoutput.txt’.

When you call the python file you should pass your chosen model and query as arguments.

Step 3: Create a file named “do_rag.py” and paste the following python codes

import os
import requests
import json
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Define default variables for model and query
chosen_model = "phi4:latest" # default
your_query = "Summarize each document in less than four words." # default


# Check API key availability and validity
def check_api_key(api_key):
    if not api_key:
    print(
        "API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable."
    )
    return False

# Test the API key with a simple request
test_url = "https://openwebui.crc.nd.edu/api/v1/files/"
headers = {"Authorization": f"Bearer {api_key}"}
response = requests.get(test_url, headers=headers)

if response.status_code == 200:
    return True
elif response.status_code == 401:
    print("Invalid API key. Please check your API key and try again.")
    return False
else:
    print(f"Failed to verify API key. Status code: {response.status_code}")
    return False


# Retrieve file IDs
def get_file_ids(api_key):
    url = "https://openwebui.crc.nd.edu/api/v1/files/"
    headers = {"Authorization": f"Bearer {api_key}"}
    response = requests.get(url, headers=headers)

    if response.status_code == 200:
        file_ids = [file["id"] for file in response.json()]
        print(file_ids)
        return file_ids
    else:
    print(f"Failed to retrieve files. Status code: {response.status_code}")
    return []


# Create JSON for files
def create_files_json(file_ids):
    files_json = [{"type": "file", "id": file_id} for file_id in file_ids]
    return files_json


# Upload files
def get_response(
    api_key, chosen_model, your_query, files_json, output_file="myragoutput.txt"
):
    url = "https://openwebui.crc.nd.edu/api/chat/completions"
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    data = {
        "model": chosen_model,
        "messages": [{"role": "user", "content": your_query}],
        "files": files_json,
    }

    response = requests.post(url, headers=headers, data=json.dumps(data))
    response_text = response.text
    print(response_text)

    with open(output_file, "w") as file:
        file.write(response_text)


# Main function to call rag
def rag(chosen_model, your_query, output_file="myragoutput.txt"):
    if check_api_key(api_key):
        file_ids = get_file_ids(api_key)
        if file_ids:
            files_json = create_files_json(file_ids)
            get_response(api_key, chosen_model, your_query, files_json, output_file)


# Example calls to the rag function
if __name__ == "__main__":
    rag(chosen_model, your_query)

Step 4 Call the function and pass your chosen model and query as arguments.

Option A:

From Python

import do_rag
new_model = "mistral-large:latest"
new_query = "What is the essence of each document?"
rag(new_model, new_query)

Option B:

Edit the model choice and query in the python file and run from the terminal

python do_rag.py