CRC Open WebUI User Guide

1. CRC Open WebUI GenAI Platform

1.1. Signing up on the CRC Open WebUI GenAI Platform

The Center for Research Computing (CRC) launched a GenAI platform in December 2024 for faculty, students and staff of the ND community.

This platform supports many large language models (LLMs), utilizing systems resources, such as, servers, memory, GPUs; software resources such as, stable OS, frameworks, libraries; network resources including wide bandwidth, firewalls and other resources.

You can sign up on the CRC Hosted Open WebUI Platform (on campus or VPN, Notre Dame Google login required).

1.2. About the CRC Open WebUI GenAI Platform

The CRC GenAI integrated platform leverages the Open WebUI application. Open WebUI (OWUI) is an extensible, feature-rich, and user-friendly AI platform that supports various Large Language Model (LLM) runners like Ollama and OpenAI-compatible APIs, with a built-in inference engine for Retrieval-Augmented Generation (RAG). Refer to the Open WebUI documentation for more information.

Users can have certain features enabled for them individually, or preferrably as groups, on the platform, if they request.

1.3. About the CRC Open WebUI User Guide

Why this Guide?

Considering the queries received on issues and challenges in accessing the CRC Open WebUI through an API or using the CRC Open WebUI programmatically, we prepared this guide. The guide is comprehensive even to users with little programming experience.

Brief Description of this User Guide

The following Sections comprise a comprehensive guide on using your Open WebUI API key to access models on the CRC hosted Open WebUI GenAI platform, from the command prompt or a suitable integrated development environment (IDE). The examples are written in two programming languages: shell (Sections 2 to 5) and python (Sections 6 through 9).

The CLI guide is intended for users that need more control in using the models, for. e.g., accessing other keys in a chat response, other than the content.

These codes can be run on IDEs such as Google Colab, Jupyter Lab, Visual Studio Code, etc.

More details of interacting with models via API access to Open WebUI can be found here:

2. Accessing Open WebUI with an API

2.1. API Key Access and Storage

The steps in obtaining your API key from the GUI are as follows:

Step 1: Obtain your API key from Settings > Account on the CRC Hosted Open WebUI application

Generate / fetch your Open WebUI API

Step 2: Create an environmental file on your main project directory with the following content, and append your Open WebUI API key. The filename must start with a dot.

Filename: .env

# .env
 MY_OPEN_WEBUI_API_KEY="sk-…”

Step 3: Include the environmental file as follows from the command line:

source .env

Step 4: Curl at the url path and list available models as follows from the command line:

Note

Beware of invisible extra spaces in your commands which can cause errors in your shell commands.

curl -s -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/models | jq '.data[].name'

You may have to install the tool – ‘jq’ if required. You can do a brew install on mac.

Note

brew install jq

3. API Use in Shell

3.1. Querying a model

The following is a code block for chatting with a model

curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "codellama:70b",
    "messages": [
    {
        "role": "user",
        "content": "write python codes for calculating the mean squared error of two vector inputs of numbers"
    }
    ]
}'

You could also use variables as follows:

chosen_model="phi4:latest"
my_query="Narrate a brief history of Canada"
my_response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "'"${chosen_model}"'",
    "messages": [
    {
        "role": "user",
        "content": "'"${my_query}"'"
    }
    ]
}')

Note

To avoid errors, the model must exist in your models’ list, and the name must be correct.

Print out your response on the terminal

echo $my_response

4. Document Uploads and File Ids Retrieval from the Command Line (CLI)

The process of uploading your documents converts your documents of any type into JSON format. If you choose not to upload your documents but rather do Retrieval-Augmented Generation (RAG) directly with your documents (either locally or remotely), then you will need to convert your documents to a format acceptable to the Open WebUI API. See section 10 on more about this.

It is however, recommended that you upload your files for RAG.

4.1. Directory for RAG documents

It makes sense to create a folder for your documents for RAG.

Step 1: Create a directory named for e.g. ‘myragdocuments’

mkdir myragdocuments

Step 2: Place your files intended for RAG in the folder - ‘myragdocuments’.

Step 3: Include your API key

source .env

4.2. Single document upload

To upload one document to a given URL do the following:

curl -X POST -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" -H "Accept: application/json" \
-F "file=@myragdocuments/firstfile.docx" \
https://openwebui.crc.nd.edu/api/v1/files/

Note

Ensure you overwrite all instances of the filename ‘firstfile.docx’ with your filename

4.3. Multiple documents upload

To upload multiple documents to a given URL do the following:

Step 1: Save the following to a file named ‘upload_files.sh’

File: ‘upload_files.sh’

#!/bin/bash

# API key
source .env

# Directory containing the files to upload
DIRECTORY="myragdocuments"

# API endpoint
API_URL="https://openwebui.crc.nd.edu/api/v1/files/"

# Loop through each file in the directory and upload it
for file in "$DIRECTORY"/*; do
  if [ -f "$file" ]; then
    echo "Uploading $file..."
    curl -X POST -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" -H "Accept: application/json" \
    -F "file=@$file" $API_URL
  fi
done

Step 2: Make the file executable:

chmod +x upload_files.sh

Step 3: Run the file (must be preceded by a dot as shown)

./upload_files.sh

4.4. File ids Retrieval from the CLI

The files you uploaded are assigned file ids. You can retrieve the file ids and print them to the screen.

Step 1: Retrieve your file ids and store them in a variable

myfile_ids=$(curl -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/v1/files/ | jq -r '.[].id')

Step 2: Convert the file IDs to an array and print them to the screen

myfile_ids_array=($myfile_ids)
echo "Fetched file IDs: ${myfile_ids_array[@]}"

5. Retrieval Augmented Generation (RAG) from Shell

5.1. RAG using file uploads

An LLM, your query and your files are required for RAG. In these examples, you will retrieve your uploaded files from a given URL.

5.1.1. Choosing a model

Step 1: Curl at the url path and list available models as follows:

curl -s -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/models | jq '.data[].name'

Step 2:

You can either define your model and query right in the terminal or save them to a file. In this example, we are saving the model and query definitions to a file. If you choose to define your model and query right in the terminal, please remember to edit the file ‘do_rag.sh’ in Section 5.1.2.

If you prefer to save your model and query definitions to a file, then copy a model of your choice and overwrite “your_chosen_model”. Overwrite “your_query” with your preferred query in the following:

# chosen_model_and_query.sh
chosen_model="your_chosen_model"
query="Explain the concepts in this document."

Step 3: Create a file named ‘model_and_query.sh’ with the modified content above

Step 4 Make the file executable:

chmod +x model_and_query.sh

5.1.2. RAG with all documents IDs

The following shell block of codes loads your API key, your model and query definitions, fetches all your file ids, and does RAG on your file contents using your query and chosen model. It saves the output to a file named ‘myragoutput.txt’.

You can create a shell file and save the codes to the file, then run the file.

Step 1 Save the shell codes to a file named ‘do_rag.sh’

File: do_rag.sh

#!/bin/bash

# API key
source .env

# model choice and query
source model_and_query.sh

myfile_ids=$(curl -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/v1/files/ | jq -r '.[].id')
myfile_ids_array=($myfile_ids)
chosen_model="phi3:latest"
output_file="myragoutput.txt"

files_json=""
for file_id in "${myfile_ids_array[@]}"
do
    files_json+="{\"type\": \"file\", \"id\": \"${file_id}\"},"
done

# Remove trailing comma
files_json=${files_json%,}

response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "'"${chosen_model}"'",
    "messages": [
        {"role": "user", "content": "'"${your_query}"'"}
        ],
    "files": [
        '"${files_json}"'
        ]
    }')

echo "$response" > "$output_file"

Step 2 Make the file executable:

chmod +x do_rag.sh

Step 3 Run the file (must be preceded by a dot as shown)

./do_rag.sh

Step 4 Open the file ‘myragoutput.txt’ for your results.

cat myragoutput.txt

5.1.3. RAG with select document IDs

The following shell codes loads your API key, your model and query definitions, and does RAG on your specific file contents using your query and chosen model. It saves the output to a file named ‘myoutput.txt’.

File: select_ids_rag.sh

#!/bin/bash

your_query="Explain the concepts in these documents."
selected_file_ids=("file_id_1" "file_id_2" "file_id_3") # Add as many file IDs as needed

# API key
source .env

# model choice and query
source model_and_query.sh

output_file="myoutput.txt"

files_json=""
for file_id in "${selected_file_ids[@]}"
do
    files_json+="{\"type\": \"file\", \"id\": \"${file_id}\"},"
done

# Remove trailing comma
files_json=${files_json%,}

response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${my_api_key}" \
-H "Content-Type: application/json" \
-d '{
"model": "'"${chosen_model}"'",
"messages": [
    {"role": "user", "content": "'"${your_query}"'"}
    ],
"files": [
    '"${files_json}"'
    ]
}')

echo "$response" > "$output_file"

6. API Use in Python

6.1. Required Libraries and Imports

Certain libraries are required for the API use in Python environment. The following guide prepares the user to make API calls from the python environment.

Step 1 Ensure that your API key is already stored in your environmental file. Refer to the Section here on this if you have not done so.

Ensure that the dot_env and requests libraries are installed. You can install them using pip if required:

pip install python-dotenv

pip install requests

Step 2 Import your libraries

import os
import requests
import json
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

6.2. Models’ Listing in Python

It is helpful to list models available to you before using them. Models can be listed in python by doing the following:

Step 1

Create a new file named listmodels.py and copy the following python codes to it.

File: listmodels.py

# List available models
import os
import requests
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Check API key availability and validity
def check_api_key(api_key):
    if not api_key:
        print("API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable.")
        return False
    return True

# List available models
def list_models():
    if not check_api_key(api_key):
        return

    url = "https://openwebui.crc.nd.edu/api/models"
    headers = {"Authorization": f"Bearer {api_key}"}
    response = requests.get(url, headers=headers)

    if response.status_code == 200:
        models = response.json().get('data', [])
        for model in models:
            print(model['name'])
    elif response.status_code == 401:
        print("Invalid API key. Please check your API key and try again.")
    else:
        print("Failed to retrieve models. Status code:", response.status_code)

if __name__ == "__main__":
    list_models()

Step 2

Call the function from python

from listmodels import *
list_models()

Or run the file from the terminal

python listmodels.py

Or run from Jupyter notebook, Google Colab

!python listmodels.py

7. Chatting with Models in Python

7.1. Function definition

The following comprise a basic python code block for querying a model.

# Interact with the model
def interact_with_model(chosen_model, my_query):
    url = "https://openwebui.crc.nd.edu/api/chat/completions"
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    payload = {
        "model": chosen_model,
        "messages": [{"role": "user", "content": my_query}],
    }
    response = requests.post(url, headers=headers, data=json.dumps(payload))
    if response.status_code == 200:
        print(response.json())
    else:
        print("Failed to interact with the model")

7.2. Function calling

The function can be called as follows:

chosen_model = "phi4:latest"
my_query = "Narrate a brief history of America"
interact_with_model(chosen_model, my_query)

To avoid errors, the model must exist in your models’ list, and the name must be correct.

8. Document Uploads in Python

The process of uploading your documents converts your documents of any type into JSON format. If you choose not to upload your documents but rather do Retrieval-Augmented Generation (RAG) directly with your documents (either locally or remotely), then you will need to convert your documents to a format acceptable to the Open WebUI API. See section 10 on more about this.

It is however, recommended that you upload your files for RAG.

8.1. Directory for RAG documents

It is good practise to create a folder for your documents for RAG.

Step 1: Create a directory named for e.g. ‘myragdocuments’

Step 2: Place your files intended for RAG in the folder - ‘myragdocuments’.

8.2. Multiple documents upload

The following python code block uploads documents from your folder named ‘‘myragdocuments’’ to api_url “https://openwebui.crc.nd.edu/api/v1/files/” provided in the codes.

Step 1: Save the following to a file named ‘uploadfiles.py’

File: ‘uploadfiles.py’

import os
import requests
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Directory containing the files to upload
directory = "myragdocuments" # default directory

# API endpoint
api_url = "https://openwebui.crc.nd.edu/api/v1/files/"


# Loop through each file in the directory and upload it
def upload_files(directory):
    if not os.path.exists(directory):
        print(f"Directory '{directory}' does not exist.")
        return

    files_found = False
    for file_name in os.listdir(directory):
        file_path = os.path.join(directory, file_name)
        if os.path.isfile(file_path):
            files_found = True
            print(f"Uploading {file_path}...")
            with open(file_path, "rb") as file:
                response = requests.post(
                    api_url,
                    headers={
                        "Authorization": f"Bearer {api_key}",
                        "Accept": "application/json",
                    },
                    files={"file": file},
                )
                if response.status_code == 200:
                    print(f"Uploaded {file_path} successfully!")
                else:
                    print(
                        f"Failed to upload {file_path}. Status code: {response.status_code}"
                    )

    if not files_found:
        print(f"No files found in the directory '{directory}'.")


if __name__ == "__main__":
    upload_files(directory)

Step 2

Call the main function from python

from uploadfiles import *
directory = "myragdocuments"
upload_files(directory)

Or run the file from the terminal

python uploadfiles.py

Or run from Jupyter notebook, Google Colab

!python uploadfiles.py

9. RAG using file uploads in Python

An LLM, your query and your files are required for RAG. In these examples, you will retrieve your uploaded files from a given URL.

The files you uploaded are automatically asigned file ids. During RAG the file Ids will be retrieved in the python codes in ‘do_rag.py’.

9.1. Choosing a model

list your models as shown in the Section 6.2. here.

9.2. RAG using all file Ids

The following python block of codes loads your API key, your model and query definitions, fetches all your file ids, and does RAG on your file contents using your query and chosen model. It saves the output to a file named ‘myragoutput.txt and also returns a JSON dictionary object when the main function is called.

When you call the python file you should pass your chosen model and query as arguments.

Step 3: Create a file named “do_rag.py” and paste the following python codes

import os
import requests
import json
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Define default variables for model and query
chosen_model = "phi4:latest" # default
your_query = "Summarize each document in less than four words." # default


# Check API key availability and validity
def check_api_key(api_key):
    if not api_key:
        print(
            "API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable."
        )
        return False

    # Test the API key with a simple request
    test_url = "https://openwebui.crc.nd.edu/api/v1/files/"
    headers = {"Authorization": f"Bearer {api_key}"}
    response = requests.get(test_url, headers=headers)

    if response.status_code == 200:
        return True
    elif response.status_code == 401:
        print("Invalid API key. Please check your API key and try again.")
        return False
    else:
        print(f"Failed to verify API key. Status code: {response.status_code}")
        return False


# Retrieve file IDs
def get_file_ids(api_key):
    url = "https://openwebui.crc.nd.edu/api/v1/files/"
    headers = {"Authorization": f"Bearer {api_key}"}
    response = requests.get(url, headers=headers)

    if response.status_code == 200:
        file_ids = [file["id"] for file in response.json()]
        print(file_ids)
        return file_ids
    else:
        print(f"Failed to retrieve files. Status code: {response.status_code}")
    return []


# Create JSON for files
def create_files_json(file_ids):
    files_json = [{"type": "file", "id": file_id} for file_id in file_ids]
    return files_json


# Upload files
def get_response(
    api_key, chosen_model, your_query, files_json, output_file="myragoutput.txt"
):
    url = "https://openwebui.crc.nd.edu/api/chat/completions"
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    data = {
        "model": chosen_model,
        "messages": [{"role": "user", "content": your_query}],
        "files": files_json,
    }

    response = requests.post(url, headers=headers, data=json.dumps(data))
    response_text = response.text
    print(response_text)

    with open(output_file, "w") as file:
        file.write(response_text)


    # Return the response as a dictionary
    try:
        response_dict = response.json()  # Convert response text to dictionary
        return response_dict
    except json.JSONDecodeError:
        print("Failed to decode response as JSON.")
        return None


# Main function to call rag
def rag(chosen_model, your_query, output_file="myragoutput.txt"):
    if check_api_key(api_key):
        file_ids = get_file_ids(api_key)
        if file_ids:
            files_json = create_files_json(file_ids)
            response_dict = get_response(api_key, chosen_model, your_query, files_json, output_file)
            return response_dict  # Return the dictionary for further use
        else:
            print("No files found.")
            return None
    else:
        print("API key check failed.")
        return None


# Example calls to the rag function
if __name__ == "__main__":
    rag(chosen_model, your_query)

Step 4 Call the main function and pass your chosen model and query as arguments. Assign to a variable, ‘my_dict_response’

Option A:

From Python

from do_rag import *
new_model = "mistral-large:latest"
new_query = "Summarize each document in less than ten words."
my_dict_response = rag(new_model, new_query)

The contents of the dictionary can be accessed as desired for example:

print(my_dict_response.keys())  # and further
print(print(my_dict_response['choices'][0]))
print(my_dict_response['choices'][0]['message']['content'])

Or you can display in markdown

from IPython.display import Markdown, display

my_dict_response = rag(new_model, new_query)
# Access the content field
content = my_dict_response['choices'][0]['message']['content']

# Display the content as Markdown
display(Markdown(content))

Option B:

Edit the model choice and query in the python file and run from the terminal

python do_rag.py

10. Preprocessing Documents for RAG

10.1 Documents on your local or remote folders

The process of uploading your documents converts your documents of any type into JSON format. If you choose not to upload your documents but rather do Retrieval-Augmented Generation (RAG) directly with your documents (either locally or remotely), then you will need to convert your documents to a format the Open webUI API can handle.

For best results, it is recommended that you upload your files. See section 4 on uploading files from shell and section 8 on uploading files in Python.

10.2. Preprocessing Documents in Shell

Step 1

If preprocessing an MS Word document, then you need to have docx2txt installed. Do a brew install on Mac OS

brew install docx2txt

Step 2

Convert your document to txt

docx2txt myragdocuments/firstfile.docx > myragdocuments/firstfile.txt

Step 3

clear your document of white spaces while assigning the contents to a variable

encoded_content=$(cat myragdocuments/firstfile.txt | jq -Rs '.')

Step 4

Your modified codes for your local or remote documents not uploaded to the api/v1/files will then be:

chosen_model="phi4:latest"
my_query="Summarize the contents ..."
newresponse=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
-H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
    "model": "'"${chosen_model}"'",
    "messages": [
        {"role": "user", "content": "'"${my_query}"'"}
    ],
    "files": [
        {"name": "firstfile.txt", "content": "'"${encoded_content}"'"}
    ]
}')

Step 5

Print your response

echo $newresponse

10.3. Preprocessing Documents in Python

You will need to install and import libraries for converting your documents to an acceptable JSON format.

These libraries will help pre-process your documents into a format that the API can handle. Since the Open WebUI API uses JSON payloads for communication.

To extract content from a Microsoft office file, or PDF file, you need to install python-docx and PyPDF2 or pdfplumber, respectively.

Note

# Install python-docx pip install python-docx pip install PyPDF2

The following python codes does retrieval augmented generation with Microsoft or PDF files locally stored in a folder named ‘myragdocuments’. It uses the python library named python-docx to all convert MS word files in the myragdocuments and uses PyPDF2 to convert all PDF files in the same myragdocuments folder to JSON format usable to the Open WebUI API.

Step 1

Save the following to new python file ‘process_dorag.py’

filename: process_dorag.py

import os
import json
from docx import Document
import PyPDF2
from dotenv import load_dotenv

# Load the API key from the .env file
load_dotenv()
api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

# Define default variables for model and query
chosen_model = "phi4:latest"  # default
your_query = "Summarize the contents ..."  # default


# Check API key availability and validity
def check_api_key(api_key):
    if not api_key:
        print("API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable.")
        return False
    return True  # Skip the online check


# Extract text from a Word document
def extract_text_from_docx(file_path):
    doc = Document(file_path)
    return "\n".join(paragraph.text for paragraph in doc.paragraphs)


# Extract text from a PDF document
def extract_text_from_pdf(file_path):
    text = ""
    with open(file_path, "rb") as file:
        reader = PyPDF2.PdfReader(file)
        text = "\n".join(page.extract_text() for page in reader.pages if page.extract_text())
    return text


# Create JSON from local files
def create_files_json(folder_path):
    files_json = []
    for file_name in os.listdir(folder_path):
        file_path = os.path.join(folder_path, file_name)
        if file_name.endswith(".docx"):
            content = extract_text_from_docx(file_path)
            files_json.append({"type": "file", "name": file_name, "content": content})
        elif file_name.endswith(".pdf"):
            content = extract_text_from_pdf(file_path)
            files_json.append({"type": "file", "name": file_name, "content": content})
    return files_json


# Send RAG request to the Open WebUI API
def get_response(api_key, chosen_model, your_query, files_json, output_file="myragoutput.txt"):
    url = "https://openwebui.crc.nd.edu/api/chat/completions"
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    data = {
        "model": chosen_model,
        "messages": [{"role": "user", "content": your_query}],
        "files": files_json,
    }

    response = requests.post(url, headers=headers, data=json.dumps(data))
    response_text = response.text
    print(response_text)

    with open(output_file, "w") as file:
        file.write(response_text)



# Main function to perform RAG
def rag(folder_path, chosen_model, your_query, output_file="myragoutput.txt"):
    if check_api_key(api_key):
        files_json = create_files_json(folder_path)
        if files_json:
            get_response(api_key, chosen_model, your_query, files_json, output_file)
        else:
            print("No valid files found in the folder.")


# Example call to the RAG function
if __name__ == "__main__":
    folder_path = "myragdocuments"  # Folder containing the files
    rag(folder_path, chosen_model, your_query)

Step 2

Call the main function

from process_dorag import *
new_model = "llama3.3:latest"
new_query = "Please summarize. "
directory = "myragdocuments"
rag(directory, new_model, new_query)