.. _crcowui:

===================================
    CRC Open WebUI User Guide
===================================

1. CRC Open WebUI GenAI Platform
----------------------------------

1.1. Signing up on the CRC Open WebUI GenAI Platform
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Center for Research Computing (CRC) launched a GenAI platform in December 2024 for faculty, students and staff of the ND community. 

This platform supports many large language models (LLMs), utilizing systems resources, such as, servers, memory, GPUs; software resources such as, stable OS, frameworks, libraries; network resources including wide bandwidth, firewalls and other resources.

You can sign up on the `CRC Hosted Open WebUI Platform <https://openwebui.crc.nd.edu/>`_ (on campus or VPN, Notre Dame Google login required).


1.2. About the CRC Open WebUI GenAI Platform
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The CRC GenAI integrated platform leverages the Open WebUI application. Open WebUI (OWUI) is an extensible, feature-rich, and user-friendly AI platform that supports various Large Language Model (LLM) runners like Ollama and OpenAI-compatible APIs, with a built-in inference engine for Retrieval-Augmented Generation (RAG). Refer to the `Open WebUI documentation <https://www.openwebui.com/>`_ for more information.

Users can have certain features enabled for them individually, or preferrably as groups, on the platform, if they request.


1.3. About the CRC Open WebUI User Guide
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Why this Guide?**

Considering the queries received on issues and challenges in accessing the CRC Open WebUI through an API or using the CRC Open WebUI programmatically, we prepared this guide. The guide is comprehensive even to users with little programming experience.

**Brief Description of this User Guide**

The following Sections comprise a comprehensive guide on using your Open WebUI API key to access models on the CRC hosted Open WebUI GenAI platform, from the command prompt or a suitable integrated development environment (IDE). The examples are written in two programming languages: shell (Sections 2 to 5) and python (Sections 6 through 9).

The CLI guide is intended for users that need more control in using the models, for. e.g., accessing other keys in a chat response, other than the content.

These codes can be run on IDEs such as Google Colab, Jupyter Lab, Visual Studio Code, etc.

More details of interacting with models via API access to Open WebUI can be found `here: <https://docs.openwebui.com/getting-started/api-endpoints/>`_


2. Accessing Open WebUI with an API
----------------------------------------


.. _api_key_access:

2.1. API Key Access and Storage 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The steps in obtaining your API key from the GUI are as follows:

**Step 1:**
Obtain your API key from Settings > Account on the CRC Hosted Open WebUI application
   
.. image:: images/setting_user.png
   :alt: CRC Open WebUI User
   :scale: 70%

.. :width: 400px # commented out
.. :height: 300px

Generate / fetch your Open WebUI API 

.. image:: images/settings_api.png
   :alt: CRC Open WebUI User
   :scale: 90%


**Step 2:**
Create an environmental file on your main project directory with the following content, and append your Open WebUI API key. The filename must start with a dot.

Filename: **.env**

.. code-block:: shell

   # .env
    MY_OPEN_WEBUI_API_KEY="sk-…”


**Step 3:**
Include the environmental file as follows from the command line:

.. code-block:: shell
    
    source .env

**Step 4:**
Curl at the url path and list available models as follows from the command line:

.. code-block:: shell
    
    curl -s -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/models | jq '.data[].name'

You may have to install the tool – ‘jq’ if required. You can do a brew install on mac.

.. note::

    brew install jq



3. API Use in Shell
----------------------


3.1. Querying a model 
~~~~~~~~~~~~~~~~~~~~~~

The following is a code block for chatting with a model 

.. code-block:: shell

    curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
    -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "codellama:70b",
        "messages": [
        {
            "role": "user",
            "content": "write python codes for calculating the mean squared error of two vector inputs of numbers"
        }
        ]
    }'

You could also use variables as follows:

.. code-block:: shell
    
    chosen_model="phi4:latest"
    my_query="Narrate a brief history of Canada"
    curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
    -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "'"${chosen_model}"'",
        "messages": [
        {
            "role": "user",
            "content": "'"${my_query}"'"
        }
        ]
    }'

To avoid errors, the model must exist in your models’ list, and the name must be correct. 


4. Document Uploads and File Ids Retrieval from the Command Line (CLI)
------------------------------------------------------------------------

4.1. Directory for RAG documents 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It makes sense to create a folder for your documents for RAG.

**Step 1:**
Create a directory named for e.g. ‘myragdocuments’

.. code-block:: shell
    
    mkdir myragdocuments

**Step 2:**
Place your files intended for RAG in the folder - ‘myragdocuments’.

**Step 3:**
Include your API key

.. code-block:: shell
    
    source .env

4.2. Single document upload
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To upload one document to a given URL do the following:

.. code-block:: shell
    
    curl -X POST -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" -H "Accept: application/json" \ -F "file=@myragdocuments/firstfile.docx" https://openwebui.crc.nd.edu/api/v1/files/

4.3. Multiple documents upload
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To upload multiple documents to a given URL do the following:

**Step 1:**
Save the following to a file named ‘upload_files.sh’

File: ‘upload_files.sh’

.. code-block:: shell
    
    #!/bin/bash

    # API key
    source .env

    # Directory containing the files to upload
    DIRECTORY="myragdocuments"

    # API endpoint
    API_URL="https://openwebui.crc.nd.edu/api/v1/files/"

    # Loop through each file in the directory and upload it
    for file in "$DIRECTORY"/*; do
      if [ -f "$file" ]; then
        echo "Uploading $file..."
        curl -X POST -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" -H "Accept: application/json" \
        -F "file=@$file" $API_URL
      fi
    done


**Step 2:**
Make the file executable:

.. code-block:: shell
    
    chmod +x upload_files.sh

**Step 3:**
Run the file (must be preceded by a dot as shown)

.. code-block:: shell
    
    ./upload_files.sh


4.4. File ids Retrieval from the CLI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The files you uploaded are assigned file ids. You can retrieve the file ids and print them to the screen.

Step 1: Retrieve your file ids and store them in a variable

.. code-block:: shell
    
    myfile_ids=$(curl -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/v1/files/ | jq -r '.[].id')

Step 2: Convert the file IDs to an array and print them to the screen

.. code-block:: shell
    
    myfile_ids_array=($myfile_ids)
    echo "Fetched file IDs: ${myfile_ids_array[@]}"



5. Retrieval Augmented Generation (RAG) from Shell
----------------------------------------------------

5.1. RAG using file uploads
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

An LLM, your query and your files are required for RAG. In these examples, you will retrieve your uploaded files from a given URL.


5.1.1. Choosing a model 
***************************

Step 1: Curl at the url path and list available models as follows:

.. code-block:: shell
    
    curl -s -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/models | jq '.data[].name'

Step 2: 

You can either define your model and query right in the terminal or save them to a file. In this example, we are saving the model and query definitions to a file. If you choose to define your model and query right in the terminal, please remember to edit the file 'do_rag.sh' in Section 5.1.2.

If you prefer to save your model and query definitions to a file, then copy a model of your choice and overwrite “your_chosen_model”. Overwrite “your_query” with your preferred query in the following:

.. code-block:: shell
    
    # chosen_model_and_query.sh
    chosen_model="your_chosen_model"
    query="Explain the concepts in this document."

Step 3: Create a file named ‘model_and_query.sh’ with the modified content above

Step 4
Make the file executable:

.. code-block:: shell
    
    chmod +x model_and_query.sh


5.1.2. RAG with all documents IDs
*************************************

The following shell block of codes loads your API key, your model and query definitions, fetches all your file ids, and does RAG on your file contents using your query and chosen model. It saves the output to a file named 'myragoutput.txt'. 

You can create a shell file and save the codes to the file, then run the file.

**Step 1**  
Save the shell codes to a file named ‘do_rag.sh’

File: do_rag.sh

.. code-block:: shell
    
    #!/bin/bash

    # API key
    source .env

    # model choice and query
    source model_and_query.sh

    myfile_ids=$(curl -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" https://openwebui.crc.nd.edu/api/v1/files/ | jq -r '.[].id')
    myfile_ids_array=($myfile_ids)
    chosen_model="phi3:latest"
    output_file="myragoutput.txt"

    files_json=""
    for file_id in "${myfile_ids_array[@]}"
    do
        files_json+="{\"type\": \"file\", \"id\": \"${file_id}\"},"
    done

    # Remove trailing comma
    files_json=${files_json%,}

    response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
    -H "Authorization: Bearer ${MY_OPEN_WEBUI_API_KEY}" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "'"${chosen_model}"'",
        "messages": [
            {"role": "user", "content": "'"${your_query}"'"}
            ],
        "files": [
            '"${files_json}"'
            ]
        }')
    
    echo "$response" > "$output_file"


**Step 2**
Make the file executable:

.. code-block:: shell
    
    chmod +x do_rag.sh

**Step 3**
Run the file (must be preceded by a dot as shown)

.. code-block:: shell
    
    ./do_rag.sh

**Step 4**
Open the file ‘myragoutput.txt’ for your results.

.. code-block:: shell
    
    cat myragoutput.txt

5.1.3. RAG with select document IDs
****************************************

The following shell codes loads your API key, your model and query definitions, and does RAG on your specific file contents using your query and chosen model. It saves the output to a file named 'myoutput.txt'.

File: select_ids_rag.sh

.. code-block:: shell
    
    #!/bin/bash

    your_query="Explain the concepts in these documents."
    selected_file_ids=("file_id_1" "file_id_2" "file_id_3") # Add as many file IDs as needed

    # API key
    source .env

    # model choice and query
    source model_and_query.sh

    output_file="myoutput.txt"

    files_json=""
    for file_id in "${selected_file_ids[@]}"
    do
        files_json+="{\"type\": \"file\", \"id\": \"${file_id}\"},"
    done

    # Remove trailing comma
    files_json=${files_json%,}

    response=$(curl -X POST https://openwebui.crc.nd.edu/api/chat/completions \
    -H "Authorization: Bearer ${my_api_key}" \
    -H "Content-Type: application/json" \
    -d '{
    "model": "'"${chosen_model}"'",
    "messages": [
        {"role": "user", "content": "'"${your_query}"'"}
        ],
    "files": [
        '"${files_json}"'
        ]
    }')

    echo "$response" > "$output_file"



6. API Use in Python
------------------------

6.1. Required Libraries and Imports 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Certain libraries are required for the API use in Python environment. The following guide prepares the user to make API calls from the python environment.

**Step 1**
Ensure that your API key is already stored in your environmental file. 
Refer to the Section :ref:`here <api_key_access>` on this if you have not done so. 


Ensure that the dot_env and requests libraries are installed. You can install them using pip if required:

.. code-block:: python
    
    pip install python-dotenv


.. code-block:: python
    
    pip install requests


**Step 2**
Import your libraries

.. code-block:: python
    
    import os
    import requests
    import json
    from dotenv import load_dotenv

    # Load the API key from the .env file
    load_dotenv()
    api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")


6.2. Models' Listing in Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is helpful to list models available to you before using them. Models can be listed in python by doing the following:

**Step 1**

Create a new file named listmodels.py and copy the following python codes to it.

.. _listmodels_reference_target:

File: listmodels.py

.. code-block:: python
    
    # List available models
    import os
    import requests
    from dotenv import load_dotenv

    # Load the API key from the .env file
    load_dotenv()
    api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

    # Check API key availability and validity
    def check_api_key(api_key):
        if not api_key:
            print("API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable.")
            return False
       return True

    # List available models
    def list_models():
        if not check_api_key(api_key):
            return
    
        url = "https://openwebui.crc.nd.edu/api/models"
        headers = {"Authorization": f"Bearer {api_key}"}
        response = requests.get(url, headers=headers)

        if response.status_code == 200:
            models = response.json().get('data', [])
           for model in models:
                print(model['name'])
        elif response.status_code == 401:
           print("Invalid API key. Please check your API key and try again.")
       else:
            print("Failed to retrieve models. Status code:", response.status_code)

    if __name__ == "__main__":
        list_models()


**Step 2**

Call the function from python

.. code-block:: python

    list_models()

Or run the file from the terminal 

.. code-block:: shell

    python listmodels.py

Or run from Jupyter notebook, Google Colab 

.. code-block:: ipypb

    !python listmodels.py


7. Chatting with Models in Python
----------------------------------

7.1. Function definition 
~~~~~~~~~~~~~~~~~~~~~~~~~~

The following comprise a basic python code block for querying a model. 

.. code-block:: python

    # Interact with the model
    def interact_with_model(chosen_model, my_query):
        url = "https://openwebui.crc.nd.edu/api/chat/completions"
        headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
        payload = {
            "model": chosen_model,
            "messages": [{"role": "user", "content": my_query}],
        }
        response = requests.post(url, headers=headers, data=json.dumps(payload))
        if response.status_code == 200:
            print(response.json())
        else:
            print("Failed to interact with the model")


7.2. Function calling
~~~~~~~~~~~~~~~~~~~~~~~~~~

The function can be called as follows:


.. code-block:: python

    chosen_model = "phi4:latest"
    my_query = "Narrate a brief history of America"
    interact_with_model(chosen_model, my_query)


To avoid errors, the model must exist in your models’ list, and the name must be correct. 


8. Document Uploads in Python
-------------------------------

8.1. Directory for RAG documents 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is good practise to create a folder for your documents for RAG.

**Step 1:**
Create a directory named for e.g. ‘myragdocuments’


**Step 2:**
Place your files intended for RAG in the folder - ‘myragdocuments’.


8.2. Multiple documents upload
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following python code block uploads documents from your folder named '‘myragdocuments’' to api_url "https://openwebui.crc.nd.edu/api/v1/files/" provided in the codes.

**Step 1:**
Save the following to a file named ‘uploadfiles.py’

File: ‘uploadfiles.py’

.. code-block:: python
    
    import os
    import requests
    from dotenv import load_dotenv

    # Load the API key from the .env file
    load_dotenv()
    api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

    # Directory containing the files to upload
    directory = "myragdocuments" # default directory

    # API endpoint
    api_url = "https://openwebui.crc.nd.edu/api/v1/files/"


    # Loop through each file in the directory and upload it
    def upload_files(directory):
        if not os.path.exists(directory):
            print(f"Directory '{directory}' does not exist.")
            return

        files_found = False
        for file_name in os.listdir(directory):
            file_path = os.path.join(directory, file_name)
            if os.path.isfile(file_path):
                files_found = True
                print(f"Uploading {file_path}...")
                with open(file_path, "rb") as file:
                    response = requests.post(
                        api_url,
                        headers={
                            "Authorization": f"Bearer {api_key}",
                            "Accept": "application/json",
                        },
                        files={"file": file},
                    )
                    if response.status_code == 200:
                        print(f"Uploaded {file_path} successfully!")
                    else:
                        print(
                            f"Failed to upload {file_path}. Status code: {response.status_code}"
                        )

        if not files_found:
            print(f"No files found in the directory '{directory}'.")


    if __name__ == "__main__":
        upload_files(directory)


**Step 2**

Call the function from python

.. code-block:: python

    directory = "myragdocuments"
    upload_files(directory)

Or run the file from the terminal 

.. code-block:: shell

    python uploadfiles.py

Or run from Jupyter notebook, Google Colab 

.. code-block:: ipynb

    !python uploadfiles.py



9. RAG using file uploads in Python
------------------------------------

An LLM, your query and your files are required for RAG. In these examples, you will retrieve your uploaded files from a given URL.

The files you uploaded are automatically asigned file ids. During RAG the file Ids will be retrieved in the python codes in 'do_rag.py'.


9.1. Choosing a model 
~~~~~~~~~~~~~~~~~~~~~~~

list your models as shown in the Section 6.2. :ref:`here <listmodels_reference_target>`.


9.2. RAG using all file Ids
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following python block of codes loads your API key, your model and query definitions, fetches all your file ids, and does RAG on your file contents using your query and chosen model. It saves the output to a file named 'myragoutput.txt'. 

When you call the python file you should pass your chosen model and query as arguments.

Step 3: Create a file named "do_rag.py" and paste the following python codes

.. code-block:: python
    
    import os
    import requests
    import json
    from dotenv import load_dotenv

    # Load the API key from the .env file
    load_dotenv()
    api_key = os.getenv("MY_OPEN_WEBUI_API_KEY")

    # Define default variables for model and query
    chosen_model = "phi4:latest" # default
    your_query = "Summarize each document in less than four words." # default


    # Check API key availability and validity
    def check_api_key(api_key):
        if not api_key:
        print(
            "API key is not available. Please set the MY_OPEN_WEBUI_API_KEY environment variable."
        )
        return False

    # Test the API key with a simple request
    test_url = "https://openwebui.crc.nd.edu/api/v1/files/"
    headers = {"Authorization": f"Bearer {api_key}"}
    response = requests.get(test_url, headers=headers)

    if response.status_code == 200:
        return True
    elif response.status_code == 401:
        print("Invalid API key. Please check your API key and try again.")
        return False
    else:
        print(f"Failed to verify API key. Status code: {response.status_code}")
        return False


    # Retrieve file IDs
    def get_file_ids(api_key):
        url = "https://openwebui.crc.nd.edu/api/v1/files/"
        headers = {"Authorization": f"Bearer {api_key}"}
        response = requests.get(url, headers=headers)

        if response.status_code == 200:
            file_ids = [file["id"] for file in response.json()]
            print(file_ids)
            return file_ids
        else:
        print(f"Failed to retrieve files. Status code: {response.status_code}")
        return []


    # Create JSON for files
    def create_files_json(file_ids):
        files_json = [{"type": "file", "id": file_id} for file_id in file_ids]
        return files_json


    # Upload files
    def get_response(
        api_key, chosen_model, your_query, files_json, output_file="myragoutput.txt"
    ):
        url = "https://openwebui.crc.nd.edu/api/chat/completions"
        headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
        data = {
            "model": chosen_model,
            "messages": [{"role": "user", "content": your_query}],
            "files": files_json,
        }

        response = requests.post(url, headers=headers, data=json.dumps(data))
        response_text = response.text
        print(response_text)

        with open(output_file, "w") as file:
            file.write(response_text)


    # Main function to call rag
    def rag(chosen_model, your_query, output_file="myragoutput.txt"):
        if check_api_key(api_key):
            file_ids = get_file_ids(api_key)
            if file_ids:
                files_json = create_files_json(file_ids)
                get_response(api_key, chosen_model, your_query, files_json, output_file)


    # Example calls to the rag function
    if __name__ == "__main__":
        rag(chosen_model, your_query)



Step 4
Call the function and pass your chosen model and query as arguments.

Option A:

From Python

.. code-block:: python
    
    import do_rag
    new_model = "mistral-large:latest"
    new_query = "What is the essence of each document?"
    rag(new_model, new_query)


Option B:

Edit the model choice and query in the python file and run from the terminal

.. code-block:: shell
    
    python do_rag.py