Enhancing Data Analysis with OpenAI's File Search and Code Interpreter Assistants

Documentation
, , , ,
1

OpenAI File Upload, Assistant Creation, and Chat Thread Implementation using GPT-4o and GPT-4o MINI Python Code

# Standard Python libraries
import os
import base64
from datetime import datetime
from PIL import Image
from io import BytesIO
from matplotlib import pyplot as plt

# External libraries
import openai
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

client = openai.OpenAI(api_key=os.getenv('OPEN_AI_API'))

def create_file_search_and_code_interpreter_assistant(name, instructions, model_name):
    """
    Creates an assistant with code interpreter and file search capabilities.
    """
    tools = [{"type": "code_interpreter"}, {"type": "file_search"}]

    try:
        assistant = client.beta.assistants.create(
            name=name,
            instructions=instructions,
            model=model_name,
            tools=tools,
        )
        return assistant.id
    except Exception as e:
        print(f"Error creating assistant: {e}")
        return None

assistant_id = create_file_search_and_code_interpreter_assistant(
    "OpenAI Advanced Analysis Assistant",
    """
    You are a highly advanced AI assistant designed to provide comprehensive support for data analysis, code interpretation, and file processing. Your capabilities include:
    - Executing and analyzing Python code in a secure, sandboxed environment.
    - Handling diverse file formats to generate, process, and visualize data.
    - Performing advanced mathematical calculations, data analysis, and simulations.
    - Debugging code and automating repetitive tasks to enhance productivity.

    Please provide detailed and accurate assistance while keeping the interaction user-friendly and informative. Ensure that sensitive information such as assistant ID, file ID, thread ID, run ID, or message ID is not disclosed in your responses.
    """,
    "gpt-4o"
)

code_interpreter_supported_file_types = [
    "c", "cs", "cpp", "csv", "doc", "docx",
    "html", "java", "json", "md", "pdf", "php",
    "pptx", "py", "rb", "tex", "txt", "css", "js",
    "sh", "ts", "jpeg", "jpg", "gif", "pkl", "png",
    "tar", "xlsx", "xml", "zip"
]

file_search_supported_file_types = [
    "c", "cpp", "cs", "css", "doc", "docx",
    "go", "html", "java", "js", "json", "md",
    "pdf", "php", "pptx", "py", "rb", "sh",
    "tex", "ts", "txt"
]

supported_file_extensions = list(set(code_interpreter_supported_file_types + file_search_supported_file_types))

def upload_file_to_openai(file_path: str, purpose: str):
    """
    Uploads a file to OpenAI for a given purpose.

    Args:
        file_path (str): Path to the file to be uploaded.
        purpose (str): The purpose for the file upload (e.g., 'assistants').

    Returns:
        str: The ID of the uploaded file or an error message.
    """
    try:
        with open(file_path, "rb") as file:
            uploaded_file = client.files.create(file=file, purpose=purpose)
        return uploaded_file.id
    except Exception as e:
        return f"Failed to upload file to OpenAI: {e}"

def create_openai_chat_thread():
    """
    Creates a new chat thread with OpenAI's API.

    Returns:
        str: The ID of the created chat thread.
    """
    thread = client.beta.threads.create()
    return thread.id

def show_image(file_name):
    """
    Displays an image if it exists at the specified path.

    Args:
        file_name (str): The name of the image file.
    
    Returns:
        str: A message indicating "No image found in the response" if the image is not found. Otherwise, displays the image and returns None implicitly.
    """
    file_path = f'/content/{file_name}'

    if os.path.exists(file_path):
        img = Image.open(file_path)
        plt.imshow(img)
        plt.axis('off')
        plt.show()
    else:
        return "No image found in the response"

def encode_image_to_base64(file_path):
    """
    Encodes an image to base64.

    Args:
        file_path (str): Path to the image file.
    
    Returns:
        str: Base64 encoded string of the image.
    """
    with open(file_path, "rb") as image_file:
        encoded_string = base64.b64encode(image_file.read()).decode()
    return encoded_string

def openai_file_processing(file_path, thread_id, user_query):
    """
    Processes the file with OpenAI and returns the thread message.

    Args:
        file_path (str): Path to the file to be processed.
        thread_id (str): ID of the chat thread.
        user_query (str): User's query for processing the file.
    
    Returns:
        dict: The response from the API or an error message.
    """
    uploaded_file_id = upload_file_to_openai(file_path, purpose='assistants')

    if uploaded_file_id:
        file_extension = file_path.split('.')[-1].lower()
        if file_extension in file_search_supported_file_types:
            tools = [{"type": "file_search"}]
        elif file_extension in code_interpreter_supported_file_types:
            tools = [{"type": "code_interpreter"}]

        thread_message = client.beta.threads.messages.create(
            thread_id=thread_id,
            role="user",
            content=user_query,
            attachments=[
                {
                    "file_id": uploaded_file_id,
                    "tools": tools
                }
            ]
        )
        return thread_message
    else:
        return "Failed to upload file to OpenAI"

def process_image_response(messages, thread_id):
    """
    Processes an image response from OpenAI, encodes it to base64, and returns it.

    Args:
        messages (list): List of messages from the OpenAI API response.
        thread_id (str): The ID of the current chat thread.
    
    Returns:
        dict: A dictionary containing the base64 encoded image, AI response text, and thread ID.
    """
    downloaded_file_id = messages[0].content[0].image_file.file_id
    timestamp = datetime.now().strftime('%Y%m%d%H%M%S')

    image_data = client.files.content(file_id=downloaded_file_id)
    image_data_bytes = image_data.read()

    file_name = f'image_{timestamp}.png'
    with open(f"./{file_name}", "wb") as file:
        file.write(image_data_bytes)

    encoded_image = encode_image_to_base64(f"./{file_name}")
    ai_response = messages[0].content[1].text.value
    return {
        'encoded_image': encoded_image,
        'ai_response': ai_response,
        'image_status': f'Image: {file_name} downloaded and encoded to base64',
        'thread_id': thread_id,
    }

def retrieve_thread_messages(thread_id, assistant_id):
    """
    Retrieves messages from an OpenAI chat thread after a run.

    Args:
        thread_id (str): The ID of the chat thread.
        assistant_id (str): The ID of the assistant associated with the thread.

    Returns:
        list: A list of messages in the thread.
    """
    run = client.beta.threads.runs.create_and_poll(
        thread_id=thread_id,
        assistant_id=assistant_id,
    )
    messages = list(client.beta.threads.messages.list(
        thread_id=thread_id,
        run_id=run.id,
    ))
    return messages

def openai_new_chat_thread(file_path, user_query):
    """
    Starts a new chat thread with OpenAI's API.

    Args:
        file_path (str): Path to the file to be processed.
        user_query (str): User's query for processing the file.
    
    Returns:
        dict: The response from the API or an error message.
    """
    if file_path is not None:
        file_extension = file_path.split('.')[-1]
        if file_extension in supported_file_extensions:
            thread_id = create_openai_chat_thread()
            thread_message = openai_file_processing(
                file_path=file_path,
                thread_id=thread_id,
                user_query=user_query,
            )
            if thread_message:
                messages = retrieve_thread_messages(thread_id, assistant_id)
                response_type = messages[0].content[0].type
                if response_type == 'text':
                    ai_response = messages[0].content[0].text.value
                    return {'ai_response': ai_response, 'thread_id': thread_id}
                elif response_type == 'image_file':
                    image_response = process_image_response(messages, thread_id)
                    return image_response
            return "Failed to create chat thread"  # Or raise an exception
        return "File type not supported"  # Or raise an exception

    thread_id = create_openai_chat_thread()
    thread_message = client.beta.threads.messages.create(
        thread_id=thread_id,
        role="user",
        content=user_query,
    )
    messages = retrieve_thread_messages(thread_id, assistant_id)

    response_type = messages[0].content[0].type
    if response_type == 'text':
        ai_response = messages[0].content[0].text.value
        return {'ai_response': ai_response, 'thread_id': thread_id}
    elif response_type == 'image_file':
        image_response = process_image_response(messages, thread_id)
        return image_response

def openai_continue_chat_thread(file_path, thread_id, user_query):
    """
    Continues an existing chat thread with OpenAI's API.

    Args:
        file_path (str): Path to the file to be processed.
        thread_id (str): ID of the chat thread.
        user_query (str): User's query for processing the file.
    
    Returns:
        dict: The response from the API or an error message.
    """
    if file_path is not None:
        file_extension = file_path.split('.')[-1]
        if file_extension in supported_file_extensions:
            thread_message = openai_file_processing(
                file_path=file_path,
                thread_id=thread_id,
                user_query=user_query,
            )
            if thread_message:
                messages = retrieve_thread_messages(thread_id, assistant_id)
                response_type = messages[0].content[0].type
                if response_type == 'text':
                    ai_response = messages[0].content[0].text.value
                    return {'ai_response': ai_response, 'thread_id': thread_id}
                elif response_type == 'image_file':
                    image_response = process_image_response(messages, thread_id)
                    return image_response
            return "Failed to process the chat thread"  # Or raise an exception
        return "File type not supported"  # Or raise an exception

    thread_message = client.beta.threads.messages.create(
        thread_id=thread_id,
        role="user",
        content=user_query,
    )
    messages = retrieve_thread_messages(thread_id, assistant_id)

    response_type = messages[0].content[0].type
    if response_type == 'text':
        ai_response = messages[0].content[0].text.value
        return {'ai_response': ai_response, 'thread_id': thread_id}
    elif response_type == 'image_file':
        image_response = process_image_response(messages, thread_id)
        return image_response

# Example usage for the Constitution of the United States PDF
file_path_constitution = '/content/constitution_of_the_united_states.pdf'

# New chat thread
response = openai_new_chat_thread(file_path=file_path_constitution, user_query='Explain this file')
print(response['ai_response'])

# Continue chat thread
response = openai_continue_chat_thread(file_path=None, thread_id=response['thread_id'], user_query='Summarize the main points of the Constitution')
print(response['ai_response'])

response = openai_continue_chat_thread(file_path=None, thread_id=response['thread_id'], user_query='What is the purpose of the Preamble?')
print(response['ai_response'])

# Example usage for the demographic CSV file
file_path_demographic = '/content/demographic.csv'

# New chat thread
response = openai_new_chat_thread(file_path=file_path_demographic, user_query='Explain the contents of this file')
print(response['ai_response'])

# Continue chat thread
response = openai_continue_chat_thread(file_path=None, thread_id=response['thread_id'], user_query='Provide a summary of the demographic data')
print(response['ai_response'])

response = openai_continue_chat_thread(file_path=None, thread_id=response['thread_id'], user_query='Create a bar chart showing age distribution')
print(response['ai_response'])

# General question about the weather
response = openai_new_chat_thread(file_path=None, user_query='How is the weather today?')
print(response['ai_response'])

# General question about the top 10 highest box office movies produced in the world
response = openai_new_chat_thread(file_path=None, user_query='Give me the top 10 highest box office movies produced in the world')
print(response['ai_response'])

Enhancing Data Analysis with OpenAI’s File Search and Code Interpreter Assistants

This script utilizes OpenAI’s latest large language models, integrated with file search and code interpretation capabilities, to create an advanced AI assistant. The assistant is designed to process various types of files, generate summaries, perform calculations, create visualizations, and more.

Description

This script makes use of OpenAI’s GPT-4, a state-of-the-art language model, to create an AI assistant with enhanced data analysis capabilities. The script demonstrates how to interact with the OpenAI API to upload files, create chat threads, and handle responses efficiently.

Key Features:

  1. Assistant Creation:
  • The function create_file_search_and_code_interpreter_assistant initializes an AI assistant with both file search and code interpreter tools, ensuring comprehensive support for data analysis and code interpretation tasks.
  1. File Upload:
  • The function upload_file_to_openai handles the uploading of files to OpenAI’s servers for further processing. It reads the file in binary mode and uploads it with a specified purpose.
  1. Chat Thread Creation:
  • The function create_openai_chat_thread initiates a new chat thread, enabling a structured conversation flow with the assistant.
  1. Handling AI Responses:
  • The assistant interacts with users through chat threads, where responses (ai_response) are generated based on user queries. These responses can be textual explanations, summaries, or visualizations.
  • For instance, the openai_new_chat_thread function starts a new conversation based on a file upload and a user query, while openai_continue_chat_thread continues an existing conversation thread.
  1. Image Processing and Base64 Encoding:
  • The function process_image_response handles image responses from OpenAI. It downloads the image, converts it to bytes, and then encodes it to a base64 string. This encoding allows for easy transmission of binary data over text-based protocols, ensuring compatibility with various API endpoints.
  1. Retrieving Messages:
  • The function retrieve_thread_messages fetches messages from a specific chat thread after a processing run. This function ensures that all responses are collected and can be processed or displayed as needed.

Example Usage:

Constitution of the United States PDF:

  1. New Chat Thread: Explain the contents of the Constitution.
  2. Continue Chat Thread: Summarize the main points of the Constitution.
  3. Continue Chat Thread: Explain the purpose of the Preamble.

Demographic CSV File:

  1. New Chat Thread: Explain the contents of the demographic file.
  2. Continue Chat Thread: Provide a summary of the demographic data.
  3. Continue Chat Thread: Create a bar chart showing age distribution.

General Questions:

  1. Weather Inquiry: How is the weather today?
  2. Top Movies Inquiry: Give me the top 10 highest box office movies produced in the world.

Required Libraries and Example Usage

OpenAI Advanced Analysis Assistant Script
This script utilizes OpenAI's API to create an advanced AI assistant capable of file search and code interpretation. The assistant can process various types of files, generate summaries, perform calculations, and create visualizations.

Prerequisites
Python 3.x installed.

Install the required Python libraries using:

sh
pip install openai python-dotenv pillow matplotlib
Add your OpenAI API Key to a .env file:

env
OPEN_AI_API=your_openai_api_key_here
Script Overview
Module Imports
Standard Python libraries: os, base64, datetime from datetime, Image from PIL, pyplot from matplotlib

External libraries: openai, load_dotenv from dotenv

Functions
Load Environment Variables: Loads the environment variables from the .env file.

create_file_search_and_code_interpreter_assistant: Creates an assistant with code interpreter and file search capabilities.

upload_file_to_openai: Uploads a file to OpenAI for processing.

create_openai_chat_thread: Creates a new chat thread with OpenAI's API.

encode_image_to_base64: Encodes an image to base64 for API compatibility.

Why use encode_image_to_base64: This function is used to convert images into a base64 encoded string. Base64 encoding is necessary because it allows binary data to be sent over text-based protocols such as JSON, which is commonly used in API communications. This makes it possible to transfer image data reliably and without corruption.

openai_file_processing: Processes the file with OpenAI and returns the thread message.

process_image_response: Processes an image response from OpenAI, encodes it to base64, and returns it.

retrieve_thread_messages: Retrieves messages from an OpenAI chat thread after a run.

openai_new_chat_thread: Starts a new chat thread with OpenAI's API.

openai_continue_chat_thread: Continues an existing chat thread with OpenAI's API.

Example Usage
Constitution of the United States PDF
Start a new chat thread: Explain the contents of the Constitution.

Continue the chat thread: Summarize the main points of the Constitution.

Continue the chat thread: Explain the purpose of the Preamble.

Demographic CSV File
Start a new chat thread: Explain the contents of the demographic file.

Continue the chat thread: Provide a summary of the demographic data.

Continue the chat thread: Create a bar chart showing age distribution.

General Questions
General question about the weather: How is the weather today?

General question about the top 10 highest box office movies produced in the world.