Is there a way to enable internet browsing (like I would do in a new GPT4 chat enabling the relative functionality)?
I’m trying to see if I can find a way of doing so (it usually works very well and is supported in chat so I don’t see why is not available with Assistans).
Thanks!
8 Likes
I am interested in that too. Can we use gpt-4-1106-preview API to make it search on the internet like bing browsing on the chatgpt web interface?
2 Likes
Sure @giancarloerra. You can integrate your OpenAI assistant with search APIs using the retrieval tools. I find Tavily search API to be best for LLMs. Search for “Tavily Search API”.
2 Likes
To use search tools you should use function calls (not retriever knowledge). Check out OpenAI’s function calls documentation
2 Likes
Thank you, that’s an interesting one. I didn’t know about it. I gave it a try, but I definitely find it’s not as effective as using GPT4 with “search with bing”, not nearly the same thing.
I still think it’s bizarre that’s something is not available in Assistants. Actually I see is available in in custom GPT, but they seem to not be available to use via an API?
There’s no logical explanation to this.
1 Like
@giancarloerra I wrote a snippet below that uses Search with openai assistant:
# Step 1: Upgrade to Python SDK v1.2 with pip install --upgrade openai
# Step 2: Install Tavily Python SDK with pip install tavily-python
# Step 3: Build an OpenAI assistant with Python SDK documentation - https://platform.openai.com/docs/assistants/overview
import os
import json
import time
from openai import OpenAI
from tavily import TavilyClient
# Initialize clients with API keys
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
tavily_client = TavilyClient(api_key=os.environ["TAVILY_API_KEY"])
assistant_prompt_instruction = """You are a finance expert.
Your goal is to provide answers based on information from the internet.
You must use the provided Tavily search API function to find relevant online information.
You should never use your own knowledge to answer questions.
Please include relevant url sources in the end of your answers.
"""
# Function to perform a Tavily search
def tavily_search(query):
search_result = tavily_client.get_search_context(query, search_depth="advanced", max_tokens=8000)
return search_result
# Function to wait for a run to complete
def wait_for_run_completion(thread_id, run_id):
while True:
time.sleep(1)
run = client.beta.threads.runs.retrieve(thread_id=thread_id, run_id=run_id)
print(f"Current run status: {run.status}")
if run.status in ['completed', 'failed', 'requires_action']:
return run
# Function to handle tool output submission
def submit_tool_outputs(thread_id, run_id, tools_to_call):
tool_output_array = []
for tool in tools_to_call:
output = None
tool_call_id = tool.id
function_name = tool.function.name
function_args = tool.function.arguments
if function_name == "tavily_search":
output = tavily_search(query=json.loads(function_args)["query"])
if output:
tool_output_array.append({"tool_call_id": tool_call_id, "output": output})
return client.beta.threads.runs.submit_tool_outputs(
thread_id=thread_id,
run_id=run_id,
tool_outputs=tool_output_array
)
# Function to print messages from a thread
def print_messages_from_thread(thread_id):
messages = client.beta.threads.messages.list(thread_id=thread_id)
for msg in messages:
print(f"{msg.role}: {msg.content[0].text.value}")
# Create an assistant
assistant = client.beta.assistants.create(
instructions=assistant_prompt_instruction,
model="gpt-4-1106-preview",
tools=[{
"type": "function",
"function": {
"name": "tavily_search",
"description": "Get information on recent events from the web.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "The search query to use. For example: 'Latest news on Nvidia stock performance'"},
},
"required": ["query"]
}
}
}]
)
assistant_id = assistant.id
print(f"Assistant ID: {assistant_id}")
# Create a thread
thread = client.beta.threads.create()
print(f"Thread: {thread}")
# Ongoing conversation loop
while True:
user_input = input("You: ")
if user_input.lower() == 'exit':
break
# Create a message
message = client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content=user_input,
)
# Create a run
run = client.beta.threads.runs.create(
thread_id=thread.id,
assistant_id=assistant_id,
)
print(f"Run ID: {run.id}")
# Wait for run to complete
run = wait_for_run_completion(thread.id, run.id)
if run.status == 'failed':
print(run.error)
continue
elif run.status == 'requires_action':
run = submit_tool_outputs(thread.id, run.id, run.required_action.submit_tool_outputs.tool_calls)
run = wait_for_run_completion(thread.id, run.id)
# Print messages from the thread
print_messages_from_thread(thread.id)
6 Likes
You pay to use the API per transaction.
You can also pay to search per transaction.
https://www.microsoft.com/en-us/bing/apis/pricing
The better link: Azure services https://azure.microsoft.com/en-us/pricing/details/cognitive-services/search-api/
See S5 where you get perhaps the best bot feed.
You might not even pay $240 per year. Or more than one person could use it.
3 Likes
I think would be fine for all of us to pay for the Bing API, but the way “Search with Bing” is integrated by OpenAI is quite effective and clever: it’s quite intelligent in searching online, following links etc.
It would be nice to have the option to use it and possibly paying for the usage.
I wonder if anyone integrated Bing API using functions calls.
3 Likes
@insxlnz I gave a try to your example tailoring it a bit…in my specific case, I need a reliable system to search on specific websites the price of second-hand items…the “search with bing” is incredibly effective.
Tavily can “sort of” do the same, but is far from being as efficient and good.
Thanks for your example code, I still hope that something similar to the “browse with bing” integration emerges and can be used via API…it’s an incredibly powerful tool.
1 Like
Hey @giancarloerra , are you referring that chatgpt + web is more powerful or specifically bing? Because from my tests Tavily is much better in the results quality.
@insxlnz I agree Tavily results are good quality, BUT the integration of chatgpt+bing (not bing itself) it’s incredibly reliable and clever.
United with the simplicity of Assistants or CustomGPTs, I think there’s a huge missed opportunity in there (also for potential more paying users for Azure) into making that available to API.
For some reason OpenAI since the start tended to be two systems, one is the API, one is the ChatGPT, probably because of some technical debt that’s hard to recover now.
The potential of ChatGPT interface being an interface on their APIs, and so for example having access to all ChatGPT functionalities with the API (not just plugins but also access to the threads etc.) would be a gigantic leap forward I think, now that Assistants and CustomGPTs are so powerful.
Anyway…I hope to see Seach with Bing soon available via API as many people are asking, but thanks for Tavily as it might be something to experiment with more in the meantime.
2 Likes
i haven’t got it working as well as openai yet either. I am thinking having a very specific task is difficult for the LLM and search function.
I have been trying to get a report on the latest stock news for the past day, but it seems to pull information from many different dates with inaccurate findings.
1 Like
I could not agree more! The integration of “search with Bing” has a very smart way of finding the correct answer. I think it somehow weights the found results and it seems it can also crawl certain websites to find the correct answer. This is indeed incredible powerfull and not compareable with static search results. I also hope this function will be available within the assistants API and that my post will trigger the alg of OpenAI to have one more vote for this 
2 Likes
I agree. @logankilpatrick any plans to add ‘browse with Bing’ tool available in the API?
1 Like
Hey guys, I modified @insxlnz’s code by replacing the Tavily with Bing search. Also, added functionality for the agent to respond with a summary-analysis of the search results.
It’s up on github gists
Gist friendly name: *OpenAI Assistant 🤖 with Bing Search*.
Gist script name: openai-bing-search-assistant.py
Also, pasted below!
Hope this helps. If you give it a try, let me know how it goes 
!
Biggest props go to @insxlnz, as he did all the heavy lifting 
import os
import asyncio
import requests
import time
import json
import openai
from urllib.parse import quote_plus
from openai import OpenAI
from dotenv import load_dotenv
from azure.cognitiveservices.search.websearch import WebSearchClient
from azure.cognitiveservices.search.websearch.models import SafeSearch
from msrest.authentication import CognitiveServicesCredentials
# NOTE: Bing search is free for 1000 queries per month!
# NOTE: OpenAI is *not cheap* particularly using assisistants - just keep in mind!
# GPT-4 Turbo (gpt-4-1106-preview) is set here, but you can use GPT-3.5 Turbo (gpt-3.5-turbo-1106) which is 10x cheaper
# Implementation Overview:
# - Creates an OpenAI assistant instance set up to call Bing-search and analysis functions (funcs defined in this script)
# - Uses Bing Search API to conduct internet searches based on the processed user query. NOTE: Can easily be swapped to use the Bing Custom Search service (aka resource) instead (which allows you to define what sites it can search, and what sites it can't search)
# Bing API & Service (resource) Setup:
# 1. Sign Up for Azure
# - If you don't already have one, create a Microsoft Azure account at Azure Portal.
# 2. Go to the Azure Marketplace within the Azure Portal
# 3. Create a new Bing Search resource
# - If you wish to limit the search window to certain URLs or domains, then see comments listing 'Bing Custom Search' below.
# 4. Copy the API key from the Azure Portal
# Overview of the Process:
# 1: User Request:
# - The user provides a plain-english request for information (e.g. "What are the best stonks to buy right now?").
# 2. Automated Bing Web-Search:
# - OpenAI assistant generates a query to search Bing
# - Implements the Bing Web Search API to conduct internet searches based on the processed user query.
# - If the user request isn't clear enough to form a Bing-search query from (example query: "sup man?"), then the assistant will (likely) respond seeking more information.
# 3. Automated Search Result Processing:
# - Analyzes and processes the search results obtained from Bing, again using the OpenAI assistant instance.
# - The assistant will then provide a summary of the search results, tailored to answer the user's initial query.
# Result Analysis and Response: Provides a summary or analysis of the search results, tailored to answer the user's initial query.
# NOTE (references and notes)
#
# Bing web search (Azure) API (Use Azure marketplace and search for "Bing Search" or "Bing Custom Search" - avoid the old Bing API)
# CURRENT AZURE API REF: https://docs.microsoft.com/en-us/azure/cognitive-services/bing-web-search/quickstarts/python
# DEPRECATED API REF: https://learn.microsoft.com/en-us/bing/search-apis/bing-web-search/quickstarts/sdk/web-search-client-library-python
#
# Tavily web-search for LLMs (alternative option - probably not as good as Bing search)
# https://docs.tavily.com/docs/tavily-api/python-sdk
#
# OpenAI Tools (assistants):
# https://platform.openai.com/docs/assistants/tools
# https://platform.openai.com/docs/assistants/tools/function-calling
# RECENT: https://medium.com/@nilakashdas/how-to-build-smart-assistants-using-open-ai-apis-ebf9edc42084
# https://medium.com/@assafelovic/how-to-build-an-openai-assistant-with-internet-browsing-ee5ad7625661
# NOTE, share results with community:
# https://community.openai.com/t/new-assistants-browse-with-bing-ability/479383/12
# Multi-function assistants:
# https://dev.to/esponges/build-the-new-openai-assistant-with-function-calling-52f5
#
# Use a connection to Azure OpenAI on your data.
# https://learn.microsoft.com/en-us/microsoft-copilot-studio/nlu-generative-answers-azure-openai
# TODO:
# Have LLM scrape data from sites linked by the Bing web search, and then analyze the *scraped data* (rather than raw search results) to answer the user's question.
# Consider using local LLM for generating Bing search query (it creates an optimized keyword search query from user's plain english request)
# Consider using local LLM for performing Bing search analysis (for answering user's question)
# Load environment variables
load_dotenv()
# OpenAI API Key
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
# NOTE: OLD BING API fields
# subscription_key = "79f43664b4a343e38b4017f758ee80f3"
# search_client = WebSearchClient(endpoint="https://api.bing.microsoft.com/", credentials=CognitiveServicesCredentials(subscription_key))
# NOTE: NEW BING API fields (API migrated to azure marketplace)
# custom_config_id = "define this if you are using 'Bing Custom Search' service (aka resource) instead of 'Bing Search'"
searchTerm = "microsoft"
# NOTE: This URL is not the same as the one listed in the Azure resource portal. It has the additional v7.0/search? to specify the resource function.
url = 'https://api.bing.microsoft.com/v7.0/search?' # + 'q=' + searchTerm + '&' + 'customconfig=' + custom_config_id
# OpenAI Model Configuration
base_model = "gpt-4-1106-preview"
max_tokens = 7000
temperature = 0.2
u_request = ""
s_query = ""
s_results = ""
run = None
############################################################################################################
### OPENAI FUNCTIONS: Functions to perform a Bing search and process the results
############################################################################################################
# OPENAI FUNCTION: Function to perform a Bing search
def perform_bing_search(user_request):
global u_request
global s_query
global s_results
u_request = user_request
print(f"Generating a search_query for bing based on this user request: {user_request}")
openai_prompt = "Generate a search-engine query to satisfy this user's request: " + user_request
response = client.chat.completions.create(
model=base_model,
messages=[{"role": "user", "content": openai_prompt}],
)
# Get the response from OpenAI
bing_query = response.model_dump_json(indent=2)
s_query = bing_query
print(f"Bing search query: {bing_query}. Now executing the search...")
bing_response = run_bing_search(user_request)
s_results = bing_response
return bing_response
# OPENAI FUNCTION: Function to process Bing search results
def process_search_results(search_results):
global u_request
global s_query
global s_results
print(f"Analyzing/processing Bing search results")
# Use GPT to analyze the Bing search results
prompt = f"Analyze these Bing search results: '{s_results}'\nbased on this user request: {u_request}"
response = client.chat.completions.create(
model=base_model,
messages=[{"role": "user", "content": prompt}],
)
analysis = response.choices[0].message.content.strip()
print(f"Analysis: {analysis}")
# Return the analysis
return analysis
############################################################################################################
### ANALYSIS: Perform a Bing search and process the results
############################################################################################################
def run_bing_search(search_query):
# Returns data of type SearchResponse
# https://learn.microsoft.com/en-us/python/api/azure-cognitiveservices-search-websearch/azure.cognitiveservices.search.websearch.models.searchresponse?view=azure-python
try:
base_url = "https://api.bing.microsoft.com/v7.0/search?"
encoded_query = quote_plus(search_query)
bing_search_query = base_url + 'q=' + encoded_query # + '&' + 'customconfig=' + custom_config_id --> uncomment this if you are using 'Bing Custom Search'
r = requests.get(bing_search_query, headers={'Ocp-Apim-Subscription-Key': subscription_key})
except Exception as err:
print("Encountered exception. {}".format(err))
raise err
# Old API
#try:
# web_data = search_client.web.search(query=search_query)
#except Exception as err:
# print("Encountered exception. {}".format(err))
# raise err
response_data = json.loads(r.text)
results_text = ""
for result in response_data.get("webPages", {}).get("value", []):
results_text += result["name"] + "\n"
results_text += result["url"] + "\n"
results_text += result["snippet"] + "\n\n"
print(f"Title: {result['name']}")
print(f"URL: {result['url']}")
print(f"Snippet: {result['snippet']}\n")
return results_text
############################################################################################################
### OPENAI ASSISTANT RUN MANAGEMENT
############################################################################################################
# Function to wait for a run to complete
def wait_for_run_completion(thread_id, run_id):
while True:
time.sleep(1)
run = client.beta.threads.runs.retrieve(thread_id=thread_id, run_id=run_id)
print(f"Current run status: {run.status}")
if run.status in ['completed', 'failed', 'requires_action']:
return run
# Function to handle tool output submission
def submit_tool_outputs(thread_id, run_id, tools_to_call, run, tool_output_array=None, func_override=None):
global s_results
print(f"Submitting tool outputs for thread_id: {thread_id}, run_id: {run_id}, tools_to_call: {tools_to_call}")
if tool_output_array == None:
tool_output_array = []
for tool in tools_to_call:
output = None
tool_call_id = tool.id
function_name = func_override if func_override else tool.function.name
function_args = tool.function.arguments
if function_name == "perform_bing_search":
print("[function call] perform_bing_search...")
output = perform_bing_search(user_request = json.loads(function_args)["user_request"])
elif function_name == "process_search_results":
print("[function call] process_search_results...")
output = process_search_results(json.loads(function_args)["search_results"]) #search_results = s_results) #json.loads(function_args)["search_results"]) #(search_results = s_results)
if output:
print("[function result] Appending tool output array...")
tool_output_array.append({"tool_call_id": tool_call_id, "output": output})
return client.beta.threads.runs.submit_tool_outputs(
thread_id=thread_id,
run_id=run_id,
tool_outputs=tool_output_array
)
# Function to print messages from a thread
def print_messages_from_thread(thread_id):
messages = client.beta.threads.messages.list(thread_id=thread_id)
message = ""
print("\n====== Assistant Response ======\n")
for msg in messages:
if msg.role == "assistant":
print(f"{msg.role}: {msg.content[0].text.value}")
message += f"{msg.role}: {msg.content[0].text.value}\n"
return message
# Initialize the assistant and its features and tools
assistant = client.beta.assistants.create(
instructions="""You are a Q&A bot who performs web searches to respond to user queries.
Call function 'perform_bing_search' when provided a user query.
Call function 'process_search_results' if the search results contain the info needed to answer the user query.
Call function 'perform_bing_search' again if the search results do not contain the info needed to answer the user query.
""",
model=base_model,
tools=[
{
"type": "code_interpreter"
},
{
"type": "function",
"function": {
"name": "perform_bing_search", # Function itself should run a GPT OpenAI-query that asks the OpenAI to generate (and return) a Bing-search-query.
"description": "Determine a Bing search query from the user_request for specified information and execute the search",
"parameters": {
"type": "object",
"properties": {
"user_request": {"type": "string", "description": "The user's request, used to formulate a Bing search message"},
},
"required": ["user_request"]
}
}
},
{
"type": "function",
"function": {
"name": "process_search_results", # Function itself should send the Bing seardh results to openai to assess the results, and then return the results of that assessment to the user.
"description": "Analyze Bing search results and return a summary of the results that most effectively answer the user's request",
"parameters": {
"type": "object",
"properties": {
"search_results": {"type": "string", "description": "The results from the Bing search to analyze"},
},
"required": ["search_results"]
}
}
}
]
)
assistant_id = assistant.id
print(f"Assistant ID: {assistant_id}")
# Create a thread
thread = client.beta.threads.create()
print(f"Thread: {thread}")
# Ongoing conversation loop
while True:
prompt = input("\nYour request: ")
if prompt.lower() == 'exit':
break
status = "na"
#while status != "completed":
# Create a message and run
message = client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content=prompt,
)
run = client.beta.threads.runs.create(
thread_id=thread.id,
assistant_id=assistant_id,
)
print(f"Run ID: {run.id}")
# Wait for run to complete
run = wait_for_run_completion(thread.id, run.id)
while run.status == 'requires_action':
print("Run requires action 1")
run = submit_tool_outputs(thread.id, run.id, run.required_action.submit_tool_outputs.tool_calls, run) # **error on this line**
run = wait_for_run_completion(thread.id, run.id)
time.sleep(1)
if run.status == 'failed':
print(run.error)
continue
# Print messages from the thread
#prompt = print_messages_from_thread(thread.id)
print_messages_from_thread(thread.id)
time.sleep(1)
8 Likes
@logankilpatrick maybe this got lost in the notifications. Please update us!
Most OpenAI staff have notifications turned off, mainly because of unsolicited tagging like this.
“Browse and search with whatever you want” is something you can build yourself with function tools of an assistant or just with the chat AI models themselves. There has been no peep of this or any other internet access method being available by API.
You can build yourself a retrieval system as well. We could argue that one could build a sandbox system for the Code Interpreter too.
It’s obvious that many of us who participated in this thread are interested in the shenanigans OpenAI puts in its search tool. The agentic-like system of doing the search is well different that just querying Bing and feeding the first three results in the prompt.
We don’t currently support browsing as a tool but it is on our radar: https://platform.openai.com/docs/assistants/how-it-works/limitations
6 Likes
Here is a sample code to integrate function calling (Google search) using serp API https://youtu.be/BAfCt_R61Ms?si=0VOirITMwhfnBw_0