As a complete newbie I have my server running on 127.0.0.1:7860, which means I can only run it locally. So I installed a proxy (HAProxy) that allows access to port 80 on the LAN interface and forwards to 127.0.0.1:7860, and, along with a suitable DNS entry, that allows me to enter my question in the main page , however just when I expect to receive the answer, there is an error “invalid character 1 of the JSON file”. So something goes wrong with the operation that manages the display of the answer on the browser. Any idea what might be causing this? How can I set up port 80 as the main port in place of 7860?
Hi and welcome to the developer forum!
Can you please share the code you’re using to generate the prompts and any setup code it relies upon.
This could be a local networking issue or something in the code.
Things to think about:
Use a tool like Postman or the browser’s built-in developer tools (usually accessed with F12) to inspect the HTTP response from the server. This will allow you to see exactly what data is being returned, which can help identify if the issue is with the server or the client, and check if the response is indeed a valid JSON. An unexpected character or malformed JSON structure could do it.
Ensure that the HAProxy configuration is correct and is not inadvertently modifying the data in transit. Look especially for any configurations that might change the content type or encoding.
Check that the server is sending the correct Content-Type header (usually application/json for JSON responses). If this header is incorrect or missing, the client may not know how to properly process the response.
Thanks for taking the time to answer!
The response I get to the POST request to chat-dot-mydomain on the “predict” URI is a “504 gateway timeout”
HAProxy is configured to send traffic it receives on port 80 to localhost collon 7860, I think it’s properly configured, it’s so simple anyway.
The code that I run is below. When I execute it it tells me that my app is accessible at http-localhost -7860. Instead I want it to be accessible at chat-dot-mydomain for obvious reasons. And it’s working, until the instant when the response is supposed to pop up on the “reply” textbox.
import openai
import gradio as gr
import requests
from bs4 import BeautifulSoup
openai.api_key = "<our-API-key>"
def search_manual():
urls = [
<list of URLs>
]
all_headings = []
for url in urls:
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")
headings = [heading.text.strip() for heading in soup.find_all("h2")]
all_headings.extend(headings)
return all_headings
def chatbot(input):
headings = search_manual()
messages = [
{"role": "system", "content": f"You are a helpful and kind AI Assistant. Here are some headings from the manual: {', '.join(headings)}"}
]
if input:
messages.append({"role": "user", "content": input})
chat = openai.ChatCompletion.create(
model="gpt-3.5-turbo", messages=messages
)
reply = chat.choices[0].message.content
messages.append({"role": "assistant", "content": reply})
return reply
inputs = gr.inputs.Textbox(lines=7, label="Chat with AI")
outputs = gr.outputs.Textbox(label="Reply")
gr.Interface(fn=chatbot, inputs=inputs, outputs=outputs, title="AI Chatbot",
description="Ask anything you want",
theme="compact").launch(share=True)
UPDATE:
I figured out how to make the application accessible on port 80 on 0.0.0.0 and the problem is now solved (HAProxy is no longer required).
I wish I could keep it running on port 7860 so as to not have to give the user sudo access, but with port 7860 (and HAProxy doing the forwarding again from 80 to 7860) I kept on having the exact same problem. Perhaps the involvement of HAProxy is to blame, I don’t know.
I now have a different question: my code seems to re-index all URLs with every question. This means that all questions take about 60-90sec to be answered, which is obviously too much.
How can I update the code to only index once, thereby suffering this delay once? Ideally I’d like to index on a schedule, e.g. once per day.