I have a chatbot that is prompted with the following :
Ignoring the titles and subtitles of the text and use the body of the given text, generate three multiple-choice questions, each offering four answer choices. For every question, ensure one of the answer choices is the correct option, clearly marked as such. Extract questions from the text's content to showcase a profound understanding of the material. Aim to construct questions demanding thoughtful consideration, unveiling critical thinking. Address diverse facets of the topic explored in the text. Your objective is to craft questions evaluating comprehension, guaranteeing they are lucid, pertinent, and closely aligned with the content. Steer clear of overly simplistic inquiries, and endeavor to fabricate questions that gauge the reader's mastery of the material and their adeptness in engaging with it on a substantial level.
This is supposed to output three questions based on a text provided along with four multiple choice questions, with the correct response labeled as such. But it is giving me weird questions. For example i provided a text (a paragraph or two) about the history of a maojr US city and questions 2 and 3 were ok, but question 1 was about monarch butterflies. I ran the text about the city through the chatgpt site and asked it did it see anything that would trigger a question about monarch butterflies, it responded with something approximate to âthere is nothing in this text about monarch butterfliesâ. Any thoughts? Thanks in advance.
Does the instruction really have to be so convoluted and distracting from the AI function? I think no.
ChatGPT prompt
AI instruction //
For the provided text, you create three multiple-choice questions giving four answers: one correct, three wrong but very convincing. Determining the correct answer shall take critical reading and thinking skills by the student, requiring mastery of materials.
text //
(1200 tokens of Carson City history)
output:
Question 1: What was the primary reason for Carson Cityâs growth and development as a commercial center?
A) The construction of the V&T railroad to Virginia City.
B) The discovery of gold and silver on the Comstock Lode.
C) The completion of the United States Mint in Carson City.
D) The establishment of the Warm Springs Hotel.
Question 2: What role did Abraham Curry play in the development of Carson City?
A) He named the nearby river after Kit Carson.
B) He established the Warm Springs Hotel for legislative meetings.
C) He purchased the Comstock Lode mines.
D) He surveyed and plotted the town site of Carson City.
Question 3: Why did Carson Cityâs population decline significantly by 1930?
A) The decline of the Comstock mines.
B) The construction of the Southern Pacific railroad.
C) The consolidation of government services with Ormsby County.
D) The conversion of the V&T railroad to a narrow gauge.
Also, ChatGPT gets past conversation fed back into it, and now is dumb and canât break away from what it previously output. For your first prompt, start a new chat. For followups, keep on editing the original or start a new chat to ensure no distractions from the past.
Programmers that are using the API to access OpenAI models can both choose how information is submitted and the parameters of language generation:
Instead of a chatbot conversation, one can submit input as a straightforward single input that produces an output
settings (top_p and temperature) can reduce the chance that very unlikely words are randomly chosen when generating text, with the AI then needing to adapt in order to continue after the strange word or word fragment.
I tried the prompt you provided and my first question is still suspect. So here is the text:
Baltimore, a city with a rich and diverse history, holds a significant place in the tapestry of American development. Established in 1729, it quickly became an important seaport, capitalizing on its strategic location along the Chesapeake Bay. During the Revolutionary War, Baltimore played a pivotal role, hosting the Continental Congress and contributing ships to the fledgling American Navy. The cityâs prominence grew further with the advent of the Industrial Revolution, as it became a hub for manufacturing and shipping, particularly in the textile and steel industries.
In the War of 1812, Baltimore earned enduring fame for successfully repelling a British naval assault on Fort McHenry, inspiring Francis Scott Key to pen âThe Star-Spangled Banner.â This event solidified the cityâs patriotic spirit and its role in shaping the nationâs identity.
Throughout the 19th century, Baltimore thrived as a center of commerce, immigration, and innovation. It was a key destination for European immigrants, contributing to its cultural diversity and urban growth. The introduction of the Baltimore and Ohio Railroad in the 1830s further bolstered its significance as a transportation hub, connecting the city to the expanding frontier.
However, like many American cities, Baltimore faced its share of challenges in the 20th century. Economic shifts led to deindustrialization and urban decline, leading to periods of social unrest and economic disparities. The cityâs demographics shifted as well, with waves of suburbanization impacting the urban landscape.
In recent decades, Baltimore has embarked on revitalization efforts, focusing on economic diversification, urban renewal, and the preservation of its historic districts. The Inner Harbor, once dilapidated, has been transformed into a thriving cultural and entertainment hub, drawing visitors and residents alike. While the city still grapples with certain socio-economic issues, its history of resilience, its cultural heritage, and its ongoing redevelopment projects continue to shape Baltimore into a dynamic and evolving metropolis.
And here are the responses:
Question 1: What is the main theme of the text? A. The importance of critical reading and thinking skills B. The power of mastering materials C. The ability to create multiple-choice questions D. The use of non-trivial topics for learning Correct Answer: A. The importance of critical reading and thinking skills
Question 2: What year was Baltimore established? A. 1729 (correct) B. 1776 C. 1820 D. 1902
Question 3: What was the name of the poem written by Francis Scott Key after the War of 1812? A: The Star-Spangled Banner B: The American Flag C: The Red, White, and Blue D: The Star-Spangled Spectacle
See how the questions 2 and 3 are on topic but question 1 is not. How can fine tune this?
When you say âone can submit input as a straightforward single input that produces an outputâ, could you provide a link to a resource or tutorial about this approach?
You can just dump more rules on it in a clear list (minus signs) until your are able to reshape the complianceâŚor give up on the modelâs abilities.
you will only write questions sourced from, and with answer contained in, the provided text just seen
think about the scope and topic of the text and ensure questions fit the theme
prior knowledge and training is not a knowledge source. Only read the text.
Also, when I removed the blank lines from the top of the document, the weird questions stopped. Even though when I asked Chatgpt3 a question about blank lines it respondedâŚ
Blank lines, also known as empty lines or whitespace lines, are typically ignored by most chatbot models when generating responses. These blank lines do not usually affect the modelâs response, as they are considered part of the formatting or structure of the text rather than meaningful content.
This comment is if you are willing and able to move from just asking ChatGPT your buddy, to submitting text for ai processing by your custom software to the api, paying per use. (different billing system).
Instead of past chat being a confusion for the ai, each call is independent.
echo "<h3>Document: $documentName</h3>";
echo "<div class='my_bot_resp'>$decodedContent</div>";
// Split the document content into paragraphs
$paragraphs = explode("\n", $decodedContent);
// Initialize question number
$questionNumber = 1;
// Initialize a variable to store all question-answer sets
$allSetsContent = "";
// Prompt for generating questions
$questionPrompt = "
1. analyze the text and get the core understanding of the topic and subject (example, is it history, math, science, art, technology, english)
2. create 3 2-part questions from this text based on the subject
3. each of the questions created in step 2 must have 4 multiple choice answers labeled A,B,C,D:
- with one being obviously wrong
- two of them being close
- and one being correct, label the correct answer the term '(Correct)' placed to the right of the correct answer";
// Loop through paragraphs and generate question-answer sets
foreach ($paragraphs as $paragraph) {
// Use the paragraph as the question
$userMessage = $questionPrompt . "\n\n" . $paragraph;
// API key for OpenAI
$apiKey = "***************************************";
// Call the OpenAI API to get answers based on user input
$data = array(
"prompt" => $userMessage . "\n\nQ:",
"max_tokens" => 2000, // Adjust as needed
"temperature" => 1.0, // Adjust as needed
"stop" => "\n\nQ:"
);
$headers = array(
"Content-Type: application/json",
"Authorization: Bearer $apiKey"
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://api.openai.com/v1/engines/text-davinci-003/completions");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$response = curl_exec($ch);
curl_close($ch);
$botResponse = json_decode($response, true)["choices"][0]["text"];
Please tell me if there is a better approach. Thanks for the assist!
The basic thing is you didnât write a chatbot where a user keeps on adding to a conversation and the past conversation turns are also included.
Instead, you loop through the separated API calls where you batch inputs of passages to receive the independently-generated multiple choice test, and store them in a database or use their answers.
It looks like you are doing some auto-splitting of paragraphs. I thought instead youâd want to be making a quiz on a whole story, like an SAT reading comprehension exam. A single paragraph may have poor context in relation to the whole story.
You now can decrease the temperature to 0.1, or instead use the differently-performing parameter top_p = 0.1, unless you have need for any identical input to give you a different unpredictable type of output between runs.