How to add a custom prompt before form input?

API
,
1

I followed the Vercel AI SDK guide for OpenAI step-by-step,
I’d like to send a fixed prompt before receiving user input. How can I achieve this?

1 Like
2

Welcome to the community!

I imagine there are multiple guides out there - which one did you follow specifically?

Are you talking about instruct/completion mode or chat mode?

If I’m not mistaken, vercel just makes you use the vanilla openai libraries. the docs to that are here: https://platform.openai.com/docs/api-reference/chat/create

long story short:

for instruct, your prompt is just a string. You can edit that string before sending it off.

for chat, you have a messages array. You can intercept and edit that array before sending it off to OpenAI.

4

I’ve been playing around with the Vercel SDK and checking out the OpenAI API. Specifically, I’ve been trying out the chat mode feature using Vercel and OpenAI, as explained in their documentation. I’ve been tinkering with the code to see what I can do with it.

import OpenAI from 'openai';
import { OpenAIStream, StreamingTextResponse } from 'ai';

// Create an OpenAI API client
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export const runtime = 'edge';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const fixedPrompt = {
    role: 'system',
    content: 'give me a TL;DR of the following text:  '
  };

  // Prepend the fixed prompt to the messages array
  const modifiedMessages = [fixedPrompt, ...messages];

  // Ask OpenAI for a streaming chat completion given the modified prompt
  const response = await openai.chat.completions.create({
    model: 'gpt-3.5-turbo',
    stream: true,
    messages: modifiedMessages,
  });

  // Convert the response into a friendly text-stream
  const stream = OpenAIStream(response);
  // Respond with the stream
  return new StreamingTextResponse(stream);
}
1 Like
5

Ah. Well, it generally looks ok, but I guess the prompt could use some work.

If you haven’t tried yet, a good testbench to get started is the playground:

https://platform.openai.com/playground?mode=chat

there you can manually test how well your prompt works.

And if you’re a little lost on where to start with prompting, here’s the prompting guide from openai: https://platform.openai.com/docs/guides/prompt-engineering
It contains a bunch of basic tricks to help you get started.

3 Likes