Request for OpenAI Team: Need Split Frontend and Backend Example for Real-Time API

Tomas.K · December 16, 2024, 8:49am

Dear OpenAI Team and community,

I’m reaching out to request your assistance in providing a working example of OpenAI Real-Time API with a split frontend and backend setup. The current documentation only includes a frontend solution, which poses a risk of exposing the API key. A comprehensive example demonstrating how to implement both frontend and backend components would be immensely helpful for developers looking to integrate your API without compromising security.

Thank you for your attention to this matter.

Best regards,

Tomas K
Selendia AI

robertb · December 16, 2024, 10:01am

While not an official example, here is our Real-Time API front-end and back-end solution. We are leveraging the open-source Your Priorities platform and the Policy Synth AI library.

Our solution integrates a hybrid voice/text interface. We are using WebSockets to send audio to the client but plan to upgrade this functionality to WebRTC soon.

Back-end:

github.com

CitizensFoundation/your-priorities-app/blob/master/server_api/agents/assistants/voiceAssistant.ts

import { response } from "express";
import { YpBaseChatBot } from "../../active-citizen/llms/baseChatBot.js";
import { YpBaseAssistant } from "./baseAssistant.js";
import WebSocket from "ws";

interface VoiceMessage {
  delta: any;
  type: string;
  content?: string;
  audio?: string; // base64 encoded audio
  metadata?: any;
}

interface VoiceState {
  speaking: boolean;
  listening: boolean;
  processingAudio: boolean;
  lastAudioTimestamp?: number;
}

This file has been truncated. show original

Front-end:

github.com

CitizensFoundation/your-priorities-app/blob/master/webApps/client/src/yp-assistants/yp-assistant-base.ts

import { css, html, nothing, PropertyValueMap } from "lit";
import { property, customElement, state, query } from "lit/decorators.js";
import { YpChatbotBase } from "../yp-chatbots/yp-chatbot-base.js";
import "./yp-assistant-item-base.js";
import { literal } from "lit/static-html.js";
import { WavRecorder } from "../tools/wavTools/wav_recorder.js";
import { WavStreamPlayer } from "../tools/wavTools/wav_stream_player.js";
import { YpAssistantServerApi } from "./AssistantServerApi.js";
import { WavRenderer } from "./wave-renderer.js";
import { YpConfirmationDialog } from "../yp-dialog-container/yp-confirmation-dialog.js";
import { YpServerApi } from "../common/YpServerApi.js";

import "./yp-assistant-welcome.js";
import { cache } from "lit/directives/cache.js";
import { resolveMarkdown } from "../common/litMarkdown/litMarkdown.js";
import { YpSnackbar } from "../yp-app/yp-snackbar.js";

@customElement("yp-assistant-base")
export abstract class YpAssistantBase extends YpChatbotBase {
  @property({ type: Boolean })

This file has been truncated. show original

One example of this code in action:

The AI agent library driving our solution:

Tomas.K · December 16, 2024, 1:14pm

Hi @robertb , thank you for fast reply. Unfortunatelly you are running javascript code on frontend so its big security risk.

robertb · December 16, 2024, 1:15pm

The OpenAI API key is never exposed to the front-end and we use our own user/role system to control access. So pretty much exactly like in the quote above about secure usage of the API.

Tomas.K · December 16, 2024, 1:16pm

We would like to see working example code of connection between backend and openai websocket for Real Time API.

Tomas.K · December 19, 2024, 11:57am

Update: solved by update from Openai, rollout webRTC (Implementing an ephemeral API key). Thank you

Tomas.K · December 19, 2024, 12:10pm

solution is also over node.js server.

vb · December 21, 2024, 12:10pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Real Time API Connectiion problem API	4	297	November 8, 2024
Realtime api example in c or c++ API api-realtime-speech	0	109	November 18, 2024
Realtime API Pyton SDK and Assistant support API assistants-api , realtime	6	856	October 11, 2024
Private and public API Key to build browser client apps API api	2	149	July 18, 2024
Realtime api - get conversation transcription from backend while using webrtc API realtime	1	88	January 16, 2025

Request for OpenAI Team: Need Split Frontend and Backend Example for Real-Time API

Related topics