New Voice Agents STT and TTS not working through front end

API
1

What is the recommended/reliable method in browser JavaScript to receive a complete WAV or MP3 audio file (generated server-side by OpenAI TTS via the Agents SDK VoicePipeline, converted using pydub, and sent via an HTTP response from a Python Flask backend) as a Blob/ArrayBuffer and play it back immediately following a user interaction? We need to ensure playback actually occurs and completion events fire reliably across browsers like Chrome and Safari on macOS. Standard attempts using the HTML <audio> tag with Blob URLs and AudioContext.decodeAudioData have resulted in silent playback failures without clear console errors after the audio data is successfully received.

repo for code here: GitHub - samerGMTM22/Habeebi_AI

I am processing audio as a whole i.e. not in chuncks and in backend it is working but everything stops working when frontend index.js functions come into play and I cant figure out

2

@OpenAI_Support can you help out here please

3

You are asking for programming techniques that are generalizable to ANY application that would play back audio.

That is not a “help me use your API, because you haven’t provided adequate documentation”, that is a “provide consultation and development to fill in gaps in my skill set”.

That’s when I turn to ChatGPT.

4

It is a problem none ai could solve for me even ai within an ide
for some reason (could be obvious for non-vibe coders here aka people who know what they are doing) in the OpenAI demo they showed doing the chunks and pipeline which as a backend function worked for me, but when I try to run through a front end agent voice doesn’t play

I added the github with really simple code there. what do you mean documentation other than the code