# All the questions addressed by the API team during the December 17, 2024 AMA **URL:** \ **Category:** Community\ **Tags:** community, ama, shipmas\ **Created:** [December 17, 2024, 7:19pm UTC](https://community.openai.com/t/all-the-questions-addressed-by-the-api-team-during-the-december-17-2024-ama/1059780 "2024-12-17T19:19:33Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![VeitB](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/veitb/32/712987_2.png) [@VeitB](https://community.openai.com/u/VeitB)\ **Post date:** [December 17, 2024, 7:19pm UTC](https://community.openai.com/t/all-the-questions-addressed-by-the-api-team-during-the-december-17-2024-ama/1059780/1 "2024-12-17T19:19:33Z")
Link to the original topic with all questions and answers: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527): > > Little Dev Day confirmed. Join the AMA with the API team right after the release! To prepare for the event—or if you’re unable to attend—you can post your questions here in advance. The AMA will begin at approximately 2024-12-17T18:30:00Z (UTC), right after the Shipmas live stream. You can also follow @OpenAIDevs on X (Twitter) to learn when the event will start. [https://x.com/OpenAIDevs](https://x.com/OpenAIDevs) We’re looking forward to your questions, helpful answers, and an all-around great time! See you there! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/22): > > Would love to hear from the API team if the endless newline bug was fixed with streaming structured outputs, where the API spits out newlines for 90 seconds and then disconnects. Gif of it [here](https://x.com/louzell_/status/1850014719224185215). In the community forums [here](https://community.openai.com/t/structured-outputs-infinite-n-newline-characters/953491). Thanks! > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/95): > > Hey Lou – sorry about this unfortunate behavior. This can happen when the model outputs are sufficiently improbable so the only valid token per your schema is the newline token. It’s not really a bug in that engineering changes cannot fix it. When this happens, I would suggest trying to use a simpler schema or changing your prompting so it’s more clear to the model what it should respond with. Let me know if this helps! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/51): > > Will SORA have an API, if yes When? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/96): > > We don’t have plans for a Sora API yet, but we’d love to hear more! What will you build if we ship one? 🙂 Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/49): > > Assistants API for over a year now in BETA. Are the Assistants API dead-end? is it just proof of technology? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/98): > > We’re seeing a lot of developers building with the Assistants API, and we’re continuing to invest in the tools developers need to build agents. Next year, we plan to bring o1 to the Assistants API and will have more to share soon! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/11): > > At the moment, I am interested in information on the following questions: > > 1. Updating the API whisper on V3? > The current version of V2 is not as accurate and contains hallucinations. > 2. Expanding the capabilities for speech control in the API TTS? (pauses, emotions, etc.) > 3. Voice cloning > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/104): > > Nothing to share yet on V3 Whisper in the API. But for both audio understanding and TTS, do check out the new [GPT-4o mini audio preview model](https://platform.openai.com/playground/chat?models=gpt-4o-mini-audio-preview-2024-12-17). It’s got state of the art speech understanding and you can prompt the model directly to control how it hears and speaks! For example, give it a prompt like "Say the following in a somber tone, and make sure to pause your speech appropriately: " Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/21): > > When do you increase the maximum TOKEN (input) in the API? We are developing a data analysis tool and this is a must. As it is now, it will be replaced by gemini. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/108): > > It’s something we care about! Giving the model more context and examples is a great way to get smarter responses. Nothing to announce just yet but stay tuned in 2025 Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/47): > > is there an option to send image frames to realtime api like in the new vision mode in advanced mode ? i am working on smart glasses for blind > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/112): > > Not yet! The model has been trained for this capability (you can play with it in ChatGPT now!). Stay tuned for it coming to API next year Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/83): > > I’ve tried the Realtime API a few days ago in Portuguese and unfortunately the results were pretty bad. I assume it uses whisper small under the hood which makes it useless in Portuguese. Should we expect a better performance with todays release? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/115): > > It’s definitely worth retrying! Both GPT-4o and 4o mini have improved meaningfully in multi-lingual understanding with the latest snapshots. We still use the same Whisper model to transcribe what the user said, but then GPT-4o processes the audio directly and responds (without going through a transcription). Would love to hear what you find? Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/63): > > Can there be an easier way of doing continuation (with structured outputs and streaming)? Right now it’s rather difficult to stitch 2 assistant responses appropriately > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/116): > > Interesting question, and not one I’ve encountered before! It really depends on your schema and what you’re looking for. You may want to consider a different schema for your second response, which has something like a “continuation” key to make it clear to the model that it is supposed to add to its earlier response. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/5): > > Great news, devs have been waiting patiently for our Shipmas present! > > With o1 now having vision capabilities (which I presume will be released in the API in the future) how should we be thinking about using it for use-cases that typically require in-context learning? > Is it effective to provide the model a multi-shot prompt with each example having the ∼5 images that will be encountered in the real world, or is a different strategy recommended? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/118): > > We launched o1 with vision inputs today! Yes, we’ve seen few shot prompting to be quite effective to explain to the model how it should respond to new images. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/109): > > Im sorry if this question has been asked already, and I absolutely appreciate you all and the accomplishment you have attributed to so many communities and the world as a whole. However I’m wondering if o1 is now available in Playground under Chat? Because at this time it’s only showing o1 preview still. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/122): > > Which tier are you on? o1 has started rolling out to developers on [usage tier](https://platform.openai.com/docs/guides/rate-limits/usage-tiers) 5 today, and rollout will complete after a few weeks. Team’s working on adding expanding to more tiers. (You’ll get an email notifying you once it’s available to you in the API and Playground.) Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/10): > > - What is the API team’s take on [Model Context Protocol · GitHub](https://github.com/modelcontextprotocol) by Anthropic? > - Dall-E seed parameter, pretty please! We need to be able to generate consistent images. > - Output vision improvements in GPT-4o and future models. Specifically, the most important capability would be finding (x,y) coordinates in a specific image. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/123): > > - It’s a cool protocol! > - For DALL-E, thanks for the input! > - I hear you on the vision improvements. We’re working on something in this space now and hope to have some improved vision models soon. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/110): > > Is an ephemeral API key subject to abuse? For example, could a bad actor scrape a website every minute to keep getting new ephemeral keys to use for their own purposes? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/126): > > The keys are locked to a specific Realtime API session, so their use is substantially limited compared to typical OpenAI API keys. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/61): > > can preference tuning be used to encourage longer answers (say 10+ pages) and better use of output context ? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/131): > > Preference fine-tuning can encourage longer responses, however we still have a maximum number of output content tokens for each model. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/64): > > I had a look at the docs for the new ‘developer’ message in the chat completions. I’m not quite clear on if you handle the system vs. developer message at your end, or if it’s something we as developers still need to handle via `if 'o1' in model_name: do this else: do that` kinda thing? And a follow-up if I may - there was a mention of work you’d been doing on the relative ‘weighting’ of system vs developer vs user messages - is that documented? I’d be really interested in digging in if it was! Thanks! > > P.S No PHP SDK. 😭 The unloved and yet massively in-production step-child of the developer world. Santa? Can you hear me?.. 😉 > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/132): > > Great question on the `developer` message! The best resource to read more is our [Model Spec](https://openai.com/index/introducing-the-model-spec/) detailing the hierarchy: > > > Follow the chain of command. Subject to its rules, the Model Spec explicitly delegates all remaining power to the developer (for API use cases) and end user. In some cases, the user and developer will provide conflicting instructions; in such cases, the developer message should take precedence. > > Thank you for the note on PHP! With the addition of Java and Go today, our goal is to ultimately support the most popular languages and tech stacks. No timeline to share yet for a PHP SDK, but in the meantime, we recommend [community-supported libraries](https://platform.openai.com/docs/libraries#php) to get started. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/45): > > What was the most difficult task you’ve worked on so far at OpenAI? What is the most fun task you’ve worked on so far? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/134): > > Building Structured Outputs was one of my personal highlights! The engineering side was actually super challenging, in order to get constraining to work at scale but also without impacting latency. There was a ton of work there to bring everything together in a performant way. If you want to learn more about it, we talked about it here: [https://www.youtube.com/watch?v=kE4BkATIl9c](https://www.youtube.com/watch?v=kE4BkATIl9c) 🙂 Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/40): > > Is it possible to do both supervised and preference fine tuning on a model? > > Can you do preference fine tuning on an existing model fine tuned with supervision? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/136): > > Yes! It is possible to do supervised fine-tuning and then run preference fine-tuning. We have seen good results from running SFT first and then DPO after. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/24): > > Hi there! > As I am preparing for a rollout of o1 fine-tuning, how can I best prepare data, and build an intution, for this new mechanism? > My applciation is not gene matching; is there a way to conceptually guide / reward thinking steps? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/137): > > We are actively working on a guide around how to approach reinforcement fine-tuning, and will publish it once it’s ready! But there are still some details we wanted to work out first. > > At a high level though, I would say to keep several things in mind: > > 1. The underlying dataset are “tasks” – a set of instructions paired with an output that is the result of the task > 2. Make sure the task is autogradable via the options we make available in the API (it should be easy to verify if the task was done correctly or not). We currently support a set of graders, but will expand over time. I know this is a bit hard because you can’t see the set of graders available until we launch more broadly – but easily gradable tasks (i.e. string match) are more likely to work out of the box. > 3. Make sure that the task is clear enough that if expert humans do it, they also converge onto the same answer. > > More to come here soon! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/42): > > Hi there, what micro controller was used for the fawn? > > Thanks, > > Kyle > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/138): > > The microcontroller was an ESP32-S3. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/157): > > For the demo I used a [https://sonatino.com](https://sonatino.com). I have used the Realtime API Embedded SDK on multiple ESP32-S3 devices. You might even be able to use it in other ESP32s, performance wise I had a decent amount of overhead left. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/56): > > When will you be including search capabilities in the API. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/140): > > We’ve heard a lot of demand for search and browse capabilities in the API – especially from folks who want to build applications that are grounded in super recent information. We’re actively working on this and expect to launch something next year! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/74): > > Great work, team! > > Do you plan to keep lowering the cost of the realtime API? A 60% price drop is simply amazing but it keeps the costs too high for some of our products. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/143): > > We are always working on decreasing the prices of our models – so I would expect more here over time as well! I would also recommend trying the new 4o-mini realtime model we launched today, it’s quite a bit cheaper too. 🙂 Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/69): > > On structured outputs, > > 1. Should we be confident building products with client.beta.chat.completions.parse for Pydantic structured outputs? Or go the JSON schema route? > 2. Will o1-mini API soon be able to do structured outputs? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/145): > > Yes, you should be confident building with `client.beta.chat.completions.parse` for Pydantic Structured Outputs! > > Also yes, we are working on bringing structured outputs (and also other useful API features) to all reasoning models, including the o-mini series. Stay tuned. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/80): > > In the Playground, the beta feature for designing a json schema is really useful. Could it be given to us as an endpoint? Maybe we could then hook it up to a custom GPT as a function call so we can “chat with” our responseFormat. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/146): > > If you’re referring to the “generate” feature that auto-generates a json schema for you – this is just a simple call to our chat completions API with a specific prompt! You can actually see the prompt we use in our documentation: [Prompt generation | OpenAI API](https://platform.openai.com/docs/guides/prompt-generation#meta-schemas) Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/86): > > - Do we expect o1 and the realtime API to eventually converge? Or if we want an audio model that’s smart, will we need to call o1 as a tool call from the realtime API? > - Related: Do you have it on the roadmap for the realtime API to take video input? Or the o1 API to take audio or video input (similar to Gemini 2.0)? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/148): > > Today, the best way is with a tool call that you use to trigger o1 (probably using the [new out of band conversation feature](https://platform.openai.com/docs/guides/realtime-model-capabilities#create-responses-outside-the-default-conversation)). We’ll keep investing in making it easier to use more intelligence within the Realtime API > > Re video. Stay tuned it’ll come next year. The model has been trained for this capability so in the mean time, you can experiment with it in ChatGPT Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/92): > > Any plans to make prompt caching better? Specifically, allow specification of what content to cache, give it a variable name, and allow reference to the cache in subsequent API calls for a certain time duration. I know another company is doing this 🙂 > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/152): > > Nothing to share just yet. We focused the first pass of prompt caching on making it as easy as possible to use (no API changes needed, no extra cost for cache writes). It’s a feature we care a lot about though … would be curious how you’d want to use a more structured cache? Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/62): > > Where can I buy the Reindeer that speaks? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/153): > > The demo itself used a [https://sonatino.com](https://sonatino.com). You could use any ESP32-S3 though! > > It was connected to Sennheiser Enterprise Solution SP 20 ML that I got off eBay. You can just use any headphones though. > > The stuffed toy is [Record Your Own Plush 16 inch Reindeer - Ready 2 Love in a Few Easy Steps - Walmart.com](https://www.walmart.com/ip/Record-Your-Own-Plush-16-inch-Reindeer-Ready-2-Love-in-a-Few-Easy-Steps/734020747?wmlspartner=wlpa&selectedSellerId=101100540&gQT=1) Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/93): > > Great updates on Realtime API. > > - Any timeline as to when the issue with the realtime model dropping out of voice mode when reconstructing conversation history is going to get resolved? > - Also the audio getting randomly cutoff at the end from time to time. > These two are probably the most critical issues of the API right now. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/154): > > We made a bunch of improvements to audio truncations with the new models… would be curious if you still see problems there? > > It’s still possible to get the model stuck in text only mode if you give it a huge amount of text upfront. Known issue that we’ll keep improving in future releases. In the mean time, putting audio in the latest user turns can help the model rediscover its voice Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/78): > > - will OpenAI also support the “prefill” for greater output control that Claude already has? It seems quite easy to implement, or is there a reason OpenAI doesnot support this? > - for the direct preference optimization (DPO), is it better to sample the preferred response and the non preferred response from the model that is finetuned or does it not matter where they come from. It seems more logical to sample them from the same model, but it can be hard to get 2 responses in a clear different style. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/159): > > We don’t have a current roadmap for response pre-filling, but we will keep this in mind! For the DPO datasets, this can be obtained through human annotation, or some kind of A/B testing flow. For synthetic data generation, you can also explore some kind of rejection sampling with a model and an evaluation to help generate preferred and non-preferred outputs from the same prompt. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/73): > > Have you tried generating your own datasets for DPO using O1? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/161): > > Personally, I haven’t 🙂 - but this would be an interesting approach to try if you have a use case in mind! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/117): > > When can we expect the voices from advanced mode will be made available to the API. Right now they are two different voice. And when will you make available tools to control the voice emotion, tone, innotation, etc? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/162): > > No current plans to bring the ChatGPT voices to API > > In terms of controlling the voices, we’ve done a ton to make them steerable., especially on today’s new models. Definitely try prompting them to speak the way you want (accent, speed, excitement, age, emotion, etc…) Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/87): > > Here’s a clearer version of your question: > > Would it be possible to automate the DPO (Direct Preference Optimization) training data collection process? I’m considering two approaches: > > 1. Using a fine-tuned model as a preference judge > 2. Using an OpenAI model as a general preference judge > > The workflow would be: > > - Input: List of prompts > - Output: Multiple model responses per prompt > - Judge model: Assigns preferences between responses > - Result: Automated DPO training dataset > > Additionally, can DPO handle multi-turn conversations where we provide a complete dialogue (user-assistant-user-assistant) with only the final assistant response varying? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/163): > > Yes, it is definitely possible to do synthetic data generation for DPO, an important thing to keep in mind is to make sure your evaluator for generated samples is consistent across data points. Also, it is possible to fine-tune for multi-turn conversations! Just put the final assistant message in the preferred / non-preferred output. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/120): > > Just FYI the python/JS code blocks in the docs here are reversed > > [Reasoning models | OpenAI API](https://platform.openai.com/docs/guides/reasoning) > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/164): > > Thank you for the catch! We’ll fix this right away. 🙂 Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/144): > > Thanks for the GO and Java SDK’s. Any updates for Official Open AI PHP SDK? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/166): > > No timeline yet for a PHP SDK, but we’re excited to ultimately support all the most popular languages! We’ll keep you updated, and in the meantime, we recommend [community-supported libraries](https://platform.openai.com/docs/libraries#php) to get started. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/119): > > Is Realtime API out of BETA now? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/167): > > It’s still in Beta. There are a few more goodies we’d like to add before declaring it a GA Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/121): > > Is there a plan to have fine tuning for the audio models? And if so, when can we expect it? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/168): > > It’s something we want, but nothing to share just yet. Our biggest focus at the moment is making sure the base audio models are very high quality Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/85): > > Great mini dev day! > > Would love more clarity on using System Message vs Developer Message in the api? Has System Message gone away? Or Developer Messages are an addition to it? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/172): > > Developer messages are new in o1 as part of our recent work on [instruction hierarchy.](https://openai.com/index/the-instruction-hierarchy/) You can read more about them in the [model spec](https://cdn.openai.com/spec/model-spec-2024-05-08.html#follow-the-chain-of-command). You don’t have to think too much about the difference between these two, it’s effectively a no op for devs but we wanted to let you know in case any of your messages reference “system messages” so you can update it to reference “developer messages” instead. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/32): > > Could you please update the list of [API Community Libraries](https://platform.openai.com/docs/libraries#community-libraries)? There are some that are outdated and archived. > > For example, in Java, the listed library was abandoned months ago by its author, and on the other hand, there are more up-to-date alternatives such as: [simple-openai](https://github.com/sashirestela/simple-openai) (I’m the author) or Lambdua/openai4j to name a few. > > By the way, I followed your [procedure](https://help.openai.com/en/articles/6684216-how-can-i-add-my-api-client-to-the-community-libraries-page) to add my library to the list several times, but it never worked and I never received any feedback. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/176): > > Thanks for calling this out - I will both make this change, and revise the process for sharing these updates. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/141): > > When will we be able to fine-tune voice models? > Is this something in the works or on the roadmap? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/179): > > We have heard good feedback regarding audio / multi-modal finetuning, but right now we don’t have plans to release audio fine-tuning soon. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/50): > > Thanks for this AMA! Is there any word on when assistants code interpreter tool will be out of preview/beta? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/183): > > We plan to improve the Assistants API next year. You can continue to use our [`code_interpreter` tool](https://platform.openai.com/docs/assistants/tools/code-interpreter) in your apps and we’ll continue to ensure its performance! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/9): > > Assistants api has been in beta with associated rate limits for over a year now - why should we as developers continue to waste time prototyping solutions with OpenAI when we can’t actually ship anything. No roadmap or communication. Either kill it or release it - permanent betas are just toxic. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/192): > > I understand this has taken longer than expected but we plan to iterate on and improve the Assistants API next year. In the meantime, we’re committed to making sure that your apps on the Assistants API work performantly. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/29): > > Assistant API: citations with quoted text and bounding boxes of the quoted text. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/194): > > Sounds like a cool idea – we’ll discuss it with the team! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/30): > > Can we expect visual grounding for references cited by Assistants API, or visual grounding more generally? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/195): > > Not immediately, but this is an interesting idea! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/44): > > Is caching available in the assistants api? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/196): > > Yes, it is available in Assistants Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/46): > > 1. Can the Vector Stores associated with Assistants API be used by third party LLMs like OSS models/Gemini for context retrieval? Or it has to be one of the OpenAI models only? > 2. Is there a limit on the creation of number of Assistants? And can each Assistant along with the associated Vector Stores be associated with a unique organization, so that each organization has a separate RAG store of its own segmented by Assistant ID? > > Wish, you people brought some improvements to the Assistant API as well today. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/197): > > - Vector stores can’t be used outside the Assistants API at the moment, but we’ll take this as a feature request! > - No limits on number of Assistants. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/48): > > Hey team!! > Congrats on all the releases 🥳 > Just curious, are there any plans to allow PDF input for the Chat Completion API? Claude added it to their API for sonnet 3.5 and allowing PDF upload and it’s much better than having the developer handle the PDF to image conversion and sending it as images. > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/198): > > This is a really useful feature and we’ll work on bringing this to the OpenAI API. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/68): > > If I can control how long/hard o1 thinks in the API, what’s the difference with o1-pro (the upcoming API version), which is designed to think longer/harder? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/199): > > A high `reasoning_effort` tells the model to think longer in a single model turn. > > o1-pro uses techniques that go beyond thinking for longer and we’re planning on supporting o1 pro mode in the API! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/181): > > Does the new WebRTC requires (for Realtime voice) VAD implement notation or is that handled by OpenAI backend? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/200): > > VAD functionality is automatically handled when using the WebRTC API. Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/71): > > what’s the difference between o1-pro and o1 with long reasoning\_effort? > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/201): > > o1-pro uses techniques that go beyond thinking for longer. We’re working on bringing o1 pro mode to the API soon! Question: > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/102): > > More generally, what are we as developers not doing as much as you think we should? What do you wish we did differently, or more or less of? We take constructive criticism too 🙃 > [@AMA on the 17th of December with OpenAI's API Team: Post Your Questions Here](https://community.openai.com/t/ama-on-the-17th-of-december-with-openais-api-team-post-your-questions-here/1057527/203): > > One big one is evals! I see tons of developers not using evals at all and relying on vibes for rolling out changes to prod. Would highly recommend creating some simple evals using our evals product (or open source offerings) so you can update with confidence when we release new models. ---
**Author:** ![platypus](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/platypus/32/759972_2.png) [@platypus](https://community.openai.com/u/platypus)\ **Post date:** [December 17, 2024, 7:37pm UTC](https://community.openai.com/t/all-the-questions-addressed-by-the-api-team-during-the-december-17-2024-ama/1059780/3 "2024-12-17T19:37:10Z")
This is awesome, thank you @VeitB and all OpenAI staff who took time to answer these great Qs. ---
**Author:** ![sashirestela](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sashirestela/32/191115_2.png) [@sashirestela](https://community.openai.com/u/sashirestela)\ **Post date:** [December 17, 2024, 7:37pm UTC](https://community.openai.com/t/all-the-questions-addressed-by-the-api-team-during-the-december-17-2024-ama/1059780/4 "2024-12-17T19:37:40Z")
Hey OpenAI Staff guys, very thanks for your time to answer most of our questions. We hope you could take them to improve the OpenAI API.!! ❤ 🎉 Sashir ---
**Author:** ![Foxalabs](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/foxalabs/32/738355_2.png) [@Foxalabs](https://community.openai.com/u/Foxalabs)\ **Post date:** [December 17, 2024, 7:39pm UTC](https://community.openai.com/t/all-the-questions-addressed-by-the-api-team-during-the-december-17-2024-ama/1059780/5 "2024-12-17T19:39:44Z")
Awesome work collating all of that, VB!