This is a great initiative, thank you!
I am building something that uses a wide range of OpenAI APIs, from whisper to reasoning models to the amazing new images generation!
It started as an idea to help me out creating slideshows for the stories that my mum sends me on Whatsapp, and it is now becoming quite a complete dashboard that takes a narrated story as an audio input and outputs an mp4 video.
The project can be found on GitHub: openai-audio-to-video
An example video made with this tool is this one: Farmita en la Farmacia
My expectations are:
- I do not want to make any money with this, I just want a decent tool to make these stories happen in a nice and charming way, without “wasting” too much time repeating prompts over and over again.
- It would be great to come up with a free and open source solution that anyone can use, to make their stories become alive too. The world needs more stories!
- In that sense, it would be awesome to get help, either with coding or with prompting tips or whatever!
- And it may be too much dreaming, but it would be hyper awesome if OpenAI offers some kind of hosting for open source projects like this one where people can plug in their own API Keys and use it… the possibilities for open source ideas that use OpenAI APIs and build a cool wrapper tool around it are endless.
My challenges are:
- Style and character consistency: most difficult thing to manage, but it is getting way better now that I added the references gallery, which allows me to easily add reference images to each prompt. This has been the best addition since dalle-3, thanks!
- A lack of a real 16:9 proportion: i fixed this by adding an image cropper that lets me crop every generated image into a proper 16:9 proportion. I wonder why OpenAI does not offer this format, being it such a common one. There probably is a reasonable reason, but I wonder what it is.
- How to properly mention and use reference images in the prompt: I started a separate topic on this matter but didn’t get any response yet.