Using Sora to create new musical instruments and simulate what it would sounds like with true acoustical physics

I imagine Sora will be able to insert sound in the video related to the contents after listening to millions of videos but even teaching it physical acoustic properties of metals, glass, wood, plastic, rock based on shape, volume, density and material.

I feel this would reinvent musical instruments beyond what we could imagine. I hope Sora would also be able to create an STL volume based on 360 film for 3D printing and mold making. I always wondered as a digital sculptor what the instrument I shape would sound like based on materials used and have searched for and requested software for engineering and simulating the sound accurately.

1 Like

From The Forum SORA: wait but does it come with sound?!

AI Music is becoming advanced, yes, but words could be implemented…

Multiple recordings of voices from different genres could be recorded with all the words in the dictionary and some slang. Cusses and inappropriate language could be excluded. Then, have them recorded in different tones, feed them into the system, and have the AI find the best fit by comparing the tags from the request and the ID of the word. This could help it sound more realistic.

A more complicated solution would be using today-music-generation tech to process the sounds into words, but it’s pretty out there with today’s technology.

Again, you could record instruments in different pitches and do different things to ensure the notes’ diversity, then incorporate those like with the words, or you could do the out-there generation.

You could overlay the normal generation, words, and instruments to create a full song or do them separately to make a good sound.

You could also record sound effects, which would add to the overall feel of the video, such as activity effects, which are caused by something happening, or background effects, like birds chirping.

These are but some ways that sound could be added to OpenAI’s Sora’s Video Generations.

1 Like