Marble Run AI Project Prototype

[norvum]

Being a fan of marble runs/races and wanting to experiment/learn more about the latest AI tech that has been surfacing I decided to mix the two together to bring you this early prototype.

Marble Run where GPT is the sports commentator.

Tech Used.
Unreal Engine 5
OpenAI API (Testing done with GPT 3.5 and 4, video is recorded on 4)
11Labs for the voice synthesis.

The only thing that is baked in so far is the intro, the actual race is entirely real time.
I have some ideas for improving the AI latency aside from just switching to GPT 3.5 which seems generally faster but of course any other tips that comes to mind feel free to share.

Marbelous Run 1st Race

Fresh upload so won’t be HD for another 45 minutes but SD is up.

[N2U]

Cool project mate, love to see it!

I think your project does a far better job at commentating than most twitch streamers

Are going to put this on GitHub or somewhere else? I think a lot of people could learn something from your project, and have a lot of fun doing so.

@norvum

I have some ideas for improving the AI latency aside from just switching to GPT 3.5 which seems generally faster but of course anything other tips that comes to mind feel free to share.

I know that many tv channels delay their live feed by 5-15 seconds, to allow censoring of profanity and live editing, such as inserting an information overlay. Maybe you could do the same to make the commentary “real time”?