Video and AI - Help needed

Hi everyone,

I am new to all of this, I thought I would join to talk to some smart people !!!

I have a challenge.

I am a sports coach and I’m interested in the ability to have a sequence of play in a game (say rugby union) which is captured in video and convert it into animation.

The animation I’m looking for is where all the players become dots and move around the field. Currently we have the ability to do this manually which takes time.

Does anyone know a piece of software or AI platform that can help?

I have attached an example of the type of output i would like to generate from the video.

look forward to hearing from you clever people !!!

‘cheers
Al

1 Like

Not an expert on this but I can share something.

There is a tool called YOLO that detects objects in a video. I think with some fine tuning you can make it to detect players with a high accuracy.

I don’t know how you can turn this into an animation but I bet chatGPT or Google has an answer

1 Like

Thank you very much.

I appreciate you taking the time to reply.
I’ll look into it.

Appreciate it

Cheers
Al

I was thinking about this during the Olympics, when the USA basketball team wasn’t performing all that well. But I was thinking of something bigger, more complex, and on the cutting-edge I suppose. What you are looking for is in 2d, which is really good, as that isn’t super hard to do, won’t require a huge amount of compute to do so either.

For your task specifically, I would turn this into a simple video game, perhaps using a python library called pygame since it is 2d. In sequence, I would put all the rules there so it follows what you do manually do while following all the rules. I would then train it, perhaps using adversarial or evolutionary based algorithms, always checking if they are improving or stagnating.

So, in my view, ML (machine learning) is still the way to go about this. Perhaps using a python library called pytorch. Maybe in the future there will be ways to do this using generative pre-trained models and fine tuning without costing a bunch of money or without having to do a bunch of work on it to get it just right.

A great example is this video, but that is 3d. This is a simple and shorter video, in 2d.

I think perhaps this is the closest video to what you are looking for.

Now, there are some issues with just using 2d. It lacks a lot of information about what is going on, so you might get amazing results from it that don’t necessarily translate into the field all that well. In case you do decide to build something yourself, in 2d, there is some things to consider. While training, you might also find that the AI is finding bugs in your game and exploiting them, so you’d have to fix the bug to make sure they don’t create a strategy around something along those lines. An example of this exploitation of a non intent possibility can be seen in this video from 3 minute papers where the AI learned to jump over the walls.

In terms of 3d sports playing there is this video also from two minute papers, but I don’t think this is open source, as in, you won’t be able to access this to use it for your use case.

best platform for you to build the functionality you are looking for? in my opinion, ChatGPT.

2 Likes

Hi Thiago,

thanks a million for this. great.
I’m not a programmer unfortunately and i have tinkered with motion capture software etc.

I was wondering if AI is at a stage where i could just grab a video clip and automate it. The insights with ChatGpt i’m getting at the moment are very good based on my prompting (which i’m learning) but it’s only using these stills. Wouldn’t it be great to put a video in and get the same thing? thats where my curiosity lies.

Appreciate you taking time to respond.

Cheers
Al

1 Like