You’ll likely need multiple different AI systems for that, an LLM can definitely handle the everything that’s language based, aka taking queries and talking to users 
You’ll need to build an entirely different AI for classification of movement data and such, an LLM won’t be able to do that particularly well, no matter how much you fine tune it 
Yeah I agree, but it might be easier for an AI classifier to work with a bunch of stick figures without having to interpolate all the vertex data