Hello everyone,
This past summer, I did some work on reviving and modernizing OpenAI’s Spinning Up in Deep RL repository and creating the relevant documentation to make it more accessible for today’s newest researchers, educators, and enthusiasts.
Spinning Up has been a cornerstone resource for those of us learning reinforcement learning, but many of its implementations were tied to older dependencies. I updated core algorithms (like SAC with Ant-v5 in Gymnasium), integrated Hugging Face + Kaggle pipelines for reproducible benchmarking, and created demos to visualize RL agents in action.
Links to my work and writeups are available:
-
GitHub (modernized repo): GitHub - monigarr/spinningup: MoniGarr’s Branch: fully modernized and operational fork of OpenAI's Spinning Up, now compatible with current Python (3.8+), PyTorch, Gymnasium, and MuJoCo ecosystems.
-
Hugging Face Space demo: Spinning Up SAC Agent (live agent simulation) monigarr (MoniGarr)
-
Kaggle experiments: SAC Ant-v5 RL Benchmarks MoniGarrr | Kaggle
-
Blog posts documenting the modernization process: researchengineer.wordpress.com
-
Reviving Spinning Up: A Modern, Accessible RL Framework
-
Benchmarking SAC Agents with OpenAI Spinning Up
-
What I Learned Benchmarking SAC Agents
-
Why this matters
My long-term goal is to push the boundaries of accessible AI research tools while contributing to OpenAI’s vision of safe, useful, and open AI systems.
This isn’t my first project. I previously produced the world’s first multimodal Mohawk language chatbots and AI system (Onkwehonwehneha AI) before modern LLMs existed. It was recognized globally in the late 1990s and earned awards for cultural + technical innovation. That work continues today as part of my Indigenous language NLP projects (polysynthetic low resource language revival in English speaking communities).
By modernizing Spinning Up and sharing benchmarks + reproducible experiments openly, I hope to provide value for:
-
New researchers who need working RL baselines
-
Educators teaching reinforcement learning
-
Teams building applied RL systems today
Let’s Collaborate
I’m looking to connect with:
-
Hiring managers exploring reinforcement learning, multimodal AI, or safe AI research.
-
Peers and open-source contributors who want to build on the modernized repo or share results.
-
OpenAI residency reviewers who are scouting for candidates passionate about applied research, reproducibility, and cross-disciplinary AI.
I’d love to hear feedback, ideas, or collaborations from this community.
Thanks for reading,
MoniGarr
(AI Researcher, Developer, Engineer, blending reinforcement learning, low-resource polysynthetic language revival & retention in English speaking communities, and open-source culture)