Hello everyone,
I’m pleased to announce the stable release of LM-Proxy — an OpenAI-compatible opensource HTTP proxy for multi-provider LLM inference (Google, Anthropic, OpenAI, PyTorch).
Built with Python, FastAPI and MicroCore, LM-Proxy seamlessly integrates cloud providers like Google, Anthropic, and OpenAI, as well as local PyTorch-based inference, while maintaining full compatibility with OpenAI’s API format.
It works as a drop-in replacement for OpenAI’s API, allowing you to switch between cloud providers and local models without modifying your existing client code.
LM-Proxy supports real-time token streaming, secure Virtual API key management, and can be used both as an importable Python library and as a standalone HTTP service. Whether you’re building production applications or experimenting with different models, LM-Proxy eliminates integration complexity and keeps your codebase provider-agnostic.
I’d be happy to welcome your contributions and can provide friendly advice on integration, security, or answer any questions you might have.
Explore the project here:
PYPI: https://pypi.org/project/lm-proxy/
GitHub: https://github.com/Nayjest/lm-proxy
Support vendor-agnostic open source by starring the repository 