# Faster Api throughput using Cerebras

**URL:** <https://community.openai.com/t/faster-api-throughput-using-cerebras/1357998>\
**Category:** Community\
**Tags:** api\
**Created:** [September 14, 2025, 3:56pm UTC](https://community.openai.com/t/faster-api-throughput-using-cerebras/1357998 "2025-09-14T15:56:23Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![jeffvpace](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jeffvpace/32/741732_2.png) [@jeffvpace](https://community.openai.com/u/jeffvpace)\
**Post date:** [September 20, 2025, 11:17pm UTC](https://community.openai.com/t/faster-api-throughput-using-cerebras/1357998/6 "2025-09-20T23:17:26Z")

</div>

Well, it’s true that enterprise customers need a protected environment and gpt-oss-120b could be a compelling solution when hosted by Cerebras…

But enterprises can ceate their own H100 stack and host their own gpt-oss-120b, right?

Enterprises can create their own H100 stack and host `gpt-oss-120b` on-premise. Hosting the model in-house offers greater control over data, security, and costs, but it requires significant investment in specialized hardware and infrastructure.

**Requirements for hosting `gpt-oss-120b` on-premise**

**Hardware**

- **H100 GPUs:** A single NVIDIA H100 80GB GPU is sufficient for running `gpt-oss-120b` for inference due to its highly optimized Mixture-of-Experts (MoE) architecture. For higher throughput and performance, especially under heavy load, deploying the model across multiple H100 GPUs (e.g., 4x or 8x) is recommended.

- **GPU memory (VRAM):** A single H100 provides the 80GB of VRAM required to load the model and its active parameters.

- **System memory (RAM):** The server or workstation requires a minimum of 128GB of system RAM for the 120B model, in addition to the VRAM on the GPU.

- **Networking:** High-speed networking, such as the 350Gbps offered on cloud services, is essential for minimizing bottlenecks, especially in multi-GPU deployments. **Reference designs:** Hardware vendors like Dell offer validated solutions, such as the “Dell AI Factory,” which combines PowerEdge servers with NVIDIA GPUs for implementing AI workloads on-premises.

**Software and deployment**

- **Inference servers:** To run the model efficiently, enterprises need to deploy an inference server. Common options include:

- **Model optimization:** Frameworks like NVIDIA’s TensorRT-LLM can be used to optimize the model for specific hardware configurations and improve performance.

- **Licensing:** Since `gpt-oss-120b` is released under a permissive Apache 2.0 license, enterprises can use it for experimentation, commercial deployment, and fine-tuning without significant restrictions.

**Business and security considerations**

- **Data sovereignty:** For enterprises handling sensitive or regulated data, hosting the model on-premise ensures that data processing remains within a controlled and chosen region, meeting data sovereignty requirements.

- **Customization:** Self-hosting gives enterprises complete control to fine-tune and adapt the model with their proprietary data, creating a more tailored and powerful solution.

- **Cost avoidance:** Enterprises can avoid recurring API fees and manage costs more predictably than with a pay-per-use cloud service.

---

_[View the full topic](https://community.openai.com/t/faster-api-throughput-using-cerebras/1357998)._