Saw a lot of confusion around VRAM vs system RAM for running gpt-oss-20b locally, so I tested it across a few setups to get real numbers instead of guesses.
Setups tested: [fill in your actual specs — e.g., 16GB VRAM discrete GPU, 32GB unified memory Apple Silicon, 8GB VRAM + CPU offload]
Key takeaways:
- VRAM matters far more than total system RAM until you start offloading layers to CPU — after that, RAM bandwidth becomes the bottleneck, not capacity.
- Quantization level (4-bit vs 8-bit) had a bigger impact on usable speed than I expected — [fill in tokens/sec numbers if you have them].
- Unified memory setups (Apple Silicon) handled larger context windows more gracefully than discrete GPU + CPU offload, at the cost of raw throughput.
Happy to share exact configs/settings if useful — curious what setups others are running this on and what corners you had to cut.