It has already been hardcoded in, don’t waste your time with naming conventions, check out ie compare and contrast w\ GitHub - NVIDIA/Megatron-LM: Ongoing research training transformer language models at scale, including: BERT & GPT-2
| GitHub - microsoft/DeepSpeed: DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.