I just applied for OpenAI software engineer, ML framework Job, and I wondered if OpenAI offers training for new employees since I already know a fair amount of python but would like to be a good fit for the position.
Good luck Michael!!!
It looks like a role where writing performance critical code (ie fast, scalable, bug free code) is critical!!
If I had to guess, since GPT4 seems to be GPT3 but more efficient, presumably you’ll be tasked with optimizing core components of GPT3 (bfloat16, quantization, sharding, switch transformers, fast MoE, lottery ticket hypothesis, better memory management, better FLOP usage, better parallelization etc).
I’m not an OpenAI employee sadly, but I’m more than happy to give tips if you’re interested! I previously made TSNE 2000x, made SVD use 50% less memory and was researching Randomized Sparse SVD, faster NMF, PCA, linear regression etc methods for NVIDIA!