1 Like
They do reference the work of others in the original paper who were able to achieve 50-75% efficiency gains. Before this paper, that would have been impressive. This model was 2 orders of magnitude more efficient. I think the key difference is that it performs well on trained tasks rather than few or zero shot tasks. In other words, it is not a good general model like GPT-3.
Here is the github repo.
1 Like