Ok, I just tried it out. It turns out that yes, simply buying credits increases the usage tier. You don’t need to start spending the credits that you bought for the usage tier to go up.
Edit Jan 2024: Someone else reported not being upgraded after prepaying. Perhaps more people can leave a reply on this thread to let everyone know if they got upgraded or not, so we get more data points.