Welcome to the community!
Since storage and compute is relatively cheap for me, I go with the full dims always, but it’s probably time to eval this again.
In this thread It looks like 'text-embedding-3' embeddings are truncated/scaled versions from higher dim version - #14 by LinqLover @LinqLover’s experiments seem to suggest that the error seems to grow rapidly below 512 dims in his use case, although the exact inflection point isn’t clear.
It’s possible that 256 dims, or perhaps even less, is super good enough for what you’re doing. After all, you’re not trying to find similar posts, you’re trying to ensure that no post is too similar.
You’ll need to find a proper cutoff anyways. If it turns out that you get too many false positives, and narrowing your cutoff doesn’t help, you’ll know that that you’ve picked too few dimensions.