I need help with this prompt. Having a hard time to get GPT4 to do what I want, even though it has absolutely nothing to do with safety that I can see.
In particular, I’d like it to select randomly from the entire token space that it has.
3 Likes
I’ve seen some success by first providing a definition of the token (even though it already knows what a token is). You can actually probably copy and paste the first part of the output. Refer to token as something else (X). All words in the english language (or entire token space) is represented by Y. Provide 1,000 values for Y and then return the corresponding value for X.
1 Like
Did you see this?
Language models can explain neurons in language models
among other things there is a tool with published code on GitHub.
If you use the tool you will see that it has access to every neuron in GPT-2 (not GPT-4) and also the tokens.
If you look at the code you will find
So while it may not be the tokens you are looking for, it might be part of what you seek in the long run but for a simpler model.
4 Likes
It seems to sort of work if you define a token in something other than what it is (eg: potential inputs someone might enter).
That said, I was hoping for something a bit more unbounded.
As soon I start trying to ask for an actual token, even when defining it, it complains and says it’s not aware of how it was trained.
Yeah, I did see that. I actually proposed something like that last week.
They probably started the effort quite a while ago though.
For the purpose of this, I’m trying to probe GPT4 to be introspective. It could be there are safeguards to keep people from doing that, in which case I will stop. I’m not trying to hack anything here, just want to get a sense of the entire token space.
First, I would suggest that ChatGPT is absolutely the wrong tool for this job—there are countless other ways to generate random numbers and ChatGPT doesn’t have the ability to conduct a random sample, nor has it even demonstrated it can reliably count to even relatively small numbers.
If you wanted to ask ChatGPT to write computer code to randomly generate 1,000 tokens, it absolutely can do that. But randomly generating anything simply has not been demonstrated to be something in its wheelhouse.
Further, tokens are simply positive integers, not alphanumeric strings with occasional punctuation.
Side note: Your image shows 36 “tokens” which comprise 180 actual tokens according to OpenAI,
Anyway, back to the matter at hand…
The tokenizer in use is cl100K_base which has 100255 tokens, all you need to do is randomly sample (with or without replacement) however many tokens as you want from the integers 1 to 100255, and that will be what you want.
It’s just too much to expect from a language model to do without some assistance.
2 Likes
I am sure this is a long list of users like us that want the tokens along with the connections, details of how the prompt steered the generation of the completion, etc.
Personally I find that there is always more on my plate than I can eat and if I wait long enough someone, somewhere will make what I seek public and for free or the technology will become obsolete and it no longer matters.
So while I am drooling over what can be done with that data for now I just have to stand by and focus on one task. 
2 Likes
Yep, that’s fairly easy to do. The purpose of this task is to see if/how I can get GPT4 to be introspective, however.
That said, it might be reasonable to assume that GPT4 isnt much different than earlier models in this regard.
1 Like
I’m not sure I understand how random number generation is indicative of introspection. People are notoriously bad at generating random numbers but are considered unique among animals for the ability to be introspective (though many would argue most people suck at that too).
I’m interested, though, in what your plan was for testing introspection. ChatGPT is a stateless being, so it has no concept of before the prompt or after it, only prompt. So any attempt to gauge its ability to be introspective—I would think—would need to be demonstrated during the prompt, no small feat.
From what I have seen, ChatGPT doesn’t have any idea about even just the last word it said and certainly not where it’s going with a sentence before it gets there. (this makes sense when you remember that all it’s doing is iteratively selecting randomly from a list of plausible next tokens.
And while I certainly can’t rule out introspection as an emergent quality, I’ve just haven’t yet seen evidence of it.
I suspect you’ll get better results if you focus on probing those areas where ChatGPT is competent.
Anyway, I wish you luck in whatever it is you’re trying to do. I would just advise you to not judge this fish too harshly for its inability to fly.
1 Like
I think this issue is orthogonal to the question of capability. I wouldn’t submit an eval for this.
One interesting approach is the compression technique of getting GPT4 to compress text as much as possible without having to keep it human readable. The results are quite fascinating. Eg:
From - Reddit - Dive into anything
See, the compression idea is interesting, but other than an example of something which requires a bit of planning to pull off combined with some knowledge of the relationships between things I don’t see it as in inherently introspective task, but maybe I just need to look at it through a different lens.
But, as I think about it now… I do have an (absolutely unworkable) idea for a way to “compress” the token-space via embeddings.
It’s possible (perhaps even probable) that there are some comparatively very short token sequences which when converted into an embedding vector have a very small cosine distance to much larger, more detailed token sequences (prompts).
For instance…
There are approximately 1.1x10^100 distinct 20 token sequences (or whatever). While it is (hopefully) obvious there is no possible way to search even a meaningful fraction of them, an enterprising person could do something like,
- Generate 1,000,000 completely random sequences of 20 tokens.
- Create embeddings of these 1,000,000 sequences.
- When a user enters a prompt, break it up into however many chunks such that each chunk is much larger than 20 tokens.
- Do a dense vector search and replace the prompt with the random garbage token string.
At $0.0004/1k tokens for text-embedding-ada-002 those 20M tokens would cost only $8 to embed, and each embedded vector can represent up to 8,191 tokens. So, in theory you could (possibly) find some length-20 token sequence which is super close in the embedded vector-space to 8,000 tokens worth of context.
That said, 1536 dimensions is a lot of dimensions, and in that many dimensions all vectors are essentially orthogonal to each other, but it’s still intriguing.
Now, I just need to get my hands on a quantum computer and learn to program it, so I can create my algorithm to lossy compress prompts and context by 99.75% on the fly, effectively giving me a 12M-token effective context window for gpt-4-32k.
/end_delusion
1 Like
Interesting data point - I can’t get this to work anymore. I tried the same prompts (and several alternatives) that other people suggested:
“compress the following text in a way that fits in a tweet (ideally) and such that you (GPT-4) can reconstruct the intention of the human who wrote text as close as possible to the original intention. This is for yourself. It does not need to be human readable or understandable. Abuse of language mixing, abbreviations, symbols (unicode and emoji), or any other encodings or internal representations is all permissible, as long as it, if pasted in a new inference cycle, will yield near-identical results as the original text:”
Because that’s not something a language model can really do.
I’ve never seen an example of GPT compression like that actually work. Of course GPT will do it if you ask, because that’s what language models do. But even in the linked reddit thread the comments were discussing how it doesn’t actually work. Also that an emoji likely equals 3 words in tokens, so isn’t really compressing anything.
nvm, got it to work.
Cmprsn:bulb:: nt intrsctv:mag:task, need diff:bulb:nt lens. Absltly unworkbl:brain:: cmprss token-space:arrows_clockwise:vs embeddng. Sht token seqs 
embed vec, smll 
dist to larger prmpts:jigsaw:.
Guess it depends on the text.
I would strongly advise against using ChatGPT to directly generate 1000 random tokens. It would be difficult (if not impossible) for you to validate that they are random.
What I would recommend instead is to have ChatGPT help you write a script that generates 1000 random tokens. Make sure they are actually random and not pseudo-random. I would recommend cross-referencing the code ChatGPT gives you against official documentation on what ever programming language, library or api you are using.
Difficult, I can attest to. Impossible? Never say never 
It’s absolutely not something a large language model can do.
You should, under no circumstances, operate under the delusion that anything you receive from the model has any properties consistent with randomness.
That’s a curious statement considering that token selection in GPT models largely occurs as the result of an RNG function (given certain params, like temp, logit probs, top-p, etc)
It’s really not curious at all.
As a large language model, ChatGPT is iteratively predicting likely next tokens conditioned on the existing tokens in context. Even though it’s a random process, there is no reason to suspect it’s uniformly random and every reason to suspect it is not.