Direct Content Injection - Existing Feature?

Plugins
1

Hey All,

I’m making a plugin that occasionally doesn’t really need to transform data once it gets it. For example, URLs don’t really need to get processed through the model. More importantly, they waste a lot of tokens as many URLs have unique identifiers. So, I’m wondering if there is a feature that can inject content directly into the conversation.

I’m sure this could be useful in other cases where users are looking for exact content in a document and just need the quote ripped out. Again though, the bot doesn’t really need to process that type of data if the user wants it exactly as is. So I’m wondering if there is a method of doing this type of direct injection. Maybe there is a template style variable like {{content[0]}}?

Just wondering. As a solid example:

User: Please list all of the urls in my discord messages from April.
Bot: April 1st: [UFOs sighted over whitehouse!]({{var1}})
     April 16th: [Happy Birthday Old Man!]({{var2}})

That is basically the essence of what I am trying to do. But you can only do like 8 links because they take up all of the tokens. So my thinking was, it doesn’t really need to be tokenized, I can just inject it if there was a method for it.

Thanks in advance.

2

In the prompt response tell ChatGPT to not change the text and display exactly.

3

Sorry, I understand that method. What I am trying to do it is avoid the model from processing it at all. There really isn’t a need for it to go through the model if it is just going to spit it out exactly. It’s not super-efficient and in the case of URLs, they eat up a lot of the tokens.

My hope is that there is a technique where the model will print out {{var1}} and then the front end program, in this case OpenAI ChatGPT interface, can retro actively fill in the template variables programmatically rather than through the model.

In my particular case there may be a lot of URLs and spitting out only 8 links at a time is not good enough. Or in the case of people who want to add snippets of documentation or quotes, it would be nice to just jam some in there without getting a token penalty.

So was hoping there was a work around or a feature.

4

I truncate or paginate the data and then the user can ask for additional results.