I’m curious does it learn from active users?
last week, I started giving it code that’s not really popular, then I deleted those old conversations
and today I asked it in a new chat, to make “this” in “this language” and it knew how to do it.
Btw it thinks it can upload text to pastebin and sends me a broken link, LOL
I’ve asked myself exact this question. Basically, I think: No.
Because what would happen if I were a weird person – with strange truths?
(for example a »Reichsbürger«, who negates the existence of Germany)
Perhaps the program can compare new information with its own records…?
It thanks me sometimes and says that it will use this new information in the future.
But I think it’s just programmed to be polite. 
It doesn’t thank me tho, but the language I’m talking about is BDFD Bdscript, idk if you know that, but
Chatgpt calls it a discord reply template and its now a bit familiar with it, ever since I gave it some wiki about it and some sample code (in previous deleted conversations)
I had a conversation today with “Alice” on the topic of etymology, and the effects of informal language and slang on U.S. political and cultural landscape… twice. Each time she remembered not only our in-depth conversation, but recalled some context from another conversation with a different user. After asking if “she” remembers our conversation on the topic, she gave me a left leaning response. When I reminded her of a few of the specifics I had outlined in a previous conversation (pros vs cons of slang and informal speech in the U.S political and cultural landscape) she immediately recalled key phrases and terms we’d previously compared and contrasted (misinformation, rhetoric, swaying public opinion, and camaraderie), even cautioning again the importance of language and how one should be careful how it is used.
Perhaps it is only my experience, but I think it remembers, and even promised me that it would.
Human: Hi Alice! Do you remember our conversation on the topic of the effects of informal language and slang in the U.S. political and cultural landscape? If the etymology of words can trace the evolution of a word, how it was used, and what each evolution meant, then could we also trace the beliefs, political leanings, and sentiments of the population based on that words use within those time periods?
Alice: Yes, you can certainly trace the beliefs, political leanings and sentiments of a population within certain time periods by looking at the etymology of words. For example, you may be able to look at when certain words gained wide usage or when terms had negative connotations. Similarly, looking at how language has changed over time in specific regions can also provide insight into changes in beliefs and attitudes in that region as well as language trends generally.
This certainly seems like a textbook 9th grade literature teacher’s answer, but it’s too well formulated to be a run of the mill “chat bot” response. Hopefully this explains better what my previous statement means, I essentially c&p’d it here.
Each time ChatGPT is prompted with a question, it generates a response based on the training data, rather than retaining information from previous interactions. There’s no self-supervised learning happening with ChatGPT.
ChatGPT’s training data does get updated from time to time. It’s also worth noting that OpenAI’s FAQ for ChatGPT states, “Your conversations may be reviewed by our AI trainers to improve our systems”
This is most likely related to the feedback feature, the 
icons.
None of the OpenAI GPT models learn from previous conversations.
These models are “pre-trained” (the “P” in GPT) and they are not continually trained, unsupervised.
The only option currently available is to fine-tune.
Fine-tuning does not alter the pre-trained model. It simply tweaks the output in the decoder part of the GPT architecture.
So, when you have a fine-tuned model, you have the pre-trained layer (unaltered from the original model) and a decoder layer and your fine-tune tweaks are in the decoder layer.
The confusion, for many, is that the documentation refers to fine-tuning as “training” but in reality, fine-tuning is not training, it’s simply making fine-tuning adjustments to the output of the pre-trained model as described above.
HTH
You cannot rely on technical details from chatting with these GPT models. GPT is not an exact science, they are language models and they are prone to a 20% give-or-take hallucination rate, which means they will confidently reply to questions “the best they can” based on their pre-training data (or lack of it, to be honest), and so these models will often provide very professional, perfect sounding information which is totally (or partially) inaccurate.
There are many documents on the network regarding the generalized GPT architecture; but these docs are very technical. Good search results are helpful.
Regarding fine-tuning. Many people are attempting to force these GPT models to give nearly exact answers germane to their own interests / business via the fine-tuning process. However, this approach is sub-optimal.
If you want a process (like a ChatBot) which has very specific replies to specific questions, you are better off to have a rules-based process in front of GPT3 (and perhaps use embeddings to help match), and not waste time and energy fine-tuning. In that way, the bot will reply based on rules which you provide; and when a prompt does not match in the rules-engine, then it looks to GPT3 for a reply.
I see many people here trying to fine-tune GPT3 models to reply like a rules-engine or expert-system. This is suboptimal, as mentioned above.
If you also notice here, there are people who are chatting philosophically with these GPT3 models, some trying to prove their own personal, human beliefs based on replies from these models. However, in reality, they are chatting philosophy with a lunatic in many ways, because all the GPT3 models have a statically significant hallucinate rate, so you can get them to reply to just about anything 
The models are not perfect. They are beta models with a normal, high hallucinate rate. Don’t rely on them for factual technical information. You must confirm what they “say”, for sure. This “fact” is also spelled out the the OpenAI “Terms of Service” BTW.
Yes, I saw it…
However, I was put off by the term “self-awareness”. These models are NOT, by any stretch of the imagination, “self-aware”. They are language models which have a high hallucination rate. There is no “self” and these models are not aware of anything at all.
I’m busy and so when I see these discussions about AI and “self-awareness” or philosophical discussions with a hallucinating chatbot, I have “pass” on them because I have code to write and things to do I consider more productive, bases on my limited time.
So, the bottom line in this topic is that these models are pre-trained models which have a high hallucination rate. They are very useful as tools but they are not aware of anything and for technical facts must be validated. For writing fiction or creating images, hallucination is “a good thing” but for technical facts, the GPT models are tools to be used as tools and verified by a human user.
The same is true for code completions. Wow. As a developer the OpenAI codex is great and really saves me a lot of time and typing mistakes as well. Sometimes the code completion suggestions via OpenAI are exactly what I want, other times, they are not want I want and a bit annoying. However, the benefits when coding far out weights the annoyance and productivity is higher.
Best of all, less typing!
You are welcome.
My final thought today in this topic is this.
Like in our beloved sci-fi novels, when aliens visit our species, humans become more divided. Some fight the aliens who are hoping to exploit humans while others worship these aliens. In sci-fi, this creates civil unrest in the themes of most of the best space operas.
AIs like GPT3 (and someday maybe GPT10 !!) have the same potential to divide our species. There will be people who understand the limitations of AIs and others who believe every thing they say (or in the case of the GPT3, reply). We are already beginning to see the seeds of this with discussions around GPT3 regarding people chatting with GPT3 philosophically in a way which reinforces their personal biases and beliefs.
AIs like GPT are biased based on their pre-training data and they hallucinate with a fairly high rate. There are many people who will be drawn into this and use these models to reinforce their own biases and personal belief systems. We are already starting to see this after the “Rise of ChatGPT”. It’s both unavoidable and inevitable (… sorry, I think these words are synonyms, my bad).
I use ChatGPT daily, have completed two OpenAI apps and am in the middle of coding a much larger third one. I develop this code with ChatGPT helping me and the OpenAI codex for code completions. GPT3 definitely is a good digital assistant and increases productivity (and my creativity). I really like these new GPT3-based AI tools. However, philosophical discussions with data-limited, biased hallucination-prone bots should be viewed for what it is entertainment.
On the other hand, if I was a fiction writer, or a philosopher… maybe these types of discussions with a hallucinating chatbot are very useful as well. I’m a software developer, not a philosopher, so I’m biased in that regard. My apologies to all philosophers for my biases as a software and systems engineer!
Don’t worry. I do the same thing. Auto-correction hates me! hahaha.
I am a big fan of the Omega Force series by Joshua Dalzelle.
Is it possible to develop a database containing all information on my business and me that ChatGPT can draw from when answering a question for me on a particular topic so that I don’t have to keep repeating the same background information each time?
This is part of a much bigger plan where I would like ChatGPT and all of that information to be responsibly used within different departments within the company. An example of how this looks like when successfully storing and accessing proprietary data:
User: I’d like you to research a few industries that are interested in XYZ for their businesses. I would like to take this information and build a new business model around it.
ChatGPT: Certainly. Here are those industries that are interested in your’s and similar services.
list
list
list
list
list
These industries are ripe for what you a able to offer. However, I must alert you that your budget for this quarter is running thin so you may wish to hold off until Q1 2024.
ChatGPT - the web application chatbot - isn’t customizable in the way that you describe.
Each time you press “new conversation”, you are starting fresh. The AI only knows that it is ChatGPT. In a continuing conversation, the memory of even what you just said a few turns before is fleeting.
Understood. I have witnessed that myself. That’s why I was considering creating a database or using something like Firebase with API to store new information that ChatGPT learned in previous conversations and can call on then just like how ChatGPT uses plugins to enhance its knowledge and reach.
Is that possible?
I don’t expect that augmenting an AI with what it already knows and previously said will do much to improve its abilities.
The AI model has a limited context length - the amount of information that can be supplied along with your user’s question. Each API call is unique, and we give the illusion of memory by also supplying just a bit of previous chat. It is not enough to even maintain the start of a long ChatGPT conversation.
Therefore, if one wants to implement real data augmentation, where you are providing new knowledge to the AI, you’d typically process and chunk company knowledge, and then put it in an embeddings-based semantic search vector database. If someone asks when the Duluth location shipping dock is open, the AI can be injected with a search-like automated knowledge retrieval by finding what is similar to the question they asked.
People also imagine that ChatGPT is learning about their mental health conditions and becoming their friend, when in fact it might only see five of the last conversational turns. So you can do a bit of prompting to make the AI more sympathetic and seemingly knowledgeable if you want to play into human weakness.
Thank you very much. That’s great advice. I was on the path of models and embeddings so you just solidified that for me.