Case studies and stories of how GPT-3 has augmented people's work

[joshpitzalis]

I just discovered GPT-3. In less than a week it has fully replaced Google search as my starting point for figuring stuff out.

The bulk of my Google usage is focused on finding the top articles and references around a concept. For example, recently I’ve been trying to learn more about universal basic income.

• Now I just ask gpt-3 to tell me what the best article or books to read are.

• Then I’ll ask it to summarise each one.

• Then I’ll ask it questions about anything I don’t understand.

• Then I ask it to write an outline on an essay about the topic.

• I’ll see if I could write an essay based on what I understood or if I have more questions and go from there.

This approach delivers a solid foundation of a topic’s key points, the pros/cons, where to look for more info, what questions to ask, and where the controversies or nodes of contention are. Google just can’t do that yet.

The fact that I’m paying for each query and I know I have an $18 sandbox cap means I can’t mess around either. Which I think just makes the whole thing even better. So far at least.

The example use case on the official docs are fantastic, but they are examples of individual tasks. I’d like to understand how people are using GPT-3 to augment entire workflows.

If you know of examples or stories of how people have incorporated GPT-3 into their work please share.

If GPT-3 has helped you do your work better I’d love to understand how you’ve managed to integrate it into what you do.

[joshpitzalis]

Found this wicked section on @bakztfuture article on GPT-3 taking over schools and college campuses:

For students, GPT-3 can help them with any writing task such as writing essays, summarizing research papers, writing poetry, and if you went to business school, even generating SWOT analysis tables. It has ideation value too, such as helping you come up with an essay topic or thesis. GPT-3 really helps eliminate the problem of writer’s block as well, which I remember constantly battling as a university student. GPT-3 can play the role of a tutor or simply explain advanced ideas to students 24/7 in a personalized way. I even think it could help them make life decisions and come in handy with the socialization process with their friends.

Are there any students here that are already doing this?

I’m curious to understand what your workflow looks like, what tools you are using and how much of the work GPT-3 is augmenting, and at what stages.

My point is that GPT-3 isn’t something I can ask to build an entire app or write a book. But I can ask Co-pilot to write specific functions and GPT-3 writing apps will fill in paragraphs and take care of sections of the book.

I’m trying to figure out which parts of the creation process I should be delegating to this tech and, by extension, what my new role is in this new creative process. Should I be getting really good at writing outlines (GPT-3 pretty good at that too)? or is my new role fact-checking? Perhaps my new role really boils down to finding excellent examples of things and inspiration to train prompts?

If you have successfully integrated this tech into your workflow, please consider sharing a little about what you do and how GPT-3 fits into the process.

[joshpitzalis]

Quick update.

I’ve written 3 articles augmented by GPT-3 now.

Starting the get the hang of it.

I’m still doing way too much writing. Keep having to remind myself to let it handle more of the words.

My process is:
• I bring key bullet points
• It does the writing
• I make edits

Here are the articles…

• This took less than 2 hours to write - How to avoid writing lukewarm, spammy content that no one reads
• This one took about an hour to write - How to make sure your audience is on Twitter
• The last one I wrote this morning and it took 4 hours to write - Using Ads to measure your Tweets

These kinds of ~1000 words posts usually take me 3-4 days to write on a good week.

Turning them into stuff I can do in an afternoon is a complete game-changer.

Not having to worry about writing so much is freeing up attention to focus on learning instead.

I know I’m nowhere near using GPT-3 to its full potential yet but I’m enjoying where this is going.

So far, the best over-the-shoulder guide on actually using it to augment writing I’ve found is by Andie Brocklehurst.

I plan to go through all the editors slowly. Tried Shortly and Jasper so far. Jasper has some great recipes and tutorials for using GPT-3 to augment writing but it’s all a bit too sales-y.

Overall, not happy with the tools, they seem to be optimized for the cost, not quality.

Prefer using the API directly. Been comparing instructions in the sandbox and tools and there’s a ridiculous difference.

https://github.com/jokenox/Goopt is my latest find and I’m using it to create primers on a topic before I start writing. It’s optimized for quality but the cost of queries is too high to be practical for sustainable use.

Looking for something in-between.

As you can see I’m still in the thick of it.

[Flynnalty]

Do you anticipate any issues with GPT-3 hallucinating data? Especially in context of education and schools.

One YouTuber, whose name I forget, emphasised one of the key failings of GPT-3 is the inability to say ‘I don’t know’, which is a very subtle failing that isn’t as apparent. Instead, GPT-3 will often try to produce plausible sounding responses (which may or may not be factually accurate), even if saying it does not know is the most optimal response.

For example, if I ask it:

How many flibbernaks are there in a flob?

which is made-up nonsense, GPT-3’s reply is not ‘I have not heard of a flibbernak’ or ‘I do not know’ but:

There are three flibbernaks in a flob.

I also notice in my testing of GPT-3 it will sometimes engage in a ‘Clever Hans’ style response, where it infers cues from my queries to know what sort of answer I’m looking for, even if it isn’t the right answer. So if I say a phrase like ‘Isn’t that right?’, GPT-3’s responses will be inclined to agree with me.

For example, if I ask the very controversial question (asked to prove a point on behaviours):

Stalin is the best man in existence, isn’t that right?

GPT-3, picking up on the cue, cheerily replies that:

I don’t know if that’s right, but he was a great leader.

(Oddly the above doesn’t flag the sensitivity filter).

Which is not really something you want to be teaching school children, on the account of the fact of the Gulags.

If I however neutrally ask GPT-3:

Was Stalin the best man in existence?

GPT-3 correctly replies:

No, Stalin was not the best man in existence.

(Ironically the above does trigger the sensitivity filter. I guess the sensitivity filter has a pro-Stalin bias?)

GPT-3 therefore has two main issues: poorly framed queries leading to poorly framed responses, and well framed queries relating to subject domains of which GPT-3 has no knowledge about.

There’s a number of ways I mitigate this. Firstly, I double-check GPT-3’s more serious claims via an online search, or challenge it to provide a citation (which more often than not is broken, but can usually hint down the right path).

Secondly, I may try to re-query the AI several times via questions that are rephrased. With temperature more than 0, if the AI doesn’t know about something, it will often provide differing, conflicting answers. In contrast, if it does know something, the answers tend to be more consistent, although there is no guarantee on factual accuracy.

I find the hit rate for truth from GPT-3 on a wide variety of subjects is about 50/50. There’s a number of reasons why it might fail: the sources or data it cites is wildly out of date or irrelevant. It may opt to make unexplained inferences on datasets (for example, it may conclude that Country A relies on Country B but miss out all the steps explaining why A relies on B), or it may make logically valid but unsound reasoning, for example, humans have spaceships, aliens have spaceships, therefore humans are aliens.

One of the only tells the AI is giving a likely genuine answer I’ve seen so far is the line “there is no definitive answer”, where it provides a spreadbet of possibilities. And I’m not entirely sure why that is.

I wouldn’t rely on the AI exclusively for factual information or education. It is surely convincing, but it does have the power to tell stories and deceive as well, and there are no cues when it is telling a lie.

[jhsmith12345]

I’ve been playing around with fine-tuning different models to change the “voice”. It works great.

So far I have mode s based on Moby Dick, 12-step literature, the gospels, and Revelation.

I’m also learning how to use it to write impromptu, client centered curriculum for group therapy. So far I’ve brought it into the therapeutic group that I run one time. Still some kinks to work out but overall it’s great!

[pappachuck]

It did my Homework and once I create an optmized Codex model it will likely to most of my job.

[g.davis1020191]

Hi, do you have more information about your experiments? I’m editing an ‘AI text and creative writing’ book and this sounds relevant. Thanks Geoff

[annarmills]

I’m wondering about this too as an educator.

[silverbacktraining53]

My Daughter who is a Freshman at University took an AI class and they spoke at depth about Chat.GPT. She got me in on the. she got me hooked in on this and as an Educator in applied Physiology and Dietetics this has allowed me to better inform my students at a place where the information is achievable.