I was surprised by how much of an effect small changes to the prompts made on the output. I used data from Hacker News (news.ycombinator.com/). Here are my results:
I use ‘note-taking’ as an example domain for research. I ranked users based on how many note-taking related posts they commented on (since the start of 2022). I then used GPT-3 to generate reports about the top ranked users. Here are two reports of the highest ranking users (renamed ‘Alpha’ and ‘Bravo’).
User Alpha is an active user on Hacker News and has been engaging in conversations related to note-taking for the past 12 months. He is knowledgeable about Obsidian, Confluence, Markdown, WYSIWYG editors, and open source alternatives to Evernote. He is also familiar with Chrome, Chromebooks, Rust, and Microsoft Windows. He values file uploads, does not like tags for querying, and prefers a directory approach to organization. He is also aware of potential risks of using new software and is willing to ask questions. He mentions holding multiple roles in academia.
User Bravo is an active user on Hacker News who has posted and commented on various topics related to note-taking over the past 12 months. He is an ex-Roam, daily Obsidian user who uses Obsidian for navigation and discovery, with a template to fill in most of what he needs. He is impressed by Obsidian and Dataview, and likes the Excalidraw plugin for its power for visually-oriented users. He prefers to keep notes offline, local, and portable. He is interested in Xournalpp++ and the ReMarkable tablet, and is open to trying new software. He highly recommends Obsidian as a note-taking tool, and finds it powerful, flexible, and extensible. He is interested in self-dialogue as a journaling strategy, mental health, and self-improvement. He enjoys humor and is open to other people’s opinions. He is interested in productivity, creative output, and personal development. He knows about Obsidian’s Outliner plugin and Workflowy.
I describe how I got here on my blog: Using GPT-3 and Hacker News for slightly creepy market research
Here are the prompts I used:
- To summarize a user’s conversation in each post:
Hacker News is a website where users share links and discuss the content using comments. The following is a conversation between user '<USER A>' and user '<USER B>' under a post titled '<POST TITLE>': original comment by <USER A> : <root comment> <USER B> replying to <USER A>'s comment: <child comment> Summarize <USER A>'s conversation in a first person point of view. After the summary include a list of relevant details about <USER A> that would be useful to a marketer. Summary:
- To summarize a year’s worth of activity:
Hacker News is a website where users share links and discuss the content using comments. The following is a timeline containing summaries and relevant details of <USER A>'s activity over the last 12 months. post: <POST 1> posted: 12 Months Ago <USER A>'s activity: <SUMMARY GENERATED FROM PREVIOUS STEP> relevant details: <LIST OF DETAILS GENERATED FROM PREVIOUS STEP> post: <POST 2> posted: 10 Months Ago <USER A>'s activity: <SUMMARY GENERATED FROM PREVIOUS STEP> relevant details: <LIST OF DETAILS GENERATED FROM PREVIOUS STEP> I am a researcher trying to conduct market research online. I'm currently trying to understand the note-taking app market. Here's my detailed summary of what I learned about <USER A> with regard to note-taking. Summary:
What could I have done better? Please and thank you.