[Democratic inputs to AI] An Incentive To Label

tl;dr I wrote a blog post about having an “incentive to label” and wanted to re-share here.

I’m not sure if cross-posting a personal blog is “kosher” but am happy to take down this topic if it causes an issue.

Last April I wrote a blog post titled An Incentive To Label [1]. It is the result of work I did at Magic Leap & Waymo prior to moving into Web3; I know someone must be rolling their eyes but please keep reading.

The topic of AI Alignment has subsided from mainstream day-to-day but I still keep thinking about how:

  • Sometimes I, personally, do not align with myself when I know I should do something but I don’t.
  • Alignment is hard in small teams, let alone larger orgs requires strong leadership.
  • If we scale to nations or the entire world, the difficulty scales exponentially.
  • Getting alignment within a specific context requires VERY HIGH QUALITY data

The blog post I wrote covers an approach to:

  • Create public data sets in a decentralized/permissionless fashion by putting incentives in place.
  • Set a foundation to democratic inputs to AI that have “alignment” within specific contexts.

I have no immediate asks or agenda, but simply wanted to share some of my thinking/work in case anyone has been interested to work on a similar project.

[1] olshansky.substack (dot) com/p/an-incentive-to-label


Welcome to the community!

If you stick around a little, you’ll eventually earn the privilege of posting links :cowboy_hat_face:

All that said I have a question for you:

why do you think that?


Getting alignment within a specific context requires VERY HIGH QUALITY
d̶a̶t̶a̶ queries



Looking forward to it @Diet :slight_smile:

why do you think that?

My approach to this is build moreso around experience & intuition rather than a data-driven evaluative approach.

GPT4 is my go to LLM for my daily work because “it works”. I recognize it’s at the top of the leader boards, but it also passes my personal “vibes” check, and I believe it’s a result of the RLHF data being tuned to how I think & prompt LLMs.

When I was doing eval for adversarial planner evaluation, we didn’t need “big data” as much as we needed a very “high quality golden data set” against which we could do eval.

I have a gut feeling that the dataset OpenAI used in the RLHF fine-tuning step wasn’t retrieved through mechanical turk, but was likely a highly curated list from focused individuals.


I don’t really fundamentally disagree with you.

I guess it depends on what you define as “extremely high quality” - because quality is unfortunately subjective.

My concern is that “extremely high quality” becomes “aligned with our corporate/ideological/whatever goals” - I’d instead hope for neutral and unopinionated training sets, wherefrom the capacity to adapt to any given scenario emerges naturally.


@Diet Completely agree.

The idea between the design I proposed in the blog post is that there’s an incentive to create many different fine-tuning data sets depending on the model’s/architects need.

Have you had a chance to check it out?