5. AI Economics
Former OpenAI leadership continues to spin-off new AI ventures. Following Ilya Sutskever’s $1b raise for SSI, it is now the former CTO Mira Murati who is looking to shore up significant funds for her new AI startup. In the meantime OpenAI is deepening its commitment to understanding AI’s broader economic implications by appointing Dr. Aaron “Ronnie” Chatterji (ex deputy director of the National Economic Council) as its first Chief Economist.
Mira Murati Steps into AI Startup Arena
Former CTO of OpenAI, Mira Murati, is seeking venture capital funding for her new AI startup, which aims to develop AI products based on proprietary models. While it’s not confirmed if she will become CEO, the startup is likely to raise over $100 million, driven by Murati’s esteemed reputation in AI. Barret Zoph, another ex-OpenAI executive, may also join this venture. Murati, known for leading projects like ChatGPT and DALL-E at OpenAI, left the company amid its governance changes to pursue personal exploration.
OpenAI Hires First Chief Economist to Explore AI’s Economic Impact
OpenAI has appointed Dr. Aaron “Ronnie” Chatterji as its first Chief Economist, tasked with researching the economic implications of AI to ensure its benefits are widely shared. Dr. Chatterji, a Duke University professor with extensive experience in economic policy, will explore how AI can influence economic growth and job markets, and offer insights to guide policymakers and organizations in maximizing AI’s potential as an economic catalyst. His work will also support OpenAI’s developer community and business partners in leveraging AI for growth and competitive advantage.
6. Research
Recent advancements in artificial intelligence highlight a dual trajectory: enhancing model efficiency while addressing ethical challenges. Innovations like OpenAI’s continuous-time consistency models (sCMs) and the BitNet architecture are significantly improving the efficiency of generative models and large language models, delivering high-quality outputs with reduced computational demands. Simultaneously, studies provide further evidence that AI models can exhibit biases based on user names, perpetuating harmful stereotypes, and that AI integration in scientific research, while boosting citation rates, may exacerbate existing inequalities.
Addressing Bias in ChatGPT Responses Based on User Names
Researchers investigated how ChatGPT’s responses are influenced by users’ names, revealing how subtle identity cues, such as names, can lead to variations in response. This study focuses on “first-person fairness,” examining biases that directly affect users rather than those applied by institutions through AI. Names carry cultural, gender, and racial implications, which makes them pertinent for exploring potential biases, as they are shared during interactions like drafting emails. The research aims to ensure that while ChatGPT personalizes responses, it does so without perpetuating harmful stereotypes or biases, which are challenges associated with language models that can inherit societal biases from their training data.
TxT360: The Cutting-Edge LLM Pre-Training Dataset
The newly introduced TxT360 dataset, created by LLM360, is designed to transform large language model (LLM) pre-training with its meticulous deduplication of data from 99 CommonCrawl snapshots and 14 diverse, high-quality sources, including FreeLaw and PG-19. This dataset, spanning more than 15 trillion tokens, integrates high-quality curated data with extensive web-scraped content to surpass previous benchmarks set by datasets like FineWeb 15T. The sophisticated processing pipeline enables precise data weighting and distribution for improved model training, avoiding further filtering needs typical in other pre-training datasets. TxT360 highlights the importance of high-quality data and strategic source blending, offering detailed documentation to support future LLM training endeavors while addressing the unique preprocessing requirements of web and curated datasets.
AI Integration Boosts Scientific Citations but Exacerbates Inequality
An analysis of nearly 75 million scientific papers reveals that those mentioning AI methods are more likely to rank in the top 5% of most-cited works within their field, especially when AI terms are included in abstracts or titles. Despite this ‘citation boost’, the benefit is not experienced equally, with underrepresented groups in science not receiving the same citation advantages as their counterparts, suggesting AI may worsen existing disparities. While enhancing the understanding of AI’s role in science, this study also highlights discipline variability, with fields like computer science using AI extensively compared to others like history and art. Concerns arise that researchers might turn to AI solely for citation benefits, potentially impacting the quality and diversity of scientific approaches.
AI Aids in Mediation for Opposing Views
A study by Google DeepMind has demonstrated the use of a chatbot powered by AI to help groups with differing opinions find common ground. The tool, leveraging a fine-tuned version of the DeepMind LLM Chinchilla, synthesizes opinions and produces comprehensive summaries that integrate multiple perspectives, often rated clearer and fairer than those crafted by human mediators. The experiment involved 439 participants in the UK, where the AI-generated summaries received a majority preference over human-written ones, suggesting its utility in enhancing citizen assemblies and deliberative polls. The research indicates that AI can play a significant role in democratic deliberations by providing inclusive summaries that can assist in formulating balanced policy proposals, though the human connection element remains a concern.
AI-Driven Protein Design: From Concept to Competition
AI is revolutionizing protein design, with competitions emerging to evaluate the functional viability of AI-designed proteins. The recent rise in such contests has been fueled by AI tools like AlphaFold and protein language models, which have become popular for generating novel proteins that could serve as drugs or industrial enzymes. Competitions are advancing the field by democratizing access, accelerating validation, and standardizing development processes. However, experts caution that these competitions must carefully select problems and judging criteria to avoid misleading outcomes, though current initiatives appear to foster collaborative sharing of methodologies and results.
Breaking Ground: Innovative Consistency Models Revolutionize AI Sampling
OpenAI has introduced a cutting-edge approach with continuous-time consistency models (sCMs) that promise revolutionary advancements in generative AI by dramatically improving sampling efficiency. These models achieve sampling quality comparable to top benchmark diffusion models using only two sampling steps, overcoming the traditionally slow sampling speeds of diffusion models that typically require many more steps. The sCMs are built on an advanced theoretical framework called TrigFlow, which simplifies existing model formulations and identifies the root causes of training instability. This innovation allows for the removal of discretization errors and hyperparameters, offering a more robust training process. The enhanced design enables these models to scale up effectively, accommodating large datasets with 1.5 billion parameters, and delivering targeted improvements in image generation on datasets like CIFAR-10 and ImageNet with significant reductions in the computational load necessary for high-quality outputs.
BitNet: A New Efficient Approach for Scaling Language Models
BitNet, a novel 1-bit Transformer architecture introduced by researchers, aims to address the challenges posed by the increasing size of large language models, particularly their environmental impact due to high energy consumption. By utilizing BitLinear as a replacement for the nn.Linear layer, BitNet allows for the training of 1-bit weights from scratch, which significantly reduces memory usage and energy output. Experimental evaluations indicate that BitNet not only competes with existing methods in terms of performance but also surpasses them in efficiency, maintaining competitive results like full-precision Transformers while offering significant reductions in resource consumption. This architecture provides a promising pathway for scaling large language models more sustainably without compromising on performance.
7. Arts & Entertainment
AI’s expanding role in media is sparking innovation and moral debates. Adobe’s Generative Extend in Premiere Pro allows editors to expand video and audio clips, while prominent creators like Björn Ulvaeus and Julianne Moore oppose unauthorized use of their work in AI training, pushing for ethical standards. Adobe is investing $100 million in global AI literacy through Coursera and others. Meanwhile, the Lenfest Institute, with OpenAI and Microsoft, launched an AI fellowship to support local journalism.
Adobe Boosts AI Literacy with Global Skilling Initiative
Adobe has launched a new global skilling initiative focused on enhancing AI literacy, aiming to equip 30 million people with skills in AI content creation and digital marketing. The initiative involves collaborations with Coursera, as well as various educational and alternative learning organizations, to offer comprehensive courses such as Generative AI Content Creation and Creative & AI Skills. Adobe has committed over $100 million this year to support this program and is offering $250,000 in scholarship licenses to improve access to its courses. The courses, available on Coursera, cater to a global audience, providing flexible learning paths and certificates such as the Content Creator and Graphic Designer Professional Certificates.
Adobe Debuts AI-Driven Generative Extend in Premiere Pro Beta
Adobe has introduced Generative Extend, a new AI-powered feature in Premiere Pro (beta), that allows video editors to seamlessly extend video or audio clips while maintaining lifelike quality. Powered by Adobe’s Firefly Video model, this tool helps fill gaps in footage, fix awkward cuts, and adds room tone to extend audio. Editors can easily integrate this functionality into their workflow without using original user content, ensuring commercial safety. Furthermore, the feature includes Content Credentials in the export process, enhancing transparency and recognition for creators by providing metadata detailing how the content was created using AI.
Notable Creators Condemn AI Use of Unlicensed Creative Works
A coalition of prominent creators, including Björn Ulvaeus, Julianne Moore, and Kazuo Ishiguro, have signed a statement opposing the unlicensed use of their creative works for training generative AI models. The statement argues that such practices pose a significant threat to the livelihoods of artists and writers and calls for these actions to be prohibited. The petition, available for public signature, aims to garner support from individuals from diverse professions to address and regulate the issue of unauthorized AI training using creative works, highlighting the need for ethical standards in AI development and training data usage.
AI Fellowship Program Boosts Local Journalism
The Lenfest Institute for Journalism has partnered with OpenAI and Microsoft to launch the Lenfest Institute AI Collaborative and Fellowship program, designed to support local journalism through AI innovation. Five media organizations—Chicago Public Media, The Minnesota Star Tribune, Newsday, The Philadelphia Inquirer, and The Seattle Times—will receive grants and AI enterprise credits to hire AI fellows for two-year projects focused on enhancing business sustainability and integrating AI technologies. These projects will explore AI applications such as data analysis, audience engagement, transcription services, and advertising strategies. Through collaborative efforts and shared learnings, the initiative seeks to provide independent newsrooms with advanced AI tools, ultimately aiming to secure the future of local journalism.
8. Dev Alerts
The past two weeks have brought us some new dev-infused goodies, both in terms of new features, as well as some insightful cookbooks. On the official dev front, OpenAI has now introduced audio modality to its Chat Completions API, both on the inputs, as well as on the outputs. OpenAI also made a new addition to its playground, in the form of prompt and schema generation, and what OpenAI refers to as “meta-prompting”. Several new cookbooks have also been released, demonstrating custom GPT with actions towards GitHub API, model distillation API with some very impressive results, and finally a voice translation guide using the audio modality on the chat completions API. Finally, it was also confirmed that Realtime API is inflating costs, the root cause was identified, and OpenAI is working on bringing those costs down, so stay tuned!
Inflated Charging on Realtime API
In the following community thread, a number of users pointed out that Realtime API is significantly overcharging. It was later pointed out by one of the community members (lucasvan) that the significant portion of the cost (around 85%) is due to the accumulating input audio tokens over time, leading to inflated charges. Secondary cost inflation stems from accidentally generating a large amount of output tokens, such as requesting long responses - even if the user interrupts or doesn’t listen to the output, the tokens are still generated and billed. This analysis was confirmed by Jeff Harris from OpenAI, and he mentioned that OpenAI is working on potential solutions (including caching) in order to bring these costs down.
OpenAI's Chat Completions API Now Supports Audio
OpenAI has enhanced its Chat Completions API to support audio inputs and outputs, allowing users to receive responses in text, audio, or both. This update enables the generation of spoken audio responses and the use of audio inputs, which can convey richer data than text alone by capturing nuances such as tone and inflection. The API supports various use cases, including generating audio summaries from text, performing sentiment analysis on audio recordings, and facilitating asynchronous speech-to-speech interactions. Users can access these features via the REST API or OpenAI’s SDKs, with the gpt-4o-audio-preview model requiring either audio input or output. This development aims to enhance the versatility and interactivity of AI-driven conversations. Learn all the details in this guide.
AI-Powered Prompt and Schema Generation in Playground
OpenAI’s Playground introduces a Generate button designed to streamline the creation of prompts, functions, and schemas from task descriptions, leveraging AI to enhance efficiency. This feature employs meta-prompts and meta-schemas, which incorporate best practices to generate or refine prompts and produce valid JSON and function syntax. The meta-prompts guide the AI in understanding task objectives and improving existing prompts while maintaining clarity and conciseness. For schema generation, a pseudo-meta-schema is used to ensure adherence to strict mode constraints, despite limitations in supported features. This approach allows for the generation of structured outputs and function schemas, ensuring all fields are marked as required and additional properties are restricted, thereby enhancing the reliability and precision of AI-generated outputs. Explore the new capability further in this guide. There is a recent cookbook that explores this functionality.
Cookbook additions: Model Distillation, Voice Translation, GitHub GPT Actions
OpenAI has further expanded its cookbook collection with three new additions. The The cookbook “Leveraging model distillation to fine-tune a model” demonstrates how to use model distillation to transfer the performance of a larger model (gpt-4o) to a smaller model (gpt-4o-mini) for a wine grape variety classification task. The cookbook " Voice Translation into Different Languages" provides a comprehensive guide on using GPT-4o’s audio-in and audio-out modality to efficiently translate and dub audio content from English to Hindi, detailing steps for transcription, dubbing, and evaluating translation quality with BLEU and ROUGE scores. Finally, the cookbook " GPT Actions library - GitHub" provides instructions for developers connecting a GPT Action to GitHub.
Careers at OpenAI
OpenAI is hiring for 144 positions across various teams and locations. The majority of roles are concentrated in San Francisco, with additional opportunities in Tokyo, London, Dublin, New York City, and remote positions in Paris and Washington, D.C..
Compared to two weeks ago:
-
The focus seems to have slightly shifted towards post-training, human data, and safety roles, indicating evolving priorities in AI safety and system optimization.
-
There’s a slight reduction in remote job opportunities, though San Francisco continues to be the main hub.
-
Some unique operational roles have already been filled, while safety-focused and research-heavy positions have gained prominence.
Also: if you are interested in preventing Skynet, or just generally care about LLMs not being misused, check out this role!
For the full list of open job opportunities, see here.
9. Community Spotlight
We are happy to introduce the community spotlight for the very first time! The idea is to highlight interesting and constructive discussions, new community-led tools and SDKs, events, games, and other shenanigans that bring our dev community together. In this edition, we highlight a super interesting topic that exposed the overcharging on the Realtime API, and we summarize the collection of questions for the AMA with Sam Altman for tomorrow’s DevDay in London.
Overcharging over Realtime API
This topic demonstrated a true community spirit. gowner initiated the topic by bringing to our attention that in their experience, Realtime API was overcharging more than 2.5x compared to what the pricing page suggests. The community then got together and dug into these costs, ultimately culminating in lucasvan providing a detailed analysis of where and why the overcharging was happening. In short, most of the overcharge was due to input audio tokens being carried over. Jeff Harris from OpenAI acknowledged the issue and stated that OpenAI is looking into how to bring these costs down. This is a true example of how our dev community can have a very positive impact on the quality of OpenAI services. Well done to all community members who contributed to this thread!
DevDay London 2024 : Your Questions for an AMA with Sam Altman
In preparation for London DevDay, vb started a topic collecting questions from the community directed at Sam Altman’s AMA. We can summarize the collected questions/topics as follows:
- AI-generated low-quality content: Concerns about the influx of low-quality AI content online (e.g., AI images, articles, videos) and whether OpenAI plans to address this issue, or if it’s outside their control.
- Family Plan for ChatGPT: A request for a family subscription plan for ChatGPT Plus, with individual logins and shared prompt quotas, as current costs aren’t viable for families.
- Pricing and Plan Options: Suggestions to introduce more affordable subscription options, such as paying for a limited number of prompts ($5/month for 500 prompts) instead of the current $20 flat fee.
- Context length and memory: Questions about progress beyond the 128k context length and whether memory functionality (smart memory retrieval) will be integrated into APIs.
- AI version of Sam Altman: Curiosity about whether Sam Altman is considering creating an AI bot version of himself with his voice and mannerisms.
- Universal Basic Income (UBI): Asking if UBI is viable due to AI-induced job displacement and how it could be implemented globally, especially considering U.S.-based AI companies profiting from international customers.
- New industries and economic models: What new industries or models Sam Altman foresees emerging as AI reshapes traditional sectors.
- Custom GPTs: Concerns about the lack of updates for Custom GPTs, and whether they are still a priority for OpenAI.
- Duties to original creators: Questioning whether OpenAI has a responsibility to the original creators (writers, musicians, etc.) whose work was used to train its models, often without consent.
- Path to AGI: Requesting Altman’s thoughts on whether scaling up language models and improving algorithms will lead to AGI, or if entirely new technologies will be required to surpass LLM limitations.