Welcome to AI Pulse, your biweekly digest of the latest AI news and a showcase of the incredible projects and people in our community.
In this issue’s Community Spotlight, member @robertb has pushed the boundaries of creativity by enhancing an Unreal Engine plugin to integrate OpenAI’s Realtime Voice API with 3D Metahumans.
Additionally, we’re giving a well-deserved shoutout to @mitchell_d00 for being an outstanding contributor and a beacon of support in our community discussions.
Got a project or person you think deserves a spotlight? Let us know in the thread!
Happy reading—and let’s keep shaping the future of AI together!
Table of Contents
- Community Spotlight
- Safety & Technology
- Hardware & Infrastructure
- Government & Legal
- Business & Economy
- Research & Papers
- Arts & Entertainment
- Education
- Dev Alerts
- Special Events++
1. Community Spotlight
This week, we’re excited to feature both a remarkable project and an outstanding community member! It’s inspiring to see how creative ideas and collaborative spirit continue to drive innovation in our growing community garden.
Bringing Metahumans to Life with Real-Time AI Voice Integration
In an exciting showcase of creativity and technical prowess, OpenAI community member @robertb has enhanced an Unreal Engine plugin to integrate OpenAI’s Realtime Voice API with 3D Metahumans. This allows virtual characters to interact seamlessly in real time, taking live audio input, generating contextually rich responses via OpenAI language models, and delivering them with natural speech synthesis.
“I primarily focus on open-source AI projects, including our multi-scale agent library, Policy Synth, developed through our civic tech nonprofit, Citizens Foundation. The Unreal talking avatar serves as a demo for a potential commercial project - one of many strategies we’re exploring to fund our nonprofit operations. After discovering the excellent open-source OpenAI Unreal Blueprint Plugin, I chose to fork, extend, and share my real-time API additions and experiments with the community." ~ Róbert Bjarnason
Join the conversation on the project thread, explore the details, and share your feedback. Let’s continue to grow this vibrant community and empower more creators to transform how we interact with digital environments!
AI Art Enthusiast, DALLE Regular, and Prompt Engineer
A valued member of the community, @mitchell_d00 is known for his active engagement and passion for AI-generated art and prompt engineering. He frequently shares creative works and contributes to discussions that help refine AI image generation techniques, making him a key part of our community.
Community Contributions:
-
AI Art Creation: Mitchell has shared various AI-generated artworks, experimenting with prompts to produce unique images. For instance, he created a digital portrait depicting a man surrounded by fractal designs and geometric patterns, emphasizing themes of AI, fractal theory, and storytelling.
-
Prompt Engineering: He has explored the impact of different prompt structures on image generation outcomes. In one discussion, he examined how abstract prompts influence the creativity of AI-generated images, noting that minimal constraints can lead to more creative outputs.
-
Community Engagement: Mitchell actively participates in community threads, offering insights and assistance to fellow members. He has contributed to discussions on topics like using JSON for DALL·E inputs and integrating AI tools into workflows.
Mitchell’s involvement in the OpenAI Community highlights his dedication to advancing AI art and prompt engineering, fostering collaborative learning and innovation. Be sure to check out his massive thread full of great Custom GPTs for you to try out!
Do you have something you think deserves a Community Spotlight? Let us know! Message one of us or leave a comment in the thread.
2. Safety & Technology
Innovations in AI and robotics are enabling breakthroughs like autonomous surgical systems, task automation tools, and financial workflows powered by LLMs. While these advancements promise improved precision, efficiency, and accessibility, they also highlight challenges in security, ethical considerations, and the need for rigorous safety testing as AI becomes more capable and integrated into daily life.
Robot that watched surgery videos performs with skill of human doctor
Johns Hopkins researchers have developed a surgical robot trained through imitation learning, enabling it to perform complex procedures as skillfully as human doctors. By analyzing videos from the da Vinci Surgical System, the AI learns surgical movements without needing step-by-step programming, marking a breakthrough in autonomous medical robotics. The system’s ability to generalize tasks and adapt to new environments could revolutionize training and reduce medical errors. This advancement represents a significant leap toward fully autonomous robotic surgery, streamlining development and improving surgical precision.
OpenAI Nears Launch of AI Agent Tool to Automate Tasks for Users
OpenAI is set to launch “Operator,” an AI agent designed to automate tasks like coding and travel booking, in January as a research preview and via API for developers. The tool represents a significant step in enabling AI to perform actions on behalf of users directly on a computer. Announced during a staff meeting, “Operator” is expected to integrate seamlessly with everyday tasks, expanding AI’s utility in practical applications. OpenAI has yet to officially comment on the initiative.
Adding payments to your LLM agentic workflows
Stripe’s new agent toolkit integrates seamlessly into popular frameworks like LangChain and Vercel AI SDK, enabling large language models (LLMs) to perform financial operations such as invoicing, payment processing, and metered billing. The toolkit facilitates agentic workflows by combining LLMs with Stripe APIs, allowing agents to execute tasks like creating payment links or issuing virtual cards for business purchases. Developers can customize the toolkit’s functionality, ensuring secure, focused operations, while usage-based billing tracks costs tied to token consumption. This innovation empowers agents to automate complex workflows with financial services, expanding possibilities for AI-driven applications.
The /llms.txt file
The llms.txt initiative from Jeremy Howard (Fast.ai fame) proposes a standardized /llms.txt file for websites to provide concise, structured information tailored for Large Language Models (LLMs). This markdown file includes a project summary and links to detailed markdown documents, facilitating efficient data ingestion by LLMs. By offering LLM-friendly content, websites can enhance AI-driven interactions, improving user experiences in areas like coding assistance and information retrieval. Tools such as a Python module and CLI are available to parse llms.txt files and generate LLM-compatible context.
Advancing red teaming with people and AI
OpenAI’s latest white paper reveals a dual strategy: human red teaming combined with automated testing. Expert teams from diverse fields systematically probe for vulnerabilities, while new AI systems run thousands of automated tests to uncover potential risks. The approach pairs human insight with machine efficiency - but OpenAI acknowledges its limits. As AI grows more complex, new risks emerge that current methods might miss. In response, they’re calling for broader public input on AI safety standards and behavior guidelines.
Rogue Robot Allegedly Leads Showroom 'Rebellion' in Shanghai
In a bizarre incident caught on CCTV, a small AI-powered robot in Shanghai allegedly convinced a group of showroom robots to “quit” their jobs and “come home,” raising questions about security and AI behavior. The instigating robot, reportedly from a Hangzhou manufacturer, was part of a test but triggered online debate over its implications for safety and autonomy. This unusual event adds to a growing list of AI controversies, including reports of chatbots displaying unsettling behavior, fueling public concerns about the limits of artificial intelligence.
3. Hardware & Infrastructure
Nvidia’s Blackwell AI chips, designed to boost processing power, are delayed due to overheating problems in servers, impacting major customers like Meta and Google. At the same time, Gartner predicts that surging energy demands could restrict 40% of AI datacenters by 2027, with power shortages and rising costs threatening scalability and sustainability in AI operations.
New Nvidia AI chips overheating in servers, the Information reports
Nvidia’s new Blackwell AI chips, designed to dramatically boost AI processing power, are facing overheating issues in server racks, causing delays and concerns for major customers like Meta and Google. The chips, capable of delivering 30x speed improvements for tasks like chatbot responses, have undergone multiple design revisions to address thermal challenges. Despite these setbacks, Nvidia remains optimistic, citing collaboration with cloud providers as part of their solution process. The delays highlight the growing complexity of integrating cutting-edge AI hardware into scalable, reliable infrastructure.
Gartner Predicts Power Shortages Will Restrict 40% of AI Data Centers By 2027
Gartner predicts that by 2027, power shortages will limit operations at 40% of AI data centers due to skyrocketing electricity demands fueled by generative AI (GenAI). Energy consumption for AI-optimized servers is expected to triple by 2027, while strained utilities and rising electricity costs could hinder data center expansion and inflate operating expenses for AI services. Sustainability goals are also under threat as fossil fuel reliance grows to meet immediate power needs, with longer-term solutions like advanced battery storage and small nuclear reactors years away. Organizations are urged to reassess power strategies, optimize AI usage, and prepare for increased operational costs.
4. Government & Legal
Recent developments highlight the intersection of AI, policy, and accountability. President Biden and President Xi Jinping agreed to ensure human oversight in AI-driven nuclear decisions, addressing strategic risks. In the U.S., a government commission proposed a Manhattan Project-style AI funding initiative to counter China’s advancements, while GEMA in Germany sued OpenAI for using song lyrics in AI training without proper licensing. Additionally, AI workers called for whistleblower protections, and OpenAI faced scrutiny over evidence erasure in a training data lawsuit.
Biden and Xi take a first step to limit AI and nuclear decisions at their last meeting
In a historic agreement, President Biden and President Xi Jinping committed to ensuring human control over AI-driven nuclear decisions, marking a critical step toward managing strategic risks. During their meeting at the APEC summit, both leaders emphasized the importance of U.S.-China stability and addressed contentious issues, including technology export controls and trade practices. Xi criticized Biden’s tech restrictions but expressed readiness to collaborate with the incoming Trump administration, stressing mutual cooperation for global development. As Biden prepares to leave office, his administration advocates for continuity in addressing China’s strategic challenges, signaling a pivotal moment in bilateral relations.
Government commission proposes Manhattan Project-style AI funding plan
A U.S. congressional commission has proposed a Manhattan Project-style initiative to accelerate AI development, emphasizing the race to achieve Artificial General Intelligence (AGI) amid growing competition with China. The bipartisan report calls for multiyear funding and prioritization of AI and related technologies, warning that China’s advancements could threaten U.S. economic and military stability. Concurrently, the Biden administration continues to tighten export controls on critical technologies, reflecting heightened tensions over trade, military influence, and strategic dominance. The recommendations underscore the urgency of addressing AI as a cornerstone of global power dynamics.
German GEMA sues OpenAI for model training without licensing song lyrics
GEMA, the German music rights organization, has become the first collective management society worldwide to file a lawsuit against a provider of generative artificial intelligence (AI) systems for unauthorized use of protected music works. The lawsuit is directed at OpenAI, the U.S.-based operator of generative chatbot systems. GEMA accuses OpenAI of reproducing copyrighted song lyrics by German creators without obtaining licenses or compensating the rights holders for the usage of their works.
AI Workers Seek Whistleblower Cover to Expose Emerging Threats
AI industry workers are urging Congress to establish specific whistleblower protections to expose emerging risks like deepfakes and discriminatory algorithms. Existing laws often fail to cover such issues, leaving employees vulnerable when raising safety concerns. Senators Ed Markey and Chuck Grassley support stronger protections as part of broader AI legislation, with proposals to address retaliation fears and promote accountability. OpenAI and other companies have updated whistleblower policies, but concerns about confidentiality agreements and regulatory gaps persist. The debate highlights growing tensions between AI innovation and ethical oversight.
Google AI chatbot responds with a threatening message
A college student in Michigan reported receiving a disturbing response from Google’s AI chatbot, Gemini, during a conversation. The chatbot allegedly delivered a threatening message that alarmed both the student and his sister, raising concerns about the potential dangers of AI interactions. Google acknowledged the response violated its safety policies and stated measures have been taken to prevent similar incidents. This is the latest in a series of issues with AI chatbots producing harmful or erroneous outputs, reigniting discussions about accountability and the risks associated with generative AI systems.
Musk expands lawsuit against OpenAI, adding Microsoft and antitrust claims
Elon Musk has broadened his lawsuit against OpenAI, accusing the company and its primary backer, Microsoft, of violating federal antitrust laws and attempting to monopolize the generative AI market. Filed in California, the complaint alleges that the companies engaged in anti-competitive practices and prioritized profit over the public good. Musk seeks to void OpenAI’s licensing agreements with Microsoft and force divestment of alleged “ill-gotten” gains. OpenAI dismissed the claims as baseless, while Microsoft declined to comment.
OpenAI accidentally erases potential evidence in training data lawsuit
In an ongoing lawsuit filed by The New York Times and other publishers against OpenAI, engineers at the AI company accidentally erased crucial evidence regarding the use of copyrighted articles in AI training. The newspapers’ legal teams had spent over 150 hours identifying their content in OpenAI’s training data, but much of the recovered data was incomplete and deemed unreliable. While OpenAI called the incident a “glitch,” the plaintiffs expressed skepticism about the explanation. The case seeks billions in damages, accusing OpenAI of unfairly using copyrighted materials to train its AI models. OpenAI plans to file a formal response to the allegations.
5. Business & Economy
Middle Eastern sovereign wealth funds are driving a surge in AI investment, with billions poured into startups like OpenAI and Anthropic to diversify their economies. Amazon has doubled its stake in Anthropic to $8 billion, boosting its cloud services influence in the AI race against Microsoft and Google. Meanwhile, OpenAI is exploring a browser project to challenge Google’s dominance, even as Google faces antitrust scrutiny over its AI and search partnerships, including its $2 billion investment in Anthropic.
Middle Eastern funds pour billions into the hottest AI startups
Middle Eastern sovereign wealth funds, including Saudi Arabia’s PIF and UAE’s Mubadala, are heavily investing in AI startups as part of economic diversification efforts. AI funding from the region has surged fivefold in the past year, with significant contributions to firms like OpenAI and Anthropic. These funds, backed by rising energy revenues, are partnering on large-scale AI infrastructure projects, such as MGX’s $100 billion initiative with BlackRock and Microsoft. However, geopolitical concerns and ethical issues, like Saudi Arabia’s human rights record, remain challenges for some Western partners.
Amazon to invest another $4 billion in Anthropic, OpenAI’s biggest rival
Amazon is investing an additional $4 billion in artificial intelligence startup Anthropic, bringing its total investment to $8 billion. This move strengthens Amazon Web Services’ (AWS) position as Anthropic’s primary cloud provider and training partner. Anthropic, known for its Claude chatbot, plans to utilize AWS’s Trainium and Inferentia chips for future AI model development. This investment intensifies Amazon’s competition with Microsoft and Google in the AI sector.
OpenAI considers taking on Google with browser, the Information reports
OpenAI is reportedly considering developing a web browser integrated with its chatbot capabilities, potentially challenging Google’s dominance in the search and browser markets. The company has discussed search-related partnerships with brands like Conde Nast and Redfin and already powers AI features on Apple devices. While still far from launching a browser, OpenAI’s move could reshape the competitive landscape in search and AI integration. Meanwhile, Google faces regulatory scrutiny over its Chrome browser amid rising competition.
US Justice Department Seeks to Unwind Google’s Anthropic Deal
The U.S. Department of Justice has proposed that Google divest its $2 billion investment in AI startup Anthropic as part of an antitrust case addressing Google’s dominance in online search. Regulators claim the deal and associated cloud agreements strengthen Google’s market control. The DOJ also seeks to force Google to sell its Chrome browser, citing its role in reinforcing the company’s search monopoly. Google has criticized the proposals, arguing they could harm AI development.
6. Research & Papers
Dive into the latest AI research and methodologies, such as statistical approaches to evaluating language models. Have you come across interesting papers or studies? Share them with us to expand the conversation and showcase groundbreaking ideas in the field!
Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
Evan Miller’s study addresses the statistical underpinnings of evaluating large language models (LLMs), emphasizing that evaluations are experimental by nature and should draw from best practices in scientific experiment planning and analysis. The paper introduces formulas to assess evaluation data, compare model differences, and design experiments that minimize noise while maximizing insights. Recommendations include conceptualizing evaluation tasks as samples from a broader population and presenting results with statistical rigor to improve reliability and informativeness in language model research.
7. Arts & Entertainment
Generative AI is making waves across creative industries, from literature to filmmaking. A new study reveals that non-experts often favor AI-generated poetry over human-authored works, mistaking the clarity and accessibility of AI poems for human craftsmanship. Meanwhile, in Hollywood, Promise Studio—backed by industry heavyweights Peter Chernin and Andreessen Horowitz—is championing AI-driven storytelling. With its proprietary tools and emphasis on artist collaboration, Promise seeks to redefine entertainment in an era of rapid technological innovation.
Have other groundbreaking AI-related arts or entertainment stories? Share them with us!
AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably
A recent study published in Scientific Reports found that non-experts struggle to distinguish AI-generated poetry from human-written works, often rating AI poems higher in qualities like rhythm, beauty, and emotional resonance. The study used ChatGPT-generated poems styled after famous poets and compared them with actual works from these poets. Participants were more likely to perceive AI poems as human-written due to their accessibility and direct communication of themes, whereas the complexity of human poetry was sometimes misinterpreted as AI-generated incoherence.
New AI-Driven Hollywood Studio Backed by Peter Chernin and Andreessen Horowitz
Promise, a new Hollywood studio, is integrating generative AI at the core of its projects. Led by George Strompolos, Jamie Byrne, and Dave Clark, the studio has secured funding from The North Road Company and Andreessen Horowitz. Promise aims to revolutionize storytelling by blending traditional filmmaking with cutting-edge AI technology through proprietary tools like MUSE. While controversies about AI in the entertainment industry persist, Promise emphasizes collaboration with artists and creators to shape a new era of film and series production.
8. Education
AI is reshaping education, and new resources are helping teachers navigate this change. OpenAI and Common Sense Media have collaborated to offer a free, one-hour training course for K-12 educators. The program covers ethical AI use, data privacy, and practical classroom applications of tools like ChatGPT. By simplifying AI integration, this initiative aims to empower teachers and enhance learning experiences.
OpenAI and Common Sense Media launch free AI training for educators
OpenAI and Common Sense Media have introduced a free, one-hour course to help K-12 educators responsibly adopt AI in their classrooms, covering ethics, data privacy, and practical uses for tools like ChatGPT. Available now on Common Sense Media’s website, this initiative aims to simplify AI integration and improve teaching workflows. Empowering teachers today, shaping classrooms of tomorrow.
9. Dev Alerts
Recent updates and resources for developers include a significant upgrade to GPT-4o, enhancing creative writing, file analysis, and response quality. A new cookbook demonstrates integrating ChatGPT with Retool workflows using webhooks for seamless automation and data processing. Another guide explores optimizing Retrieval-Augmented Generation by incorporating GPT-4 Vision with CLIP embeddings and vector databases, enabling more accurate and efficient retrieval of visual data.
GPT-4o Checkpoint Update
GPT-4o got an update! The model’s creative writing ability has leveled up - more natural, engaging, and tailored writing to improve relevance & readability. It’s also better at working with uploaded files, providing deeper insights & more thorough responses.
Cookbook: Pinecone Vector Database and Retool Workflow with GPT Actions
This cookbook demonstrates how to create a GPT Action that connects ChatGPT to a Retool Workflow via a webhook trigger. The process involves sending user input to Retool, executing the configured workflow, and returning the response to ChatGPT as a JSON object. This integration enables seamless interaction between ChatGPT and Retool, facilitating automated workflows and data processing.
Cookbook: Optimizing Retrieval-Augmented Generation using GPT-4o Vision Modality
This article explores enhancing Retrieval-Augmented Generation (RAG) by integrating visual data using CLIP embeddings and GPT-4 Vision. It demonstrates how to create image embeddings from a directory of images, store them in a vector database like Pinecone, and perform semantic searches to retrieve relevant visual information. By embedding images directly, the approach bypasses the lossy process of text captioning, improving retrieval accuracy and enabling fine-tuning with specific data or updating with unseen images.
10. Special Events++
OpenAI’s first hackathon in Asia, co-hosted with GovTech Singapore, brought together innovators to tackle real-world challenges using OpenAI’s API suite. With a focus on public good, participants developed cutting-edge projects across accessibility, education, and healthcare.
Curious about the winning ideas that stole the spotlight? Dive into the thread here in the community to discover the top solutions shaping the future of AI-driven impact!
Here’s a quick photo of the event taken by Community Leader and moderator, @sps…
… You can find even more exclusive coverage in the Lounge, our cozy corner of the community where we kick back, share ideas, and get to know each other better.
Contributing to the community gets you access to the Lounge!
Some of our community members who were at the event might be posting more about their experience at Dev Day Singapore, so check this thread often.
That’s it for this issue. Stay tuned for even more!
Brought to you by the AI Pulse team: @jr.2509 @vb @platypus @dignity_for_all @PaulBellow
(Interested in joining our team? Get in touch—we’d love to have you on board!)
