AI Pulse – Edition #4
We are back with another edition of AI Pulse 
and you’re in for a treat!
OpenAI is shaking things up with Canvas, a new tool integrated into ChatGPT that transforms collaboration in writing and coding while Meta wows with Movie Gen and a Digital Twin Catalog, pushing the boundaries of video generation and 3D reconstruction.
Infrastructure investments continue to soar with Microsoft committing $4.8 billion to build a massive data center in Italy, Google betting on nuclear energy to power its ambitious AI projects, and OpenAI seeking additional compute capacity beyond Microsoft to fuel its rapid growth. Likewise, Governments maintain their focus on AI. The U.S. invests $100 million in AI for sustainable semiconductor innovation and the UAE strengthens its AI leadership with a new policy integrating AI into foreign relations.
On the legal front, Adobe’s web app helps protect creators from AI misuse, ensuring proper attribution of artistic works. In the world of AI economics, startups are riding a wave of investor enthusiasm, attracting billions in funding. OpenAI secures a staggering $6.6 billion to propel AI advancements, Sierra aims high with ambitious fundraising goals On the research side, OpenAI’s MLE-bench pushes the limits of machine learning agents, while AI continues to earn global recognition with Nobel Prizes in Chemistry and Physics.
In arts and entertainment, AI is reshaping creativity, with partnerships enhancing curated journalism and artists experimenting with AI-generated choirs.
Also don’t miss our Dev Alerts for the latest updates on tools, models, and career opportunities at OpenAI and our plans for a new Community Spotlight section.
Happy reading 
!
Your AI Pulse Team: @jr.2509 @PaulBellow @vb @platypus @thinktank @dignity_for_all @trenton.dambrowitz
Table of contents
1. Technology
2. Infrastructure
3. Government & Policy
4. Legal Matters
5. AI Economics
6. Research
7. Entertainment
8. Dev Alerts
9. Community Spotlight
1. Technology
AI technology is rapidly reshaping creativity, collaboration, and industry innovation. In the past two weeks, we’ve witnessed further advancements that enhance user experiences and drive sustainability across sectors. OpenAI’s Canvas is transforming collaborative writing and coding. Meta’s latest models and Digital Twin Catalog are pushing the boundaries of high-definition video generation and augmented reality with the potential to significantly boost augmented reality and e-commerce. Moreover, traditional industries are harnessing AI like never before: Walmart is personalizing shopping with AI and AR; KoBold Metals is accelerating the discovery of critical minerals for sustainable development; Uber is guiding drivers toward electric vehicles with a new AI assistant; and Japanese tech companies are developing AI systems to assist call center workers in handling abusive customers.
Meet Canvas: A New Collaborative Tool in ChatGPT
OpenAI introduces Canvas, a fresh interface designed to enhance the collaborative efforts of ChatGPT users in writing and coding projects. Built on GPT-4o, Canvas allows users to interact in-depth with ChatGPT by enabling targeted edits and suggestions within a separate window tailored for more complex tasks beyond traditional chat. The new tool, initially available to ChatGPT Plus and Team users, offers features like inline feedback, version control, and the ability to manage specific sections of text or code, enhancing ChatGPT’s ability to understand context and provide precise support. The rollout focuses on training GPT-4o to open Canvas appropriately for writing and coding projects, continuously refining its editing behavior and accuracy as part of its ongoing development.
Meta Introduces Movie Gen and Digital Twin Catalog
Meta has introduced Movie Gen, a suite of foundation models designed to generate high-quality 1080p HD videos with synchronized audio, excelling in tasks like text-to-video synthesis, video personalization, and audio generation. The largest model, with 30B parameters, produces videos of up to 16 seconds at 16 frames-per-second, pushing the boundaries of AI-driven media creation. In parallel, Meta’s Reality Labs Research launched the Digital Twin Catalog (DTC), a dataset of over 2,400 photorealistic 3D models aimed at democratizing 3D object reconstruction. Through a partnership with Shopify, Meta is facilitating an academic competition with 7,000 digital twins to improve AI-based reconstruction algorithms, lowering the cost and complexity of creating digital models for e-commerce and augmented reality.
AI Powers KoBold Metals to Major Funding Success
KoBold Metals, a startup leveraging artificial intelligence to discover critical minerals essential for the energy transition, has raised $491 million in its latest funding round, nearing its $527 million target. The company utilizes AI to identify and develop high-grade mineral deposits, including a recent significant copper find, which could produce hundreds of thousands of metric tons annually. This funding will support KoBold in advancing its extensive portfolio of exploration projects and aims to boost its valuation to $2 billion, highlighting the growing role of AI in mineral exploration.
Uber's AI Assistant Drives Towards an Electric Future
Uber Technologies is launching an AI assistant, powered by OpenAI’s GPT-4o, to assist drivers in transitioning to electric vehicles. Announced at the Go-Get Zero event in London, the AI assistant will provide personalized guidance based on drivers’ locations and available incentives, aiming to support Uber’s commitment to fully electrify its fleet by 2040. Initially set to launch in the U.S. next year, the assistant will expand globally and extend its capabilities beyond EV-related queries by 2025. This initiative is part of Uber’s broader strategy to lower emissions and support its driver partners in adopting electric vehicles.
Walmart's AI-Powered Retail Transformation
Walmart is advancing its retail strategy by integrating proprietary artificial intelligence (AI), generative AI (GenAI), augmented reality (AR), and immersive commerce technologies to create highly personalized shopping experiences. The company has introduced Wallaby, a series of retail-specific large language models (LLMs) trained with decades of Walmart data, to enhance customer interactions and support. Additionally, Walmart’s Content Decision Platform uses AI to tailor online shopping experiences, predicting customer preferences to provide personalized content. The retailer is also leveraging its Retina AR platform to develop immersive virtual environments and commerce experiences, including collaborations with platforms like Unity and ZEPETO, aiming to meet customers in new digital spaces and expand its adaptive retail approach.
Japanese Tech Firms Turn to AI to Ease Call Center Strain
Japanese tech companies NTT Communications and SoftBank are developing AI systems to assist call center workers in handling abusive customers. NTT’s system monitors conversations and displays appropriate response suggestions on operators’ screens, aiming to reduce employees’ psychological burden and help them remain calm during difficult interactions. SoftBank’s AI technology softens the tone of customers’ voices when operators respond, with plans to commercialize the system by fiscal 2025. These initiatives address the growing issue of customer harassment, known as “kasu-hara,” in Japan’s hospitality-focused culture.
Source: Kyodo News
2. Infrastructure
The challenges of meeting AI’s unquenchable thirst for compute power are pushing tech giants into unprecedented infrastructural ventures. Microsoft is pouring $4.8 billion into building one of Europe’s largest data centers in Italy. OpenAI, venturing beyond its exclusive cloud partnership with Microsoft, is seeking additional compute capacity from other providers to satiate its growing needs. Meanwhile, Google is investing in nuclear energy to fuel its AI technologies.
Microsoft's $4.8 Billion AI Investment in Italy
Microsoft is set to invest $4.8 billion to enhance its AI and cloud infrastructure in northern Italy over the next two years, marking its largest investment in the country. This initiative will expand the ItalyNorth cloud region into one of Microsoft’s largest data centers in Europe, serving as a data hub for the Mediterranean and North Africa. The investment aligns with Microsoft’s collaboration with BlackRock, which also involves a $30 billion fund focusing on AI-centric data centers and AI supply chains. The growing demand for AI and cloud services is driving firms in diverse sectors to integrate custom AI models for competitive differentiation.
OpenAI Expands Compute Capacity Beyond Microsoft
OpenAI is expanding its data center partnerships beyond Microsoft to meet its growing compute demands, following a recent $6.6 billion capital raise. The company has already initiated a deal with Oracle to rent servers from a data center in Abilene, Texas, which could expand to 2GW if Oracle secures additional power. This move comes as OpenAI seeks greater involvement in securing data centers and AI chips, with CFO Sarah Friar noting Microsoft’s slow pace in meeting their needs. OpenAI is also considering its ambitious ‘Stargate’ project, a potential 5GW data center, which would require substantial financing and power resources, and could involve deploying custom chips to reduce costs.
Google Invests in Nuclear Energy to Power AI Technologies
Google has signed the world’s first corporate agreement to purchase nuclear energy from multiple small modular reactors (SMRs) developed by Kairos Power, aiming to bring the first SMR online by 2030 and additional reactors through 2035. This initiative is designed to meet the increasing energy demands of AI technologies, which require reliable and carbon-free power to drive scientific advances, improve services, and foster economic growth. By investing in advanced nuclear solutions, Google seeks to provide up to 500 MW of new 24/7 carbon-free power to U.S. electricity grids, ensuring the energy infrastructure can support AI developments while pursuing its ambitious 24/7 carbon-free energy and net-zero goals.
Source: Google Blog
3. Government & Policy
The global surge in government spending on AI continue. The U.S. is dedicating significant funds to integrate AI into sustainable semiconductor development, aiming to revolutionize manufacturing while addressing environmental concerns. Simultaneously, the UAE is forging ahead with a comprehensive international AI policy to reinforce its position as a global tech leader. As these nations pour resources into ambitious AI initiatives, questions arise about their effectiveness and the transformative impact they may have. One thing is certain: the governmental drive toward AI is intensifying.
US Invests in AI for Sustainable Semiconductor Innovation
The U.S. Commerce Department is allocating up to $100 million to enhance the use of artificial intelligence in the development of sustainable semiconductor materials. This initiative is part of a broader $52.7 billion effort to bolster U.S. chips manufacturing and research. The funding is aimed at supporting universities, national laboratories, and private sector entities in utilizing AI-powered autonomous experimentation to expedite the creation of semiconductor materials that require fewer resources. The primary objective is to reduce the development time for these advanced materials, promoting sustainability in semiconductor manufacturing.
UAE Strengthens Global AI Leadership with New Policy
The UAE Cabinet has approved the country’s official stance on international AI policy, aiming to bolster its global leadership in technology. Developed by the Office of the Assistant Foreign Minister for Advanced Science and Technology and the Office of the Minister of State for Artificial Intelligence, the policy integrates AI into the UAE’s foreign policy framework, addressing global AI challenges. The policy emphasizes the UAE’s role in shaping global AI governance, contributing to multilateral platforms, and aligning with international standards to ensure ethical, social, and environmental priorities. It includes participation in international AI forums, advocating transparency, supporting global alliances for AI governance, and promoting responsible AI use to enhance peace and stability.
4. Legal Matters
The sources of training data for new and advanced generative AI models have been a contentious issue since ChatGPT became the first AI app to attract a large daily user base. Content creators are fighting to keep their work out of the datasets used for training, while companies seek the highest quality inputs. Rights holders are navigating the complex landscape, balancing between protecting their customers and creators or seeking compensation for allowing their content to be used in training data. Adobe, still recovering from a recent controversy caused by changes to its Terms of Service, is now attempting to gain a foothold in the market by protecting content creators.
Adobe's New Web App Protects Creators from AI Misuse
Adobe has launched the Adobe Content Authenticity web app, enabling creators to attach Content Credentials to their digital work, safeguarding it from unauthorized use and ensuring proper attribution. This new tool allows creators to specify if they do not want their content used to train generative AI models, addressing a major concern among artists about their work being exploited without consent. The app integrates with Adobe Creative Cloud applications and uses digital fingerprinting, watermarking, and cryptographic metadata to maintain content integrity across the digital ecosystem. Adobe aims for industry-wide adoption of these protective measures to foster a transparent and trustworthy digital environment.
5. AI Economics
The financial stakes in AI are rising, with OpenAI securing billions in funding to expand its leadership in the field. In parallel, OpenAI’s strategic request to limit investor backing for rival startups reflects competitive tensions in the industry. Additionally, startups like Sierra continue to draw significant funding, reinforcing investor confidence in AI’s potential to transform business operations.
OpenAI Secures $6.6B to Propel AI Advancements
OpenAI has secured $6.6 billion in new funding, valuing the company at $157 billion post-money, to further its mission of ensuring artificial general intelligence benefits all of humanity. This substantial investment will enhance OpenAI’s leadership in frontier AI research, expand compute capacity, and facilitate the development of tools aiding in problem-solving across industries. Alongside the funding announcement, news have surfaced that OpenAI is seeking exclusive commitments from its investors, asking them to refrain from funding five AI startups considered as competitors including Anthropic, xAI, and Safe Superintelligence (SSI), the latter co-founded by OpenAI’s Ilya Sutskever.
Source 1: OpenAI, Source 2: Reuters
Sierra's AI Ambitions: Raising Funds at $4B Valuation
Sierra, co-founded by OpenAI Chairman Bret Taylor and former Google executive Clay Bavor, is seeking to raise hundreds of millions in new funding, potentially tripling its valuation beyond $4 billion. The startup focuses on developing AI solutions for corporate customer service, utilizing multiple AI models to automate tasks such as voice calls and message processing. Despite Taylor’s dual roles at Sierra and OpenAI, he does not perceive a conflict of interest, as Sierra is not seen as a direct competitor. The funding efforts come amid continued enthusiasm for generative AI by the investor community with several high-profile AI start-ups besides OpenAI including xAI, CoreWeave and Scale AI having secured funding in the range of $500M to $6B since the start of 2024, backed by major venture capital firms including Andreessen Horowitz, and Sequoia Capital.
AI as a Catalyst for Business Growth: Insights from SoftBank CEO
SoftBank Corp. President & CEO Junichi Miyakawa emphasized the transformative potential of AI in driving value creation across industries during his keynote at SoftBank World 2024. He highlighted the risks of lagging in AI adoption, drawing parallels with historical technological hesitance, and predicted imminent advancements towards Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI). Miyakawa stressed the competitive advantage AI offers to businesses, urging proactive integration of AI to expand company capabilities and domains. He detailed SoftBank’s AI-driven initiatives, such as improving call centers and financial services, and underscored the importance of evolving communication infrastructure to support a society that coexists with AI.
6. Research
The lines between human ingenuity and AI capabilities are increasingly blurring, as evidenced by the 2024 Nobel Prizes recognizing breakthroughs in protein engineering and neural networks, both driven by AI advancements. Additionally, OpenAI’s new MLE-bench, which tests AI agents on real-world machine learning tasks, underscores how closely intertwined human and machine innovation has become, prompting us to reconsider how AI will shape the future of discovery and progress.Meanwhile, Rhymes AI’s open-source multimodal model Aria exemplifies the growing movement to democratize access to the latest AI technology.
Pioneers of AI: 2024 Nobel Prizes in Physics and Chemistry
The 2024 Nobel Prize in Chemistry has been awarded to David Baker, Demis Hassabis, and John Jumper for pioneering protein engineering and AI-driven protein structure prediction. Hassabis and Jumper’s development of AlphaFold2, an AI model capable of predicting the structures of nearly all known proteins, has revolutionized understanding of protein functions and facilitated advancements in areas such as antibiotic resistance and plastic decomposition. Meanwhile, the 2024 Nobel Prize in Physics went to John J. Hopfield and Geoffrey E. Hinton for their foundational contributions to machine learning via artificial neural networks. Hopfield developed an associative memory network capable of storing and reconstructing patterns, utilizing principles from physics related to atomic spin. Hinton expanded on this by creating the Boltzmann machine, which autonomously identifies data properties and performs tasks like image classification.
Introducing MLE-bench: Benchmarking AI in ML Engineering
OpenAI has introduced MLE-bench, a new benchmark designed to evaluate AI agents’ capabilities in machine learning engineering. This benchmark is built on 75 Kaggle competitions, challenging AI agents with real-world tasks such as model training and dataset preparation. MLE-bench uses open-source scaffolding to assess frontier language models, with OpenAI’s o1-preview and AIDE combination achieving a bronze medal level in 16.9% of competitions. The study also explores resource scaling and pre-training contamination impacts, providing an open-source codebase to encourage further research into AI’s ML engineering capabilities.
Aria: Pioneering Open Multimodal AI
Aria, developed by Rhymes AI, is the first open-source multimodal native Mixture-of-Experts (MoE) model, designed to handle text, images, video, and code in a unified framework. This model stands out for its efficient parameter usage, activating only 3.9 billion parameters per token, and offering a long context window of 64K tokens for processing extended multimodal inputs effectively. Pre-trained from scratch on a blend of multimodal and language data, Aria delivers competitive performance against proprietary models like GPT-4o and Gemini-1.5, excelling in tasks such as document understanding and video analysis.
7. Arts & Entertainment
The convergence of artificial intelligence and human creativity is redefining the landscape of arts and entertainment. OpenAI’s partnership with Hearst is bringing curated journalism from renowned magazines and newspapers into AI platforms. At the same time, visionary artists like Holly Herndon are pushing musical boundaries, using AI to foster collaborative creations such as AI-generated choirs.
OpenAI Partners with Hearst for Enhanced Content Integration
OpenAI has announced a new partnership with Hearst to integrate the latter’s extensive newspaper and magazine content into OpenAI’s AI products, such as ChatGPT. This collaboration will incorporate content from over 20 magazine brands and 40 newspapers, including renowned names like Esquire, Cosmopolitan, and the Houston Chronicle, enriching the AI’s offerings with diverse lifestyle and local news content. The integration aims to provide ChatGPT’s 200 million weekly users with reliable and curated journalism, ensuring that professional journalism remains central as generative AI evolves. Hearst content will include appropriate citations and direct links, maintaining transparency and easy access to original sources.
AI Choirs Redefine Art with Machine Learning
Holly Herndon and Mat Dryhurst are pioneering the use of artificial intelligence (AI) in art through projects like Holly+, which allows others to sing using Herndon’s voice, transforming individual expression into a collaborative art form. Their latest project, The Call, in collaboration with Serpentine Arts Technologies, involves training AI models with recordings from 15 UK choirs, creating an immersive audio installation that explores AI’s role in communal singing and data ownership ethics. Central to the project is a data trust experiment, proposing a model for sharing power and profits between data contributors, like the choirs, and AI model users, highlighting the potential for collective ownership and compensation in AI data use.
8. Dev Alerts
Cookbook additions: Swarm offers an educational framework exploring ergonomic, lightweight multi-agent orchestration
Ilan Bigio, Applied AI Engineer at OpenAI, recently made a new cookbook on agent orchestration. He defines the concepts of routines - a list of natural language instructions, and handoffs - an agent-initiated transfer of the conversation to another agent. With this cookbook, Ilan demonstrates how routines and handoffs can be used to orchestrate multiple agents in a simple, powerful, and controllable way. These concepts are then packaged in an experimental Github repo, Swarm.
Realtime API: Developers showcase emerging use case
Since the release of the realtime API on DevDay in San Francisco, discussions have been buzzing in the Community and developers have not hesitated to put the new model to test. From live voice translation, browsing the web and even painting with voice, this post showcases some of the initial and unique use cases that have emerged.
Model changes: GPT-4o now officially points to GPT-4o-2024-08-06
GPT-4o is now automatically pointing to GPT-4o-2024-08-06, which has prompt caching enabled by default (> 1024 tokens, and based on eviction criteria), is conformant to OpenAI structured outputs. The model also comes at roughly at 50% of the cost compared to previous GPT-4o versions.
Deprecations: Assistant v1 to be formally shut down in December
OpenAI will officially shut down access to its Assistants API beta v1 on December 18. This comes after the release of the v2 beta in April. Developers can refer to the Assistants API v2 beta migration guide for instructions on migrating tool usage to the latest version. For other up-to-date information on OpenAI deprecation, please refer to the official page here.
Careers at OpenAI
OpenAI’s job landscape is heavily focused on technical and engineering roles, especially around AI development, data science, and platform engineering. San Francisco is the primary hub. However, the recent announcement of office openings in NYC, Seattle, Paris, Brussels, and Singapore, alongside the existing presence in London, Dublin, and Tokyo marks a shift towards a greater global presence. Visit the careers page for all currently open roles.
Condensed OpenAI Jobs Landscape
Most Active Roles:
- Software Engineers (Backend, Frontend, Full Stack)
- Data Scientists and Research Engineers
- Machine Learning Engineers
Most Active Job Areas:
- Applied AI Engineering and Platform Engineering
- Data Science and AI Research
- Customer Success and Technical Solutions
Most Active Locations:
- San Francisco (HQ)
- Remote (especially Singapore, Tokyo, London)
- London, UK and Tokyo, Japan
Interesting Unique Positions:
- Director of Income Tax Compliance and Accounting
- Human-AI Interaction Specialist
- Food Program Operations Manager
- Trusted Compute and Cryptography Leader
Organizational changes: OpenAI hires Microsoft’s Generative AI Research Lead and Appoints Lead for International Expansion
Microsoft’s Vice President of generative AI research, Sebastian Bubeck, has left the company to join OpenAI. Bubeck is known for his significant contributions to Microsoft’s Phi models—lightweight language and vision models designed to enable AI applications on edge devices. While his specific role at OpenAI remains undisclosed, it’s anticipated that he will focus on advancing OpenAI’s capabilities in efficient, small-scale AI models. Besides the addition of Bubeck, OpenAI also announced that Oliver Jay (formerly the chief revenue officer at Asana and head of APAC and LATAM at Dropbox) will oversee international operations and facilitate its global expansion from Singapore.
9. Community Spotlight 
Starting next issue, we’re excited to introduce a new Community Spotlight section, where we’ll showcase highlights from our talented developer community. This is your chance to highlight innovative solutions, standout projects, or any other unique problems in the OpenAI Developer Community.
If you’ve come across an awesome project or clever solution, we’d love to hear about it. Just send your nomination and why it stands out to someone on the AI Pulse team. The only catch? You can’t nominate yourself!
