7. Arts & Entertainment
Runway and Lionsgate’s AI Collaboration
Runway and Lionsgate join forces to develop an AI model to enhance film production with AI-generated cinematic video
Details
The AI model is designed to assist Lionsgate Studios, including its filmmakers and creative talent, in augmenting their work by generating cinematic videos. The model will be customized with Lionsgate’s proprietary catalog and output can be further refined with Runway’s suite of controllable tools. Runway is also considering licensing the models to individual creators.
AI-powered Coral Marvel at the United Nations
Refik Anadol’s ‘Large Nature Model: Coral’ uses AI to transform 100 million coral images into a massive immersive artwork, spotlighting environmental issues at the United Nations General Assembly
Details
The installation is being showcased at the UN headquarters during the General Assembly’s high-level week, organized in partnership with the Executive Office of the UN Secretary-General and the Patrick J. McGovern Foundation. It serves as a poignant reminder of the vulnerability of coral ecosystems to climate change, aiming to inspire a global dialogue on environmental conservation. Additionally, Refik Anadol plans to open Dataland, the world’s first museum dedicated to AI-generated art, in Los Angeles in 2025, featuring “living paintings” created by training an AI model on half a billion images, sounds, and scents to produce immersive artworks evocative of rainforests and underwater coral.
YouTube Arms Creators with Generative AI
YouTube introduces new AI features including Google DeepMind’s Veo for Shorts and an enhanced Inspiration Tab for creators
Details
Veo enables creators to generate high-quality video backgrounds and standalone six-second clips for Shorts. AI-generated content will include SynthID watermarks and labels to indicate its origin. The revamped Inspiration Tab in YouTube Studio will use generative AI to provide video ideas, titles, thumbnails, and outlines tailored to each creator’s style.
8. Dev Alerts
New real-time API (beta)
- API Features: Supports low-latency, multimodal inputs and outputs (text, audio); enables function calling
- Speech-to-Speech: Native speech processing without text intermediary, reducing latency
- Natural Voices: Models offer inflection control, including laughter, whispers, and tone adherence
- Multimodal Output: Simultaneous text (for moderation) and audio playback faster than real-time
- WebSocket Interface: Stateful, event-based API with JSON-formatted event exchange
- Detailed guide here
New model distillation suite
- Stored outputs: Stored outputs from large models are used as training data for smaller models
- Evals: Dedicated new evals platform to create and run custom evaluations using stored completions or existing datasets to measure model performance
- Detailed guide here
Expanded access to o1 models
- API Access & Rate Limits: Expanded to Tier 3; Tier 5 increases to 10,000/min for o1-preview, 30,000/min for o1-mini; Tier 4 set at 10,000/min and Tier 3 at 5,000/min for both o1-preview and o1-mini
- Deep Dive: Extended cut on o1 model development.
New multimodal moderation model
- Model Features: Supports text & image inputs; adds illicit & illicit/violent categories
- Performance: 42% accuracy improvement; notable enhancements in low-resource languages
- API Access: Free use with tier-based rate limits
- Detailed guidance here
Free fine-tuning extended and new vision fine-tuning capabilities
- Vision fine-tuning: Supports up to 50,000 image-containing examples in JSONL files with image data provided via HTTP or base64 URLs
- Offer Details: Free fine-tuning for gpt-4o models (including vision) until October 31
- Token Allotments: 1M daily for GPT-4o, 2M for GPT-4o mini; overage charges apply.
Rate limit increases for Whisper, TTS and DALL-E
- New Limits: Tier 1 – 500; Tier 2 – 2,500; Tier 3 – 5,000; Tier 4 – 7,500; Tier 5 – 10,000.
New request IDs for easier debugging
- Feature Update: Top-level _request_id in OpenAI SDKs for efficient debugging
- Detailed guidance here
New cookbook additions
- Prompt caching 101: Examples for the use of caching in multi-turn conversations and with image input (Link)
- o1 Structured Outputs: Methods for JSON responses and data structuring (Link)
- BYOB Tool for Web Browsing: Setup guide for real-time web browsing and summarization (Link)
- Canvas LMS Integration: ChatGPT actions for Canvas LMS management (Link)
Model deprecation, shutdown and change dates to keep on your radar
- Model updates: Update of gpt-4o to the latest version pt-4o-2024-08-06 as of October 2
- Fine-tuning cessation: No new runs on babbage-002 and davinci-002 after October 28, 2024; switch recommended to gpt-4o-mini.
- Model shutdowns: gpt-4-vision-preview and gpt-4-1106-vision-preview ending December 6, 2024; replacement with gpt-4o recommended.
You and OpenAI
OpenAI continues to have 169 open roles, of which over 80% are based in SF and the remainder spread across Seattle, NYC, London, Dublin, Singapore and Tokyo. Focus of roles remains on Applied AI, Research and Go To Market in addition to some other unique roles.
Discover the full details here
- Most Active Roles:
- Software Engineering dominates the list, with numerous roles for Backend, Frontend, Full Stack, and specialized positions like Model Inference and Cloud Infrastructure.
- Data Science and Machine Learning are also well-represented, reflecting the growing demand for AI-focused roles.
- Most Active Areas:
- Applied AI Engineering is the most frequent area, covering a range of roles from Software Engineering to Machine Learning.
- Platform Engineering and Security also feature prominently, indicating a focus on infrastructure and cybersecurity.
- IT has several openings, especially for support and network-related roles.
- Customer Success and Finance have fewer roles, but they include leadership and partner-facing positions.
- Most Active Locations:
- San Francisco is by far the most common location, underscoring its role as a tech hub.
- Other key locations include London, UK, New York City, Seattle, and Tokyo, with scattered remote roles, especially in Singapore and Japan.
- Interesting Unique Positions:
- Roles like “Growth PM, Web Optimization”, “Technical Advisor to the Head of Security”, and “Trusted Compute and Cryptography Leader” stand out as highly specialized or leadership positions.
- There are also unique positions focused on AI research and human-data interaction, such as “Human-AI Interaction Specialist” and “Research Engineer, AI Security & Privacy.”