Seeking Advice: Extracting Text from Keynote and MP4 Files for RAG Implementation

[_j]

mp4 is a container that internally has audio and video streams that can be demuxed. You can have a clever AI write you a script. Or a clever human can find his links…

Whisper doesn't work with mp4

I have an AI I can ask… It’s possible to demux an MP4 file and extract the first audio stream using Node.js. You can use the mp4box module to achieve this. Here’s a step-by-step guide on how to do it: Install the mp4box module: npm install mp4box Create a script to demux the MP4 file and extract the first audio stream: const fs = require('fs'); const MP4Box = require('mp4box'); function demuxMP4(inputFile, outputFile) { const mp4box = new MP4Box.MP4Box(); mp4box.onReady = (info) => …

You will then want to re-encode the audio with a library that supports most anything like ffmpeg, because there can be bloated multichannel files in a codec that still would be unsupported by the API.

Whisper API cut output short when input is large (smaller than 25 MB)

First: I would see if it is the file size, or the audio length. For transcriptions, you can send Opus audio using a voice codec. This is three hours at under 20MB: ffmpeg -i audio.mp3 -vn -map_metadata -1 -ac 1 -c:a libopus -b:a 12k -application voip audio.opus It’s more efficient for everybody, and limiting to voice bandwidth can improve the transcription. Then: is it terminating at a silence? Too much silence would normally get you some hallucinations after a long period, not a premature f…

I don’t know what “Keynote files” are, so discover it is Apple presentation software. I would focus on one of these, like HTML, then process with a tag stripper. " Exports to: PDF, QuickTime, JPEG, TIFF, PNG, HTML (with JPEG images) and PowerPoint. Keynote also uses .key (presentation files) and .kth (theme files) bundles based on XML."