Hyperframes Goes Open Source: HeyGen Launches HTML-Driven AI Video Generation Framework
AI video generation company HeyGen recently announced and open-sourced a video rendering framework called Hyperframes. The project’s core philosophy is “write HTML, render video.” It brings the mature web technology stack into the video production domain, with a special emphasis on native support for AI Agents, offering a novel solution for programmatic video generation.
A Video Workflow Designed for AI Agents
Hyperframes was designed from the ground up to serve AI agents, enabling them to understand and execute video creation tasks. Traditional video editing software relies on a graphical user interface (GUI), making it difficult to automate programmatically. In contrast, Hyperframes defines a video as a structured HTML document, leveraging the inherent HTML generation capabilities of large language models.
Developers can install a “skill” for their coding agents (like Claude Code, Cursor, Gemini CLI, etc.) with a single command: npx skills add heygen-com/hyperframes. This skill encapsulates the Hyperframes syntax, commands, and best practices, allowing developers to create videos using natural language prompts, such as:
- Create from Scratch: Describe a task like, “Create a 10-second product intro video with a fade-in title and background music.”
- Content Transformation: Instruct the agent to “Summarize a specific GitHub repository or PDF document into a 45-second pitch video.”
- Data Visualization: Convert the contents of a CSV file into an “animated bar chart race video.”
- Iterative Edits: Make revision requests as if speaking to a video editor, such as “Make the title 2x larger and add a fade-out effect at the end.”
This AI-first workflow aims to lower the barrier to video production, shifting the required skill from professional editing expertise to the ability to describe requirements clearly.
Architecture Based on HTML and Web Technologies
The technical core of Hyperframes involves mapping various video elements—such as video clips, images, audio, and text—to HTML tags. By adding data-* attributes to these tags, developers can precisely control their behavior on the timeline.
For example, data-start defines an element’s start time, data-duration defines its duration, and data-track-index specifies its track (layer). This declarative approach makes video composition intuitive and easy to version control.
Here’s how it works:
- Define: A developer or AI agent writes an HTML file containing the video’s timeline logic.
- Preview: The
npx hyperframes preview command launches a live preview environment in the browser with hot-reloading support.
- Render: When the
npx hyperframes render command is executed, the framework starts a headless browser environment in the background, using Puppeteer to capture the rendered output of the HTML page frame by frame.
- Encode: The captured image sequence and audio tracks are finally combined and encoded into a standard MP4 video file using FFmpeg.
Additionally, Hyperframes uses a “Frame Adapter” pattern, which avoids locking developers into a specific animation technology. This allows for the freedom to use GSAP, Lottie, CSS Animations, or even Three.js to create complex visual effects.
Modular Ecosystem and Deterministic Rendering
To boost development efficiency, Hyperframes offers a catalog of over 50 pre-built components, including social media overlays, shader transitions, and data charts. These can be quickly integrated into a project using the npx hyperframes add [component-name] command.
The project itself features a highly modular package structure, with its main NPM packages including:
hyperframes: The core command-line interface (CLI).@hyperframes/core: Defines the framework’s types, parsers, and runtime.@hyperframes/engine: The page-to-video capture engine based on Puppeteer and FFmpeg.@hyperframes/producer: The complete rendering pipeline that integrates capture, encoding, and audio mixing.
A key feature is its deterministic rendering capability, meaning the same inputs (HTML, CSS, asset files) will always produce an identical output video. This is crucial for enterprise applications that need to incorporate video generation into automated testing or CI/CD pipelines, allowing videos to be managed as reliably as other software build artifacts.
The open-sourcing of Hyperframes (project available at: https://github.com/heygen-com/hyperframes) signals that video creation is moving further towards engineering and automation. It not only provides developers with a powerful tool for programmatic video but also endows AI agents with new capabilities to create dynamic visual content.
