Induction Labs Photon-1: A Foundation Model That Learns Tasks From Raw Video Alone

Induction Labs has released Photon-1, a foundation model that learns to perform computer tasks by watching raw video of human screen activity. Unlike most AI agents that require a labelled action for every frame, Photon-1 pretrains solely on video sequences using a next-latent-token prediction objective. The model never sees a mouse click, keystroke, or instruction during pretraining, yet it learns to complete tasks such as dragging files, launching applications, and even playing checkers — all by predicting what the next frame should look like.

The research team calls this approach an “imagination model.” The architecture is a sparse 106B-parameter mixture-of-experts (MoE) transformer with 5B active parameters per token. It was trained from scratch on 575 million frames of computer screen recordings, sampled at one frame per second, drawn from an internal index of 2 billion publicly available videos. The final dataset represents roughly 18 years of continuous video.

Datos clave

Metric	Value
Model size	106B-A5B sparse MoE
Training data	575 million frames (552B tokens)
Pretraining compute	30,000 H200 GPU-hours (4.4 × 10²² FLOPs)
Input cost	$0.06 per 1M input tokens

How the model works

Photon-1 uses a vision encoder that compresses each frame into 960 discrete tokens, each an 8-dimensional vector with values drawn from a set of five: −1, −1/2, 0, 1/2, 1. This finite scalar quantisation gives a codebook of 5⁸ possible codes, yielding about 2.2 KB per frame. Induction Labs reports over 100× better compression than existing OCR and multimodal-model representations, while preserving text, layout, and state changes.

A differential latent encoder further improves compression by encoding video frames as pairs — the latents describe differences between frames rather than the full content. During pretraining, the model learns to predict the next latent state autoregressively. It does not generate pixels; everything is modelled in a learned representation space.

After pretraining, the research team finetuned Photon-1 on fewer than 35,000 computer-use trajectories to teach it the action and instruction format. At inference, the model first predicts the next frame’s state, then outputs the action that gets there. Online reinforcement learning follows, with rollouts running in real time on virtual machines at scale.

Benchmark performance and caveats

On an internal computer-use benchmark, Induction Labs reports that Photon-1 beats Gemini 3.1 Flash-Lite while using far less pretraining compute and costing roughly 3× less to serve. The Gemini comparison is based on Induction Labs’ own conservative estimate of 8B active parameters and 25T pretraining tokens. The benchmark is internal and unreleased, so the result is not independently reproducible today.

The more interesting test involved domains the model had never seen during pretraining. On 20,000 tournament checkers games from the Open Checkers Archive 2.0, Photon-1 beat both a vision encoder baseline (same architecture, no imagination pretraining) and an LLM baseline (Ling-flash-2.0 from Inclusion AI, pretrained on 20T tokens) on world simulation and move quality. On 10,000 synthetically generated billiard games simulated at 5 fps, Photon-1 produced a mean absolute error of 0.47 against the ground-truth physics engine, versus 1.15 for the LLM baseline and 1.44 for the vision encoder baseline.

Implications for AI agents and Indian research

The core claim — that predicting future states teaches a model to complete tasks without ever seeing an action during pretraining — challenges the dominant paradigm in agentic AI. If validated, this approach could reduce the enormous cost of collecting action-labelled data, a bottleneck that particularly affects smaller labs and researchers in emerging AI hubs like India.

Induction Labs has published a full technical writeup and the announcement thread on X. The model weights and training code are not yet publicly available, but the research team has indicated plans to release a smaller version for community experimentation.

Why this matters

Photon-1 suggests that raw video contains enough structure for a model to learn an implicit policy of how to interact with a computer interface. The model picks up human priors — it learned to use an in-VM ChatGPT clone to draft artifacts and answer knowledge questions, steering the LLM the way a person would. This opens the door to AI agents that can be trained on any screen recording, without requiring manual annotation of every action.

Source: MarkTechPost, https://www.marktechpost.com/2026/07/26/induction-labs-photon-1-simulates-desktops-plays-checkers-and-models-billiard-physics-from-one-pretraining-run/