Generalist AI's GEN-1.5 teaches robots a task from one demo

Robotics startup Generalist AI has unveiled GEN-1.5, a model that teaches a robot a new task from a single demonstration lasting 3 to 12 seconds, the company said. The clip loads into the model's context window as what the company calls a "physical prompt," functioning as short-term memory the robot draws on to perform the task with no additional training.

Across ten tests, including opening a jar and pulling money from a wallet, Generalist reports an average success rate of 59% with zero training. Give the model ten training steps built from five minutes of extra data on the same task, and that figure climbs to 83%. The model can also chain two demonstrations into a longer sequence, learn from simulated demos instead of real ones, and partially imitate human hand movements it observes.

• Demo length needed to prompt a new task: 3 to 12 seconds
• Zero-training success rate across 10 tasks: 59%
• Success rate after 10 training steps on 5 minutes of data: 83%
• Pretraining period Generalist credits for these emergent abilities: over 8 months
• Independent verification of the results: none so far

Generalist says the robot never received explicit training to chain prompts or imitate hand movements. The company attributes both abilities to more than eight months of pretraining on interaction data, where they emerged on their own rather than being built in by design. Other research groups have already demonstrated similar in-context learning for robots, but Generalist claims to be the first to make the approach work across a wide range of task types rather than a narrow handful.

The tasks shown are short and simple, and Generalist produced every number in this report itself. No outside lab has replicated or verified the success rates, a gap that matters for a company whose pitch rests on robots learning general-purpose skills rather than narrow, hand-tuned ones. Founded by former Google DeepMind senior researcher Pete Florence, who worked on the PaLM-E and RT-2 robotics models, Generalist raised $400 million in a round led by Radical Ventures earlier this year, valuing the company at $2 billion with backing from Nvidia's venture arm and Bezos Expeditions.

GEN-1.5 follows GEN-1, the foundation model Generalist introduced first, which the company says handles physical tasks with a 99% reliability rate against an industry benchmark it puts closer to 64%. Where GEN-1 aims at reliably repeating tasks a robot has already learned, GEN-1.5 targets the harder problem of picking up a brand-new task on the spot, the same in-context learning trick that let large language models follow a written instruction without retraining on it first. Applying that idea to a physical robot, one that has to translate a short video into a working sequence of joint movements rather than a string of text, is a meaningfully different engineering problem, which is part of why Generalist's claim to generalize across a wide range of task types stands out from prior work limited to a handful. Warehouse operators and manufacturers, the customers a startup like Generalist ultimately needs to convince, care less about a demo reel than about how a system behaves on tasks nobody scripted in advance, which is exactly the scenario GEN-1.5 is built to handle and exactly the claim outside labs have yet to test for themselves.

This article is for informational purposes only and does not constitute investment advice.