A waiter that reads the room.
Customers come in, order at the counter — or sit down and call the waiter to order — talk, take calls, and leave; some want another drink, water, napkins or the bill and call for it, and much else they do only looks like a call. The G1 takes their orders and carries the drinks the barista puts on its tray. Every second System 1 is asked what to do with what the robot's head camera sees — people, their joints — and what it hears: whom to serve, who is calling, what to say, and what was asked for.
Simulated in MuJoCo (Google DeepMind, Apache-2.0) — not a sim-to-real result · the robot is Unitree Robotics' G1 as shipped in NVIDIA's GR00T-WholeBodyControl (Apache-2.0), with Unitree's rubber hands from MuJoCo Menagerie (BSD-3-Clause, licence); its legs and waist run GR00T-WholeBodyControl's Walk and Balance policies, licensed by NVIDIA Corporation under the NVIDIA Open Model License (Agreement,notice) · the people are Microsoft Rocketbox avatars and animations (MIT, © 2020 Microsoft, licence) · the furniture, furnishings, surfaces and the street outside (Piazza Bologni, Palermo) are Poly Haven's (CC0) · stand-ins, named: people are read from a rendered mask whose instance numbers stand in for segmentation and re-identification, their joints are the avatars' projected into the camera for a pose estimator, and what is heard is the café's utterance text for speech recognition · the arms, the grip and the order of the moves are this page's executor; the judgment is System 1's · the System 1 layer runs on the decision model chosen above, through OpenRouter