HeyHomees Lab Notes · No. 16 · Meta · Published November 4, 2026
By HeyHomees Newsroom ·
In 30 seconds
- It simulates first-person view, dialogue and actions in 44 homes using 6,831 objects.
- The benchmark, InteractiveBench, freezes 120 multi-turn tasks across six categories.
- Running requires an NVIDIA GPU (tested on A100 80 GB, ~4 GB per simulator worker) and ~100 GB of disk.
What is it?
InteractiveWorld is a reusable simulator that presents a first-person view of a human avatar, lets the avatar act via short commands, and updates a stateful 3D world. It includes a benchmark layer, InteractiveBench, which defines 120 multi-turn tasks for evaluating assistant models. The environment checks goal completion against the world state rather than textual judgment.
How does it work?
A human avatar walks, grasps, opens and uses objects according to 36 action families (e.g., [Grab], [OpenDoor]) defined in docs/ACTIONS.md. Commands are issued via a command-line or Python API; the simulator launches Isaac Sim in a separate process while the user’s code stays independent. InteractiveBench pairs an Assistant model (seeing only the avatar’s view and dialogue) with a fixed Human model that executes the Assistant’s commands. A task passes when all goals hold in the final world state.
What does it need to run?
It runs on Linux with an NVIDIA GPU (tested on A100 80 GB; one simulator worker uses about 4 GB of GPU memory) and requires about 100 GB of disk. Setup involves downloading Isaac Sim 5.1, the BEHAVIOR-1K dataset (31 GB), a SMPL+H body model, and an ACCAD walking motion; the latter two are licensed for non-commercial research and must be registered separately. Baking the 6,831 objects takes about two minutes with eight workers.
How well does it do?
The release provides 120 frozen tasks (20 in each of six categories: Guidance, Planning, Accessibility, Decision, Prevention, Finding). Success is measured by whether every goal holds in the final world state; no external accuracy scores are reported in the source.
By the numbers
- Number of homes and venues simulated: 44
- GPU memory per simulator worker: about 4 GB
- Disk space required: about 100 GB
- Number of action families: 36
- Number of frozen benchmark tasks: 120
- Tasks per category: 20
The honest read
The figures show the scale of the environment (44 homes, 6,831 objects) and the hardware footprint (≈4 GB GPU memory per worker, ≈100 GB disk). Success is defined only by goal satisfaction in the world state; no quantitative model performance numbers are provided, so the release does not report how well any assistant model does on the benchmark. All licensing statements note that the SMPL+H body model and ACCAD motion are for non-commercial research only and cannot be redistributed. The benchmark tasks are frozen and publicly available, but the source does not give any baseline scores or comparisons, so users must supply their own models and evaluate them to obtain results. The hardware requirements are limited to Linux with an NVIDIA GPU; no CPU-only or non-NVIDIA options are mentioned.
What we don't know yet
- Exact performance of assistant models on InteractiveBench
- Support for non-Linux platforms or non-NVIDIA GPUs
- Whether the simulator can run without Isaac Sim (the source says discovery commands start instantly but does not detail full functionality)
Sources
- README, GitHub (primary)
Researched and drafted with AI from the sources above; every figure is checked against them by code before it is published, and a line that fails is dropped.