Coordinate frames

The scene is modelled as a pose graph: every component is positioned relative to whichever frame is most natural for it, and scenes differ only in where the graph’s root sits in the world. This page is the contract. The constants themselves live in so101_mvbench.utils.so101_transforms.

Read it as two chains that meet at the cube {P}. The arm chain runs world {0} -> arm base {B} -> end effector {E} -> wrist camera {C_w}, given by the scene and the robot’s kinematics. The workspace chain runs {0} -> workspace {W} -> external camera {C_E}, and the bin spawner places the cube relative to {W}. Three edges carry the task: the two cameras observe the cube, and the policy has to learn {E} -> {P}, the transform that closes the loop, from those observations alone. {W} -> {C_E} is the experimental variable — changing it is what a camera configuration does.

Moving the rig into another scene changes {0} -> {W} and {0} -> {B} by the same rigid translation, so every edge inside the two chains is untouched. That is why a recorded joint trajectory replays unchanged in a different scene.

The workspace frame {W}

All task geometry is authored in one frame, the workspace frame {W}: the bin grid is centred on it, external cameras orbit it on a hemisphere (azimuth, elevation, radius), the cube spawns on it, and the arm base sits at a fixed offset from it. A scene places this frame in the world by declaring one value:

"workspace_origin_m": [0.0, 0.0, 0.8663]

in its scene config (SceneConfig.workspace_origin_m). That triple is the {W} origin expressed in the scene’s world frame — nothing else about the task geometry changes between scenes.

Fixed {W}-relative anchors (identical in every scene):

Anchor	Position in {W}
Bin grid centre	(0, 0)
Arm base	(-0.20, 0, -0.12)
Cube spawn height	z = -0.07 (CUBE_SPAWN_Z_IN_W)
Camera hemisphere centre	(0, 0, 0)

The two origins that matter

Constant	Value	Meaning
HEMISPHERE_ORIGIN_W	(0.20, 0.0, 0.12)	{W} origin of the retired floor-level table scene. Kept solely to interpret legacy datasets.
BASE_SCENE_ORIGIN_W	(0.0, 0.0, 0.8663)	{W} origin of the cylindrical room — the scene LiftCubeSceneCfg is authored in.

Their difference, (-0.20, 0.0, 0.7463), is the historical ARM_BASE_OFFSET: the rigid translation that carried the whole floor-scene rig onto the pedestal table.

Recorded data declares its frame

Every recording writes its scene’s {W} origin into both sidecars:

• binmap.json → top-level workspace_origin_m

• meta/episode_*_scene_state.json → workspace_origin_m

A dataset without the field predates the mechanism and by definition means the floor-level table scene (HEMISPHERE_ORIGIN_W). That is why the released corpus records the bin_c0_r0 cube at world x = 0.32 while a fresh cylroom recording of the same bin records x = 0.12: both are 0.12 in {W}.

Two shifts, not one

Replay and evaluation derive two distinct translations, and conflating them is the documented failure mode (it once put the arm on the floor):

• RIG_SHIFT = target origin − BASE_SCENE_ORIGIN_W. Re-anchors the scene furniture (robot base, action pad, cube spawn, external camera). Zero for the shipped configs, because the base scene already is the cylindrical room.

• DATA_SHIFT = target origin − source origin. Transports recorded coordinates (binmap x/y, spawn height) from the scene the data was captured in. Zero when replaying a cylroom recording in the cylroom, and the full table-to-cylroom delta for a legacy dataset.

Because the whole rig moves rigidly, the arm-to-cube geometry is identical in every scene and recorded joint trajectories replay unchanged — this is the pose-graph property the entire cross-scene pipeline rests on. Verified end-to-end: the cube lands at the predicted world position with err_xy = 0.0 m for both legacy and native recordings (multicam_replay --verify_placement).

Conventions worth pinning

• Robot forward = +X. The SO-ARM101 USD mesh defaults to −Y, and a +90° yaw in init_state.rot turns it to +X.

• Quaternion convention is XYZW since Isaac Lab PR #4437. Helpers in so101_white.py and lift_cube_env_cfg.py swizzle the euler_angles_to_quat() output (which is WXYZ) accordingly.

• XFormPrim.set_local_poses() is WXYZ. Different convention than OffsetCfg.rot and init_state.rot — easy to mix up.