Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis

Shayegan Omidshafiei, Andrei Kapishnikov, Yannick Assogba, Lucas Dixon, Been Kim

---

MultiAgent-Mujoco HalfCheetah Environment

Description

Domain

2-Agent HalfCheetah Multiagent-MuJoCo domain, where agent 0 and 1, respectively, control the back and front limbs of a cheetah to coordinate movement. Each agent's state consists of a 6-dimensional vector summarizing velocity and position information for the 3 joints it controls, with its 3-dimensional action space corresponding to motor torques applied to said joints. The agents coordinate to maximize the speed of the cheetah in the forward (+x) direction over episodes of 200 timesteps each.

Plots

• The large plot on the left is a PCA projection of the joint embedding space learned using MOHBA, and the smaller plots on the right are PCA projections of the local embedding spaces learned for each agent.
• Each point on the plot corresponds to a complete trajectory. Colors indicate the sum of agent returns, which may be helpful for narrowing down more interesting trajectories.
• The same trajectories are displayed in each scatter plot.
• 15856 trajectories are plotted.

Interaction Instructions

• Click and Drag to select a region of the embedding space. For each selected trajectory, a corresponding animated gif summarizing the agents behaviour will be displayed below the charts. Click and drag on an empty region on the scatter plot to de-select trajectories. Once a region is selected you may need to scroll down the page to see all the images.
  Warning: The animated gif for each trajectory is about ~5mb and multiple will be downloaded upon selecting a region and scrolling down the page. Images are loaded lazily if your browser supports it.
• Use the mouse-wheel on the scatter plots to zoom in and out.
• Activate the pan tool in the tools menu on the left to switch the click and drag interaction to panning around the space.