Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis

Shayegan Omidshafiei, Andrei Kapishnikov, Yannick Assogba, Lucas Dixon, Been Kim

---

3-Agent Hill-Climbing Environment

Description

Domain

3-agent hill-climbing environment, where states and actions correspond, respectively, to the 2D positions and forces imparted by each agent for movement. Agents spawn at the origin and are each rewarded for climbing any of three hills within the domain in episodes of 50 timesteps.

Plots

• The large plot on the left is a PCA projection of the joint embedding space learned using MOHBA, and the smaller plots on the right are PCA projections of the local embedding spaces learned for each agent.
• Each point on the plot corresponds to a complete trajectory. Colors indicate the sum of agent returns, which may be helpful for narrowing down more interesting trajectories.
• The same trajectories are displayed in each scatter plot.
• 16384 trajectories are plotted.

Interaction Instructions

• Click and Drag to select a region of the embedding space. For each selected trajectory, a corresponding image summarizing the agents behaviour will be displayed below the charts. Click and drag on an empty region on the scatter plot to de-select trajectories. Once a region is selected you may need to scroll down the page to see all the images.
• Use the mouse-wheel on the scatter plots to zoom in and out.
• Activate the pan tool in the tools menu on the left to switch the click and drag interaction to panning around the space.