Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis

Shayegan Omidshafiei, Andrei Kapishnikov, Yannick Assogba, Lucas Dixon, Been Kim

---

2-agent Coordination Environment

Description

Domain

2-agent coordination environment, where states and actions correspond, respectively, to the 2D positions and forces imparted by each agent for movement. Three regions are defined: A and B (two circular regions), and C (the area exterior to the circles). Agents spawn at the center and are each rewarded +1 whenever they simultaneously navigate to any combination of circular regions, with the exception of (B, B).

Plots

• The large plot on the left is a PCA projection of the joint embedding space learned using MOHBA, and the smaller plots on the right are PCA projections of the local embedding spaces learned for each agent.
• Each point on the plot corresponds to a complete trajectory. Colors indicate the sum of agent returns, which may be helpful for narrowing down more interesting trajectories.
• The same trajectories are displayed in each scatter plot.
• 16384 trajectories are plotted.

Interaction Instructions

• Click and Drag to select a region of the embedding space. For each selected trajectory, a corresponding image summarizing the agents behaviour will be displayed below the charts. Click and drag on an empty region on the scatter plot to de-select trajectories. Once a region is selected you may need to scroll down the page to see all the images.
• Use the mouse-wheel on the scatter plots to zoom in and out.
• Activate the pan tool in the tools menu on the left to switch the click and drag interaction to panning around the space.