Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis

MultiAgent-MuJoCo AntWalker Environment

4-Agent AntWalker Multiagent-MuJoCo domain, wherein 4 agents each control one of 4 ant legs to coordinate movement towards the +x direction.

The large plot on the left is a T-SNE projection of the joint embedding space learned using MOHBA, and the smaller plots on the right are T-SNE projections of the local embedding spaces learned for each agent.
Each point on the plot corresponds to a complete trajectory. Colors indicate the sum of agent returns, which may be helpful for narrowing down more interesting trajectories.
The same trajectories are displayed in each scatter plot.
8192 trajectories are plotted.

Click and Drag to select a region of the embedding space. For each selected trajectory, a corresponding animated gif summarizing the agents behaviour will be displayed below the charts. Click and drag on an empty region on the scatter plot to de-select trajectories. Once a region is selected you may need to scroll down the page to see all the images.
Warning: The animated gif for each trajectory is about ~5mb and multiple will be downloaded upon selecting a region and scrolling down the page. Images are loaded lazily if your browser supports it.
Use the mouse-wheel on the scatter plots to zoom in and out.
Activate the pan tool in the tools menu on the left to switch the click and drag interaction to panning around the space.