Predicting Board Game Collections

J_3MBG’s Collection

About

This report details the results of training and evaluating a classification model for predicting games for a user’s boardgame collection.

Note

To view games predicted by the model, go to Section 5.

Collection

The data in this project comes from BoardGameGeek.com. The data used is at the game level, where an individual observation contains features about a game, such as its publisher, categories, and playing time, among many others.

I train a classification model at the user level to learn the relationship between game features and games that a user owns - what predicts a user’s collection?

username	status	games
J_3MBG	ever_owned	574
J_3MBG	own	372
J_3MBG	rated	606

I evaluate the model’s performance on a training set of historical games via resampling, then validate the model’s performance on a set aside set of newer relases. I then refit the model on the training and validation in order and predict upcoming releases in order to find new games that the user is most likely to add to their collection.

username	years	type	Own
			no	yes
J_3MBG	-3500-2020	train	24240	278
J_3MBG	2021-2022	valid	9844	53
J_3MBG	2023-2028	test	9058	41

Types of Games

What types of game does the user own? The following plot displays the most frequent publishers, mechanics, designers, artists, etc that appear in a user’s collection.

collection |>
        filter(own == 1) |>
        collection_by_category(
                games = games_raw
        ) |>
        plot_collection_by_category()+
        ylab("feature")

The following plot shows the years in which games in the user’s collection were published. This can usually indicate when someone first entered the hobby.

Games in Collection

What games does the user currently have in their collection? The following table can be used to examine games the user owns, along with some helpful information for selecting the right game for a game night!

Use the filters above the table to sort/filter based on information about the game, such as year published, recommended player counts, or playing time.

collection |>
        filter(own == 1) |>
        prep_collection_datatable(
                games = games_raw
        ) |>
        filter(!is.na(image)) |>
        collection_datatable()

Modeling

I’ll now the examine predictive models trained on the user’s collection.

For an individual user, I train a predictive model on their collection in order to predict whether a user owns a game. The outcome, in this case, is binary: does the user have a game listed in their collection or not? This is the setting for training a classification model, where the model aims to learn the probability that a user will add a game to their collection based on its observable features.

How does a model learn what a user is likely to own? The training process is a matter of examining historical games and finding patterns that exist between game features (designers, mechanics, playing time, etc) and games in the user’s collection.

I make use of many potential features for games, the vast majority of which are dummies indicating the presence or absence of the presence or absence of things such as a publisher/artist/designer. The “standard” BGG features for every game contain information that is typically listed on the box its playing time, player counts, or its recommended minimum age.

Note

I train models to predict whether a user owns a game based only on information that could be observed about the game at its release: playing time, player count, mechanics, categories, genres, and selected designers, artists, and publishers. I do not make use of BGG community information, such as its average rating, weight, or number of user ratings. This is to ensure the model can predict newly released games without relying on information from the BGG community.

What Predicts A Collection?

A predictive model gives us more than just predictions. We can also ask, what did the model learn from the data? What predicts the outcome? In the case of predicting a boardgame collection, what did the model find to be predictive of games a user has in their collection?

To answer this, I examine the coefficients from a model logistic regression with ridge regularization (which I will refer to as a penalized logistic regression).

Positive values indicate that a feature increases a user’s probability of owning/rating a game, while negative values indicate a feature decreases the probability. To be precise, the coefficients indicate the effect of a particular feature on the log-odds of a user owning a game.

The following visualization shows the path of each feature as it enters the model, with highly influential features tending to enter the model early with large positive or negative effects. The dotted line indicates the level of regularization that was selected during tuning.

model_glmnet |> 
        pluck("wflow", 1) |>
        trace_plot.glmnet(max.overlaps = 30)+
        facet_wrap(~params$username)

Partial Effects

What are the effects of individual features?

Use the buttons below to examine the effects different types of predictors had in predicting the user’s collection.

Categories

Mechanics

Themes

Families

Publishers

Designers

Artists

Components

Mechanisms

Assessment

How well did the model do in predicting the user’s collection?

This section contains a variety of visualizations and metrics for assessing the performance of the model(s). If you’re not particularly interested in predictive modeling, skip down further to the predictions from the model.

The following displays the model’s performance in resampling on a training set, a validation set, and a holdout set of upcoming games.

metrics |>
        mutate_if(is.numeric, round, 3) |>
        pivot_wider(
                names_from = c(".metric"),
                values_from = c(".estimate")) |>
        gt::gt() |>
        gt::sub_missing() |>
        gt_options()

username	wflow_id	type	.estimator	mn_log_loss	roc_auc	pr_auc
J_3MBG	glmnet	resamples	binary	0.045	0.916	0.200
J_3MBG	glmnet	test	binary	0.027	0.886	0.051
J_3MBG	glmnet	valid	binary	0.028	0.909	0.128

An easy way to visually examine the performance of classification model is to view a separation plot.

I plot the predicted probabilities from the model for every game (during resampling) from lowest to highest. I then overlay a blue line for any game that the user does own. A good classifier is one that is able to separate the blue (games owned by the user) from the white (games not owned by the user), with most of the blue occurring at the highest probabilities (left side of the chart).

preds |>
        filter(type %in% c('resamples', 'valid')) |>
        plot_separation(outcome = params$outcome)

I can more formally assess how well each model did in resampling by looking at the area under the ROC curve (roc_auc). A perfect model would receive a score of 1, while a model that cannot predict the outcome will default to a score of 0.5. The extent to which something is a good score depends on the setting, but generally anything in the .8 to .9 range is very good while the .7 to .8 range is perfectly acceptable.

preds |>
        nest(data = -c(username, wflow_id, type)) |>
        mutate(roc_curve = map(data, safely( ~ .x |> safe_roc_curve(truth = params$outcome)))) |>
        mutate(result = map(roc_curve, ~ .x |> pluck("result"))) |>
        select(username, wflow_id, type, result) |>
        unnest(result) |>
        plot_roc_curve()

Top Games in Training

What were the model’s top games in the training set?

preds |>
        filter(type == 'resamples') |>
        prep_predictions_datatable(
                games = games, 
                outcome = params$outcome
        ) |>
        predictions_datatable(outcome = params$outcome,
                remove_description = T, 
                remove_image = T, 
                pagelength = 15)

Top Games in Validation

What were the model’s top games in the validation set?

preds |>
        filter(type %in% c("valid")) |>
        prep_predictions_datatable(
                games = games,
                outcome = params$outcome
        ) |>
        predictions_datatable(
                outcome = params$outcome,
                remove_description = T, 
                remove_image = T, 
                pagelength = 15)

Top Games by Year

Displaying the model’s top games for individual years in recent years.

preds |>
        filter(type %in% c('resamples', 'valid')) |>
        top_n_preds(
                games = games,
                outcome = params$outcome,
                top_n = 15,
                n_years = 15
        ) |>
        gt_top_n(collection = collection |> prep_collection())

Rank	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021	2022
1	Battlestar Galactica: The Board Game	Dominion: Intrigue	DungeonQuest (Third Edition)	Village	Terra Mystica	Lewis & Clark: The Expedition	Imperial Settlers	The Gallerist	Star Wars: Rebellion	This War of Mine: The Board Game	Cosmic Encounter: 42nd Anniversary Edition	Paladins of the West Kingdom	Dwellings of Eldervale	The Great Wall	Nemesis: Lockdown
2	A Game of Thrones: The Card Game	Cyclades	Troyes	A Game of Thrones: The Board Game (Second Edition)	Wiz-War (Eighth Edition)	Eldritch Horror	AquaSphere	7 Wonders Duel	Terraforming Mars	My Little Scythe	Underwater Cities	Chocolate Factory	Etherfields	Terraforming Mars: Ares Expedition	Wayfarers of the South Tigris
3	Pandemic	Chaos in the Old World	Dominant Species	The New Era	The Manhattan Project	Relic	King of New York	Viticulture Essential Edition	Scythe	Anachrony	Newton	Tainted Grail: The Fall of Avalon	Century: Golem Edition – An Endless World	Ark Nova	ISS Vanguard
4	Roll Through the Ages: The Bronze Age	Small World	Runewars	Rune Age	Robinson Crusoe: Adventures on the Cursed Island	Forbidden Desert	Pandemic: Contagion	Pandemic Legacy: Season 1	Islebound	Pandemic Legacy: Season 2	Nemesis	Star Wars: Outer Rim	Dune: Imperium	Rome: Rising Empires	Carnegie
5	Space Alert	Shipyard	Labyrinth: The War on Terror, 2001 – ?	Mage Knight Board Game	Keyflower	Rococo	Praetor	Tiny Epic Galaxies	Lorenzo il Magnifico	Azul	Everdell	Clank! Legacy: Acquisitions Incorporated	Viscounts of the West Kingdom	Canvas	Frostpunk: The Board Game
6	Dominion	Imperial 2030	51st State	The Lord of the Rings: The Card Game	Empires of the Void	Legacy: The Testament of Duke de Crecy	Arcadia Quest	Mission: Red Planet (Second Edition)	The Manhattan Project: Energy Empire	Century: Spice Road	Arkham Horror (Third Edition)	Tapestry	The Search for Planet X	Sleeping Gods	Starship Captains
7	Call of Cthulhu: The Card Game	Greed Incorporated	7 Wonders	Puerto Rico	Android: Infiltration	Eight-Minute Empire: Legends	La Granja	Rattle, Battle, Grab the Loot	Days of Ire: Budapest 1956	Twilight Imperium: Fourth Edition	Brass: Birmingham	Wingspan	Guild Master	Cascadia	Woodcraft: Roll and Write
8	The Rich and the Good	Thunderstone	Forbidden Island	Elder Sign	Suburbia	City of Iron	Pandemic: The Cure	Thunderbirds	Star Trek: The Dice Game	Gaia Project	Architects of the West Kingdom	Aftermath	Pandemic Legacy: Season 0	Galactic Era	The Age of Atlantis
9	Android	Axis & Allies: 1942	Alien Frontiers	Space Empires 4X	Android: Netrunner	City of Remnants	Artifacts, Inc.	Above and Below	Kanagawa	Century: Golem Edition	Rising Sun	Pax Pamir: Second Edition	Hallertau	Unsettled	Bardsung
10	Stone Age	War of the Ring	Dominion: Big Box	War of the Ring: Second Edition	Archipelago	Glass Road	Sons of Anarchy: Men of Mayhem	Risk: Europe	Dead of Winter: The Long Night	Gloomhaven	Azul: Stained Glass of Sintra	Era: Medieval Age	On Mars	Arkham Horror: The Card Game (Revised Edition)	Shogun No Katana Deluxe Edition
11	Cosmic Encounter	Finca	Sid Meier's Civilization: The Board Game	Belfort	Zombicide	Russian Railroads	Fields of Arle	Super Motherload	Arkham Horror: The Card Game	Fallout	Everdell: Collector's Edition	Fantastic Factories	Santorini: New York	Roll Camera!: The Filmmaking Board Game	Endless Winter: Paleoamericans
12	Cavum	Ad Astra	Dungeons & Dragons: Castle Ravenloft Board Game	Nightfall	Rex: Final Days of an Empire	Theseus: The Dark Orbit	Star Wars: Imperial Assault	Minerva	Great Western Trail	First Martians: Adventures on the Red Planet	Pax Emancipation	PARKS	Deep Vents	Genotype: A Mendelian Genetics Game	SPYBAM
13	Planet Steam	Age of Conan: The Strategy Board Game	Battles of Westeros	Ora et Labora	Andean Abyss	Gearworld: The Borderlands	Five Tribes: The Djinns of Naqala	Zombicide: Black Plague	Evolution: Climate	Sagrada	Between Two Castles of Mad King Ludwig	Horrified	Raiders of Scythia	Bloodborne: The Board Game	Frosthaven
14	Byzanz	Terra Prime	Warhammer: The Island of Blood	King of Tokyo	Star Wars: The Card Game	Zombicide Season 2: Prison Outbreak	Roll for the Galaxy	Blood Rage	Perdition's Mouth: Abyssal Rift	Spirit Island	New Frontiers	Core Space	The Lost Worlds of Josh Kirby	Flourish	Merchants of the Dark Road
15	Axis & Allies Anniversary Edition	Arcana	Prêt-à-Porter	A Few Acres of Snow	Descent: Journeys in the Dark (Second Edition)	Dominion: Special Edition	Three Kingdoms Redux	Raiders of the North Sea	Mansions of Madness: Second Edition	Charterstone	Space Park	Amul	Eclipse: Second Dawn for the Galaxy	Radlands	Planet Unknown

Predictions

New and Upcoming Games

What were the model’s top predictions for new and upcoming board game releases?

new_preds |>
        filter(type == 'upcoming') |>
        # imposing a minimum threshold to filter out games with no info
        filter(usersrated >= 1) |>
        # removing this goddamn boxing game that has every mechanic listed
        filter(game_id != 420629) |>
        prep_predictions_datatable(
                games = games_new,
                outcome = params$outcome
        ) |>
        predictions_datatable(outcome = params$outcome)

Older Games

What were the model’s top predictions for older games?