Predicting Board Game Collections

rahdo’s Collection

About

This report details the results of training and evaluating a classification model for predicting games for a user’s boardgame collection.

Note

To view games predicted by the model, go to Section 5.

Collection

The data in this project comes from BoardGameGeek.com. The data used is at the game level, where an individual observation contains features about a game, such as its publisher, categories, and playing time, among many others.

I train a classification model at the user level to learn the relationship between game features and games that a user owns - what predicts a user’s collection?

username	status	games
rahdo	ever_owned	1776
rahdo	own	290
rahdo	rated	272

I evaluate the model’s performance on a training set of historical games via resampling, then validate the model’s performance on a set aside set of newer relases. I then refit the model on the training and validation in order and predict upcoming releases in order to find new games that the user is most likely to add to their collection.

username	years	type	Own
			no	yes
rahdo	-3500-2021	train	26158	204
rahdo	2022-2023	valid	10251	57
rahdo	2024-2028	test	8564	29

Types of Games

What types of game does the user own? The following plot displays the most frequent publishers, mechanics, designers, artists, etc that appear in a user’s collection.

collection |>
    filter(own == 1) |>
    collection_by_category(
        games = games_raw
    ) |>
    plot_collection_by_category() +
    ylab("feature")

The following plot shows the years in which games in the user’s collection were published. This can usually indicate when someone first entered the hobby.

Games in Collection

What games does the user currently have in their collection? The following table can be used to examine games the user owns, along with some helpful information for selecting the right game for a game night!

Use the filters above the table to sort/filter based on information about the game, such as year published, recommended player counts, or playing time.

collection |>
    filter(own == 1) |>
    prep_collection_datatable(
        games = games_raw
    ) |>
    filter(!is.na(image)) |>
    collection_datatable()

Modeling

I’ll now the examine predictive models trained on the user’s collection.

For an individual user, I train a predictive model on their collection in order to predict whether a user owns a game. The outcome, in this case, is binary: does the user have a game listed in their collection or not? This is the setting for training a classification model, where the model aims to learn the probability that a user will add a game to their collection based on its observable features.

How does a model learn what a user is likely to own? The training process is a matter of examining historical games and finding patterns that exist between game features (designers, mechanics, playing time, etc) and games in the user’s collection.

I make use of many potential features for games, the vast majority of which are dummies indicating the presence or absence of the presence or absence of things such as a publisher/artist/designer. The “standard” BGG features for every game contain information that is typically listed on the box its playing time, player counts, or its recommended minimum age.

Note

I train models to predict whether a user owns a game based only on information that could be observed about the game at its release: playing time, player count, mechanics, categories, genres, and selected designers, artists, and publishers. I do not make use of BGG community information, such as its average rating, weight, or number of user ratings. This is to ensure the model can predict newly released games without relying on information from the BGG community.

What Predicts A Collection?

A predictive model gives us more than just predictions. We can also ask, what did the model learn from the data? What predicts the outcome? In the case of predicting a boardgame collection, what did the model find to be predictive of games a user has in their collection?

To answer this, I examine the coefficients from a model logistic regression with ridge regularization (which I will refer to as a penalized logistic regression).

Positive values indicate that a feature increases a user’s probability of owning/rating a game, while negative values indicate a feature decreases the probability. To be precise, the coefficients indicate the effect of a particular feature on the log-odds of a user owning a game.

The following visualization shows the path of each feature as it enters the model, with highly influential features tending to enter the model early with large positive or negative effects. The dotted line indicates the level of regularization that was selected during tuning.

#|
model_glmnet |>
    pluck("wflow", 1) |>
    trace_plot.glmnet(max.overlaps = 30) +
    facet_wrap(~ params$username)

Partial Effects

What are the effects of individual features?

Use the buttons below to examine the effects different types of predictors had in predicting the user’s collection.

Categories

Mechanics

Themes

Families

Publishers

Designers

Artists

Components

Mechanisms

Assessment

How well did the model do in predicting the user’s collection?

This section contains a variety of visualizations and metrics for assessing the performance of the model(s). If you’re not particularly interested in predictive modeling, skip down further to the predictions from the model.

The following displays the model’s performance in resampling on a training set, a validation set, and a holdout set of upcoming games.

metrics |>
    mutate_if(is.numeric, round, 3) |>
    pivot_wider(
        names_from = c(".metric"),
        values_from = c(".estimate")
    ) |>
    gt::gt() |>
    gt::sub_missing() |>
    gt_options()

username	wflow_id	type	.estimator	mn_log_loss	roc_auc	pr_auc
rahdo	glmnet	resamples	binary	0.033	0.942	0.153
rahdo	glmnet	test	binary	0.021	0.884	0.043
rahdo	glmnet	valid	binary	0.026	0.944	0.134

An easy way to visually examine the performance of classification model is to view a separation plot.

I plot the predicted probabilities from the model for every game (during resampling) from lowest to highest. I then overlay a blue line for any game that the user does own. A good classifier is one that is able to separate the blue (games owned by the user) from the white (games not owned by the user), with most of the blue occurring at the highest probabilities (left side of the chart).

preds |>
    filter(type %in% c("resamples", "valid")) |>
    plot_separation(outcome = params$outcome)

I can more formally assess how well each model did in resampling by looking at the area under the ROC curve (roc_auc). A perfect model would receive a score of 1, while a model that cannot predict the outcome will default to a score of 0.5. The extent to which something is a good score depends on the setting, but generally anything in the .8 to .9 range is very good while the .7 to .8 range is perfectly acceptable.

preds |>
    nest(data = -c(username, wflow_id, type)) |>
    mutate(
        roc_curve = map(
            data,
            safely(~ .x |> safe_roc_curve(truth = params$outcome))
        )
    ) |>
    mutate(result = map(roc_curve, ~ .x |> pluck("result"))) |>
    select(username, wflow_id, type, result) |>
    unnest(result) |>
    plot_roc_curve()

Top Games in Training

What were the model’s top games in the training set?

preds |>
    filter(type == "resamples") |>
    prep_predictions_datatable(
        games = games,
        outcome = params$outcome
    ) |>
    predictions_datatable(
        outcome = params$outcome,
        remove_description = T,
        remove_image = T,
        pagelength = 15
    )

Top Games in Validation

What were the model’s top games in the validation set?

preds |>
    filter(type %in% c("valid")) |>
    prep_predictions_datatable(
        games = games,
        outcome = params$outcome
    ) |>
    predictions_datatable(
        outcome = params$outcome,
        remove_description = T,
        remove_image = T,
        pagelength = 15
    )

Top Games by Year

Displaying the model’s top games for individual years in recent years.

preds |>
    filter(type %in% c("resamples", "valid")) |>
    top_n_preds(
        games = games,
        outcome = params$outcome,
        top_n = 15,
        n_years = 15
    ) |>
    gt_top_n(collection = collection |> prep_collection())

Rank	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021	2022	2023
1	Shipyard	7 Wonders	The Castles of Burgundy	The Manhattan Project	Lewis & Clark: The Expedition	Port Royal	Oh My Goods!	Honshū	The Castles of Burgundy: The Dice Game	Newton	The Magnificent	Lost Ruins of Arnak	Ark Nova	Planet Unknown	La Granja: Deluxe Master Set
2	Macao	Innovation	Puerto Rico	Suburbia	Amerigo	La Granja	7 Wonders Duel	Agricola (Revised Edition)	NUT	Underwater Cities	The Castles of Burgundy	Puerto Rico	Boonlake	Beer & Bread	Expeditions
3	At the Gates of Loyang	Luna	Ora et Labora	Tzolk'in: The Mayan Calendar	Glass Road	Imperial Settlers	Viticulture Essential Edition	Terraforming Mars	Circle the Wagons	Architects of the West Kingdom	Wingspan	Tekhenu: Obelisk of the Sun	Lorenzo il Magnifico: Big Box	Marrakesh	Welcome To...: Collector's Edition
4	Dungeon Lords	Merkator	Mage Knight Board Game	Robinson Crusoe: Adventures on the Cursed Island	Bruges	AquaSphere	Tiny Epic Galaxies	The Castles of Burgundy: The Card Game	Spirit Island	Everdell	Maracaibo	Gloomhaven: Jaws of the Lion	Great Western Trail: Second Edition	The Lord of the Rings: The Card Game – Revised Core Set	Empire's End
5	Endeavor	Troyes	Eminent Domain	Galaxy Trucker: Anniversary Edition	Händler der Karibik	Fields of Arle	Mombasa	Lorenzo il Magnifico	Santa Maria	Carpe Diem	Era: Medieval Age	Faiyum	Cascadia	Tiletum	Shipyard (Second Edition)
6	Finca	Dominion: Big Box	Strasbourg	Ginkgopolis	Dominion: Special Edition	Patchwork	Between Two Cities	Scythe	Tybor the Builder	Concordia Venus	Tapestry	Monster Expedition	Agropolis	Woodcraft: Roll and Write	Art Society
7	Bunny Bunny Moose Moose	Navegador	Tournay	Terra Mystica	Suburbia + Inc.	Orléans	Through the Ages: A New Story of Civilization	Broom Service: The Card Game	Unfair	New Frontiers	Marco Polo II: In the Service of the Khan	Bonfire	Corrosion	Woodcraft	Night Flowers
8	Dominion: Intrigue	Spiel mit Lukas: Dribbel-Fieber	The New Era	Keyflower	Nations	Viticulture: Complete Collector's Edition	Pandemic Legacy: Season 1	Last Will	Charterstone	Sprawlopolis	PARKS	The Castles of Tuscany	Llamaland	Frostpunk: The Board Game	Ticket to Ride Legacy: Legends of the West
9	The Pillars of the Earth: Builders Duel	Glen More	City Tycoon	Agricola: All Creatures Big and Small	Bora Bora	Roll for the Galaxy	The Voyages of Marco Polo	Cottage Garden	Anachrony	Welcome To...	Paladins of the West Kingdom	Merlin: Big Box	Terraforming Mars: Ares Expedition	Carnegie	Nucleum
10	Vasco da Gama	20th Century	The City	Il Vecchio	Patchistory	Isle of Trains	Minerva	The Oracle of Delphi	Sagrada	The Estates	Fields of Arle: Big Box	New York Zoo	Rolling Realms	Agricola 15	Astro Knights
11	Arena: Roma II	Vinhos	Walnut Grove	Würfel Bohnanza	Viticulture	Fresco: Big Box	Steampunk Rally	Codenames: Pictures	Gaia Project	Railroad Ink: Blazing Red Edition	Ragusa	Furnace	Galaxy Trucker (Second Edition)	Endless Winter: Paleoamericans	Cuzco
12	Alea Iacta Est	The Speicherstadt	PAX	Wallenstein (Second Edition)	Euphoria: Build a Better Dystopia	Nations: The Dice Game	Mombasa (Limited Edition)	A Feast for Odin	Altiplano	Space Park	Ultra Tiny Epic Galaxies	Welcome to New Las Vegas	Newton & Great Discoveries	Hamburg	The White Castle
13	Automobile	It Happens..	Trajan	Uchronia	Caverna: The Cave Farmers	La Isla	Het Koninkrijk Dominion	Kodama: The Tree Spirits	In the Year of the Dragon: 10th Anniversary	Between Two Castles of Mad King Ludwig	Barrage	Calico	Corduba 27 a.C.	Amsterdam	The Castles of Burgundy: Special Edition
14	Homesteaders	The Mines of Zavandor	Pergamemnon	Seasons	Concordia	Alchemists	Broom Service	Codenames: Deep Undercover	Notre Dame: 10th Anniversary	Orchard: 9 card solitaire game	Clank! Legacy: Acquisitions Incorporated	Pandemic Legacy: Season 0	Tumble Town	Wayfarers of the South Tigris	Aldebaran Duel
15	Alhambra: Big Box	Funfair	Friday	Myrmes	Cinque Terre	Five Tribes: The Djinns of Naqala	The Prodigals Club	Jórvík	Near and Far	Hokkaido	Tiny Towns	Dune: Imperium	Imperium: Classics	New York City	Faraway

Predictions

New and Upcoming Games

What were the model’s top predictions for new and upcoming board game releases?

new_preds |>
    filter(type == "upcoming") |>
    # imposing a minimum threshold to filter out games with no info
    filter(usersrated >= 1) |>
    # removing this goddamn boxing game that has every mechanic listed
    filter(game_id != 420629) |>
    prep_predictions_datatable(
        games = games_new,
        outcome = params$outcome
    ) |>
    predictions_datatable(outcome = params$outcome)

Older Games

What were the model’s top predictions for older games?