Supervised Topic Modeling
Supervised Topic Modeling Although topic modeling is typically done by discovering topics in an unsupervised manner, there might be times when you already have a bunch of clusters or classes from which you want to model the topics . For example, the often used 20 NewsGroups dataset is already split up into 20 classes. Similarly, you might already have created some labels yourself through ...
Topic modeling is an unsupervised NLP technique that aims to extract hidden themes within a corpus of textual documents. This paper provides a thorough and comprehensive review of topic modeling techniques from classical methods such as latent sematic analysis to most cutting-edge neural approaches and transformer-based methods.
Rethinking How I Do Supervised Topic Modeling, Using ModernBERT and GPT ...
I wrote a post in July 2023 describing my process for building a supervised text classification pipeline. In short, the process first involves reading the text, writing a thematic content coding guide, and having humans label text. Then, I define a variety of ways to pre-process text (e.g., word vs. word-and-bigram tokenizing, stemming vs. not, stop words vs. not, filtering on the number of ...
Such details provide a deeper understanding and appreciation for Supervised Topic Modeling.
However, due to the lack of guidance from labels, unsupervised neural topic models are less powerful in this situation. Existing supervised neural topic models often adopt a label-free prior to generate the latent document- topic distributions and use them to predict the labels and thus achieve label- topic alignment indirectly.
Useful Notes on Supervised Topic Modeling
In semi- supervised topic modeling , we only have some labels for our documents. The documents for which we have labels are used to guide BERTopic to the extraction of topics for those labels.
