Knowledge Distribution in German Drama: An Annotated Corpus

What do characters in theater plays know about character relations, and how does the distribution of knowledge evolve over a play’s course? We present a dataset of 30 German plays annotated with information about the distribution of knowledge about character relations (such as “A learns from B that C is the parent of D”). All plays were manually annotated by two independent annotators in the Q:TRACK project, which aims to systematically model character knowledge. The dataset is available on GitHub and Zenodo and can be reused, for example, for systematic studies of knowledge in plays or for analyzing annotator disagreements.


2.
In the second round, the other 14 plays were annotated independently following the guideline.These plays were used to calculate the inter-annotator agreement using the measure gamma by Mathet, Widlöcher, and Métivier (2015), as presented (and discussed critically) in Andresen, Krautter, Pagel, and Reiter (2022b).
3. In a final round, every play was discussed and double checked by at least one annotator.In this round, three more relations were added for murder, death and pregnancy.
The final version of the corpus (round 3) comprises 37 files, as for seven plays, both annotators performed the last step of finalizing the annotations, resulting in two final versions for these plays.We decided to keep two versions instead of creating a single gold standard, because in many cases more than one way of annotating the play was justified (see below).In total, there are 1277 annotated text passages, which corresponds to an average number of 34.5 annotations per text, with a considerable standard deviation of 18.8.

SAMPLING STRATEGY
The plays were manually selected to cover • plays of which we knew that knowledge about character relations is important for the plot, (?) as well as plays where this was not the case, • tragedies as well as comedies, • plays from different literary epochs .
Accordingly, the dataset is not designed to be representative of a specific group of texts, but to cover a wide range of relevant phenomena.

QUALITY CONTROL
All plays were annotated by two people independently, making it possible to calculate the inter-annotator agreement.The agreement is rather low for many of the plays, see Table 2.This is due to the high complexity and interpretation dependency of the task.In many cases more than one way of modeling the data is plausible.Also, measuring inter-annotator agreement in a way that makes the scores comparable to other studies is challenging for annotations without predefined annotation spans.See Andresen et al. (2022b) for a more in-depth discussion and the repository for more detailed scores.We publish several versions of each annotation as well as the annotation guidelines (Andresen et al., 2021, in German) for comparability and transparency.

REUSE POTENTIAL
The dataset can be reused in a number of ways.Literary scholars might take the data as a starting point for a systematic analysis of knowing and not-knowing, knowledge distribution and knowledge transmission between characters in one or several individual plays.This is often considered a crucial piece of information for the interpretation of dramatic texts (Gutjahr, 2012;Kiss, 2010).Horstmann (2018, pp. 184-209) has proposed to narratologically reinforce theater studies by including focalization, understood as relations of knowledge, into the analysis.
Analyses of individual plays can be supported by the visualization of the data as we have suggested in Andresen, Krautter, Pagel, and Reiter (2022a) and Andresen et al. (2022b).
Quantitative analyses of the frequency of specific types of knowledge transfers, for instance, are limited by the size of the dataset, but are still possible on a small scale.This allows insights into which relations are discussed most often, which characters are the most important for knowledge transfer and similar questions.The annotations could also be aligned with the attempt to model character relationships based on topic modeling as presented in Iyyer, Guha, Chaturvedi, Boyd-Graber, and Daumé III (2016).
To solve the problem of data scarcity in the long term, the dataset can be used as training and/ or test data for attempts to automate this type of annotation, for instance by prompting large language models (Liu et al., 2023;Ziems et al., 2023).As we provide the annotations of two annotators for most plays, the data can also be used to investigate annotation disagreement.One may investigate if annotation disagreements point to ambiguous and potentially crucial text passages or look into the causes of disagreements (Andresen, Vauth, & Zinsmeister, 2020;Gius & Jacke, 2017).

Table 1
List of all plays included in the corpus.Andresen et al.