“A Database of Intertexts in Valerius Flaccus’ Argonautica 1: A Benchmarking Resource for the Evaluation of Computational Intertextual Search of Latin Corpora”

Characterization of intertextual references among authors is fundamental for the study of Latin literature. In this paper, we describe a large-scale intertextuality dataset compiled from three modern commentaries on Valerius Flaccus’ epic poem Argonautica . The dataset includes 945 references to earlier and contemporary Roman authors, as well as associated metadata required for use of multiple intertext search tools. To illustrate the dataset’s reuse potential, we perform a new

Since the leading Latin intertextual search tool, developed by the Tesserae Project, analyzes two-word phrases, our dataset follows the same practice for ease of comparison (Coffee et al., 2012).The Tesserae team also corroborated a common intuition among philologists that Latin intertexts frequently occur in the form of two-word phrases.On occasion, however, commentators refer to a single-word intertext or an intertext composed of several words.In these cases, our research team used expert judgment to choose a relevant two-word phrase.These phrases are composed of the key intertextual term plus a natural complement in close proximity (e.g., an adjective-noun, verb-object, or preposition-noun unit).Our dataset uses the term "query phrase" to indicate a two-word phrase of interest in VF 1 derived from the commentaries and "result phrase" to indicate the two-word comparison phrase likewise derived from the commentaries.The terminology of "query" and "result" is intended to emphasize that the phrases are being deployed in a search and retrieval process that looks outward from VF 1 to a set of comparison texts.This unidirectional process is a technical simplification of human reading, of course, which takes account of multiple texts simultaneously in determining which phrases in any text might be of interest.In other words, a commentator's choice of lemma can never arise from consideration of a single text in isolation but always emerges from a personal history of reading situated within a broader literary critical and cultural context.
In addition to the list of intertexts, the commentary sources, and the citation information, the dataset also includes three parameters describing the relationship between the phrase in VF 1 and the parallel phrases noted by commentators, which are labeled "Order Free," "Interval," and "Edit Distance"."Order Free" is a binary parameter indicating whether the words comprising the result phrase (i.e., the intertext or parallel phrase) are either adjacent and in the same order as the source phrase (in which case the cell is marked "False") or non-adjacent or in reverse order ("True")."Interval" indicates the number of words between the words comprising the result phrase for parallels that are labeled "Order Free;" when the result phrase consists of adjacent words, the interval is zero.For fixed-order searches, the "Interval" column is left blank."Edit Distance" indicates the number of character substitutions, additions, or deletions required to turn the query phrase into the result phrase.The first two parameters are necessary input data for the method of semantic intertextual search described in our original paper (Burns et al., 2021).The same parameters plus the edit distance parameter are required for another tool, Fīlum, which we developed to identify phonetically similar phrases (Chaudhuri et al., 2015;Chaudhuri & Dexter, 2017; see Reuse Potential for further discussion).

SAMPLING STRATEGY
Select intertextual parallels previously recorded in the commentaries of Spaltenstein (2002), Kleywegt (2005), and Zissos (2008) were included in the dataset.No new parallels were added.

QUALITY CONTROL
All entries in the dataset were reviewed by multiple members of the project team for completeness and accuracy.

LANGUAGE
The language of entries in the dataset is Latin.The language of the metadata is English.

4) REUSE POTENTIAL
The dataset presented here provides a resource for researchers investigating intertextuality in historical language traditions, especially Latin.For philologists pursuing qualitative studies, the catalog assembles in a single source a systematic list of intertextual parallels between VF 1 and several major Latin epic poets who either influenced or were influenced by Valerius Flaccus.The availability of three modern, high-quality philological commentaries attests to the fact that this particular text is an especially rich model of intertextual relationships.The collation from multiple sources, furthermore, enables researchers to analyze the practices of modern philologists, in particular the patterns of reference and degree of overlap among the commentators.Such reuse could involve qualitative review of recorded intertexts, as well as quantitative and statistical analysis of the entire dataset.For instance, although the total number of parallels recorded by each commentator differs substantially (Kleywegt 757,Zissos 373,Spaltenstein 228), it is striking that the distribution over comparison texts is fairly consistent, with the Aeneid referred to most often (55-64%) and Ovid (7-17%), Lucan (17-18%), and Statius (11-13%) each referred to with approximately similar frequency.
Perhaps the greatest future potential, however, lies in the use of the dataset as a benchmark for testing intertextual search methods.This use case was already exemplified in the original paper (Burns et al., 2021), which compared lemma matching and semantic scoring using word embeddings as two complementary approaches to Latin intertextual search.The addition of the edit distance parameter in the revised dataset now facilitates evaluation of search methods based on character similarity, such as Fīlum.
To illustrate this kind of reuse potential, we used the dataset to conduct a large-scale validation analysis of Fīlum.In prior work, we applied Fīlum to a variety of case studies involving intertextuality in classical and post-classical Latin literature (Chaudhuri et al., 2015;Chaudhuri and Dexter, 2017), but we have not previously validated the performance of the tool at scale.We ran book-level Fīlum searches for all 945 intertexts in the dataset using the three parameters recorded therein ("Order Free," "Interval," and "Edit Distance").To summarize these results, we calculated the precision@k and recall@k across a range of cutoffs (k = 1, 3, 5, 10, 25, 50, 75, 100, and 250), which are the metrics we used to validate the semantic search method (see Burns et al., 2021 for details).As shown in Figure 2, Fīlum outperforms semantic search in both precision and recall; more than half of all intertexts can be recovered with no off-target results, and more than 90% can be recovered with at most 250 off-target results.One example of the type of intertext pair well-suited to discovery using phonetic search (but not semantic search) is patuere doli ("his deceptions were revealed," VF 1.64) and [nec] latuere doli ("deceptions did [not] escape the notice," Aeneid 1.130).Although the phrases are phonetically very similaronly a single character (p/l) distinguishes them -they are semantically distinct; the two verbs are almost antonymic.As a consequence, the similarity score for the pair calculated from word embeddings is not especially high, which contrasts with the very low edit distance and, therefore, the absence of off-target results in a Fīlum search.

Figure 1
Figure 1 Overview of intertextuality dataset compiled from three commentaries on VF 1.

OBJECT NAME vf_intertext_dataset_2_0.csv FORMAT NAMES AND VERSIONS CSV CREATION DATES
The dataset was created by Joseph P. Dexter, Pramit Chaudhuri, Patrick J. Burns, Elizabeth D. Adams, Thomas J. Bolt, Adriana Cásarez, Jeffrey H. Flynt, Kyle Li, James F. Patterson, Ariane Schwartz, and Scott Shumway.Dexter et al.