# MetaQA KG–QA (Navigation-Ready Encoding)

A navigation-ready, identifier-encoded packaging of the original **MetaQA** movie-domain knowledge graph question answering dataset introduced by **Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J. Smola, and Le Song**.

MetaQA was introduced in *Variational Reasoning for Question Answering with Knowledge Graph* (AAAI 2018). The original dataset and documentation are available from the [MetaQA repository](https://github.com/yuyuz/MetaQA).

> **Attribution and scope.** This release does **not** introduce a new MetaQA dataset. The underlying knowledge graph, questions, answers, hop definitions, text variants, and train/dev/test splits originate from the original MetaQA release. This packaging only reorganizes and encodes those resources to make them easier to use in graph-navigation pipelines.

## Key Features

- Movie-domain **knowledge graph + question answering (KGQA)** benchmark
- **43,234 entities**, **9 relations**, and **134,741 triples**
- Natural-language **1-hop, 2-hop, and 3-hop** questions
- Explicit **start/topic entity** for navigation-based models
- Support for **multiple valid answers**
- Compact local **EID/RID identifiers** for entities and relations
- Original MetaQA **train/dev/test split labels** preserved
- Available question variants grouped for convenient text-based evaluation

---

## Contents

```text
kg/
└─ triplets.txt

metadata/
├─ node_data.csv
└─ relation_data.csv

qa/
├─ metaqa_1hop.csv
├─ metaqa_2hop.csv
├─ metaqa_3hop.csv
└─ metaqa_nhop.csv
```

---

## 1) Knowledge Graph (KG) Triples

The encoded MetaQA knowledge graph is stored in:

```text
kg/triplets.txt
```

Each line follows the format:

```text
head relation tail
```

- Triples are **tab-separated**.
- Entities are represented using local **EIDs**.
- Relations are represented using local **RIDs**.
- Human-readable labels are available through the files in `metadata/`.

### EID/RID encoding

The `EID` and `RID` values are **local, opaque identifiers** introduced only to provide a compact and unambiguous graph representation for agents and preprocessing code.

They do **not** carry semantic meaning. In particular:

- `EID` values are not Wikidata QIDs or another external entity ontology.
- `RID` values are not Wikidata PIDs or another external relation ontology.
- The numeric portion of an identifier does not encode entity type, relation type, importance, ordering, or graph structure.

For example, the original MetaQA triple:

```text
Kismet directed_by William Dieterle
```

is represented using the local metadata mappings as:

```text
e16009 r6 e05727
```

where:

```text
e16009 -> Kismet
r6     -> directed_by
e05727 -> William Dieterle
```

The encoding is intended only to make graph actions easier to read, parse, and execute consistently in navigation-based systems.

---

## 2) Metadata

Metadata is stored under the `metadata/` directory and maps the local identifiers back to the original human-readable MetaQA labels.

### `metadata/node_data.csv`

| Column | Description |
|---|---|
| `EID` | Local entity identifier |
| `Title` | Original human-readable entity label |

Example:

```text
EID,Title
e16009,Kismet
e05727,William Dieterle
```

### `metadata/relation_data.csv`

| Column | Description |
|---|---|
| `RID` | Local relation identifier |
| `Title` | Original human-readable relation label |

Example:

```text
RID,Title
r6,directed_by
```

The release contains **43,234 entity mappings** and **9 relation mappings**.

---

## 3) Question Answering (QA) Files

QA datasets are stored under the `qa/` directory.

| File | Description |
|---|---|
| `metaqa_1hop.csv` | Contains only 1-hop questions |
| `metaqa_2hop.csv` | Contains only 2-hop questions |
| `metaqa_3hop.csv` | Contains only 3-hop questions |
| `metaqa_nhop.csv` | Mixed dataset containing 1–3 hop questions |

### CSV schema

| Column | Description |
|---|---|
| `Question-Number` | Integer question identifier |
| `Question` | Natural-language question |
| `Question-Paraphrased` | List of available question text variants |
| `Source` | Start/topic entity in human-readable form |
| `Source-Entity` | Encoded start/topic entity (`EID`) |
| `Answer` | List of valid gold answers in human-readable form |
| `Answer-Entity` | List of encoded gold answer entities (`EID`) |
| `Hops` | Original MetaQA reasoning hop count (`1`, `2`, or `3`) |
| `SplitLabel` | Original dataset split (`train`, `dev`, or `test`) |

### Navigation-ready representation

The original MetaQA text format marks the topic entity directly in the question. In this packaging, the topic entity is additionally materialized as:

```text
Source
Source-Entity
```

and the valid targets are explicitly represented as:

```text
Answer
Answer-Entity
```

This allows a navigation agent to begin from a known graph node and evaluate arrival against the encoded answer set without recovering entity identifiers from free text.

For questions with multiple valid answers, reaching **any entity listed in `Answer-Entity`** should be treated as a correct terminal answer.

> **Reference-path note.** This packaging does not add manually annotated entity-level gold reasoning paths to MetaQA. Unlike navigation resources that contain `Paths` or `Paths-Label` columns, MetaQA is made navigation-ready here by explicitly materializing its start entity and answer entities over the encoded graph.

---

## 4) Question Text Variants

The original MetaQA release contains a **Vanilla** text dataset and an **NTM** text dataset whose questions were paraphrased using neural machine translation. The original documentation describes the NTM variants as English questions translated to French and then back to English, while preserving the referenced entity.

Where available, the corresponding text variants are grouped in the `Question-Paraphrased` field for convenient evaluation while retaining the original MetaQA question semantics.

The audio and MFCC resources distributed by the original MetaQA release are **not part of this navigation-oriented packaging**.

---

## Quick Example

A packaged QA instance has the form:

```text
Question: What movies are about ginger rogers
Source: ginger rogers
Source-Entity: e24748
Answer: ['Top Hat', 'Kitty Foyle', 'The Barkleys of Broadway']
Answer-Entity: ['e27148', 'e17053', 'e08394']
Hops: 1
SplitLabel: train
```

For navigation, the agent starts at `e24748` and attempts to reach one of the valid answer entities in `Answer-Entity`.

---

## Usage Notes

- **Identifier lookup.** Use `node_data.csv` and `relation_data.csv` whenever human-readable labels are required. Do not infer semantics from the EID/RID numbering.
- **Multiple answers.** `Answer` and `Answer-Entity` are serialized lists because MetaQA questions may have more than one valid answer.
- **Hop labels.** The `Hops` field preserves the original MetaQA 1-hop, 2-hop, and 3-hop categorization.
- **Dataset splits.** `SplitLabel` preserves the original MetaQA train/dev/test assignment.
- **Graph direction.** `triplets.txt` preserves the subject–relation–object structure of the MetaQA knowledge base. Navigation implementations that expose inverse actions should construct those actions explicitly at load time rather than assigning additional meaning to the RID values.

---

## Loading Tip

Several QA fields are serialized Python-style list literals. They can be parsed safely with `ast.literal_eval`.

```python
import ast
import pandas as pd

qa = pd.read_csv("qa/metaqa_nhop.csv")

qa["Question-Paraphrased"] = qa["Question-Paraphrased"].map(ast.literal_eval)
qa["Answer"] = qa["Answer"].map(ast.literal_eval)
qa["Answer-Entity"] = qa["Answer-Entity"].map(ast.literal_eval)
```

To recover human-readable graph labels:

```python
nodes = pd.read_csv("metadata/node_data.csv")
relations = pd.read_csv("metadata/relation_data.csv")

entity_title = dict(zip(nodes["EID"], nodes["Title"]))
relation_title = dict(zip(relations["RID"], relations["Title"]))
```

---

## Dataset Statistics

The following summary uses the MetaQA statistics reported in *Theseus in the Graph: Towards Traceable Multi-Hop Graph Navigation* for the navigation benchmark setting.

### Knowledge Graph

- Entities: **43,234**
- Relations: **9**
- Triples: **134,741**

### Question Answering

- Total QA pairs: **407,513**
- Hop range: **1–3**

---

## Original Dataset and Provenance

MetaQA was introduced by **Zhang et al.** as a multi-hop question answering benchmark over a movie-domain knowledge base derived from **MovieQA / WikiMovies**.

The original release provides:

- 1-hop, 2-hop, and 3-hop question sets;
- train/dev/test splits;
- Vanilla natural-language questions;
- NTM-generated paraphrased questions;
- audio questions and MFCC features; and
- the movie knowledge base used to generate the QA tasks.

This navigation-ready packaging preserves the original benchmark content while changing how entities, relations, source nodes, and answer nodes are represented for graph-based agents.

### Upstream resources

- **Original MetaQA repository:** https://github.com/yuyuz/MetaQA
- **Original paper:** https://arxiv.org/abs/1709.04071

---

## Citation

If you use this dataset, please cite the **original MetaQA paper**:

```bibtex
@inproceedings{zhang2017variational,
  title={Variational Reasoning for Question Answering with Knowledge Graph},
  author={Zhang, Yuyu and Dai, Hanjun and Kozareva, Zornitsa and Smola, Alexander J and Song, Le},
  booktitle={AAAI},
  year={2018}
}
```

The EID/RID representation used in this packaging is only a local encoding of the original MetaQA entities and relations and does not change the provenance or authorship of the dataset.

---

## License

The underlying MetaQA data remains subject to the licensing terms of the **original MetaQA release**.

The upstream MetaQA README states that the data is released under the **Creative Commons Public License** and that a copy of the license is included with the distributed data. Users should consult the [original MetaQA repository](https://github.com/yuyuz/MetaQA) and its distributed dataset package for the applicable license terms.

This navigation-ready encoding does not change the ownership, provenance, or attribution of the original MetaQA dataset.