An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation

Online Supplement

Bach Unconditioned Samples

Relative Transformer

Baseline Transformer

Piano-e-Competition Unconditioned Samples

Relative Transformer

Piano-e-Competition Primed Samples

Priming sequence

Relative Transformer

Baseline Transformer

Performance RNN LSTM