RoPER, in addition to using relative positions in the attention score calculation, adds relative positional information explicitly to value embeddings.| labml.ai research blog
