#| Annotated Research Paper Implementations: Transformers, StyleGAN, Stable Diff...
This is an implementation of RoPER which adds relative distance information to embeddings on top of RoPE introduced in RoFormer: Enhanced Transformer with Rotary Position Embedding| Rotary Positional Embeddings with Relative distance (RoPER)