Login
From:
Rotary Positional Embeddings with Relative distance (RoPER)
(Uncensored)
subscribe
Rotary Positional Embeddings with Relative distance (RoPER)
https://nn.labml.ai/transformers/rope/value_pe/index.html
links
backlinks
Roast topics
Find topics
Find it!
This is an implementation of RoPER which adds relative distance information to embeddings on top of RoPE introduced in RoFormer: Enhanced Transformer with Rotary Position Embedding
