There’s a lot of great research on foundation models these days that are yielding deeper understanding of their mechanics (how they work) and phenomenology (how they behave in different scenarios). But despite a growing body of research literature, in my work at Notion and conversations with founders, I’ve noticed a big gap between what we currently know about foundation models and what we still need to build valuable tools out of them. This post is a non-exhaustive list of applied resear...| thesephist.com
Foundation models gesture at a way of interacting with information that’s at once more natural and powerful than “classic” knowledge tools. But to build the kind of rich, directly interactive information interfaces I imagine, current foundation models and embeddings are far too opaque to humans. Models and their raw outputs resist understanding. Even when we go to great lengths to try to surface what the model is “thinking” through black-box methods like prompting or dimensionality ...| thesephist.com