Disclaimer: These ideas are untested and only come from my intuition. I don’t have the resources to explore them any further. intro CoT and test time compute have been proven to be the future direction of language models for better or for worse. o1 and DeepSeek-R1 demonstrate a step function in model intelligence. Coconut also provides a way for this reasoning to occur in latent space. I have been thinking about the geometric structure of the latent space where this reasoning can occur.
