Retrieval-Augmented Generation (RAG) isn’t a silver bullet. This post highlights the hidden costs associated with RAG and makes the case for a smarter, adaptive approach.| Sumit's Diary
Learned embeddings often suffer from ’embedding collapse’, where they occupy only a small subspace of the available dimensions. This article explores the causes of embedding collapse, from two-tower models to GNN-based systems, and its impact on model scalability and recommendation quality. We discuss methods to detect collapse and examine recent solutions proposed by research teams at Visa, Facebook AI, and Tencent Ads to address this challenge.| Sumit's Diary
This article provides an introduction to online advertising systems and explores research work that incorporates ads into the LLM responses to user queries of commercial nature.| blog.reachsumit.com
This article continues the discussion on the evolution of multi-task learning-based large-scale recommender systems. We take a look at strategies from Kuaishou, Tencent, YouTube, Facebook, and Amazon Prime Video to disentangle input space and address systematic biases. The article ends with sharing several tips and learnings for professionals working in this domain.| All Posts - Sumit's Diary
This article introduces the multi-task learning paradigm adopted by various large-scale video recommender systems. It introduces a general setup for such an MTL-based recommender. It highlights several associated challenges and describes solutions adopted by various state-of-the-art recommenders in the industry.| blog.reachsumit.com
This article provides an introduction and literature review for multi-task learning based recommender systems. We learn how to discover task relations, design MTL architectures and overcome some of the associated challenges.| blog.reachsumit.com
Modeling users’ past historical interactions or behavior sequences is an essential task for domains like recommender systems, click-through rate prediction, targeted advertisement, and more. This article provides a comprehensive introduction to the user behavior modeling paradigm along with highlighting several relevant and recent research works.| blog.reachsumit.com
The previous article did a deep dive into the prompting-based pointwise, pairwise, and listwise techniques that directly use LLMs to perform reranking. In this article, we will take a closer look at some of the shortcomings of the prompting methods and explore the latest efforts to train ranking-aware LLMs. The article also describes several strategies to build effective and efficient LLM-based rerankers.| blog.reachsumit.com
In recent years, there has been a significant amount of research activity in the graph representation learning domain. These learning methods help in analyzing abstract graph structures in information networks and improve the performances of state-of-the-art machine learning solutions for real-world applications, such as social recommendations, targeted advertising, user search, etc. This article provides a comprehensive introduction to the graph representation learning domain, including comm...| Sumit's Diary
The Mixture-of-Experts (MoE) is a classical ensemble learning technique originally proposed by Jacobs et. al1 in 1991. MoEs have the capability to substantially scale up the model capacity and only introduce small computation overhead. This ability combined with recent innovations in the deep learning domain has led to the wide-scale adoption of MoEs in healthcare, finance, pattern recognition, etc. They have been successfully utilized in large-scale applications such as Large Language Modeli...| Sumit's Diary
