This article continues the discussion on the evolution of multi-task learning-based large-scale recommender systems. We take a look at strategies from Kuaishou, Tencent, YouTube, Facebook, and Amazon Prime Video to disentangle input space and address systematic biases. The article ends with sharing several tips and learnings for professionals working in this domain.| All Posts - Sumit's Diary
The Mixture-of-Experts (MoE) is a classical ensemble learning technique originally proposed by Jacobs et. al1 in 1991. MoEs have the capability to substantially scale up the model capacity and only introduce small computation overhead. This ability combined with recent innovations in the deep learning domain has led to the wide-scale adoption of MoEs in healthcare, finance, pattern recognition, etc. They have been successfully utilized in large-scale applications such as Large Language Modeli...| Sumit's Diary