Training Machine Learning models on big data isn’t just about fitting the model itself — it’s about efficiency at every stage of the process. While much attention is given to optimizing model training itself, the earlier phases can be just as, if not more, critical to the overall performance. In this article, we take a deep dive into what happens before we actually invoke model.fit(), focusing on the data pivoting stage. We are taking you on a journey through various pivoting solutions,...| blog.allegro.tech
Lazy evaluation is an interesting topic that can be a little bit difficult to understand at first glance, it is quite common for me to have to help people who misunderstand when their code is executed – since they use lazy technologies such as Spark Dataframes or Django ORM.| Coderbook