Making Deep Learning Go Brrrr From First Principles| horace.io
Overview: Memory management in Python involves a private heap containing all Python objects and data structures. The management of this private heap is ensured internally by the Python memory manag...| Python documentation
We’re on a journey to advance and democratize artificial intelligence through open source and open science.| huggingface.co
What's different about LLMs versus traditional ML| ★❤✰ Vicki Boykis ★❤✰
Source code: Lib/tracemalloc.py The tracemalloc module is a debug tool to trace memory blocks allocated by Python. It provides the following information: Traceback where an object was allocated, St...| Python documentation
torch.Tensor¶| pytorch.org
We’re on a journey to advance and democratize artificial intelligence through open source and open science.| huggingface.co
I have spent many years as an software engineer who was a total outsider to machine-learning, but with some curiosity and occasional peripheral interactions with it. During this time, a recurring theme for me was horror (and, to be honest, disdain) every time I encountered the widespread usage of Python pickle in the Python ML ecosystem. In addition to their major security issues1, the use of pickle for serialization tends to be very brittle, leading to all kinds of nightmares as you evolve y...| Made of Bugs
Base class for all neural network modules.| pytorch.org
Today, we’re introducing the availability of Llama 2, the next generation of our open source large language model.| Meta
Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring and enrichment of that 2020 post — restructure the hierarchy of sections and improve many sections with more recent papers. Version 2.0 is a superset of the old version, about twice the length. Notations Symbol Meaning $d$ The model size / hidden state dimension / positional encoding size.| lilianweng.github.io