From Python 3.14, python -m pdb -p pid lets you connect a pdb session to a running Python process. | rtpg.co
From Python 3.14, python -m pdb -p pid lets you connect a pdb session to a running Python process. This post goes into a part of what makes this possible. The barrier to entry for writing general debugging tools for Python programs has always been quite low. Unlike many languages, you're rarely finding yourself working with weird internals. Instead, debugging tools can be built off of pretty straightforward knowledge of the language. This is powered by the languages treating things like excep...| Raphael
So I've been messing with Purescript recently. Purescript is a purely functional language that compiles to Javascript. It includes support for things like object literals that make Javascript interop pain-free. It also has mature tooling, making getting started quite simple.| rtpg.co
Sometimes you're looking at a snippet of code, and want to change it. my_datamy_data.some_attr When going from the first line to the second, I needed to put my cursor near the end, and then add the attribute access. Especially when working in a REPL where the previous line was just above me, this is pretty nice! my_data.some_attrjson.loads(my_data.some_attr) This is less nice. I have to jump backwards, add json.loads, and wrap the thing I care about! And here I'm assuming we had import json. ...| Raphael
When working on a live system, the feedback loop can be pretty tricky. You don't want to entirely suspend a system, but you really want to get the info out of the system. Memory leaks in particular can be a nightmare, because they can take a while to show up and, definitionally, won't survive a process restart. Let's talk about debugging memory leaks. objgraph is a great library to get a birds-eye view of whats in your memory. print("Showing growth...")objgraph.show_growth()print("...end show...| Raphael
This post summarizes my takeaways from Platform as a Reflection of Values, a very eye-opening talk that helped me understand why I would have the same disagreements over and over with some team members in the past. Let's imagine a universe where you and I are on a small engineering team. I care about testing a lot. I believe the following: Automated tests prevent regressions in production. Easy-to-test code is easy-to-debug code. After all, a debug session is often a prelude to a test-writing...| Raphael
Consider the following snippet of Rust, of a Target struct, along with a builder to go along with it: structTarget{foo:bool,bar:u64,baz:i64,}structBuilder{foo:Option<bool>,bar:Option<u64>,baz:Option<i64>,}implBuilder{fnnew()->Builder{returnBuilder{foo:None,bar:None,baz:None,};}fnfoo(mutself,v:bool)->Builder{self.foo=Some(v);returnself;}fnbar(mutself,v:u64)->Builder{self.bar=Some(v);returnself;}fnbaz(mutself,v:i64)->Builder{self.baz=Some(v);returnself;}fnbuild(self)->Target{returnTarget{foo:se...| Raphael
Disclaimer: as with every performance discussion, nothing beats measuring. Disclaimer 2: Metaphors are great at hiding important distinctions between two situations through massive context flattening. Having said that, metaphor time. You have just graduated from the Very Fancy School With An Important Alumni Network. As a graduation gift, you are given two school alumni contact books: One is a list of alumni, sorted by the year of graduation, then sorted by their name. Another is a list of gr...| Raphael
The following is something I know in my heart but always take a minute to reconstruct when it comes up, so I am posting it here for posterity. Typescript lets you write type C = A | B. This often feels more ergonomic than when many languages force you to tag the union. C = Either A B, then doing left a or right b, all followed by needing to unwrap the values at the usage site... it feels like busy work! Typescript's untagged unions often lead to cleaner user-side APIs that don't require boile...| Raphael
The other day this snippet of Typescript generated by Github copilot was floating around: typeMappedParameterTypes<TextendsParameter[]>={// Check if the parameter has an 'enum' defined[PinT[number]asP["name"]]:Pextends{enum:Array<inferE>}?Eextendsstring// Ensure the enum values are strings?P["required"]extendsfalse// Check if the parameter is optional?E|undefined// If so, include 'undefined' in the type:E// Othewise use the enum type directly:never// This case should not occur since 'enum' im...| Raphael
Property testing is a very helpful form of testing. At its core, it involves doing the following: Generate a "random" value Test whether some property for that value holds I the property holds, "shrink" the value, to try and simplify it, and see if the test still fails So in practice, you can have some idea that your system validates something like the following: forall(list:List),reversed(reversed(list))==list Your property testing framework can test this by generating a bunch of "random" va...| Raphael
I have been trying to use nushell as my daily driver, and it's pretty cool! It has a terseness that makes people come to POSIX shells, but offers a nicer syntax, just the right amount of type safety for a shell language, and ultimiately feels worth it. But I found a weird perfomance issue while exploring some of the features. When looking at open files, I wanted to get a structured table from the string (normally doable with from ssv -m 1): let $files = lsof | from ssv -m 1 This command would...| Raphael
Feature flags are a way for us to enable or disable code paths without needing to re-deploy software. In a world without feature flags, you might develop a new feature on a branch, and then when you merge it into main, the next time you deploy the feature goes out. This feature goes out on the release (so you want to time your deploy based on some marketing strategy), and on top of this the release goes out and immediately causes problems. You have to rollback (also rolling back some bugfixes...| Raphael
Node packages love to use carot versioning. read-pkg-up@"^7.0.1 means that this package requires at least7.0.1 but less than versions 8 or above. This is great in a sense because it means we can get bugfixes for the same major version, without requiring us to chase down each package maintainer to update the requirements file. But let's look at a certain dependency chain: semver@5.7.1node_modules/normalize-package-data/node_modules/semversemver@"2 || 3 || 4 || 5"fromnormalize-package-data@2.5....| Raphael
When first setting up some containers on a new project, you might be using 2 or 3 tools, and decide to just take some "official" image from Docker Hub as a baseline for a container. Official Docker images are pretty great when you have very little you want to configure. They're often more than sufficient for prototyping when it comes to things that are otherwise fidget-y to set up, like a database. But a common pattern emerges when using these tools: You follow along with the quickstart, and ...| Raphael
tl;dr don't turn on all the checks for a subset of files, check as many files as possible and add checks on a module-by-module basis This post is a bit of a response to some online complaining by Armin Ronacher about mypy. While I agree in the abstract that mypy-s payoff feels less nice than Typescript's, I spent 30 minutes or so looking at Sentry's codebase and felt that Armin's problems are not "just" mypy problems, but problems compounded by how Sentry is choosing to adopt mypy. Adding typ...| Raphael
Quines are programs that, when you run, will print out its own source text. You can read this very long article about how to write one in C. Here we're going to do it in Python and skip a lot of the details and theory. You can, of course, "just" read the data from disk: importsysprint(open(sys.argv[0],'r').read()) This is obviously cheating though. It only works if we call it the right way, and is dependent on the command line to work. Really we want something pure that can be done in most pr...| Raphael
A long time ago Javascript introduced strict mode. This gave everyone an opportunity to fix some embarassing stuff in how Javascript worked, up to outright removing some functionality that was bad. Things have of course improved a lot, especially with the advent of various transpiling efforts, more serious standardization workflows, and just a lot more eyes on Javascript as a whole by many more serious people. But we still have some issues that are hard to fix because of backwards compatibili...| Raphael
What I'm going to describe is just derived from how traits themselves work, but despite having now written a decent amount of Rust, I had not really fully absorbed this fact. Imagine you are working with some cell towers owned by various people, in various locations: structCellTower{id:String;location:Location;owner_id:String;} You might want to do various things with these so you can add an impl block with some helper functions to help make your code legible. implCellTower{fndistance_from(&s...| Raphael
There are a lot of simple ideas that float about in programming. Many that are conceptually simple, yet end up being a bit tricky for you to implement. Right until the moment you see how somebody else does it, and you have a more concrete feeling for the simplicity. What follows is an advanced common sense idea that should help you with printing inside loops. In some batch script you might have: forwordinwords:process(word) This works alright, except you don't have visibility into progress or...| Raphael
What does sudo do? Most people have a basic understanding: it lets you temporarily run a command as root (or some other user, based on the options passed in). People will discuss sudo as nice from a security perspective relative to logging in as root, because you are scoping your usage of the root account to what really needs it. And with the default sudo setup you aren't handing out the root password, but instead giving permission to certain users to use sudo for privilege escalation. One mi...| Raphael
While working with the same people in the same problem domain for a long time, it's extremely common for people to take shortcuts when describing usage of a technical system. "When the user submits the form, it fails" "When the process runs out of memory and dies, it will automatically restart" "Oh yeah the app is slow right now" "This API is harder for a user to use correctly" There's a general idea of what these statements mean, and they communicate the right idea most of the time (otherwis...| Raphael
(Update: my conjceture has been disproven, at least with current word lists.) Happy New Years everyone! I hope everyone can enter 2022 with some optimism. Everyone is playing Wordle. It is very cool and fun. It is basically Mastermind, the word guessing game, but with a cool interface and a software keyboard that makes it very addicting. On top of this lots of people are now playing Absurdle and Evil Wordle. In these adversarial variants, the game doesn't pick a word immediately. Instead, eve...| Raphael
When you're introduced to Rust's data types you will often be introduced to usage via match: fndivide(numerator:f64,denominator:f64)->Option<f64>{ifdenominator==0.0{None}else{Some(numerator/denominator)}}// The return value of the function is an optionletresult=divide(2.0,3.0);// Pattern match to retrieve the valuematchresult{// The division was validSome(x)=>println!("Result: {}",x),// The division was invalidNone=>println!("Cannot divide by 0"),} In practice, though, when writing more proce...| Raphael
Here is a mental transcript of a bunch of things I tried to do this morning when trying to figure out how to listen in on a socket and get information from a client. I have never done this in a language with manual memory allocation, and that small detail expanded into many issues. On the server-side, we have a TcpListner, giving us a TcpStream when calling accept(). Remember the canonical node.js "echo server" example? The stream should allow me to both read and write data. Since this is Fan...| Raphael
Whenever I started learning about parsing and writing interpreters, I would get to how to handle comments. The gist of it was simple: just throw them out! You find them in the raw text output as you are generating lexing tokens, and discard them, meaning that later phases won't even see them, let alone have to handle them in any real sense. And this is something that I end up seeing over and over. Just throw out the comments, you don't need them anyways, and almost by definition they can't af...| Raphael
Let's say you have a web app with some database. In your database you have an Invoice model, where you store things you are charging your customers. Your billing system is flexible, so you support setting up tax inclusive or exclusive invoices, as well as tax free invoices! In this model you store it as: #!pythonclassInvoice:...is_taxable:boolis_tax_inclusive:bool You use this for a couple of years, write a bunch of instances to your database. The one day you wake up and decide that you no lo...| Raphael
Developping for the web is fun, Webpack is a marvel of modern enginerering, and when you have a project fully set up with the latest and greatest tools, things work amazingly. But when you have a small, throwaway web projet, Webpack and all that stuff starts to feel a bit like overkill. I'm not that interested in tracking my dependencies explicitly, writing up a package.json, trying ot remember if I want css-loader or style-loader or both for the 153rd time. Seeing a bunch of npm audit result...| Raphael
Rust has an amazing dbg macro that lets you quickly set up an expression printer that will also put in the source line. It also returns the value of the expression so you can even easily inline the printing when you want to! #!rustleta=2;letb=dbg!(a*2)+1;//^--prints:[src/main.rs:2]a*2=4assert_eq!(b,5); Doing a bunch of Python, I want this in Python as well. I want a debug macro, one that gives me the file location + the expression used to calculate the value. My ideal would be to have a funct...| Raphael
Last time I spent some time digging into Pylint, I was able to identify some memory issues, but hadn't really changed the runtime of Pylint. This time around I'm going to see if I can identify something that I can make actually run faster, instead of just using less resources. My Personal Profiling Principles Function f is taking some time. You have the following options: find a way to make all calls to it faster (or at least the same) find a way to make some calls to it faster, while making ...| Raphael
Through work I have to deal with a pretty huge codebase, and run Pylint on it during our continuous integration process. It's always been a bit slow, but recently I also noticed that it was consuming a lot of memory, causing OOM failures if we tried to run it too much in parallel. I decided to roll up my sleeves and figure out: What exactly was consuming so much memory? Is there a way to avoid doing this? This is a log of how I went about these two questions, as future reference for when I do...| Raphael
I have started playing around with FPGAs, by trying to build a (very simple, 2D) video card. For those who don't know, FPGAs are basically programmable chips letting you build digital circuits without having to go get a bunch of silicon pressed on evey iteration. This lets you do stuff that would be near impossible to do with a general CPU in software, like real-time signals (making it perfect for things like video signals, or trying to make some audio format decoder). To give an idea of the ...| Raphael
Every couple of months I like to take part in a game jam. It's fun to try and build a thing, and also feel totally fine with never looking at it again afterwards. Usually these happen online, over the course of 72 hours. Probably one of the more famous ones is Ludum Dare. I've done most of these alone, though I have in the past also done jams with one other person. In the beginning of November I decided to participate in a 48 hour game jam. This time was a bit different for a couple of reason...| Raphael
Remember files? A lot of awesome stuff existed thanks to files being the atom of computation, but it's all somewhat disappeared of late So much work on desktops resolves around files. Your desktop has a bunch of files, and you use programs to work on them. So you have your spreadsheets, images, and other goodies. And you have your tools to operate on them. so you work off your files. They're pretty portable on account of being files, and other people can write tools to work on them. So people...| Raphael
Sometimes I'll find a new website where the content is so entrancing that I want to read through all the archives. But it's pretty time consuming to sit and read everything at once, and I'll always lose track of where I am. Luckily I have a Kindle, a computer, and Python. So I've started throwing together simple e-books with this content. I get the nice auto-bookmarking and screen of the Kindle, making for a very nice reading experience. Here's the quick step by step guide to packaging up som...| Raphael
ORMs have been a huge time saver for society. They line up pretty well with the "enterprise"-y software most of us write. They eliminate a good class of potential errors involved in marshalling data through SQL. The amount of things only expressible in SQL has gone way down over the years. There is a bit of a danger in how ORMs get used. The classy nature of ORMs make us reach for seductively simple OOP tools. But these tools cut us sharply, especially when working in a language without much ...| Raphael
Let's say you're building a standard web application, where you have Clients, and Documents for those clients. You want to build a page that lets you browse documents, based off of clients. Something like the following: [ClientA|V]<-ClientSelector----------------------DocumentA<---ListofClientA's documents-DocumentB You decide to load in the list of documents only after a client is selected, and to do so through an AJAX request. Your code could end up looking like this (if you're working in A...| Raphael
Property testing is a very helpful form of testing. At its core, it involves doing the following:| rtpg.co
What I'm going to describe is just derived from how traits themselves work, but despite having now written a decent amount of Rust, I had not really fully absorbed this fact.| rtpg.co