I found it striking how incapable every AI model I tried was of drawing a spider silhouette missing one leg.| will-keleher.com
I set up a quick Aider-like workflow to do grammar edits to a fantasy novel I'm working on.| will-keleher.com
In JS/TS, I occasionally see people excited about writing "functional" code write code like the following: ```ts const schools_by_id = schools.reduce((acc, school) => { acc[school.school_id] = school; return acc; }, {} as Record); ``` Personally, I find this "functional" style hard to read! The first time I encounter `acc`, I'm unsure of what it even is and then it needs to be returned? It takes me a second to parse even a simple reduce like this one. I find a `for` loop much easier to read: ...| will-keleher.com
# Don't lecture when teaching software engineering Have you ever attended a learning session or brown-bag run by another engineer at your org and realized afterwards that you didn't actually learn anything during it? Or maybe you've given a presentation like that? After your session, you saw people making the same mistakes that they'd been making before the session. It's a bad feeling. Most engineers don't know how to put together a good learning session, but teaching is a learnable skill! An...| will-keleher.com
# I avoid easy things I haven't tried When I haven't done a thing, I avoid doing it because it _might_ be hard. There's a hidden barrier in my head that makes me take a long time to adopt new tools. And sometimes those new tools are super simple to use! This is embarrassing, but it took me about five years to try recording my screen using QuickTime Player on Mac and _it's so easy_. You just need to right click on the player and select "New Screen Recording" and then you'll be able to select w...| will-keleher.com
# Why I avoid test stubs in TypeScript > stub _/stʌb/_ (verb) > 1. To replace a function on an object or class with a test-only implementation. See [sinon.js](https://sinonjs.org/) for a well-written example. > 2. To leave an impediment to future engineers that they are likely to "stub" themselves on. Related topics: footgun, anti-pattern, spies, testing. > > The etymology of "stub" is disputed, but a common theory is that it was from "stubbing one's toe," and that the reaction of many engin...| will-keleher.com
# Most engineers have git aliases; what are yours? > Summary: Typing is hard. Here are my [git aliases](#just-show-me-the-code). I like typing on the command-line. Even after a decade of programming, it still makes me feel like a hacker. I'm not someone who needs a fancy GUI; I can type a few quick commands and bam!—my computer does something powerful. And it's fun to clicky-clack my way through a long CLI command. But `git` tests my limits. At first, it was fun to type out `git pull --reba...| will-keleher.com
## When helping elves, you should use complex numbers to navigate in 2D > I'm hugely grateful to [4HbQ](https://old.reddit.com/user/4HbQ) for sharing their [solutions and python tricks](https://old.reddit.com/user/4HbQ/?sort=top) in r/adventofcode! They're the inspiration for this post. The first time I saw someone use complex numbers of navigate in two dimensions, I was horrified. Why would someone _want_ to use `x + y * 1j` rather than `x` and `y` to represent a point in space? It seemed li...| will-keleher.com
# 18,957 tests in under 6 minutes: ClassDojo's approach to backend testing We're pretty proud of our backend test suite. We have a lot of tests, and developers can run the full test suite locally in under six minutes. These aren't simple unit tests—they're tests that hit multiple databases and routes with only a minimal amount of stubbing for external dependencies. 9 years ago, [we were proud of running 2,000 tests in 3 minutes](https://engineering.classdojo.com/blog/2015/01/20/continuous-t...| will-keleher.com
# Python patterns I learned during 2023's Advent of Code Last year, I used python for Advent of Code and loved it. It felt like there was always a beautiful built-in way of doing whatever it was I was trying to do. I wanted to count something? `Counter`? Compare it against something else? It just works. Create a set? Easy. Navigate in 2d space? Imaginary numbers to the rescue. I doubt any of this will be useful for someone who writes python day to day or who wants to use python to solve probl...| will-keleher.com
# Indexes are sorted arrays (sort of) Most indexes for databases like PostgreSQL, MySQL, and MongoDB are B-trees or B+-trees. These indexes support O(log(n)) look-ups because B-trees are a self-balancing binary search tree. Sorted arrays can partially represent a binary search tree, so thinking about indexes in terms of a sorted array can help you think about performance and index optimization. There's nothing magic about indexes, and they're easier to grok if you think about them in terms of...| will-keleher.com
# JSDoc comments can make fixtures easier to work with  Our tests used to be full of hundreds of random ids like `5233ac4c7220e9000000000d`, `642c65770f6aa00887d97974`, `53c450afac801fe5b8000019` that all had relationships to one another. `5233ac4c7220e900000...| will-keleher.com
# So, what can you do with a process ID? For the longest time, the only thing I knew how to do with the process ID (PID) for a running process was kill it. It turns out, if you have a PID, you can gather a ton of info about what that process is up to! I'm sure these commands will seem blindingly obvious to someone with more of sysadmin background. This is all coming from the perspective of someone who occasionally hops on to a machine and has a lot of trouble navigating around and exploring w...| will-keleher.com
# Why does `zip(*[iter(s)]*n)` chunk `s` into `n` chunks in Python? > This makes possible an idiom for clustering a data series into n-length groups using `zip(*[iter(s)]*n)`. > > — _[python.org zip docs](https://docs.python.org/3.3/library/functions.html#zip)_ During this year's [Advent of Code](https://adventofcode.com/), I've been learning python, and that snippet above blew my tiny little python mind. Let's take a look at everything that's going on here! ```py chunk = lambda s, n: zip(*...| will-keleher.com
# Culling Containers with a Leaky Bucket ClassDojo occasionally has a few containers get into bad states that they're not able to recover from. This normally happens when a connection for a database gets into a bad state—we've seen this with Redis, MySQL, MongoDB, and RabbitMQ connections. We do our best to fix these problems, but we also want to make it so that our containers have a chance of recovering on their own without manual intervention. We don't want to wake people up at night if w...| will-keleher.com
# Writing FizzBuzz in MySQL 5.7 for no good reason > Why are you in this doorless room? There's nothing but a keyboard -- black, unmarked keycaps, promising auditory violence -- and a monitor showing a terminal connection to a MySQL database. You type `show engine innodb status;` (each keystroke sharp & piercing) and a challenge scrolls into view: "Write FizzBuzz from 1 to 1000." You smile. Writing FizzBuzz in MySQL 5.7 is totally useless, but it was a fun excuse to explore some MySQL feature...| will-keleher.com
# Using NodeJS's AsyncLocalStorage to Instrument a Webserver In the dark times before [`AsyncLocalStorage`](https://nodejs.org/api/async_context.html#class-asynclocalstorage), it could be hard to tell why a request to a server would occasionally time out. Were there multiple relatively slow queries somewhere in that route's code? Was another request on the same container saturating a database connection pool? Did another request block the event loop? It was possible to use tracing, profiling,...| will-keleher.com
# Large Analytics SQL Queries are a Code Smell A single large query in SQL can be hard to understand, test, debug, or change in the same way that an over-large function in code can be. A large query is also much harder to write! Feedback loops while writing large queries are slow and you'll often find yourself needing to guess at where the problem in your query is. When I started writing analytics queries, I wrote some pretty rough ones that are now hard to debug and maintain! Apologies to ev...| will-keleher.com
# Shell patterns for easy automated code migrations Automated and semi-automated code migrations using shell text manipulation tools are great! Turning a migration task that might take multiple days or weeks of engineering effort into one that you can accomplish in a few minutes can be a huge win. I'm not remotely an expert at these migrations, but I thought it'd still be useful to write up the patterns that I use consistently. ### Use `ag`, `rg`, or `git grep` to list files Before anything e...| will-keleher.com
# Red and Blue Function Mistakes in JavaScript Bob Nystrom's [What Color is Your Function](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/) does an amazing job of describing why it can be painful when programming languages have different rules for calling synchronous and asynchronous functions. Promises and async/await have simplified things in JavaScript, but it's still a language with "red" (async) and "blue" (sync) functions, and I consistently see a few understa...| will-keleher.com
# Slack is the Worst Info-Radiator When the ClassDojo engineering team was in the office, we loved our information radiators: we had multiple huge monitors showing broken jenkins builds, alerts, and important performance statistics. They worked amazingly well for helping us keep our CI/CD pipelines fast & unblocked, helped us keep the site up & fast, and helped us build an engineering culture that prioritized the things we showed on the info radiators. They worked well while the whole team wa...| will-keleher.com
# Bash Patterns I Use Weekly ### 1. Find and replace a pattern in a codebase with capture groups **`git grep -l pattern | xargs gsed -ri 's|pat(tern)|\1s are birds|g'`** - `git grep -l`: make sure we're only looking for files in our codebase (`ag -l` is another good option) - `xargs`: allow running this with `gsed -i` - `gsed -i`: edit files (default mac `sed` is bad, so gnu-sed is essential) - `gsed -r`: use regular expression for the pattern to allow capture groups - `s|`: the first charact...| will-keleher.com
# Clean SQL Tricks I'm not great at analytical SQL, but when I try to treat my SQL the way I treat code by breaking out intermediate values, prioritizing tight feedback loops, and using clear constant values, I find that I'm able to write better, more maintainable SQL, faster. I consistently use a few techniques to make this happen: 1. `create temporary table`: creating lots of small, focused tables has been the most important technique to improve my SQL queries. Smaller queries are much simp...| will-keleher.com
# Canary Containers at ClassDojo in Too Much Detail [Canary releases](https://martinfowler.com/bliki/CanaryRelease.html) are pretty great! ClassDojo uses them as part of our continuous delivery pipeline: having a subset of real users use & validate our app before continuing with deploys allows us to safely & automatically deploy many times a day. Our canary releases are conceptually simple: 1. we start canary containers with a new container image 2. we then route some production traffic to th...| will-keleher.com
# MongoDB Performance is Disk-graceful > These are all my views, not the views of my employer. A less inflammatory thesis might be "MongoDB requires different schema patterns & understanding of its storage engine to optimize performance"... but that's less fun MongoDB does not scale well. Yes, you can easily shard a database. Yes, average write speed can be fast. Yes, there are schema patterns you can use to get impressive read performance. But, fundamentally, I've found that MongoDB is much ...| will-keleher.com
# Node's `AsyncLocalStorage` Makes the Commons Legible When multiple teams share a resource like a code-base, a database, or an analytics pipeline, that resource can often suffer from the Tragedy of the Commons. Well intentioned engineers can add slow code, not prioritize important improvements because those improvements aren't aligned with team incentives, and just generally not leave the codebase or database better than they found it. ClassDojo has a monolithic backend web-server API, and i...| will-keleher.com
# The 3 things I didn't understand about TypeScript When the ClassDojo engineering team started using TypeScript on our backend, I was incredibly frustrated. It felt like I needed to constantly work around the type system, and it felt like the types weren't providing any value at all. And, they weren't providing any value to me: I had fundamental misconceptions about how TypeScript worked, and my attempts to work around the Type system by using lots of `any` and `as` created some truly abomin...| will-keleher.com
# Manipulating Text & Files solves problems A decent number of programming tasks boil down to manipulating text and files: - automating code migrations - exporting & transforming data - exploring logs - looking for patterns in a codebase Knowing the basics of shell tools that are designed to manipulate text & files can make these tasks simple, and the interactive pipe-based programming environment of the shell makes figuring out the right command fun. It makes sense that tons of engineers eva...| will-keleher.com
# HyperLogLog-orrhea: how do HyperLogsLogs work? Say you had a few petabytes of ids lying around somewhere, and someone told you that they were going to use a few kb of memory to estimate the "cardinality", or the number of distinct ids, of those petabytes of ids with decently high accuracy. It'd feel like magic! The HyperLogLog algorithm makes that magic cardinality estimation happen through the Power of Random Numbers, and some clever use of data structures. I wanted to set up a [mongo-back...| will-keleher.com
# ERROR, WARN, and INFO aren't actionable logging levels The standard syslog-based way to handle logs is to divide logs into categories like `FATAL`, `ERROR`, `WARN`, `INFO`, and `DEBUG` and use those categories to adjust which logs you see. Having a standard logging taxonomy is incredibly useful when you're dealing with logs for many different systems, but for backend web-servers, these syslog-based divisions aren't necessarily actionable or easily searchable; it's better to create categorie...| will-keleher.com
Graceful shutdowns behind a load balancer are hard to do right, but they enable solid auto-scaling and safe frequent deploys.| will-keleher.com
Automating things, even when it doesn't save time, is how you learn the skills to automate the things that will.| will-keleher.com