Here are all the posts that have appeared on quanttype.| quanttype.net
A talk given at Helsinki Python Meetup.| quanttype.net
Here's how Gorilla and the later floating-point compression algorithms related to each other.| quanttype.net
In my last post, I wrote about my plans for this year: This time I’m trying to be more intentional, especially career-wise. I concluded that it is time for me to move on from my position as a software engineer at Oura Health. I had a great two and a half years there, building and operating the backend-slash-database that is the main data store of the company, and I learned a lot about Python, Rust, AWS services, cloud databases, and scaling backend systems. However, I want to focus even mor...| quanttype
It’s 2025 now. This century is now 25% complete! As is my tradition, I’m going to reflect on the year gone by. On the photos: there’s one photo for each month, in chronological order. Software engineering At work, I continued to build cloud backends to manage data. I ended up spending a lot of time with Rust, OpenTelemetry, Docker, GitHub Actions (GHA). Out of these technologies, learning Rust was great and trying to use OpenTelemetry only led to frustration. Docker and GHA got the job ...| quanttype
This week I wrote a bunch of architectural decision records (ADRs) for a project I’m working on. ADRs were popularized by Michael Nygard’s blog post Documenting architecture decisions. Nygard does a great job explaining why they will be useful in the future. They will give useful context about decisions for the people who weren’t around when the decisions were made, and also for people who were there but already forgot about it. This in turns allows being intentional about accepting or ...| quanttype
When do you add comments to your code? Do you do it all? Everyone has seen one of these comments: counter+=1# increment counter by 1 This comment is explains what is already obvious from the code, so it’s not useful. What’s worse is when somebody then changes the code to be like this: counter+=2# increment counter by 1 Now the comment does not match the code anymore. People bring up examples like this when they disparage comments. However, there are other kinds of code comments that are m...| quanttype
In the past few weeks, I’ve spent a lot of time building Docker images on GitHub Actions. Anyone who has done it knows what it is like: I’ve been waiting a lot. To avoid waiting, you’ll want to set up build caching. Here’s a few things I learned about it. Building Docker Compose targets For one of our services, we have a Docker Compose based setup for running it locally. To ensure that it continues to work, I created a CI workflow with GitHub Actions that builds the images, starts the...| quanttype
This week I reviewed my notes about code review and noticed there’s something I haven’t written about: code review often happens too late. Typically a developer requests code review for their pull request once it’s pretty much done. This is too late. If you have developed a big change and the reviewer thinks the whole approach is wrong, it’s wasteful to throw the whole change away at this point, and it’s not fun feedback to get. This puts a limit on how big changes the reviewer can ...| quanttype
Every time ThoughtWorks publishes their Technology Radar, I take a look. It’s a nice source for finding out about new technologies and I think the Hold/Assess/Trial/Adopt system makes sense even if I often disagree with the classification choices. At work, I’ve been building some internal tooling for working with timeseries data and I’ve had a chance to try some new things. I decided to make my own mini-radar based on these experiences. Adopt These are great, you should use them. Polars...| quanttype
Apparently it was Fred Brooks in The Mythical Man Month who wrote that, when building a new software system, you should “plan to throw one away”. It sounds like great advice, but I haven’t seen anyone follow it. I recently built a prototype of an internal tool at work and I thought this time I will throw it away. The tecnology I chose was Python. Normally when I write Python, I insist on using mypy the type checker. This time I didn’t use it, and I didn’t write any tests either. It...| quanttype
This Tuesday was the day of the first Systems from HEL meetup. It’s a meetup about systems programming – there’s a bunch of systems meetups in the US and now there’s one in Helsinki, too. Pekka Enberg, the founder of the meetup, was also the first person to give a presentation. He talked about deterministic simulation testing (DST). He gave an overview of the technique and demoed his prototype implementation penberg/hiisi. Pekka’s company Turso has recently started using DST in ange...| quanttype
Last time I thought I had a trick in my pocket to ensure that I get some writing done: The solution I’m going to try is free writing, similar to morning pages: set a timer for ten minutes and just write whatever is on your mind. It did not quite work the way I hoped for. Free writing is great, but it’s just that my work last week was in the weeds and I cannot write about it in detail. --- Instead, I’m going to tell you about my Mastodon data exporter. Before Mastodon had search, I wante...| quanttype
I’ve now done ten weeks of weeknotes. Let’s talk about how it has gone. In the first note, I wrote this: I haven’t posted much. I’d like to change that. Instead of trying harder, I’d like to try solving it. A friend suggested posting weeknotes, so here goes. Certainly it worked in the sense that I’ve posted every week. In general I’ve seen three kinds of benefits from blogging: Writing clarifies my thoughts for myself. Sharing the writing with people I know sparks conversations....| quanttype
My week wasn’t too focused as I was slightly ill, so I’m going to offer a selection of small thoughts. --- My main concern at work has been how to get the tool I started working on a while ago to the users. The work has been slow due to my conferencetrips and other priorities, but I’ve gotten to the point where I must get feedback from the potential users. I’ve tried to figure out what is needed to make their work easier. However, I haven’t myself done the work they’re doing, so i...| quanttype
This week I attended EuroRust 2024 in Vienna, Austria. It was a two-day conference about the Rust programming language, organized by Mainmatter, a Rust consultancy1, and this was the second time it was organized. This was my first Rust conference. Unfortunately, it ended up being more of a miss than a hit for me, and it might be the last one in a while. It was on me, really. I go to conferences to meet people, not for the talks. However, I was not in the mood for getting to know new people an...| quanttype
It is the perf season at work. We’ve been reviewing our own and each other’s job performance. The reviews will feed into managers’ assessment of our performance which in turns affects whether we will get a raise or a promotion. I struggled a bit with writing the reviews. I realized I haven’t been giving as much feedback as I should have been and now I’ve got a backlog and the perf review is an awkward place to bring up new feedback. Praise is fine, but it’s not fun to get surprise...| quanttype
I wanted to track a memory leak in a Python program. The program was leaking only in production and so I had to figure out how to use Memray to attach to a process in Kubernetes. There were a few hurdles on the way, so here’s what I did. Add Memray to your container image. We use Poetry for our Python projects, so I added Memray as a dependency. poetry add memray If you’re using the official Docker Python images as a base, be sure to use the non-slim variant. The debug symbols have been s...| quanttype
Last week I attended Heart of Clojure. It was held the last time in 2019 and I had such great time that I decided to go again even though I don’t use Clojure these days. I knew what to expect, but nevertheless I ended up surprised. I gave a lightning talk where I told everyone to blog as it clarifies their thoughts and feelings. Well, here’s a plot twist: Heart of Clojure ended up being an emotionally complex event for me and I have hard time putting it into words, at least in public. The...| quanttype
I was thinking about debugging tools last week. The same theme continues this week. My big insight is that many development and debugging tools must work with broken or incomplete inputs. Here are a few examples: A debugger must work with programs that crash. In fact, that is exactly the time when many of us reach for a debugger. Your editor should work even if there are some syntax or type errors in your code. You expect features like syntax highlighting and autocompetion to work while you...| quanttype
I’m switching the gears a bit at work and instead of working on databases, I’ll be working on a new internal developer tool for making sense of some timeseries data. That is what has been on my mind this week. Essentially I’m making a debugging tool. What’s the bar for a successful debugging tool? At least it must be better than printf. There are endless articles where experienced engineers admit that they debug software by inserting printf calls instead of reaching for a debugger lik...| quanttype
This week I’ve been thinking about object storage services such as Amazon S3. Despite being called “object” storage, in my mind they are used for storing files. However, that’s not the only way to think about them. Another perspective is that they are key-value stores for storing binary blobs Nowadays a lot of people want to use them as a storage backend for their databases because they’re very durable and the data storage is cheap. We do, too, at work. Retrofitting object storage t...| quanttype
This week at work we were talking about improving the developer experience around the company. It is about how it feels to work with the technical systems as a software developer. Is it fun or frustrating to develop a new feature? What about debugging a production issue? If you were truly ambitious about it, you might ask how to make it more fun and more exciting and more meaningful. We were more pragmatic and asked how to make it less frustrating. Doesn’t lessening frustration make things ...| quanttype
Preface: I’m trying out weeknotes I believe writing regularly in public about your ideas is valuable. Writing helps you to clarify your thinking and sharing it lets you get feedback. You get something you can refer back to and link to. I’ve been writing regularly. However, as regular readers may have noticed, I haven’t posted much. I’d like to change that. Instead of trying harder, I’d like to try solving it. A friend suggested posting weeknotes, so here goes. Weeknotes are weekly u...| quanttype
A few months ago I wrote about how to build AWS Lambda deployment packages with Pex. It works, but it left me wondering: why isn’t there a one-command solution for building the packages for simple Python projects? I decided to build one. It’s called paketoi. It takes a requirements.txt file and your source files and bundles them into a zip file that you can deploy on AWS Lambda. I’ve released the initial version on PyPI. It’s a bit rough right now - it is version 0.1 after all - but I...| quanttype
There’s a lot of debate about software engineering methods and paradigms such as test-driven development (TDD), pair programming, agile methods, functional programming, and so on. Which one is the best? This is how the debate is often framed: it’s about finding the one true method that will deliver the best results and should be used always. Someone will write a post putting forward their favorite method; another person will counter that it won’t work in their specific situation, thus c...| quanttype
At work, we use GitHub to collaborate on code. We create short-lived feature branches and merge them back to the main branch via pull requests (PRs). This is a fairly standard workflow. Unfortunately I’m not too happy with it. I’ve had trouble finding a perfect way of working with git and GitHub’s PR view. I’d like to have the following: Useful git history. There are many opinions on what makes git history great. Myself, I look at the blame annotations regularly, so for me descriptive...| quanttype
Hello and happy new year. Like last year, I’m going to indulge in self-reflection and tell you about my year. On the photos: there’s one photo for each month, in chronological order. Software engineering I poured a lot of energy into technical work. I built a system for managing large data migrations and spent a lot of time on polishing and operating a large cloud backend written in Python. It was nice to spend a lot of time on actually implementing things. I learned a lot about new techn...| quanttype
I’ve been using Poetry for package management in Python projects for a while now and, for what it’s worth, it’s working well for me. However, some regular tasks require multiple commands with specific arguments. Here are a few recipes you might find handy. Updating the lock file after editing pyproject.toml After you edit pyproject.toml, you’ll want to update your lockfile and your virtualenv. Here are the right commands: poetry lock --no-update poetry install --sync Without --no-upda...| quanttype
I’ve been experimenting with the so-called branchless version-control workflows. The idea is that instead of using named branches, you just juggle a bunch of commits on top of the main branch. Here are a couple of tools implementing the idea - while Sapling and Jujutsu bill themselves as new VCSs, they both work with git repos: git-branchless, which builds on top of git Sapling (sl), a new VCS published by Meta Jujutsu (jj), a new VCS which has some Google backing git-branchless is the one ...| quanttype
Year 2023 is here. So, how was 2022? Allow me to review my year. On the photos: there’s one photo for each month, in chronological order. Professionally In January 2022 I wrote this: For many years, I’ve focused on building web services in Clojure. However, I feel that I’ve done enough of it for now and more interesting problems and bigger impact await elsewhere. Thus I’d like to turn a new page in my career. I didn’t quite manage to do it in 2021, but there were a few starts. The n...| quanttype
When you’re writing code with a modern editor or an IDE, you can count on having a number of convenient features such as syntax highlighting, autocomplete, and code navigation. Unless you’re writing SQL, of course! A lot of editors and IDEs, such as IntelliJ IDEA, have really nice support for SQL. However, it’s not so easy to benefit from it in practice. If you’re writing Clojure and you want to use SQL to query a database, you have a few options: Embed SQL into strings in your Clojur...| quanttype
Dear fellow Clojure enthusiasts, do you know what the following two code snippets evaluate to? And why is that the result? (identical? ##NaN##NaN)(let [x##NaN](identical? xx)) I didn’t know it a couple of days ago and it took me a while to understand it. Now I want to share my understanding. Go on, make a guess and check it with a REPL. When you’re ready – or if you already saw me post about it on Twitter – scroll past the photos below for an explanation. The photos in this post are f...| quanttype
The last time I wrote about how database schema migration tools could do more to help us. The way I see it, any schema migration solution has to cover three tasks: creating, managing, and executing migrations. Creating migrations. The first task is to create a migration script. If your scripts consist of DDL commands in SQL files, you’ll probably write them by hand. A linter like Squawk can help you to ensure the migrations do not cause unnecessary downtime. Things get more interesting if y...| quanttype
If you’re developing an application that is backed by a SQL database, sooner or later you will need to do a schema migration. Maybe you’ll need to add a new table or a new column, create a new index, or change some constraint. Luckily there are plenty of tools to help you! Migratus for Clojure, Flyway for Java, and dbmate as a more language-independent option are examples of the most common design pattern: you write migrations as DDL commands in SQL files and the tool keeps track of which...| quanttype
Epistemic status: Anecdotes and opinions. I’ve used Clojure for over a decade now: I first learned it in 2012 and started using it professionally in 2013. I’ve been reflecting on what has happened and what the future looks like. In this post, I want to share a few musings about it. Missed opportunity: production REPLs REPL is so central to Clojure that people even talk about REPL-driven development. It’s not just about having a prompt for typing commands like in Python1. Instead, it’s...| quanttype
2021 is over. It was the second full year of COVID-19, possibly establishing the new normal. I felt pretty good about 2020. This year was more of mixed bag. I missed my friends a lot and felt stuck professionally, but I had a plenty of good time, too. On the photos: there’s one photo for each month, in chronological order. Professional life For many years, I’ve focused on building web services in Clojure. However, I feel that I’ve done enough of it for now and more interesting problems ...| quanttype
On Dhole Moments, there’s a nice post about a recent Lobste.rs password reset vulnerability. Via the post, I learned about a simple technique called split tokens for making your password reset token validation more resistant to timing attacks. I wanted to poke at it a bit and ended up creating a tiny Clojure library for generating and validating split tokens, called split-token. Check it out if you’re into generating random tokens!| quanttype
I brewed some coffee while on the go.Last year, Finland closed down the week I had my winter vacation. This year, the government was debating movement restrictions. Since COVID-19 broke out in Finland, I’ve thought so many times that “surely this will be over by date X” just to see the date X to come and go. I’m not going to speculate about the unprecedented restrictions we’re going to see on my winter vacation the next year. I spent a night at Liesjärvi. My mom said that it must h...| quanttype
Clojure’s standard library includes the namespace clojure.xml, which implements a XML parser. It’s not used much – which is great, because it’s vulnerable to XML external entity (XXE) attacks. It’s something that you want to be aware of if you’re using clojure.xml to process untrusted input. Update (2022-03-27): XXE processing has been disabled in Clojure 1.11.0. Juha Jokimäki tweeted about this already back in 2014. However, I still see clojure.xml occassionally used, so I thoug...| quanttype
If you’re going to use clojure.spec to validate or conform untrusted input, you should be careful. It’s easy to write code that looks correct, but opens the door for denial-of-service (DoS) attacks. For example, if you have implemented a HTTP API in Clojure and you use spec to check the incoming requests, you should be aware of this. I believe that this is well-known among the experienced practitioners. For example, Dominic Monroe recently mentioned the issue in the defn podcast recently ...| quanttype
Recently I read Sönke Ahrens’s book How to Take Smart Notes. It is about notetaking using the Zettelkasten method. The gist of the method is that you note each idea on a separate card. Then you organize the cards into a hierarchy and interlink them using a clever numbering system. This interlinking allows you to generate new ideas. Nowadays you can use a computer program to do the same, of course. The book is thin on how the method actually works and focuses on why you should use it. I’v...| quanttype
As software engineers, how do we evaluate new technologies, programming languages, and practices such as code review? We must keep our goal in mind. Our goal is to deliver working software. We need to achieve this goal with limited resources: we have only so much time, manpower, and computing capacity available. The goal is not to perfectly follow a proceduce described by book. The goal is not to craft the perfect masterpiece of code. The goal is not to make you feel smart, either. Forgetting...| quanttype
Mustikkamaa is one my favorite spots in Helsinki. It’s a small island that is connected to Helsinki by Isoisänsilta, the white bridge that is in many of my blog pictures. There’s a nice walking path along the shore that I like to follow. This winter has presented us with a rare treat: the sea has frozen. The ice is covered by snow, with a couple of buoys and sea marks sticking up here and there. Only at the shoreline and under the bridges there are a few brown spots of ice. The snow is m...| quanttype
We software developers write a lot of code, but we write text, too. We write issues, commit messages, specifications, references, docs, and architectural decision records. We write chat messages and e-mails. Some of us even blog! That’s a lot of writing and it would be great if the writing was good. When working remotely, it matters even more, since you can’t solve everything by talking. This is a bit of a blind spot for some: a person can be a great communicator in person, yet they may n...| quanttype
For a long time, Leiningen was the Clojure project automation tool almost everyone used. Clojure itself did not come with a tool for managing dependencies – you had to use Leiningen, Maven, or Boot, or download the dependencies yourself and construct the classpath by hand. This changed when Clojure 1.9 was released in 2017. It included the new clojure command-line tool that supported the deps.edn file for declaring dependencies. Many people started to use the new tool instead of Leiningen. ...| quanttype
We didn’t have fireworks, but at least we had a jätkänkynttilä for New Year’s Eve.2020 was definitely a year. As is my habit, I want to reflect a bit on how it was. I was lucky and privileged in that my year wasn’t that bad. It was easy for me to start working remotely, the business was good, and me, my close friends and my family did not have big health problems, due to COVID-19 or otherwise. Many people had a way worse year than I did. Still, it would have been nice to meet people ...| quanttype
In February, I got a new laptop for home use and installed NixOS on it. This was the first time I have ever used NixOS. In this post, I’ll share some thoughts about how it has worked for me. The laptop is a second-hand ThinkPad X250. Installing NixOS was pretty straightforward and I haven’t had any trouble the hardware compatibility. Good. Configuring your system with a centralized configuration file is a pleasure. It offers a unified way to configure everything and a short config goes a ...| quanttype
I’ve posted about code review a number of times, about how it requires trust and needs to be fast. But, in the end of the day, code review is just one tool in the toolbox of collaborative software development. It should be evaluated in the context in which it is used. For example, when you’re starting a new project from scratch, usually there’s a lot scaffolding to set up. This is an important time for collaboration as you’re laying the foundation for building new things. However, mos...| quanttype
I’ve used zsh for at least 15 years. Occasionally I’ve toyed with the idea of switching to some of the newer shells such as fish (“Finally, a command line shell for the 90s”) or nushell. I’m stuck in my ways, though, so despite all the cool features and ease of use promised by the other shells, I’ve kept using zsh. I decided to at least freshen up my shell prompt. In 2012, I added git branch information to my prompt using zsh’s vcs_info. However, I never got around to configurin...| quanttype
A couple of months ago, I was feeling that I should think more about things. It was not that I was too busy to think – the problem was that my thinking was scatter-brained. Since then, I’ve discovered a practice that helps. It’s called morning pages and the idea is this: every morning, you sit down, grab a pen, and write three pages. Just write about whatever pops in your mind. Basically it’s a journaling pratice. The idea comes from the book The Artist’s Way by Julia Cameron. I hav...| quanttype
A lot of people see code review as a gatekeeping step in the software development process. They think that the reviewer is there to prevent bugs from getting into production. This is not a good way to think about it. Code review is an opportunity for collaboration. The task of the reviewer is to work together with the author to produce great software. Identifying flaws is a part of that, as is finding ways to address them. If the goal was to prevent shipping bugs, you could just block every c...| quanttype
The Clojure backends that I’ve worked on have often had a number of subcomponents: a HTTP server such as Jetty, a database connection pool, a scheduler, a message queue processor and so on. When you start the backend, you need to start all those components. You may have seen or written code like this to do so: (defn start[](let [connection-pool(connection-pool/create...)stop-scheduler(scheduler/create...)ring-handler(create-handlerconnection-pool)stop-server(jetty/startring-handler)](fn [](...| quanttype
Looking for something to listen? Here are two great, slightly sad albums. This year has not been like the other years. Maybe it’s the gravity of COVID-19, but I’ve found myself listening to sorrowful music than before. That’s the mood I associate with these two albums, one of them by a duo of folk musicians and the other by a jazz pianist. Mielo by Maria Kalaniemi and Eero Grundström is full of yearning to the wilderness. Kalaniemi plays an accordion and Grundström plays a harmonium a...| quanttype
Have you heard of Decentralized Identifiers (DIDs)? They’re a work-in-progress W3C recommendation that I’ve seen pop up in a couple places - mostly recently in the just-updated Thoughtworks Technology Radar. Since I’ve been interested in technologies related to identity and access management, I thought I should take a look. Why do you need identifiers? When you want to refer to a subject, for example a person, in a data processing system, you need an identifier for them. If you’re ope...| quanttype
Sometimes you want to cache the results of a function with side-effects. For example, you might cache the results of HTTP requests or database queries. If you’re using Clojure, you might reach for core.cache. The caches created by core.cache are supposed to be wrapped in an atom, so you would write something like this: (require'[clojure.core.cache:ascache])(defn http-get[url]...);; cache the results for a minute(def my-cache(atom(cache/ttl-cache-factory{}:ttl60000)))(defn fetch-data[url](->...| quanttype
Web applications have a couple of common use cases for random tokens. For example, e-mail confirmation or password reset e-mails usually have a link that contains a random token. The same goes for “share this item” links in case the item does not have a canonical URL. The token should be: Unpredictable. It would be bad if an attacker was able to guess the token for a password reset e-mail. URL-safe. You’re going to embed it in an URL. How to generate such tokens in Clojure? Random data ...| quanttype
Last week, I wrote about about creating a Clojure library for encoding and decoding Branca tokens. The library is finally ready. It’s called clj-branca and the version 0.1.1 is now out. Quick pitch Need to pass information from a service to another service, possibly going through an user’s browser? URL-safe authenticated encrypted tokens could be the solution you’re looking for. Check clj-branca out! (Please do not use it for stateless sessions, it’s a bad idea. Also, this is a side p...| quanttype
A while ago I wrote about the alternatives to JWT tokens. One of the them was Branca. It’s a base62-encoded, XChaCha20-Poly1305 encrypted token with an abritrary payload. Before writing the post, I thought out it would be nice to have a Branca library for Clojure and that’s something I’ve occassionally worked on since then. First I wrote base62 encoder and decoder and published them as a tiny library called clj-base62. That was easy enough. Then I needed to implement the encryption part...| quanttype
Dealing with difficult library upgrades has been a recurring task in my career as a software developer. This week I got a new tool into my toolbox for handling dependency conflicts. I was working on a Scala project. It heavily uses a library that is no longer maintained. We would like to migrate to a newer library. It would nice to migrate piece-by-piece: that would make both development and testing much easier. However, both the libaries depend on the different version of the same library. O...| quanttype
You probably know how to create a top-level memoized recursive function in Clojure, but how do you create a local one? By local, I mean defined with let or letfn. For the lack of better example, consider a Clojure function that returns the n-th Fibonacci number. Here’s a top-level definition created with defn: (defn fibo[n](if (< n2)n(+ (fibo(dec n))(fibo(- n2))))) It’s a classic example of recursive function. However, the implementation above is not exactly efficient. If you run (fibo 50...| quanttype
I’ve committed to blogging every week and I have even a Beeminder goal to enforce it. But really, there’s no blog post in me this week, so I’m just posting this to appease Beeminder. Sorry.| quanttype
This view in Pensar reminds me of certain famous Finnish paintings.Sometimes when people encounter a creative work that challenges them, they ask “is this art?” and struggle to answer. To find a satisfying answer, you need to go beyond that question. Epistemic status: Armchair philosophy. I’m not going to give an all-encompassing definition of art, but the word is used in at least two senses: Art as in artwork: Creative works made with the intention of expressing ideas and skill and aff...| quanttype
So are there any worthwhile alternatives to JWTs? JSON Web Tokens (JWT) are a popular way to create URL-safe access tokens for web applications. They are often used for stateless sessions1 and they’re part of OpenID Connect (OIDC) protocol. Note: JWT is just one part of the JavaScript Object Signing and Encryption (JOSE) framework. It would be more accurate to talk about JOSE but JWT has come to represent the whole suite, so I’ll go with that. Unfortunately, JWT is not a great design. It...| quanttype
What have you been wearing during COVID-19? For me, the strong isolation measures meant that I started working remotely. I wasn’t meeting people and this changed how I dressed. Instead of my normal office clothes - sweaters and chinos - I would dress in athleisure. Basically I looked all the time like I was on my way to a yoga session or a hike. Recently, however, I’ve had the chance to wear a suit. We’ve celebrated some big events in friends’ lives. Picking the right clothes was a bi...| quanttype
In visual arts, minimalism was a movement that emphasized non-figurative, non-emotive, abstract art. There’s no need to refer to the world – the shapes and the materials make the piece of art interesting in itself. In computing, minimalism refers to something else. Mostly it’s about reduction and parsimony. But I wonder: are there programs that would fit the visual arts definition of minimalism? The programs relate to the world by solving problems. The minimalist program should be afunc...| quanttype
When building a web service, logging out is often an afterthought. When you finally get to it, it turns out to be complicated. As a user, you have a couple of reasons to log out from a web service: You want to switch to another account. You want to prevent anyone else (or even yourself) from accessing your account using the same device. We now have multiple devices and the services can be used with browsers and native apps. Sometimes you want to log out of the other devices, for example if yo...| quanttype
In case you’re looking for some summer reading, here are a couple of books that I’ve read this year and want to recommend: This Is How You Lose the Time War by Amal El-Mohtar and Max Gladstone. It’s a story about two time agents who fight on the opposing sides but slowly they get to know each other. The story is told entirely through the letters they send to each other. The book is rich in language and references. I recommend it to sci-fi readers, but I think that even people who are no...| quanttype
If you try to deploy a new release of Clojure library with Leiningen, it prompts you to sign the .jar file with GPG. This step often causes confusion and breaks. I believe that it’s not worth the effort to make it work. As far as I know, nobody ever verifies the signatures in a systematic way. There are a bunch of obstacles: It’s unclear if any tools for verifying the signatures actually work. For example, I just tried to run lein deps :verify against a couple of projects and it reported ...| quanttype
Every now and then, I kayak. I joined a kayaking club for a couple of years ago, but I’m still a novice. I just haven’t been kayaking that much. This summer I’ve made a new effort to become more skilled and confident at paddling. I’ve been practicing the basic techniques - how to paddle efficiently and how to maneuver the kayak. Turning a kayak is easy enough when the water is flat, but when there’s wind and waves, it gets more involved. Just the other day I got in a situation where...| quanttype
I’m starting my summer vacation next week and this blog will also take a small break. My original plan was to hike on Kungsleden in Sweden, but COVID-19 messed that up. I’m not sure what I’m going to do – I guess I’ll settle for some hiking and paddling in Southern Finland. I’ll be back in July.| quanttype
A number of enterprises in Finland rely heavily on external contractors for software development even though the software is a core part of their business. Often the product owners, project managers, and possibly architects work for the company, but the designers, developers, ops, QA specialists, and data scientists work for consulting companies. Contractors are the obvious choice when you need extra staff quickly or when you need someone with skills that your organization does not have. For ...| quanttype
When talking about programming languages, we often talk about the cool features that we could use and the thriving ecosystems that we could leverage. But when we’re choosing programming language for a project, what really matters is the team that is going to use it. Learning a new programming language – if you want to use it in anger – takes time, even if you have a lot of experience. In addition to the language itself, you have learn about architecture, ecosystem, deployment, and opera...| quanttype
My employer, Metosin, is well-known for its Clojure open-source libraries. When people hear that I work for Metosin, they often ask if I contribute to the development of the libraries. I do, but not so much in the form of coding new features or bug fixes. My focus has been on maintainership tasks such as creating releases. At the moment, we do not have a well-defined process or schedule for releases. Personally, I believe in small releases. If a library has some unreleased work and it looks l...| quanttype
Töölönlahti is surprisingly good bird spot in central HelsinkiHow have you spent the copious free time given to you by the COVID-19 isolation? I have been watching the urban nature. Usually in May, I’m spending the weekends at the summer cottage and in Nuuksio and in the nature in general. This year travelling and even using public transport has been discouraged, so I’ve taken walks in the nearby parks. It’s funny, but I’ve probably paid more attention to the flora and the fauna th...| quanttype
Last week I took a look at three data specification libariers in Clojure: Schema, Spec, and Malli. This week, let’s talk about the essential features of these libraries. Data specification language. This is the foundation of every data specification library: a way to describe the data with a schema. You could use an existing language such as JSON Schema, but in practice that’s clunky and everybody develops their own language. ;; Let's model users. We want to know the user's name and the y...| quanttype
There are a number of data specification libraries for Clojure. The best-known ones are Schema and clojure.spec. Then there’s Malli, designed by my colleagues at Metosin. The libraries are used in two ways. They’re used for specifying the shape of data inside the program, for example the parameter and return types of a function. They’re also used for specifying the external interfaces of the program, for example what kind of JSON a REST API endpoint accepts. I’ll call these use cases ...| quanttype
Let’s say you’re leaving a software team and somebody else is joining the team to replace you. How do you efficiently transfer knowledge to the new person in short time? My answer is that you… don’t. You can bring on the new member in the usual way. Maybe somebody gives a presentation, maybe you’ll pair-code. The new person starts with some easy tasks that act as a good introduction. I hope you have a great README. The common knowledge should ideally be encoded in the shared artifac...| quanttype
The red buttons are the anchors for Peak Design’s camera straps.I got my Ricoh GR III in April 2019. Back then, I thought that it’s a pretty great pocketable camera. Now it’s time to take a second look. Background: Ricoh GR III is a fixed lens compact camera with a fast, 28 mm-equivalent lens. It’s a successor to GR II, which has cult following in the street photography circles. I still haven’t explored the camera that much and, honestly, that’s a good sign. I quickly settled with...| quanttype
When creating a new file for your programming project, how do you choose how to name it and where to put it? I recommend choosing a location that is expected by both humans and programs. Here are a couple of examples: README. There’s a decades-long convention of putting the basic overview documentation for a project in a file called README. This convention is now recognized by tools: for example, GitHub renders it in the repository front page. Docker Compose. If you use docker-compose to se...| quanttype
It takes some skill to work remotely. When migrating from office to remote work, you can’t just set up some communications tools and expect the work to work as if nothing happened. You and everybody else have to learn how to use the tools to collaborate. Now software development organizations are mass-migrating to remote work. Everybody is setting up Zoom or Teams that to enable video calls and learning to raise their hand to get their turn to speak. That’s a great start, but the real rem...| quanttype
A log in a bog.If you’re running mission-critical software, how do you minimize the risk of it breaking during these difficult times? Releases cause problems, so it can be tempting to require an approval of each release by a change advisory board. You could even declare a deployment freeze: no new versions of the software can be deployed unless absolutely necessary. Releases are inherently risky and a freeze does reduce this risk by making the releases rarer. On the other hand, it makes the...| quanttype
COVID-19 is out there and like many, I’m working from home. Allow me to offer my unique perspective. Not everyone can work from home - not all jobs can be done remotely and even if they can, there are many obstacles to working from home. For example, the people you share your home with - partners, kids, roommates - have to give you enough physical and mental space so that you can work. And they have to have space for their work, too! Myself, I’m lucky and privileged enough to pull it off....| quanttype
When you write programs, you have two audiences: the humans who read the code and the computers that analyze and execute the code. The both audiences have to understand the code. Humans read programs like a hypertext1: not from start to end but by jumping around following references. For humans, the careful choice of terminology is of utmost importance. Consistency matters. The computers do not care. They care about the grammar, though. Elegant programs are easy for humans to understand and e...| quanttype
At work, we aim to have a self-managed organization instead of a hierarchical one.1 One of the challenges is that how decisions should be made. In a hierarchical organization you could ask your boss, but what if your boss does not want to make every decision? You could seek consensus, but what if you can’t reach a consensus and a decision nevertheless has to be made? A practical solution is to use the advice process. Anyone can make a decision after seeking advice from: Everyone who will be...| quanttype
When you’re out there, struggling to create, staring at the blank page, the blank canvas, or the blank MS Paint window, you might be thinking “I have nothing to say”, and you might be right.1 Yet that is not the problem, is it? The problem is how to have great idea. Having something say, a message to the world, can be a great source of creativity, but it’s not the only one. What else is out there? I challenge you to consider art. When you look at it, listen to it, experience it, what ...| quanttype
When you’re debugging a web service, it’s handy if you can get all the log entries associated with a single HTTP request. This is easy if you generate a unique ID for each request and include it in all the log entries. If your services calls other services, you can build a simple tracing system by including this ID in all those calls. If you’re using nginx as a reverse proxy, you can use it to generate the IDs.1 At work, we’re using the Tornado web framework in Python and we wanted to...| quanttype
Thanks to some organizational surprises, I’m now developing web services in Python. The last time I used Python in anger was in 2016. Has anything changed? Records. Python has records now, thanks to attrs and dataclasses. Namedtuples were too simple and and writing classes by hand for everything was too complicated. Attrs and dataclasses are just right. I didn’t get this in 2016 when attrs was new, but I get it now. Attrs has a nice overview of how it compares to the other solutions. Type...| quanttype
I’ve had three rolls of medium-format (120) film sitting in my fridge. They’ve been there long enough that they have all expired already. It’s black-and-white film, though, so it should be fine, especially since I’ve kept it refrigerated. I’ve shot with decade-old films and the results were okay. The Mamiya is still with me, so I decided to finally go and shoot the films. I headed to Mustikkamaa for my “usual” set: ice, rocks, and reeds. This winter has been unusually warm in He...| quanttype
New year, new shenanigans, as we say in Finland. A new decade! 1 I, on the other hand, am back to my bullshit. Please allow me to talk about me in the year 2019. It was a difficult year for me, but I don’t want to delve on that too much. Instead, let’s focus on the good stuff. Working I continued to work as a software developer. In autumn, I started to work for a new client. This is my first time working for a big (in the Finnish scale) tech organization. Finally I get to see in action al...| quanttype
I work as a software consultant. What we do is that we develop software for other companies. Sometimes we do it as a team of our own and sometimes embedded in an in-house development team. Sometimes the clients come to us for our special expertise and sometimes they just need butts in the seats churning out code. Our job is to implement standard solutions for standard problems. The clients are not in the high-tech business – or if they are, their in-house developers work on the innovative s...| quanttype
Fighting over syntax formatting in code review is, mostly, waste of time. Having consistent formatting is great for readability, but code review is not the right place to enforce it. To keep code reviews fast and smooth, you’ll want to focus on high-impact issues. With formatting, the returns diminish quickly. Instead, you should use computers to enforce consistent formatting. There are two ways to go about it: Use a code formatter tool. Try to find a fast one and configure everybody’s ed...| quanttype
If you want mandatory code review to really suck, make it slow. Power games aside, one of the most frustrating aspects of code review is how long it takes and how many context switches it involves. There are two main things that contribute to this: how long it takes for the reviewers to react to new and updated pull requests (PRs). how many rounds of review are needed In my experience in a 2-5 person team, if everybody is willing to dedicate some time for reviews every day, the wait for revie...| quanttype
Last week, Camille Fournier tweeted this: Questioning the value of mandatory code review is definitely the most popular underground belief held by senior engineers I know It launched a discussion about the merits of code review and a bunch of people came forward about their bad experiences. For them, code review had become an arena of power games and a place to demonstrate how smart you are. I’ve experienced code review as a highly valuable practice and advocated for it, but it’s easy to ...| quanttype
Here is how to implement a distributed lock with S3| quanttype.net
pex avoids certain shortcomings of pip and is faster than Docker| quanttype.net
There are a few situations where reviewing hinders you more than it helps.| quanttype.net
requirements.txt is not good enough for managing Python dependencies; use Poetry instead| quanttype.net
