Like many users of Prometheus, at work we use labels with AlertManager's routing to make sure the right people get paged when something goes wrong. Prometheus allows you to provide their labels via a number of sources (if I missed any, please reach out and let me know!):| hoelz.ro
The fact that tj-actions/changed-files was compromised is kind of old news at this point, but I had an insight the other day about this problem and its remediation that I haven't seen mentioned yet.| hoelz.ro
I was working on a PromQL query at work to find disk usage on Kubernetes control plane nodes - the first step was to identify those nodes:| hoelz.ro
I use restic to manage backups on my machines, and not too long ago I ran into some surprising behavior that I thought I'd share.| hoelz.ro
I've been working through Mastering Distributed Tracing to learn more about (you guessed it) tracing. The book is from 2019, and while I've been assured that its contents are still sufficiently up-to-date, the Go-based code examples are a bit dated; namely, they use dep for dependency management, rather than the new(er) built-in module system. dep seems like a fine tool that had its time, but seeing as I'm used to Go modules, and I don't really want to learn an obsolete tool just for some boo...| hoelz.ro
At work, we have a Prometheus metric from our build system named autobuild_build_timestamp that exports the timestamp for the latest build for a given application and Git branch. The process that exports this is stateless, so if it restarts or a new version of it is deployed, we're out of luck if we want to do something like build a dashboard to show the latest builds. One thing we can do is use a range query, but we can also make use of Prometheus' recording rules to provide a nice abstracti...| hoelz.ro
I recently ran into a surprising behavior when calling a Lambda function from the command line that I thought I'd share.| hoelz.ro
Recently, I wanted to find a document in my personal wiki that mentioned the idea of "there will always be another coding task" - unfortunately, traditional search came up short here, because I couldn't remember the exact wording I used, only the "vibe" of this particular phrase. However, I've been looking for a reason to use a large language model (aka LLM) with my wiki, and while there are other techniques that could accomplish the same goal, this seemed like as good an excuse as any to try...| hoelz.ro
The trick I covered in my previous post got me reflecting on "clever" code. Most developers I know (myself included) have learned to mistrust clever code - we see it and the intuition we've developed provokes a gut reaction of "oh no, how can this blow up in my face?"| hoelz.ro
While digging in and doing research for my previous post, I came across a really cool trick that glibc uses to provide a single macro for invoking system calls, no matter how many arguments they take. I thought I'd share the trick here, along with an explanation on how it works for those unfamiliar with the C preprocessor.| hoelz.ro
A few months ago I was using bpftrace to implement a program to monitor writes to a specific directory, and while working on this program I encountered some noteworthy behavior around bpftrace and its filters that I thought I should share.| hoelz.ro
For this post, I wanted to talk about a trick I came up a few years ago with to apply Git's pickaxe option in a situation where the change I was interested in wasn't contained in a single line. I'm hoping that readers of this post take away not only the trick itself, but also a perspective on how we can leverage Git's flexibility to create even more powerful tools for searching history!| hoelz.ro
I use systemd and its timers to run various tasks on my machines on a regular basis, and since that's all happening in the background, I want to know if those jobs fail. Now, it's better to have a job-specific alert rule that checks that the state of the world is desirable (for example - instead of checking "did the e-mail backup job succeed?", check "is the most recent file in the e-mail backup destination less than 24 hours old?"), but it's nice to have the general "did the service succeed?...| hoelz.ro
A lot of the software I use embeds a scripting language for easy extension - this is great, but much of it doesn't provide facilties for "hot reloading"; that is, iterating on an extension without needing to restart the application and blow away its current state.| hoelz.ro
A few months ago, I was pairing with my coworker Graham on a Grafana dashboard, and we ran into a pitfall I thought it'd be nice to share. We were trying to create a dashboard that would show how many changes happened to a database table between points on the graph, and we were getting some mysterious results!| hoelz.ro
Last year I bought myself a ThinkPad X1 Yoga to use as my personal laptop - I've been pretty happy with it so far, but there's one problem I've run into that I've noticed a lot more lately - the laptop will sometimes wake up while the lid is closed, go back to sleep, and then repeat that whole process every few minutes. This can't be good for the hardware, so I dug into this issue and wanted to share my findings with anyone who's having the same problem running Linux on this laptop!| hoelz.ro
