In software engineering organizations, there are certain practices that keep costs under control even if those seem more expensive at first. Unfortunately, because such practices feel more expensive, teams choose to keep their status quo even when they know it is suboptimal. This choice ends up hurting productivity and morale because planned work is continuously interrupted, which in turn drags project completion. The reason I say seem and not are is because the alternatives to these cost-exp...| Julio Merino (jmmv.dev)
During my 11 years at Google, I can confidently count the number of times I had to do a “clean build” with one hand: their build system is so robust that incremental builds always work. Phrases like “clean everything and try building from scratch” are unheard of. So… you can color me skeptical when someone says that incremental build problems are due to bugs in the build files and not due to a suboptimal build system. The answer lies in having a robust build system, and in this post...| Julio Merino (jmmv.dev)
“Strategies? Will you talk about Bazel’s strategy for world domination 🙀?” No… not exactly that. Dynamic execution has been quite a hot topic in my work over the last few months and I am getting ready to publish a series of posts on it soon. But before I do that, I need to first review Bazel’s execution strategies because they play a big role in understanding what dynamic execution is and how it’s implemented.| Julio Merino (jmmv.dev)
Bazel’s dynamic execution is a feature that makes your builds faster by using remote and local resources, transparently and at the same time. We launched this feature in Bazel 0.21 back in February 2019 along an introductory blog post and have been hard at work since then to improve it. The reason dynamic execution makes builds faster is two-fold: first, because we can hide hiccups in the connectivity to the remote build service; and, second, because we can take advantage of things like per...| Julio Merino (jmmv.dev)
macOS includes a sandboxing mechanism to closely control what processes can do on the system. Sandboxing can restrict file system accesses on a path level, control which host/port pairs can be reached over the network, limit which binaries can be executed, and much more. All applications installed via the App Store are subject to sandboxing. This sandboxing functionality is exposed via the sandbox-exec(1) command-line utility, which unfortunately has been listed as deprecated for at least the...| Julio Merino (jmmv.dev)
A key feature of Bazel is its ability to produce fast, reliable builds by caching the output of actions. This system, however, relies on a fundamental principle: build actions must be deterministic. For the most part, Bazel helps ensure that they are, but in the odd cases when they aren’t, builds can fail in subtle and frustrating ways, eroding trust in the build system. This article is the first in a series on Bazel’s execution model. Having explained these concepts many times, I want to...| Julio Merino (jmmv.dev)
About six months ago, during one of my long runs, I had a wild idea: what if I built an OS disk image that booted straight into EndBASIC, bundled it with a Raspberry Pi, a display, a custom 3D-printed case, and made a tiny, self-contained retro BASIC computer? Fast-forward to today and such an idea exists in the form of “the EndBOX prototype”! This article isn’t the product announcement though—that’s elsewhere. What I want to do here is look back at the Blog System/5 articles I’ve...| Julio Merino (jmmv.dev)
Back in 2017–2020, while I was on the Blaze team at Google, I took on a 20% project that turned into a bit of an obsession: sandboxfs. Born out of my work supporting iOS development, it was my attempt to solve a persistent pain point that frustrated both internal teams and external users alike: Bazel’s poor sandboxing performance on macOS. sandboxfs was a user-space file system designed to efficiently create virtual file hierarchies backed by real files—a faster alternative to the “sy...| Julio Merino (jmmv.dev)
How does Bazel avoid melting your workstation with concurrent subprocesses? Or… tries to, because I know it still does that sometimes? There are two mechanisms as play: the jobs number and the local resources tracker. Let’s dive into them. The jobs number, given by the --jobs flag, configures the number of concurrent Skyframe evaluators during the execution phase1. What a mouthful. What this essentially means is that jobs indicates the number of threads used to walk the graph looking for ...| Julio Merino (jmmv.dev)
If you know what GAFYD stands for and have felt that signing up for a free account years ago was a mistake, you know you are in trouble right now. Those accounts are shutting down. The time has come to either pay up or move out, and you should decide what to do ASAP. It took me weeks of active effort to move my and my family’s data out of GAFYD and put it back into Google consumer accounts and other non-Google services. This post is essentially a recollection of my lab notes on what I did. ...| Julio Merino (jmmv.dev)
2022-03-07: Introduction 2022-03-08: Keyboard shortcuts 2022-03-09: Input methods 2022-03-10: Look'n'feel 2022-03-11: Window switching 2022-03-12: PowerToys 2022-03-13: Miscellaneous tools 2022-03-14: Development experience 2022-03-15: PowerShell 2022-03-16: Networked file systems 2022-03-17: System debugging 2022-03-18: Software installation 2022-03-19: Finale A bit over a week ago, I narrated my decades-long love and hate relationship with Windows. Today, it’s time to start covering my im...| Julio Merino (jmmv.dev)
Are you a macOS user occasionally dealing with Windows systems or trying to switch platforms? Are you a Windows user that believes that the Windows-native keyboard shortcuts are objectively bad? Are you annoyed by something as simple as copy/pasting text not working consistently across apps? If so, this post will equip you with an AutoHotkey configuration file that brings macOS keyboard shortcuts to Windows. Read on.| Julio Merino (jmmv.dev)
Hello readers and sorry for the 2-month radio silence. I’ve been pretty busy at work, traveling during school breaks, hacking on EndBASIC when time permitted, and… as of two weeks ago… tinkering with 3D printing as a complete beginner. So, today, I’d like to walk you through the latter because it has been a really fun and rewarding journey, albeit frustrating at times. You’d think that to use a 3D printer, you’d design a 3D model and then… just… send it to the printer? That’...| Julio Merino (jmmv.dev)
Today marks the 10th anniversary of Bazel’s public announcement so this is the perfect moment to reflect on what the next generation of build systems in the Bazel ecosystem may look like. I write this with the inspiration that comes from attending the first ever conference on Buildbarn, one of the many remote execution systems for Bazel. In the conference, Ed Schouten, the creator of Buildbarn, presented Bonanza: a skunkworks reimagination of Bazel for truly large builds.| Julio Merino (jmmv.dev)
If you have followed our recent infrastructure posts, you know by now that we are actively migrating Snowflake’s build to Bazel. What we haven’t shared yet is that we have deployed our own Build Barn cluster to support Bazel’s remote execution features. We have chosen to run our own build farm service for resource governance and security purposes, but also because the behavior of this system impacts the developer experience so directly that we want to have full in-house control and know...| Julio Merino (jmmv.dev)
A few months ago, we described how we fixed three different OOM scenarios in our ongoing migration to the Bazel build system here at Snowflake. Things had been sailing along just fine since then… but a new issue showed up recently: our IntelliJ with Bazel (IjwB) Java project started showing OOMs during its sync phase. The reason this issue surfaced now is because, as we continue our migration to Bazel, our IjwB project has grown in size. Months ago, our project only covered a Java binary, b...| Julio Merino (jmmv.dev)
Here at Snowflake, the Developer Productivity organization (DPE for short) is tackling some important problems we face as a company: namely, lengthening build times and complex development environments. A key strategy we are pursuing to resolve these is the migration of key build processes from CMake and Maven to Bazel. We are still in the early stages of this migration and cannot yet share many details or a success story, but we can start explaining some of the issues we encounter as we work...| Julio Merino (jmmv.dev)
After two years, it’s time for a change: I left Microsoft last week and I’m starting at Snowflake today. Read on for details on my stint in Azure Storage, why I ended up looking for a new role, and how I landed at this new company.| Julio Merino (jmmv.dev)
Two and a half years ago, I joined Snowflake to help their mission of migrating to Bazel. I spent the first year of this period as an Individual Contributor (IC) diving deep into the migration tasks, and then I took over the Tech Lead (TL) role of the team to see the project through completion. This week, we publicly announced that we completed our migration to Bazel for the largest part of our codebase and we provided details on our journey. I did not publish that article here for obvious re...| Julio Merino (jmmv.dev)
If you grew up in the PC scene during the 1980s or early 1990s, you know how painful it was to get hardware to work. And if you did not witness that (lucky you) here is how it went: every piece of hardware in your PC—say a sound card or a network card—had physical switches or jumpers in it. These switches configured the card’s I/O address space, interrupts, and DMA ports, and you had to be careful to select values that did not overlap with other cards. But that wasn’t all. Once you ha...| Julio Merino (jmmv.dev)
In Unix-like systems, “everything is a file and a file is defined as a byte stream you can open, read from, write to, and ultimately close”… right? Right? Well, not quite. It’s better to say file descriptors provide access to almost every system that the kernel provides, but not that they can all be manipulated with the same quartet of system calls, nor that they all behave as byte streams. Because you see: network connections are manipulated via file descriptors indeed, but you don...| Julio Merino (jmmv.dev)
If you have been following the development of EndBASIC, you know its console can display both text and graphics at once. What you may not know is that, now, it can also achieve this feat on the NetBSD console without using X11 at all. This is done by directly rendering to the wsdisplay framebuffer, and this article presents a crash course on direct graphics and keyboard access via NetBSD’s wscons framework.| Julio Merino (jmmv.dev)
Make, as arcane as a build tool can be, may still be a good first fit for certain scenarios. “Heresy!”, you say, as you hear a so-called “Bazel expert” utter these words. The specific problem I’m facing is that I need to glue together the NetBSD build system, a quilt patch set, EndBASIC’s Cargo-based Rust build, and a couple of QEMU invocations to produce a Frankenstein disk image for a Raspberry Pi. And the thing is: Make allows doing this sort of stitching with relative ease. Su...| Julio Merino (jmmv.dev)
I recently picked up an embedded project in which I needed to build a highly customized full system image with minimal boot times. As I explored my options, I came to the conclusion that NetBSD, the often-forgotten BSD variant, was the best viable choice for my project. One reason for this choice is NetBSD’s build system. Once you look and get past the fact that it feels frozen in time since 2002, you realize it is still one of the most advanced build systems you can find for an OS. And it ...| Julio Merino (jmmv.dev)
My interest in storage is longstanding—I loved playing with different file systems in my early Unix days and then I worked on Google’s and Microsoft’s distributed storage solutions—and, about four years ago, I started running a home-grown NAS leveraging FreeBSD and its excellent ZFS support. I first hosted the server on a PowerMac G5 and then upgraded it to an overkill 72-core ThinkStation that I snapped second-hand for a great price. But as stable and low maintenance as FreeBSD is, r...| Julio Merino (jmmv.dev)
I recently got a Synology DS923+ for evaluation purposes which led me to setting up NFSv4 with Kerberos. I had done this about a year ago with FreeBSD as the host, and going through this process once again reminded me of how painful it is to secure an NFS connection. You see, Samba is much easier to set up, but because NFS is the native file sharing protocol of Unix systems, I felt compelled to use it instead. However, if you opt for NFSv3 (the “easy default”), you are left with a system ...| Julio Merino (jmmv.dev)
Just like that, BazelCon 2024 came and went. So… it’s obviously time to summarize the two events of last week: BazelCon 2024 and the adjacent Build Meetup. There is A LOT to cover, but everything is here in just one article!| Julio Merino (jmmv.dev)
If you read my previous article on DOS memory models, you may have dismissed everything I wrote as “legacy cruft from the 1990s that nobody cares about any longer”. After all, computers have evolved from sporting 8-bit processors to 64-bit processors and, on the way, the amount of memory that these computers can leverage has grown orders of magnitude: the 8086, a 16-bit machine with a 20-bit address space, could only use 1MB of memory while today’s 64-bit machines can theoretically acce...| Julio Merino (jmmv.dev)
At the beginning of the year, I wrote a bunch of articles on the various tricks DOS played to overcome the tight memory limits of x86's real mode. There was one question that came up and remained unanswered: what were the various models that the compilers of the day offered? Tiny, small, medium, compact, large, huge... What did these options mean? What were their effects? And, more importantly... is any of that legacy relevant today in the world of 64-bit machines and gigabytes of RAM? To ans...| Julio Merino (jmmv.dev)
After a little over 11 years, it’s time for a much longed change: I’m leaving Google and I’m joining Microsoft as a Principal Software Engineer for Azure. These job changes are effective as of this week, but my family and I already moved from New York City to Redmond, WA about three weeks ago. Read on for a recap on my tenure at Google, the whys behind my departure, and how I ended up choosing the position in Microsoft Azure after mulling over offers from Facebook, Twitter, and Microsoft.| Julio Merino (jmmv.dev)
I joined the FreeBSD committer ranks a couple of months ago with the intention to equip FreeBSD with an out-of-the-box test suite and with a testing infrastructure. The time until now has been quite fruitful and I have been rushing to get something ready for you before the year end. With that, I am very pleased to announce that the first mockup of the FreeBSD testing cluster is up and running! Point your browser at:| Julio Merino (jmmv.dev)
About 11 years ago, I had the honor of becoming part of the NetBSD developer team. A week ago, the same offer was extended to me for FreeBSD and I could not refuse it... so, as of two days ago, I am jmmv at FreeBSD and will be working in src! But how have I ended up here? The story goes back in time... My beginnings with FreeBSD FreeBSD was the first BSD system I tried after spending a couple of years with Linux. I was able to do so because one of the PC magazines that my father used to buy i...| Julio Merino (jmmv.dev)
After a very long delay, Google has finally chosen the projects that will be part of the Summer of Code program. There seems to be no official announcement in the page yet, but I already received a mail... and... my project is accepted! :-) I briefly outlined my project some days ago, but I'll explain it in more detail now (copying some paragraphs from the application form verbatim). At the moment, NetBSD includes a memory-based file-system called mfs. mfs is is just an implementation of the ...| Julio Merino (jmmv.dev)
Blog System/5 hasn’t always been called this way and it hasn’t been my first experience with blogging either. In fact, today marks the 20th anniversary of this publication in its various incarnations so it’s time for a bit of reflection. Just to set context for when 20 years ago was: Windows XP was almost 3 years old, Ubuntu had just debuted, Apple computers were still PowerPC-based, Half Life 2 was about to launch, and Slashdot was the place to be instead of the yet-to-be-created Hacke...| Julio Merino (jmmv.dev)
In “From 0 to 1 MB in DOS”, I presented an overview of all the ways in which DOS and its applications tried to maximize the use of the 1 MB address space inherited from the 8086—even after the 80286 introduced support for 16 MB of memory and the 80386 opened the gates to 4 GB. I know I promised that this follow-up article would be about DJGPP, but before getting into that review, I realized I had to take another detour to cover three more topics. Namely: unreal mode, which I intentional...| Julio Merino (jmmv.dev)
Since the last article on the text-based IDEs of old, I’ve been meaning to write about the GCC port to DOS, namely DJGPP. As I worked on the draft for that topic, I realized that there is a ton of ground to cover to set the stage so I took most of the content on memory management out and wrote this separate post. This article is a deep dive on how DOS had to pull out tricks to maximize the use of the very limited 1 MB address space of the 8086. Those tricks could exist because of the featur...| Julio Merino (jmmv.dev)
I grew up learning to program in the late 1980s / early 1990s. Back then, I did not fully comprehend what I was doing and why the tools I used were impressive given the constraints of the hardware we had. Having gained more knowledge throughout the years, it is now really fun to pick up DOSBox to re-experience those programs and compare them with our current state of affairs. This time around, I want to look at the pure text-based IDEs that we had in that era before Windows eclipsed the PC in...| Julio Merino (jmmv.dev)
It’s more likely than you think! In a surprising twist of events, Microsoft is exploring the addition of a command-line (CLI) text editor to Windows. If you ask me, not having a CLI text editor on Windows is mind-boggling: you can access a Windows machine via SSH these days, so not having an editor that works in the console is a big handicap for remote system administration. So, should Windows bundle a CLI text editor? Of course it should.| Julio Merino (jmmv.dev)
Just yesterday, Twitter user @vkrajacic wrote: Advice for new C programmers: “Avoid null-terminated strings; they’re outdated, inefficient and impractical.” Create your own type with basic functions. It’s not that hard, and it goes a long way. One of the benefits of this approach, among others, is slicing without copying. This suggestion has its merits and I understand where it is coming from: performance. You see: the traditional way to represent strings in C is to use NUL-terminated...| Julio Merino (jmmv.dev)
While working on this static blog a few days ago, I made a change to its templates that warranted an automated test. I could have written a trivial shell script to do it, but instead I reached out for shtk’s unit-testing module. I tweeted about it right away to just say that you can, in fact, write tests in shell because lots of developers are skeptical about any script longer than 10 lines of code. Interestingly, this reply came through: a pointer to a contemporary, under-development libra...| Julio Merino (jmmv.dev)
Well, that was unexpected. I recorded a couple of crappy videos in 5 minutes, posted them on a Twitter thread, and went viral with 8.8K likes at this point. I really could not have predicted that, given that I’ve been posting what-I-believe-is interesting content for years and… nothing, almost-zero interest. Now that things have cooled down, it’s time to stir the pot and elaborate on those thoughts a bit more rationally. To summarize, the Twitter thread shows two videos: one of an old c...| Julio Merino (jmmv.dev)
Dependency injection is one of my favorite design patterns to develop highly-testable and modular code. Unfortunately, applying this pattern by taking Rust traits as arguments to public functions has unintended consequences on the visibility of private symbols. If you are not careful, most of your crate-internal APIs might need to become public just because you needed to parameterize a function with a trait. Let’s look at why this happens and what we can do about it.| Julio Merino (jmmv.dev)
