Returning Arrow tables over HTTP is relatively straightforward. The Arrow IPC streaming format is repurposed to the HTTP use case and you can return each serialized IPC batch as a chunk of the HTTP response — Transfer-Encoding: chunked — to enable streaming clients. Arrow IPC streams have a MIME type registered with IANA so you can also set the appropriate Content-Type header on your HTTP responses. 1 Content-Type: application/vnd.apache.arrow.stream You can find Python examples of this i...| Runtime Checks
In multiplication, to get a non-zero product, all factors must be greater than 0. If one factor is 0, nothing else matters. \[\begin{align} 5 \times 2 \times 229 \times 0.2 &= 458.0\\ 5 \times 2 \times 229 \times 0.0 &= 0.0 \end{align}\] This xz/liblzma incident and the public post-morten discussion is a great illustration of why incident post-mortens shouldn’t be carried as a search for a root cause, but as a comprehensive enumeration of all contributing factors. You can model successful ...| Runtime Checks
id Software co-founder John Romero tells the early story of the game company in this GDC 2016 talk and lists the programming principles that guided them towards the rapid development of many games including Doom and Quake with a very small team. Some of these principles resembles today’s common Agile practices while others do not. I like them and id definitely had good results, so the principles should at least be considered and adapted to different contexts. No prototypes. Just make the ga...| Runtime Checks
C++ 17 has introduced the std::optional<T>template class that is analogous to the Maybe/Optional monad implementation in many other languages. “Analogous” is doing a lot of work in this statement because the C++ type checker is not going to help you avoid dereferencing an empty optional like Rust, Haskell, Scala, Kotlin, TypeScript and many other languages will do. That does not make it useless. As with many things in C++, we will be careful™ when using it and write only programs that d...| Runtime Checks
Joins are an important class of operations in databases and data pipelines. There are many kinds of joins in SQL databases, but this post will focus only on the INNER JOIN. 1 It is possible to make the JOIN between two large tables very efficient if the result is going to be small. That being a common scenario in practice, makes JOINs an interesting topic of research. A naive JOIN algorithm can be very slow because JOINs are essentially nested loops. NestedLoop JOIN based on sequential scans1...| Runtime Checks
In 2017 I decided to print some PDFs to read instead of just have them in an ever-growing folder of Papers to Read. Here is the list of the ones I managed to read this year. Out of the Tar Pit / Ben Moseley, Peter Marks / 2006 This is my favorite paper. It is a bit long (around 60 pages), but well worth reading. It is very thorough in defining what software complexity is. The authors make the distinction between essential and accidental complexity (following Fred Brook’s ideas on the topic)...| Runtime Checks
When starting a new side-project I always wonder if parts of the codebase can be shared as an open-source project. A library extracted from the project might be of interest to other developers and it’s nice to have good stuff on your Github profile. That possibility may compel you to spend a lot of time thinking about the repository structure. You may create a git submodule for every library that doesn’t exist yet but might be useful to you in other projects or to other people and you don...| Runtime Checks
Every time I want to quickly understand something about an advanced type system or programming language concept I get lost when I see something like this on Wikipedia: Linear type systems are the internal language of closed symmetric monoidal categories, much in the same way that simply typed lambda calculus is the language of Cartesian closed categories. More precisely, one may construct functors between the category of linear type systems and the category of closed symmetric monoidal catego...| Runtime Checks
Ontem o Marco Gomes perguntou por que 0.4 - 0.5 é 0.09999999999999998 em vez de 0.1 em Javascript. Esse post é uma tentativa de resposta à essa pergunta. Existe um número infinito de números reais entre 0.0 e 1.0 então não dá pra representar todos os números reais com precisão infinita já que a memória do computador é finita. Ao contrário do que disseram nos comentários, dá sim pra representar 0.4, 0.5 e 0.1 em 64 bits (tamanho do float no Javascript) e em até muito menos que...| Runtime Checks
Being able to easily run and debug a simple operating system can be really useful when you want to learn how low level components are implemented. Xv6 is a very simple Unix-like operating system that allows you to do just that. sillysaurus2 exemplified this in the Hacker News’ thread on Xv6: Have you ever: Wondered how a filesystem can survive a power outage? Wondered how to organize C code? Wondered how memory allocation works? Wondered how memory paging works? Wondered about the differenc...| Runtime Checks