In this article, we explore additional methods to assign people to voting centers, such that the average distance that each person must travel is minimized. We use Optimal Transport, the Stable Assignment problem and linear programming. We review each method’s mechanism, the results that it achieves, and its computational complexity and possible drawbacks.| Posts on jreyesr's blog
In this article, we obtain data about the location of people and voting centers in Ecuador, and we start exploring possible methods for assigning people to voting centers so that the distance that people have to travel is minimized. We explore a simple scheme that assigns people to their closest voting center, and we start exploring the Earth Mover’s Distance algorithm.| Posts on jreyesr's blog
In this article, we learn about linked-data visualizations, what they are and why they may more expressive than static visualizations. We then explore Glue, an application that can be used to explore linked data by interacting with connected plots that react to changes made in each other.| Posts on jreyesr's blog
In this article, we explore Typst, a new typesetting system similar to LaTeX. We explore its syntax, current uses and possible uses; we test its scripting system, which is incredibly powerful; we compare it with LaTeX, Markdown and Word, which can be used in similar roles; and we explore its applicability to the task of automatically generating documents from templates, such as in automatic invoice, certificate or report generation.| Posts on jreyesr's blog
In this article, we make some changes to the seed counting application that was presented in the previous post, such as making detection more robust to illumination changes, allowing the application to run offline, and providing a way for users to submit images with problems to the application developer, so they can be studied and fixed.| Posts on jreyesr's blog
Hello! This is a quick post to announce that a new version of my Insomnia plugin that sends repeated requests has been released: v1.4.0 is now available for download. This version adds the ability for users to extract output data (the fields that will be written back to the CSV file) from the response headers, status code, and request elapsed time, in addition to the response body.| Posts on jreyesr's blog
In this article, we explore a dashboard that provides information about Ecuador’s electrical grid generation and production, in the context of current rolling blackouts. We explore different ways of extracting information from images, such as reading percentages from pie charts and stacked bar charts, and detecting the legends of charts.| Posts on jreyesr's blog
In this article, we review ways in which the Clickhouse analytical database engine can be made to interoperate with Steampipe, for the purpose of retrieving additional data to be added to events that are stored on Clickhouse, without requiring to list the entirety of the extra data in advance.| Posts on jreyesr's blog
In this article we extend the concept of refactorings, as used on source code, to data stored, for example, on YAML or JSON files. As an example, we extend the typical Rename Variable action to renaming a Kubernetes resource. We explore several scenarios that seem like a data-focused variant of code refactorings, why they may be useful. Then, we review some ways in which those data refactorings may be implemented, which tools could support them, and how the user experience may be like.| Posts on jreyesr's blog
In this article we review Langium, which is a language engineering tool that can be used when developing custom structured text languages. It can generate parsers for them, and provides useful services such as validation, code generation, syntax highlighting, autocompletion, a LSP server, web-only support, and the ability to generate VS Code extensions for the language.| Posts on jreyesr's blog
Hello! This is a quick post to announce that a new version of my Insomnia plugin that sends repeated requests has been released: v1.3.0 is now available for download. It adds a way to send requests in parallel, useful to speed up large batches of requests as long as the remote API can handle it.| Posts on jreyesr's blog
In this article we explore Flowpipe, a new tool to author and run workflows (i.e. sequences of steps that each fetch some data or perform some action, such as calling a REST API). It uses HCL, the same language as Terraform/OpenTofu, to define the workflows, as opposed to a UI or pure code. We review the project, the language, the available actions, try a few simple workflows, and explore tools that are somewhat similar to Flowpipe.| Posts on jreyesr's blog
In this article we explore Telegram bots, with particular attention to the developer side of things: what are bots? How are they built? What can they do? We review several patterns that bots can use to interact with users, starting with simply sending and receiving messages and ending with fully-fledged web applications.| Posts on jreyesr's blog
In this article we explore Semgrep, a static code analyzer. We investigate its use cases, its main differentiators and scenarios in which it can help. Then we learn Semgrep’s rule language, which is used to define Semgrep’s alerts, and we write custom rules that check for common errors in the Markdown files that make up this blog.| Posts on jreyesr's blog
In this article we review more authorization systems. Zanzibar systems express access rules in terms of relationships between objects. OPA is a server and language that has integrations with several CNCF projects, chief among them Kubernetes itself. Finally, we review assorted solutions such as Cedar (used in AWS products), CEL (used in Google Cloud products) and Cerbos.| Posts on jreyesr's blog
This article is the first in a deep-dive exploration of authorization/access control systems, which can be used to constrain the actions that the users of web applications can perform. In this post we introduce the problem and we take a look at simple, role-based systems that assign roles to users. We then study some situations in which role-based authorization lacks expressive power, such as users who should only have access to a subset of the data, and we review other solutions that also ta...| Posts on jreyesr's blog
In this article, we explore several alternatives to provide user-definable logic on larger applications, such as transformation functions and filtering criteria.| Posts on jreyesr's blog
In this article, we explore Streamsync’s recently released Custom Components. We try several different patterns to see how it fares, and we discover the interface that they present to the rest of the application| Posts on jreyesr's blog
In this article, we explore a blockchain system that has been used in a couple of Ecuador’s recent elections. We try to discover how it works, what exactly it certifies, and how it does it. Then, we explore what other entities have to say about election security and blockchains in particular, and we present several schemes that seem to have comparable security guarantees at a much lower complexity cost| Posts on jreyesr's blog
In this article, we present a new Steampipe plugin that can provide access to a plain PostgreSQL database. This has been requested several times and current solutions are cumbersome. This plugin extends the reach of Steampipe’s queries to static (or not) data that is hosted on ordinary tables.| Posts on jreyesr's blog
In this article, we capture time-indexed partial results of Ecuador’s recent presidential elections, we review ways of saving them, and we then explore the data. We observe the progression of results over time, we verify that results never stopped coming in, we characterize the time that an average station takes to count and report results, and we see just how few votes it took for the final results to be predictable.| Posts on jreyesr's blog
in this post, we start testing the Steampipe plugin that bridges to Terraform data sources with several random Terraform providers. We uncover a few bugs, and we find that the plugin (mostly) works as intended.| Posts on jreyesr's blog
In this article we explore the Terraform Registry API, which hosts information about Terraform providers, in particular the URLs in which the actual provider binaries are hosted. We then implement a client for that API in our Steampipe plugin, so the plugin auto-downloads whichever Terraform provider has been requested by the user, at boot time. This makes the experience of configuring a Terraform provider in Steampipe a lot closer to that of configuring it on Terraform itself| Posts on jreyesr's blog
In this article, we use Terraform data sources to automatically declare tables for a Steampipe plugin, and we forward queries that come in for each table to the corresponding Terraform datasource.| Posts on jreyesr's blog
In this article, we query the Data Sources of Terraform plugins from a standalone Go program (i.e., not via the Terraform CLI). We can thus test and use a Terraform provider in isolation, and reuse the already-existing functionality of Data Sources in other programs, such as a Steampipe plugin.| Posts on jreyesr's blog
In this article, we explore the way in which Terraform runs its plugins, AKA providers. We replicate the protocol in another program, and we run the Terraform DNS provider outside of Terraform. We extract its schema, which can tell us which credentials the provider needs, and also the data sources that it exposes, along with the fields that each data source has.| Posts on jreyesr's blog
In this article, we start a new project: giving Steampipe access to any* Terraform datasource. This would allow a project to both effect changes in cloud infrastructure (by using Terraform) and verify or monitor those changes (by querying Steampipe), reusing code between both activities| Posts on jreyesr's blog
In this devlog, we cover the implementation of a listener that can react to Telegram’s webhooks, which are sent when a bot receives a message from the user (among other events)| Posts on jreyesr's blog
In this devlog, we explore XState, a state-machine library for Javascript. We present the basics of state machines, what they are useful for, and how they could pertain to the OpenChatflow application. We also implement a first cut of a translation layer to convert OpenChatflow conversations to an executable format| Posts on jreyesr's blog
In this article we explore Kaitai, a parser for binary file formats.| Posts on jreyesr's blog
In this article, we will review a series of ways of hosting and running assorted scripts, using Python as a case study. We will start from a humble script hosted in a random user device, and progress up to a fully-featured platform to develop, run, test, monitor and authorize script runs.| Posts on jreyesr's blog
In this article, we’ll review Streamsync, a (young) framework that lets you create interactive web applications backed by Python code. We’ll compare it with some desktop UI frameworks, and also with Streamlit, and with other low-code UI development tools| Posts on jreyesr's blog
This article explores continues the exploration of Terraform, this time for dynamically generating configurations (e.g. generating one EC2 server for each entry in a DB)| Posts on jreyesr's blog
This article explores the usage of Terraform as applied to the problem of configuring generic APIs (i.e., those not related to cloud infrastructure). The motivation is to declaratively control APIs that only expose an imperative interface (such as essentially any REST-ish API)| Posts on jreyesr's blog
In this devlog, we explore the observability capabilities that are provided by NextJS, and we instrument the OpenChatflow application.| Posts on jreyesr's blog
In this devlog, we integrate a real DB (PostgreSQL with the Prisma ORM) with the OpenChatflow application, so that we can create, read and save conversations.| Posts on jreyesr's blog
In this devlog, we add more nodes, and present a first version of the application, deployed to Vercel| Posts on jreyesr's blog
In this devlog, we add several new nodes, the UI necessary to add nodes to the canvas, and we implement new types of nodes: async actions that run in the background and multiple-output nodes that route the conversation through different paths.| Posts on jreyesr's blog
In this devlog, we cover the setup of a NextJS project, creating custom nodes in Reactflow, adding a configuration dialog to one node, edge validation so that only certain node connections are allowed, and more!| Posts on jreyesr's blog
In this article, we introduce an application to design, test, run and monitor Telegram chatbots.| Posts on jreyesr's blog
In this article, we explore the traces&metrics functionality implemented in Steampipe. We also expose custom metrics from our own plugins. We also enhance Steampipe’s traces with custom tags and event logs, and even create new traces!| Posts on jreyesr's blog
In this short article, we’ll explore the caching facilities available to a Steampipe plugin.| Posts on jreyesr's blog
This article explores some advanced integration usecases in which Steampipe can be used, such as placing another Postgres instance in fron of it for greater control and more data sources.| Posts on jreyesr's blog
In this article we process the logs that we extracted from a time-tracking application, and we explore Gravwell’s facilities for extracting insights from those logs.| Posts on jreyesr's blog
Hello! This is a quick post to announce that a new version of my Insomnia plugin that sends repeated requests has been released: v1.1.0 is now available for download. It adds a way to control the delay between requests, to prevent rate-limited or slow APIs from getting nervous.| Posts on jreyesr's blog
This article presents a way of extracting information from a time-tracking and activity monitoring application, by hooking its calls to the SQLite library with Frida.| Posts on jreyesr's blog
This article continues work on the Bitcoin plugin, by adding automatic retries and exploring the contents of mods: Controls, Benchmarks and Dashboards. It also presents an enhancement to the dashboard charts, to enable them to understand time-based data.| Posts on jreyesr's blog
This article presents a plugin for Steampipe that allows you to query the Bitcoin network for information about wallets and transactions. It also covers some gotchas found while developing such a plugin.| Posts on jreyesr's blog
This article presents Steampipe, an aplication that presents disparate data (such as REST APIs) as SQL tables, allowing you to join data across API boundaries. It explains the inner workings of Steampipe plugins, that enable new data sources to be used with it. Finally, it presents a template repository that can be used as a starter by anyone wanting to create his own plugins.| Posts on jreyesr's blog
In which we introduce and describe an Insomnia plugin that can repeatedly send a request, varying arbitrary parts of it with data taken from a CSV file, and also extract response data and save it to the aforementioned CSV file.| Posts on jreyesr's blog
You have reached the blog of jreyesr! Within this blog thou shall find only the finest random articles (i.e., no specific theme is planned, mostly because I don’t think I could produce more than two articles about the same subject). The only constant will be software development, as you would expect from a Github user.| jreyesr's blog
In this article, we create a new N8N node that integrates the Docxtemplater open-source library to render Word documents from templates, as well as providing user-extensible Transforms for data, and Data Sources that provide additional template data.| jreyesr's blog
This article reviews graph databases, as a storage technology similar to relational databases like Postgres and document stores like MongoDB. We review the basic concepts of graph DBs, some example engines, use cases in which treating data as a graph can be useful. We explore a project that uses a graph DB to store data about cloud systems, and we use a graph DB to model a router+firewall’s configuration file, including queries over the stored data, which we also compare to a similar Postgr...| blog.jreyesr.com
In this article, we develop an application that can automatically count fruit seeds, using OpenCV, entirely client-side. We start from a bunch of seeds spread on a paper sheet, and we end up with automatic seed detection, segmentation and counting, plus some ideas for future enhancements.| jreyesr's blog
In this article, we create a new N8N node that integrates the Carbone open-source library to render Word documents from templates| jreyesr's blog
In the final article of this series, we extract TLS-protected messages from a time-tracking application, using both a TLS-intercepting proxy and Frida.| jreyesr's blog