We live in the age of software as a service, secret keys are everywhere. Yet somehow it is still normal to click-manage secrets. Smart people share them via signal,smarter people, share them via a password manager like Bitwarden. Super-advanced people punch them into GitHub UI and only let github actions have access to secrets. Infra-devops people spin up Vault or AWS secret manager or similar to inject secrets. In all situations secrets have a lifecycle that is completely disconnected from c...| Taras' Blog on AI, Perf, Hacks
I relax by programming. It’s especially satisfying to play with something that changes how I think about a topic. I once discovered eget and realized that most modern software ships as a single binary attached to a GitHub release. Recently my friend, David, had a further realization that what would make eget better is if it itself was easier to get (ha!). I thought it would be delightful to have a WASI eget cli to fetch native binaries. These days almost everything cross-compiles trivially ...| Taras' Blog on AI, Perf, Hacks
I decided to write down some thoughts on agentic coding and why it’s a very hyped wrong turn. Let me start with some background on my LLM experience. I adopted LLMs into my work in Aug 2020. I was sold when I saw that GPT-3 could generate usable SQL statements. Something that used to take 4-8 hours of RTFMing, now took 15min. I have since worked on chatcraft.org, various RAG frameworks, etc. I use aider heavily for work, frequently switch models, have been struggling with tool calling since...| Taras' Blog on AI, Perf, Hacks
I read a lot, blogs, papers, news, etc. My eyes are sensitive to artificial light so I prefer to read on e-ink. I also like to work outside, so things that try to fight the sun by feebly (relative to sun) shining even more light into my eyes are frustrating. I also find that e-ink screen limitations like slower updates and lack of color to be wonderful for not getting distracted by shiny pictures, videos.| Taras' Blog on AI, Perf, Hacks
It used to be that blogs all had RSS feeds. Somehow there are more blogs than ever before, but some people do not bother with setting up an RSS feed. The following blogs have nice content, but are annoying to follow without RSS. https://aider.chat/blog/ https://jkatz05.com/ https://getreuer.info/ Every time I stumble on one of these, it takes me a while to remember a fun workaround. I decided to write this down as a blog post so I have an easier time looking up the details.| taras.glek.net
I’m switching my blog over to a new hugo theme. It looks slightly nicer now, but migrations are hard and there will be some broken posts. Background When I was at Mozilla back in pre-2014 years I blogged on their corp wordpress setup. It was awful. I hated the WP editor, I hated the WP themes, I hated the WP plugins. I had cool blog posts on there, but they were all deleted when I left.| Taras' Blog on AI, Perf, Hacks
tldr: GitHub is where most CLI tools live. eget is a tool that makes it easy to download and install their binaries.| Taras' Blog on AI, Perf, Hacks
If you’re happy reading on your laptop or Apple device, you can skip this post.| Taras' Blog on AI, Perf, Hacks
I needed a way to do some programming while offline. These days I feel very unproductive without https://chatcraft.org (the best chat UI for programming) and a good LLM to chat with about coding. Chatcraft needed a few small fixes to enable llama.cpp support. Here’s how to run models with llama.cpp with chatcraft.org without internet: Instructions Install and run llama.cpp. Follow https://github.com/ggerganov/llama.cpp instructions for your platform. For mac: # install llama.cppbrew install...| Taras' Blog on AI, Perf, Hacks
TLDR: chatcraft.org is now smooth as butter and maximally performant thanks to throttling LLM chat-completion rendering via requestAnimationFrame().| Taras' Blog on AI, Perf, Hacks
I read a lot. I enjoy reading on black and white e-readers. Unfortunately many websites make it hard to read them on simple devices. https://markdown.download is my really simple solution to that. Prepend it to any website and curl: curl https://markdown.download/https://dev.to/amnish04/introducing-the-idea-of-web-handlers-in-chatcraft-1b4i That turns a user-hostile website into a thing that almost anything can render: It’s amazing how much cleaner the web is when stripped down to essence. ...| Taras' Blog on AI, Perf, Hacks
I have an intermittent issue where one of my machines on a specifc network seems to be operational locally on network but not accessible over netbird some of the time. I could not tell if the problem was due to the network the problematic machine was on or due to netbird. So to debug this I: Setup a grafana node agent to report journald logs to https://grafana.com/ Setup to docker to log to journald Wrote ssh-via-cloudflare-tunnel docker-compose stack to tunnel ssh over websocat to internet o...| Taras' Blog on AI, Perf, Hacks
Let’s face it, managing secrets in software projects can be as thrilling as being stabbed in the eye. Yet, it’s a necessary evil that we all have to deal with. Problem: we have a set of a developers and set of infrastructure that all needs to share secrets. Would like to minimize infrastructure and keep cognitive load to a minimum so we can focus on writing code. Sure you’ve got AWS Secrets Manager and Hashicorp Vault for the heavy lifting, but that’s like using a tractor to crack a n...| Taras' Blog on AI, Perf, Hacks
libkrun + krunvm Github randomly recommended me libkrun which is a library backing krunvm. It’s something similar to firecracker, but even simpler.| Taras' Blog on AI, Perf, Hacks
I have been working with Python a lot more recently, and it feels like I spend more time fighting packaging than writing code. Python’s primary package manager, pip, is roughly equivalent to the best 1990s had to offer(Perl CPAN), it makes it depressingly easy to end up with a broken environment. Pixi: A modern packaging system for Python pixi is a modern package manager along the lines of deno/pnpm, but for Python. It’s a single binary that you can download and run. It will install Pytho...| Taras' Blog on AI, Perf, Hacks
13 GiB/s per core! Sneller posted a blog on HN on how they use AVX-512 to decompress data at 13 gigabytes per second per core. This a fantastic ad for their “lets turn logs on S3 into cheap database” product. This is a solution I wanted multiple times, will definitely consider them next time the need comes up. Faster than RAM Now this post did not get overlooked, but what did get overlooked is that the post engaged the clickhouse CTO. He posted a link to a presentation on how Clickhouse u...| Taras' Blog on AI, Perf, Hacks
I’m gonna start a column on cool blog posts I found, that got 0 or minimal traction. I suspect I will also have no traction doing that 🤦♂️. The Problem I really enjoy thoughtful writing on deep technical problems. It’s even better when one sees thoughtful comments, that further contribute new directions to throughts presented. HackerNews is where most of that writing tends to land. Unfortunately it tends to not do well vs trendy, click-baity, etc content. Twitter is even worse. ...| Taras' Blog on AI, Perf, Hacks
The Problem Ukraine had a lot of power outages due to Putin’s bombing of our power infrastructure, I needed to switch to fiber + battery-backup to continue able to able to work. I’m at a rental apartment and I’m not allowed to drill walls. The idiot that layed the internet into the apartment used a 4-wire cat5 cable to save a few pennies, then cemented it in.| Taras' Blog on AI, Perf, Hacks
TLDR: I prompt-engineered the system prompt in chatcraft to turn it into a github analytics tool: github analyst chat. Longer Story I suggested to David that we have a newsfeed of new features on chatcraft.org. He replied that we should try to use chatcraft.org to generate those. Found simonw’s blog post on using a public clickhouse instance. Used David’s new “edit system prompt” feature in combination with my “Run Code” feature with the new-sh chatgpt-16k model to chat with the r...| Taras' Blog on AI, Perf, Hacks
chatcraft.org is my open source project for working with GPT. It completely changed how I work. Cool thing about chatcraft is that it’s completely client-side, almost completely stateless(except for sharing) on serverside. I am also working on a project that uses LLMs to help navigate a knowledge base. Making a wrong tech choice there could kill my project. Few thoughts from that perspective: The 3 YOLOs of LLM development “You only live once” (YOLO) is a modern adaptation of the Latin ...| Taras' Blog on AI, Perf, Hacks
George Holtz (geohot) just raised $5M for tinygrad and plans to sell a $15,000 Machine Learning AMD Epyc box with AMD 7900 XT video cards. At first glance, this seems like a really risky investment into a low-margin commodity hardware company. AMD can’t compete with NVidia, so how can we expect geohot to compete with NVidia on AMD hardware? Well, I’ve been following geohot for a long while. He delivers what he promises, but does it years later and on different hardware than initially prom...| Taras' Blog on AI, Perf, Hacks
Ah, the world of software! A place where creativity and collaboration come together to create amazing projects. But sometimes, this world can be a bit… confusing. Let me tell you a story about the frustration of open source naming, featuring our protagonist: PGVector. Once upon a time, I though that pgvector was a postgresql extension. And that langchain did not support SQL storage out of the box. After reading the source, turned out there were three things named PGVector:| Taras' Blog on AI, Perf, Hacks
I would like to introduce ChatCraft.org: a developer-focused, open source chatgpt frontend. While it’s my fav GPT UI (as it was created to make my life more convenient), it’s not so much about the features that are complete now as about the fun, collaborative journey ahead. I think we need to explore the design space of GPT-assisted collaboration to write code. The best way to do that is via open source exploration of the concept!| Taras' Blog on AI, Perf, Hacks
https://github.com/thegpvc/gpt-aria This was the most interesting prototype I’ve explored in my career so far, and I want to thank TheGP for sponsoring this project. Also thanks to Oleksandr Chugai who I partnered with on implementation and Ben Cmejla who helped with prompting. Inspiration I’ve been thinking about browser automation since the early days of my career when I worked at Firefox, where I got to work on performance problems and built things like Firefox Telemetry (Hi, HN haters!| Taras' Blog on AI, Perf, Hacks
Cloudflare Pages 😍 I got into Cloudflare pages because it lets you host static websites with all the latest optimizations conveniently and for free. Setting up a static website with a custom domain is easy, and you get the benefits of automatic SSL, CDN, and latest version of HTTP3. This is how this blog is hosted and I have a few other projects like this. I love the fact that I can get a super-fast static website up and running in a few minutes and I don’t have to worry about infrastruc...| Taras' Blog on AI, Perf, Hacks
TLDR: ZFS free-space reporting is a lagging indicator. Background I use Proxmox VM server backed by a ZFS array of hard drives for various personal infrastructure. I also have security cameras that upload motion-triggered videos to my server (via FTP!). Problem Description I would like to use 90% of my available space for most-recent security videos. Recipe: Create a dedicated ZFS volume Setup ZFS quota Run a cronjob to free space faster than it gets consumed by video uploads.| Taras' Blog on AI, Perf, Hacks
Previous blog post on how to trace Firefox IO using bpftrace via official page_fault_user tracepoint left me a bit unsatisfied with how complicated it turned out. Complexity has potential to be error-prone and the syscall-tracing dependency makes it impossible to trace IO within the main executable. I decided to try reimplement the trace using my old approach of tracing ext4 functions that handle page-faults. This turned out to be much more robust.| Taras' Blog on AI, Perf, Hacks
Modern browsers are some of the most complicated programs ever written. For example, the main Firefox library on my system is over 130Mbytes. Doing 130MB of IO poorly can be quite a performance hit, even with SSDs! :). Few people seem to understand how memory-mapped IO works. There are no pre-canned tools to observe it on Linux, thus even even fewer know how to observe it. Years ago, when I was working on Firefox startup performance, I discovered that libraries were loaded backwards (blog1, b...| Taras' Blog on AI, Perf, Hacks
My current employer does a lot of really cool systems work that’s covered by NDAs. I recently did some work to integrate a cool open source tool into our workflow. Felt it deserved a blog post. NFS Testing Requires Parallelism. I work for Pure Storage. One of the products we make is a scale-out NFS1 (and S3-compatible) server called FlashBlade. I was asked to test FlashBlade2 performance scaling. I needed to generate NFS read workloads of 15-300 Gigabytes/second.| Taras' Blog on AI, Perf, Hacks
This post is about minimizing amount of disk IO and CPU overhead when reading Zip files. I recently saw an article about a new format that was faster than zip. This is quite surprising as to my mind, zip is one of the most flexible and low-overhead formats I’ve encountered. Some googling showed me that over past 11 years people have noticed that Firefox uses optimized zip files. This inspired me to document thinking behind the optimized zip format I implemented in Firefox in the pre-pandemi...| Taras' Blog on AI, Perf, Hacks
About| Taras' Blog on AI, Perf, Hacks
| Taras' Blog on AI, Perf, Hacks
TLDR: You can predict degree of unresponsiveness of a phone via random-write-4k benchmarks. I wish review websites would fill phones to 80-90% prior to running the benchmark, especially on smaller-capacity phones where users are more likely to run out of space.| Taras' Blog on AI, Perf, Hacks
I spent most of my life in North America, but I have been living this war in Ukraine. My hope is that I can use my past experience and connections to help Ukraine win this war. Recently a friend of mine asked what would be the best way to donate $1000. This got me thinking. Wars are won by convincing the opponent that things are only gonna keep getting worse for them and better for you.| taras.glek.net