Originally Posted: Aug 24, 2022 Dealing with huge web3 data, I see myself doing this a lot: moving data directories across machines. In this case, we have a 250 GB directory with 10K files. The network connection between machines is ~5 MBps. But using these techniques, we can achieve a 6x faster transfer (~30 MBps). There’re two ways to do this: Use compressed tar (Newer and Older) Use rsync Update (Dec 6, 2022) I wrote another post comparing the performance of various compression algorithm...| Manish R Jain
This blog post was on the front page of Hacker News for a day — Link It’s been almost nine months since I left Dgraph Labs, a company that I spent 6 years of my life building — after a layoff followed by a failed Series B round (unsurprisingly, and in that order). During this time, I’ve had a chance to reflect upon what happened, what went wrong, what went right, where did we succeed, and where did we fail — and most importantly, what I learned from this entire experienc...| Manish R Jain
I was recently looking for a working solution to parsing Smart Contracts in the EVM chains. I couldn’t find a simple to follow, working solution based on Go. This post aims to fill that gap. In this post, we’d parse Polygon bridge transactions. Let’s look at this particular transaction. This txn belonging to block 31740481 in Polygon L2 chain is interacting with a smart contract here. Get a Polygon Endpoint The first thing we want to do is to grab a JSON-RPC endpoint to Polygon. I got o...| Manish R Jain
Originally Published: July 13, 2022 Update (Sep 14, 2022): I’ve switched to using ZFS because it has checksums for every block — I can’t imagine| manishrjain.com
To allow Postgres connections from a remote server, or from within a docker container, you’d want to allow it to listen to external traffic. This| manishrjain.com
Today, after visiting VFS office to apply for my child’s Overseas Citizen of India document, I was turned back because I didn’t provide both the| manishrjain.com
As companies become bigger, the “shared non-responsibility” expands vastly — where no one in particular is fully responsible, because someone else| manishrjain.com
If you’re using Cloudflare to manage traffic to your origin server, you might want to restrict incoming traffic to only come from Cloudflare. This| manishrjain.com
We’re running Postgres for the production deployment of Struct, a (not-yet-launched) chat platform we’re building to deal with the disorganized| manishrjain.com
I recently witnessed a goroutine leak in my Go server. It looked like this. Searching online, the overwhelming consensus was that there must be a| manishrjain.com
Spent the last hour wrecking my head against an npm error, trying search after search with Google, leading nowhere. Finally copied over the issue to| manishrjain.com
Originally written: Sep 7, 2022 In /etc/security/limits.conf, add Log out, log back in again, and you should have: Original Source If you want to| manishrjain.com
Setting up Nginx with Let’s Encrypt is surprisingly easy, with certbot. I won’t go into details here. This tutorial does a good job:| manishrjain.com
I’m working on serving Polygon data via Outserv. As part of the process, I downloaded the first 15 million Polygon blocks and stored them as JSON.| manishrjain.com
This would search for ISO files in Downloads, and show you the USB stick that’s plugged in. You can then follow the interface to create a bootable| manishrjain.com
Decentralization or API Keys? Pick One| manishrjain.com
A few months ago, I was talking about an exciting new product with a friend. As I described the efficiencies this product could offer, my friend| manishrjain.com
I posted in the OpenSea discord channel that I need help with getting OpenSea API key so I can write an importer for Outserv. And I got pinged by| manishrjain.com
I’m using Postgres for my new startup. The startup is still in stealth mode, so I can’t talk about it much. But, given the focus of the startup is| manishrjain.com
Today, I’m excited to launch https://www.0xfast.com — A 10x Faster, 3x Cheaper Web3 API Platform, with a mission to “Make Web3 Experience Fast”.| manishrjain.com
In a previous post, I wrote about the best way to transfer a directory across the network. In this post, I compare Brotli v Gzip v Zstd v LZ4 on| manishrjain.com
In my setup, where I have an HTTPS load balancer, sending traffic over HTTP to the server, aka reverse proxy, Caddy outperforms NGINX by 4x. That’s| manishrjain.com
As you might have picked up, I run baremetal servers. They provide the best performance-to-cost ratio. A problem I’ve been trying to solve for some| manishrjain.com
I put together a build for a 2.5 Gbps server, which can run Nginx and act as a load balancer for my other servers. Here’s all the parts in| manishrjain.com
TL;DR: You can now pass log.parse=true as a URL parameter to get human-readable parsed event logs from 0xFast Stream. Ethereum logs are hard to| manishrjain.com
I’m excited to announce the launch of Indexed Blockchain Streams at 0xFast. What are indexed streams, you ask? Indexed Streams build an index on top| manishrjain.com
Ethereum logs always stump newcomers. A typical Ethereum log looks like this: Looking at this log, it all seems strange. Address, topics, data —| manishrjain.com
ETH: https://eth-uswest.0xfast.com/stream/free?range=1337 Status Page: https://status.0xfast.com As I’ve ventured into the web3 space, I’ve been| manishrjain.com
Original: Sep 14, 2022 In a previous post, I showed how to put together a cheap load balancer server for $500. I later had an idea to use that| manishrjain.com
Decentralization — the dispersion or distribution of functions and powers. — Merriam Webster Today, I’m excited to announce the launch of Outserv| manishrjain.com
