Recording /etc/ config in git is a recommended way to track history and revert breaking changes. In many cases, content needs to be imported and transformed from other repositories before being deployed as /etc/ config on a VM . Config changes to /etc are often made in haste during development & in emergencies. While git is helpful in recording those changes locally, often content needs to be managed on one machine and pushed to another. Or content can be in an outside repo with a different d...| Tony Metzidis
Client CLIs & SDKs for GCP, Firebase and other clouds are terribly bloated. GCP includes a python distro, firebase includes node+npm. This goes unnoticed on overpowered devboxes, but impacts your cloud bill with storage, vcpu, wall-time and transfer fees. If you are trying to downsize your VMs, you will find that the client SDK/ CLI pre-requisites will often hang your machine terminal by exausting vcpu and iops budgets. Cloud container services are often storage-limited to ram-disks–so CLI ...| Tony Metzidis
WSL2 provides great disk performance, but it requires storing the files separately in a virtual disk that is not accessible by OneDrive. WSL2 can be backed up with wsl –export Debian to a VHD or TGZ, but that is a complete disk backup of 20gb or more – not scalable for hourly backups. With this approach, we use Windows Task Scheduler to trigger robocopy to incrementally sync directories from WSL2 to Onedrive’s native FS, so incremental copies are fast ( 1 s per 10k files), and OneDrive ...| Tony Metzidis
With the news that AWS will be now charging about $4 / instance-month for public IPv4 addresses, many developers who procrastinated ipv6 migration are finally updating both ends of their development setup. It’s a great time to migrate, as all the intermediate infrastructure now supports IPV6 readily. Moreover, you’ll benefit from permanent , global addresses for your development instances. Pros A single, global, stable address for EC2 instances that never changes. No need for dynamic DNS ...| Tony Metzidis
I recently set up a new SER6 and reviewed bloatware / spyware / malware prior to connecting to the internet. There were quite a few posts asking about spyware, and given it’s competitive price point, I was also a bit suspicious. I usually do a malware review before connecting any new device to the internet. What I reviewed running processes and their signatures startup apps (Task Manager formerly msconfig) Installed Services Windows Features Enabled Partition Table Review (for malware) Loca...| Tony Metzidis
By default, all Windows drives are mounted with read & write access (rw) within WSL . Though this is convenient for beginners, it opens up VM shell attacks on your Windows host files. Instead, we can disable the auto mount feature using wsl.conf and selectively add read-only drives inside the WSL VM using /etc/fstab Overview Deactivate “auto mount” in /etc/wsl.conf Enable fstab using MOUNTfStAB = true in wsl.conf test config files and mounting work well reboot the wsl VM to complete the s...| Tony Metzidis
Smokeping is a self-contained network monitoring app , capable of monitoring using ICMP/Ping, HTTP, DNS – as well as other signals generated from CLI monitoring tools (e.g. curl, dig, mtr etc). It provides a web-based monitoring UI to chart the probe measurements so no further monitoring apps (like Prometheus) are needed. Running smokeping on a $5 Raspberry Pi Zero is a fun experiment in lightweight computing . Using Apache Mod FastCGI makes the app usable on the meager hardware.| Tony Metzidis
Every app has that last inch (or mile) of code that’s not covered by tests. Usually it’s an interactive cycle of compile-run-inspect on the command line like You Test curl -X POST https://reqbin.com/echo/post/json ##👀 You Expect: {"success":"true"} Despite having 3-4 testing frameworks for unit tests, e2e, regression etc– there’s always a gap where you find yourself re-playing commands in the terminal to test. A common case is 🔥firefighting where ad-hoc tests are needed to valid...| Tony Metzidis
Recently an old friend, with great experience as an IC, PM and EM, called me to ask for some advice. He had been running his business for a while and took up a new role as an engineering manager after some time. “What areas do you focus on as an EM?, particularly when joining a new team”. I divided the conversation into three pillars: strategy & inventory, technical (aka going deep) and career / personal| Tony Metzidis
One responsibility of engineers & especially leads is managing many channels of signals : emails, blog posts (internal and external), tags , push notifications, group chats, alerts from dashboards and more. These signals tend to scale exponentially to the number of projects & people that you are responsible for. Quickly you’ll need to set up a system to make sure that you are receiving high-signal information and filtering out low-signal noise. How do you do that?| Tony Metzidis
Directory layouts are like log cabins that start from a basic shed, gradually adding a room at a time. When you start out on UNIX, everything gets thrown in your home directory. Over time you start to develop a structure for your sources, binaries, projects, data files (like CSV, images, tar files), config, etc My layout is called TDL – because it allows me to juggle open source projects, partnerships and jobs in a consistent structure across machines and time.| Tony Metzidis
Cloud instances bill by the hour (or the minute) – and right now you’re burning money. Use snooze to auto-shutdown your instances in 45 minutes. Add snooze to your ~/.bashrc alias snooze='sudo shutdown -c ; sudo shutdown -h +45 &' snooze When you want to extend your session, run snooze Broadcast message from ec2-user@ip-172-31-43-250 (/dev/pts/1) at 2:50 ... The system is going down for halt in 45 minutes! How does this work? shutdown -c cancels the shutdown, and shutdown -h +45 schedules...| Tony Metzidis
I work from about 7 different machines, including 3 laptops, ipad, chromebook and a PC desktop. Usually this means keeping credentials, config, build dependencies and IDEs in sync across all 3–and the iPad & Chromebook just can’t run my dev environment I considered a few options to enable seamless work across devices optionproscons Keep a “dev” docker image that contains everything.fully-local devonly works on Desktop OSs. Inconsistency if you forget to push the image Sync scriptfully...| Tony Metzidis
📓 The Gist You may think your 100mB Alpine images are small–but how about 100kB? Smaller images ship more quickly, and contain fewer attack vectors. Moreover, by optimizing images, you discover and isolate exactly what is needed for your app to run. Let’s Optimize. There are two key characteristics of scratch-based docker images: The Dockerfile has two build stages: a builder–which contains all of the build dependencies including source, libraries and tools and.. a final image, conta...| Tony Metzidis
In Part 1, we completed our development environment, including setting up Hugo and our repo. Here we’ll publish our site to Firebase Hosting, and create the CI tools on Google Cloud Build to build and publish upon push. Open Your Cloud Shell In Part 1, we enhanced our cloud shell with hugo and set up our repo. In Part two, we’ll use it to create the builder and configure hosting. See the Quickstart for complete instructions| Tony Metzidis
Here’s a benchmark comparing pi-hole running on a Pi Zero (with USB ethernet) vs a Pi 3b+. tl;dr There was negligible performance difference for blocked domains, but a measurable difference in mean for forwarded + cacheable domains. Although the Pi 3b+ has a 11ms better mean response time for forwarded queries, the P95 for pi zero is better in both blocked and forwarded queries. I would recommend using the Pi Zero.| Tony Metzidis
Here’s the fastest way to get your PHP app running. No MAMP, WAMP, apache or any of that nonsense. Moreover, it allows you to run multiple projects independently. I’m assuming you have docker. tl;dr This runs the php docker image, mounts the current directory, and spins up a server on port 8086 $ docker run -v $(pwd):/www -it -p8086:8086 php:5.6-alpine sh -c "cd www; php -S 0.0.0.0:8086" The Full Version Create your index.php $ cat > index.php <html><body><h1><?php print("Hello World!") ?...| Tony Metzidis
Next time you create a post with code snippets–like here on dev.to or stackoverflow–consider sharing a working and buildable gist along with it. By doing so, others can clone, reproduce your results, and commit new variants much more easily. With the process below, your gist becomes a sharable laboratory. Since the gist contains all of the code variants and test cases, any team member can create a variant and run the tests against all existing variants.| Tony Metzidis
There was a great discussion a year ago about how fast gnu’s version of “yes” is. If you’re unfamiliar, yes outputs y indefinitely. yes |head -5 y y y y y The key takeaway was that write is expensive and writing page-aligned buffers is much faster. The is true across languages, so let’s see how to do it properly in go. If you’re shocked or impressed by the results, let’s see you do it in your language – post your results in the comments.| Tony Metzidis
GCP resources are organized into projects – all resource IDs and IAM principles are grouped under a project ID. This means that by default roles assigned to a principle (e.g. a user or service account) are scoped only to project resources. This can be tricky if say your images are in one project’s storage bucket and your app is running in another If you want to provide a service principle in one project access to resources in another , the approach is not obvious, nor is it well documented.| Tony Metzidis
Static site frameworks like Hugo allow you to manage content with Markdown and publish content via scalable hosting platforms like Firebase hosting. Uptime, performance and operations cost per user can’t be beat – you can easily hit millions of pageviews for less than $10/ month In this tutorial we’ll make a production-ready personal website site, that supports multiple collaborators, built using Hugo. Moreover, we’ll publish with the free-to-start Firebase Hosting CDN, and build auto...| Tony Metzidis
Instrumentation is a critical part of any application. Along with system counters like cpu, heap, free disk, etc– it’s important to create application-level metrics to make sure health is measured closer to your customer’s experience. Example metrics could be user-registration, password-change, profile-change, etc. If you see a major spike or dip in these metrics, a wider problem could be indicated. For this example a custom metric was needed, and no infrastructure was in place for harv...| Tony Metzidis
Blink security cameras are an affordable home security camera system. Although they lack a formal public API, inventive devs have reverse-engineered their private API to allow for better integration. Here we’ll use AWS IOT Core, Lambda and node-blink-security to arm and disarm Blink security cameras using an AWS IOT Button. Activating Your IOT Button The IOT Button must be configured to your account, which includes joining it to your wifi access point, and installing the client certificates.| Tony Metzidis
Like me, you’re probably more comfortable on a CLI. Here’s a quick way to use docker to set up a Bitcoin Wallet and trade Bitcoin for free on Testnet with Electrum. You can use the same tools to manage your real Bitcoin wallet too. Setup Make sure you have Docker for your OS ( Mac, Windows, Linux) Run the electrum-cli docker image Electrum is a python-based Docker wallet with a both a gui and good cli. I’ve put together electrum-cli, a lightweight Alpine-linux Docker image with Electrum...| Tony Metzidis
Usually setting up the build dependencies is a major part of each build job. Thankfully, Atlassian’s Bitbucket Pipelines, the new CI platform that integrates into Bitbucket, supports custom docker images. To configure the build pipeline, you create bitbucket-pipeline.yml . This one uses our custom image (built below) and triggers builds whenever a releases-* tag is pushed. image: tonymet/tonym.us:latest pipelines: tags: release-*: - step: script: - make sync_down_images - make s3_upload Tha...| Tony Metzidis
A common migration pattern when moving to docker includes running some systems (e.g. dev, staging or a prod canary) on your docker image while the production app is still running your traditional tgz artifacts (e.g. your node app with node_modules) Let’s create a travis build that creates two artifacts: (1) your docker image and (2) a tgz from the docker container. Let’s assume you have a basic dockerfile with your app.js and a package.json. The key is that the app is built into /usr/src/app| Tony Metzidis
Google App Script is a little-known, yet powerful development platform for enhancing and automating google services. I use it for administration and building custom tools. Here are some things I’ve used it for a web app that scans emails for certain patterns and puts the results in email index email into a sql db to build charts & reports (e.g. 7d volume, top senders) automate account settings changes & cleanup bulk migration of email between accounts or from shared accounts to groups vario...| Tony Metzidis
2016 may be the year of free SSL, and AWS ACM (AWS Certificate Manager) is a great offering for Cloudfront & ELB users (most web apps). Not only is it free, but it’s also the simplest certificate management platform request a new certificate in minutes no server config needed no certificate , chain or private key management automatic certificate rotation Here’s how to create a certificate and then install it onto your cloudfront distribution.| Tony Metzidis
Varnish is an amazing platform – it can easily help you handle 100x traffic and is easy to add to your existing frontend or API layer with little to no change to your app. Here we’ll go over some neat tricks leveraging chef, the AWS Opsworks API and the opsworks configure lifecycle event to create a lighting fast load balancer & reverse proxy that automatically updates itself. Setup Create a new varnish layer that installs the varnish and jq packages| Tony Metzidis
AWS announced recently that it’s EC2 Container Registry (ECR) is now available. ECR simplifies hosting private images. Previously, you had to manually push your docker.io credentials to each EC2 instance – likely a deliberate pain-point encouraging you to use ECR. With ECR, EC2 container hosts can easily fetch private images using IAM authentication. Here are some of the gotchyas and stumbling blocks to help you get your repository up quickly and painlessly.| Tony Metzidis
We all know that no ports should be open to the internet for development purposes, but for convenience it’s common to find a security group with port 22 (SSH) open to 0.0.0.0/0 . Even narrower ingress rules can create backdoors. Here we’ll show you how to create an auto-updating security group that adds your active WAN IP address when you connect. This way, only your active IP is authorized. Create the “development” security group with no ingress aws ec2 create-security-group --group-...| Tony Metzidis
It’s hard to strike the right balance with admin rights–either the rights are too strict and people can’t get work done or they’re too lenient and you have security issues. As a compromise, AWS provides the AssumeRole feature which lets admins temporarily escalate their role to perform a task. It’s important when setting this up that you alert the team when it’s used. Here we’ll talk about how to set up the roles, give teams access to the roles and create an alert system when th...| Tony Metzidis
Often your creative team will produce master videos in 4k or 1080p, but you need to downcode these videos into 720p/1080p for web broadcasting. Here we automate transcoding of masters into web-friendly formats like 720p h264 mp4 & webm. AWS Elastic Transcoder is a cloud video transcoding service. At it’s simplest it transcodes video files from one bitrate, framerate, codec, container, etc–into another. By default you trigger new jobs either manually in the aws console or via the rest API....| Tony Metzidis
By default Wordpress uses it’s own pseudo-cron which triggers with every request. Obviously this is wasteful since (a) the queue needs to be inspected with every GET and (b) jobs like publishing articles will interfere with serving content. Some suggest calling the wp-cron.php GET request with curl in a cron like this * * * * * curl http://www.mysite.com/wp-cron.php but that’s sub-optimal since it needlessly ties up a worker during the cron execution.| Tony Metzidis
Redirects can account for a significant share of direct traffic so taking a few minutes to optimize them is worthwhile. Using Cloudfront & S3 for redirects will improve responsiveness, reduce server load and improve management (since they are managed via aws-cli or the console). Let’s say you have a typical .htaccess redirect like this. RewriteEngine On ### re-direct to www RewriteCond %{HTTP_HOST} !^www\. [NC] RewriteRule ^(.*)$ http://www.%{HTTP_HOST}%{REQUEST_URI} [R=301,L] Or worse, it ...| Tony Metzidis
Continuing the relentless quest to keep my phone speedy, I stumbled upon a developer tool that is useful to anyone needing to troubleshoot a slow, short-living or overheating phone – Battery Historian Battery Historian shows you a much more detailed and informative battery stats chart, highlighting the individual apps and sync services which are keeping your phone awake/busy in the background. It also shows network, wifi status, gps and more.| Tony Metzidis
I have a frustrating relationship with my phone’s performance. I can cleanup my phone for a few days, but it tends to revert to being sluggish within no time. I’ve had dozens of devices and they all suffer from this. There’s a lot of voodoo about Android Performance and Battery life–task managers, factory resets, etc. Here’s a more developer-oriented process using adb . Using ADB to identify process hogs By connecting your device to the Android SDK, you can use ADB to identify proce...| Tony Metzidis
I was desperate to try Hangouts 4.0 for Android, but suspicious of side-loading. I wanted to verify the APK signature cert had Google’s fingerprint of 38:91:8A:45:3D:07:19:93:54:F8:B1:9A:F0:5E:C6:56:2C:ED:57:88 Here’s how to check the signatures on an APK, as usual, in shell functions (JDK needed) apk-check () { jarsigner -verify -verbose -certs $1 } apk-print-cert () { keytool -list -printcert -jarfile $1 } # usage # make sure it's verified $ apk-check *apk|grep verified s = signature wa...| Tony Metzidis
Here’s a great example of using the aws-cli to speed up your life. Uses jq and aws-cli # bash / zsh function function opsworks-hosts-prod () { aws opsworks describe-instances --stack-id=fffff-fffff-ffff-fff-fffffff | jq '.Instances[].PublicDns' | grep -v null | sed s/\"//g } # usage $ opsworks-hosts-prod XXXXX.compute-1.amazonaws.com XXXXX.compute-1.amazonaws.com XXXXX.compute-1.amazonaws.com XXXXX.compute-1.amazonaws.com| Tony Metzidis
I was chatting with a buddy who was moving his web sites from dedicated hosting to AWS. Let’s just say the FTUE isn’t great. That triggered a quick brain-dump of what you should do when you first get started with AWS. understand pets v cattle. In aws all resources should be “cattle”, not pets. Periodically terminate instances to test this. activate cloudtrail (in all regions). Use Loggly to index cloudtrail (free or ~$20/mo) create restricted IAM users. Never use your root acct. Activ...| Tony Metzidis
A new lightrail line is being built in my city with bridges passing over the major boulevards. Seeing the elaborate scaffolding evoked comparisons to software engineering. What does scaffolding look like in software? Does software need to be erected like a bridge via scaffolding? Without a doubt: yes. Here are some elements of software “scaffolding”: Error log instrumentation with a formal error log schema (i.e. errors are uniquely identifiable in a MECE schema) Operational instrume...| Tony Metzidis
Opsworks is a convenient, powerful and free service provided by AWS to simplify the management of EC2 nodes. The real power of the system is exposed through customizing various stages of the instance lifecycle by creating custom-tailored chef-solo recipes. While Amazon provides a powerful deployment layer for PHP applications, it stops short once the PHP code has been checked out of git. For Laravel or other composer apps, you’ll have to customize your deployment. The most elegant and...| Tony Metzidis
Teams should consider many qualities when choosing a cloud provider like AWS, GCP (Google Cloud Platform) or Microsoft Azure. Product offerings, familiarity, pricing, and usability – among others. Compared to AWS, Google Cloud Platform (GCP) is more usable for developers due to it’s core design approach surrounding resources, projects, APIs and Identities (IAM). This project-first approach avoids common bad practices like spaghetti namespaces, excessive permissions, and accidental exposur...| Tony Metzidis