If you've been working with Go for a while, you might be familiar with the Go proverb "don't panic". It's a pithy way of saying: "handle errors gracefully, or return them to the caller to handle gracefully, instead of passing errors to the built-in panic() function". And while "don't panic" is a great guideline that you should follow, sometimes it's taken to mean that you should no-way, never, ever call panic(). And I don't think that's true. The panic() function is a tool, and there are some...| Alex Edwards
One of my favourite features of Go 1.24 is the new functionality for managing developer tooling dependencies. By this, I mean tooling that you use to assist with development, testing, build, or deployment – such as staticcheck for static code analysis, govulncheck for vulnerability scanning, or air for live-reloading applications. Historically, managing these dependencies — especially in a team setting — has been tricky. The previous solutions have been to use a tools.go file or the go ...| Alex Edwards
When working with Go, you have three main building blocks to help organize your code: files, packages and modules. But as Go developers, one of the common challenges we have is knowing how to best combine these building blocks to structure a codebase. In this post, I'll share a mix of mindset tips and practical advice that I hope will help, especially if you're new to the language. Different projects, different structures Aim for effective, not perfect Forget conventions from other languages ...| Alex Edwards
In almost all web applications that I build, I end up needing to persist some data – either for a short period of time (such as caching the result of an expensive database query), or for the lifetime of the running application until it is restarted. When your application is a single binary running on a single machine, a simple, effective, and no-dependency way to do this is by persisting the data in memory using a mutex-protected map. And since generics was introduced in Go 1.18, it's possi...| Alex Edwards
In this post we're going to talk about how (and why!) different types of function parameters behave differently in Go. If you're new (or even not-so-new) to Go, this can be a common source of confusion and questions. Why do functions generally mutate maps and slices, but not other data types? Why isn't my slice being mutated when I append to it in a function? Why doesn't assigning a new value to a pointer parameter have any effect outside the function? Once you understand how functions and th...| Alex Edwards
Whenever I start a new Go project, one of the first things I do is create a Makefile in the root of my project directory. This Makefile serves two purposes. The first is to automate common admin tasks (like running tests, checking for vulnerabilities, pushing changes to a remote repository, and deploying to production), and the second is to provide short aliases for Go commands that are long or difficult to remember. I find that it's a simple way to save time and mental overhead, as well as h...| Alex Edwards
One of my favorite things about the recent Go 1.20 release is the new http.ResponseController type, which brings with it three nice benefits: You can now override your server-wide read and write deadlines on a per request basis. The pattern for using the http.Flusher and http.Hijacker interfaces is clearer and feels less hacky. No more type assertions necessary! It makes it easier and safer to create and use custom http.ResponseWriter implementations. The first two benefits are mentioned in t...| Alex Edwards
This tutorial is written for anyone who is new to Go. In it we'll explain what packages, import statements and modules are in Go, how they work and relate to each other and — hopefully — clear up any questions that you have. We'll start at a high level, then work down to the details later. There's quite a lot of content in this tutorial, so I've broken it down into the following eight sections: Packages The main package Importing and using standard library packages Unused and missing impo...| Alex Edwards
In this post we're going to run through how to use cookies in your Go web application to persist data between HTTP requests for a specific client. We'll start simple, and slowly build up a working application which covers the following topics: Basic reading and writing of cookies Encoding special characters and maximum length Using tamper-proof (signed) cookies Using confidential (encrypted) and tamper-proof cookies Storing custom data types in cookies If you just want the final code, rather ...| Alex Edwards
In this post I'd like to talk about one of my favorite architectural patterns for building web applications and APIs in Go. It's kind of a mix between the service object and fat model patterns — so I mentally refer to it as the 'fat service' pattern, but it might have a more formal name that I'm not aware of 🙃 It's certainly not a perfect pattern (we'll discuss some of the pros and cons later) — but it is (deliberately) simple, pragmatic, and I find it often works well for small-to-med...| Alex Edwards
Last year I wrote a new HTTP router for Go called Flow. I've been using it in production on this site and in a couple of other projects since, and I'm pretty happy with how it's working out so decided to share it a bit more widely. My aim with Flow was to bring together my favourite features from other popular routers that I frequently used. It has: A very small and readable codebase (approx. 160 LOC) with pattern-matching logic similar to matryer/way. Middleware management like chi — inclu...| Alex Edwards
When you're working on a project it's common for there to be some developer tooling dependencies. These aren't code dependencies, but rather tools that you run as part of the development, testing, build or deployment processes. For example, you might use golang.org/x/text/cmd/gotext in conjunction with go:generate to generate message catalogs for translation, or honnef.co/go/tools/cmd/staticcheck to perform static analysis on your code before committing a change. This raises a couple of inter...| Alex Edwards
Now that Go 1.18 has been released with support for generics, it's easier than ever to create helper functions for your test assertions. Using helpers for your test assertions can help to: Make your test functions clean and clear; Keep test failure messages consistent; And reduce the potential for errors in your code due to typos. To illustrate this, let's say that you have a simple greet() function that you want to test: package main import "fmt" func greet(name string) (string, int) { greet...| Alex Edwards
In this post we're going to walk through how to use GitHub Actions to create a continuous integration (CI) pipeline that automatically tests, vets and lints your Go code. For solo projects I usually create a pre-commit Git hook to carry out these kinds of checks, but for team projects or open-source work — where you don't have control over everyone's development environment — using a CI workflow is a great way to flag up potential problems and help catch bugs before they make it into prod...| Alex Edwards
In this short post we're going to discuss how to add, modify or delete URL query string parameters in Go. To illustrate, we'll look at how to change this URL: https://example.com?name=alice&age=28&gender=female To this: https://example.com?name=alice&age=29&occupation=carpenter If you want to change the URL query string in place: // Use url.Parse() to parse a string into a *url.URL type. If your URL is // already a url.URL type you can skip this step. urlA, err := url.Parse("https://example.c...| Alex Edwards
When you start to build web applications with Go, one of the first questions you'll probably ask is "which router should I use?". It's not an easy question to answer, either. You've got http.ServeMux in the Go standard library, and probably more than 100 different third-party routers also available — all with distinct APIs, features, and behaviors. Is http.ServeMux going to be sufficient? Or will you need to use a different router? And if so, which one is the right choice? For this blog pos...| Alex Edwards
Recently I've been building a fully internationalized (i18n) and localized (l10n) web application for the first time with Go's golang.org/x/text packages. I've found that the packages and tools that live under golang.org/x/text are really effective and well designed, although it's been a bit of a challenge to figure out how to put it all together in a real application. In this tutorial I want to explain how you can use golang.org/x/text packages to manage translations in your application. Spe...| Alex Edwards
When searching for examples of HTTP basic authentication with Go, every result I found unfortunately contained code which was either out-of-date (i.e. doesn't use the r.BasicAuth() functionality that was introduced in Go 1.4) or failed to mitigate the risk of timing attacks. So in this post, I'd like to discuss how to use it correctly in your Go applications. We'll start with a bit of background information, but if you're not interested in that you can skip straight to the code. What is basic...| Alex Edwards
One of my favorite things about the recent Go 1.16 release is a small — but very welcome — addition to the flag package: the flag.Func() function. This makes it much easier to define and use custom command-line flags in your application. For example, if you want to parse a flag like --pause=10s directly into a time.Duration type, or parse --urls="http://example.com http://example.org" directly into a []string slice, then previously you had two options. You could either create a custom typ...| Alex Edwards
This is a list of things about Go's encoding/json package which, over the years, have either confused or surprised me when I first encountered them. Many of these things are mentioned in the official package documentation if you read it carefully enough, so in theory they shouldn't come as a surprise. But a few of them aren't mentioned in the documentation at all — or at least, they aren't pointed out explicitly — and are worth being aware of! Map entries are sorted alphabetically Byte sl...| Alex Edwards
One of the great features of Go's database/sql package is that it's possible to cancel database queries while they are still running via a context.Context instance. On the face of it, usage of this functionality is quite straightforward (here's a basic example). But once you start digging into the details there's a lot a nuance and quite a few gotchas... especially if you are using this functionality in the context of a web application or API. So in this post I want to explain how to cancel d...| Alex Edwards
Let's say that you're building a JSON API with Go. And in some of the handlers — probably as part of a POST or PUT request — you want to read a JSON object from the request body and assign it to a struct in your code. After a bit of research, there's a good chance that you'll end up with some code that looks similar to the personCreate handler here: // File: main.go package main import ( "encoding/json" "fmt" "log" "net/http" ) type Person struct { Name string Age int } func personCreate(...| Alex Edwards
For the past few months I've been running a survey which asks people what they're finding difficult about learning Go. And something that keeps coming up in the responses is the concept of interfaces. I get that. Go was the first language I ever used that had interfaces, and I remember at the time that the whole concept felt pretty confusing. So in this tutorial I want to do a few things: Provide a plain-English explanation of what interfaces are; Explain why they are useful and how you might...| Alex Edwards
PostgreSQL provides two JSON-related data types that you can use — JSON and JSONB. The principal differences are: JSON stores an exact copy of the JSON input. JSONB stores a binary representation of the JSON input. This makes it slower to insert but faster to query. It may change the key order, and will remove whitespace and delete duplicate keys. JSONB also supports the ? (existence) and @> (containment) operators, whereas JSON doesn't. The PostgreSQL documentation recommends that you shou...| Alex Edwards
Occasionally I get asked “why do you like using Go?” And one of the things I often mention is the thoughtful tooling that exists alongside the language as part of the go command. There are some tools that I use everyday — like go fmt and go build — and others like go tool pprof that I only use to help solve a specific issue. But in all cases I appreciate the fact that they make managing and maintaining my projects easier. In this post I hope to provide a little background and context ...| Alex Edwards
Thanks to Andreas Auernhammer, author of the golang.org/x/crypto/argon2 package, for checking over this post before publication. If you're planning to store user passwords it's good practice (essential really) to hash them using a computationally expensive key-derivation function (KDF) like Bcrypt, Scrypt or Argon2. Hashing and verifying passwords in Go with Bcrypt and Scrypt is already easy to do thanks to the golang.org/x/crypto/bcrypt package and Matt Silverlock's elithrar/simple-scrypt pa...| Alex Edwards
For the past couple of years I've used Sublime Text as my primary code editor, along with the GoSublime plugin to provide some extra IDE-like features. But I've recently swapped GoSublime for a more modular plugin setup and have been really happy with the way it's worked out. Although it took a while to configure, it's resulted in a coding environment that feels clearer to use and more streamlined than before. I've opted for: Tooling integration with the official sublime-build plugin. Automat...| Alex Edwards
A.K.A. HTTP method overriding. As a web developer you probably already know that HTML forms only support the GET and POST HTTP methods. If you want to send a PUT, PATCH or DELETE request you need to resort to either sending a XMLHttpRequest from JavaScript (where they are supported by most major browsers) or implement a workaround in your server-side application code to support 'spoofed' HTTP methods. The de-facto workaround — which you might be familiar with if you've used frameworks like ...| Alex Edwards
Earlier this year AWS announced that their Lambda service would now be providing first-class support for the Go language, which is a great step forward for any gophers (like myself) who fancy experimenting with serverless technology. So in this post I'm going to talk through how to create a HTTPS API backed by AWS Lambda, building it up step-by-step. I found there to be quite a few gotchas in the process — especially if you're not familiar the AWS permissions system — and some rough edges...| Alex Edwards
A nice feature of Go's http.FileServer is that it automatically generates navigable directory listings, which look a bit like this: But for certain applications you might want to prevent this behavior and disable directory listings altogether. In this post I’m going to run through three different options for doing exactly that: Using index.html files Using middleware Using a custom filesystem Using index.html files Before http.FileServer generates a directory listing it checks for the exist...| Alex Edwards
There are a lot of good tutorials which talk about Go's sql.DB type and how to use it to execute SQL database queries and statements. But most of them gloss over the SetMaxOpenConns(), SetMaxIdleConns() and SetConnMaxLifetime() methods — which you can use to configure the behavior of sql.DB and alter its performance. In this post I'd like to explain exactly what these settings do and demonstrate the (positive and negative) impact that they can have. Open and idle connections I'll begin with...| Alex Edwards
If you're running a HTTP server and want to rate limit user requests, the go-to package to use is probably Tollbooth by Didip Kerabat. It's well maintained, has a good range of features and a clean and clear API. But if you want something simple and lightweight – or just want to learn – it's not too difficult to roll your own middleware to handle rate limiting. In this post I'll run through the essentials of how to do that by using the x/time/rate package, which provides a token bucket ra...| Alex Edwards
Over the past few years I've built up a collection of snippets for validating inputs in Go. There's nothing new or groundbreaking here, but hopefully they might save you some time. The snippets assume that the data to validate is stored as strings in r.Form, but the principles are the same no matter where the data has come from. Required inputs Blank text Min and max length (bytes) Min and max length (number of characters) Starts with, ends with and contains Matches regular expression pattern...| Alex Edwards
I’ve just released SCS, a session management package for Go 1.7+. Its design leverages Go’s new context package to automatically load and save session data via middleware. Importantly, it also provides the security features that you need when using server-side session stores (like straightforward session token regeneration) and supports both absolute and inactivity timeouts. The session data is safe for concurrent use. A simple example SCS is broken up into small single-purpose packages f...| Alex Edwards
In this post I'm going to be looking at using Redis as a data persistence layer for a Go application. We'll start by explaining a few of the essential concepts, and then build a working web application which highlights some techniques for using Redis in a concurrency-safe way. This post assumes a basic knowledge of Redis itself (and a working installation, if you want to follow along). If you haven't used Redis before, I highly recommend reading the Little Book of Redis by Karl Seguin or runn...| Alex Edwards
A few weeks ago someone created a thread on Reddit asking: In the context of a web application what would you consider a Go best practice for accessing the database in (HTTP or other) handlers? The replies it got were a genuinely interesting mix. Some people advised using dependency injection, a few favoured the simplicity of using global variables, others suggested putting the connection pool pointer into the request context. Me? I think the right answer depends on the project. What's the ov...| Alex Edwards
This is the first in a series of tutorials about persisting data in Go web applications. In this post we'll be looking at SQL databases. I'll explain the basics of the database/sql package, walk through building a working application, and explore a couple of options for cleanly structuring your code. Before we get started you'll need to go get one of the drivers for the database/sql package. In this post I'll be using Postgres and the excellent pq driver. But all the code in this tutorial is ...| Alex Edwards
I've written a package for chaining context-aware handlers in Go, called Stack. It was heavily inspired by Alice. What do you mean by 'context-aware'? If you're using a middleware pattern to process HTTP requests in Go, you may want to share some data or context between middleware handlers and your application handlers. For example you might want to: Use some middleware to create a CRSF token, and later render the token to a template in your application handler. Or perhaps... Authenticate a u...| Alex Edwards
When you're building a web application there's probably some shared functionality that you want to run for many (or even all) HTTP requests. You might want to log every request, gzip every response, or check a cache before doing some expensive processing. One way of organising this shared functionality is to set it up as middleware – self-contained code which independently acts on a request before or after your normal application handlers. In Go a common place to use middleware is between a...| Alex Edwards
Often in web applications you need to temporarily store data in-between requests, such as an error or success message during the Post-Redirect-Get process for a form submission. Frameworks such as Rails and Django have the concept of transient single-use flash messages to help with this. In this post I'm going to look at a way to create your own cookie-based flash messages in Go. We'll start by creating a directory for the project, along with a flash.go file for our code and a main.go file fo...| Alex Edwards
In this post I want to outline a sensible pattern that you can use for validating and processing HTML forms in Go web applications. Over the years I've tried out a number of different approaches, but this is the basic pattern that I always keep coming back to. It's clear and uncomplicated, but also flexible and extensible enough to work well in a wide variety of projects and scenarios.To illustrate the pattern, I'll run through the start-to-finish build of a simple online contact form. So let...| Alex Edwards
Taking inspiration from the Rails layouts and rendering guide, I thought it'd be a nice idea to build a snippet collection illustrating some common HTTP responses for Go web applications. Sending Headers Only Rendering Plain Text Rendering JSON Rendering XML Serving a File Rendering a HTML Template Rendering a HTML Template to a String Using Layouts and Nested Templates Sending Headers Only File: main.go package main import ( "net/http" ) func main() { http.HandleFunc("/", foo) http.ListenAnd...| Alex Edwards
For anyone new to building web applications with Go, it's important to realise that all incoming HTTP requests are served in their own Goroutine. This means that any code in or called by your application handlers will be running concurrently, and there is a risk of race conditions occurring. In case you're new to concurrent programming, I'll quickly explain the problem. Race conditions occur when two or more Goroutines try to use a piece of shared data at the same time, but the result of thei...| Alex Edwards
I wrote a short Bash script to automatically reload Go programs. The script acts as a light wrapper around go run, stopping and restarting it whenever a .go file in your current directory or $GOPATH/src folder is saved. I've been using it mainly when developing web applications, in the same way that I use Shotgun or Guard when working with Ruby. You can grab this from the Github repository. File: go-reload #!/bin/bash # Watch all *.go files in the specified directory # Call the restart functi...| Alex Edwards
Processing HTTP requests with Go is primarily about two things: handlers and servemuxes. If you’re coming from an MVC-background, you can think of handlers as being a bit like controllers. Generally speaking, they're responsible for carrying out your application logic and writing response headers and bodies. Whereas a servemux (also known as a router) stores a mapping between the predefined URL paths for your application and the corresponding handlers. Usually you have one servemux for your...| Alex Edwards
I've recently moved the site you're reading right now from a Sinatra/Ruby application to an (almost) static site served by Go. So while it's fresh in my head, here's an explanation of principles behind creating and serving static sites with Go. Let's begin with a simple but real-world example: serving vanilla HTML and CSS files from a particular location on disk. Start by creating a directory to hold the project: $ mkdir static-site $ cd static-site And then add a main.go file to hold our cod...| Alex Edwards
When I'm building a web application in Go, I prefer to use command-line flags to pass configuration settings to the application at runtime. But sometimes, the client I'm working with wants to use environment variables to store configuration settings, or the nature of the project means that storing settings in a TOML, YAML or JSON file is a better fit. And of course that's OK — it makes sense to be flexible and vary how configuration is managed based on the specific needs of a project and/or...| www.alexedwards.net
For many years, I've used third-party packages to help organize and manage middleware in my Go web applications. In small projects, I often used alice to create middleware 'chains' that I could reuse across multiple routes. And for larger applications, with lots of middleware and routes, I typically used a router like chi or flow to create nested route 'groups' with per-group middleware. | www.alexedwards.net