I’m something of a filesystem geek, I guess. I first wrote about ZFS on Linux 14 years ago, and even before I used ZFS, I had used ext2/3/4, jfs, reiserfs, xfs, and no doubt some others. I’ve also used btrfs. I last posted about it in 2014, when I noted it has some advantages over … Continue reading btrfs on a Raspberry Pi→| The Changelog
This is a complex tale I will attempt to make simple(ish). I’ve (re)learned more than I cared to about the details of pipes, signals, and certain system calls – and the solution is still elusive. For some time now, I have been using NNCP to back up my files. These backups are sent to my … Continue reading Pipes, deadlocks, and strace annoyingly fixing them→| The Changelog
In recent posts: Doing a bit of stress work on a new HDD x8dtu: adding in the smaller drive Swapping zpools – moving from using main_tank to using data If you have a look over those posts, you’ll see why you never want to downsize a zpool. In this post: FreeBSD 14.2 removing the zfs labels from an drive which was part of a zpool which is no longer in use destroying the […]| Dan Langille's Other Diary
As mentioned in Doing a bit of stress work on a new HDD, I have a failing 5TB drive which is going to be replace by a 4TB drive. Only about 1.45TB are used, so there’s plenty of space to grow. If you get one thing from this post, don’t be downsizing zpools like this. I would have had much less work and opportunity for error, if I had returned that 4TB drive […]| Dan Langille's Other Diary
I was up at 5:30 AM today. I packed the car and headed out. I arrived within the datacenter at about 8:15 or so. By 8:50, I was on IRC and the photos of the FreeBSD racks were uploading. Since I was going there anyway, I did some inventory and disposal work (a decommissioned server, about 25 old HDD, and various bits and pieces). I must say though, I’m not liking this option. […]| Dan Langille's Other Diary
As foreshadowed in x8dtu – drive problems, I will be visiting a data center soon to replace a 4TB HDD. The replacement HDD arrived last night. It was unceremoniously tossed onto the front porch by the courier. However, it was properly packaged. I’m sure it’s fine.| Dan Langille's Other Diary
A newly found security flaw in Ubuntu could allow attackers with physical access to bypass full disk encryption. Learn how the attack works.| OMG! Ubuntu
Let’s look at these two SSDs (full smartctl output appears at the end). pool: zroot state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired […]| Dan Langille's Other Diary
For a few days now, this drive in x8dtu has been having a hard time. I think it’s time to buy a replacement. Or two. [16:37 x8dtu dvl ~] % grep ada3 /var/log/messages Jul 4 15:19:59 x8dtu smartd[8217]: Device: /dev/ada3, 1 Currently unreadable (pending) sectors Jul 4 16:19:58 x8dtu smartd[8217]: Device: /dev/ada3, 1 Currently unreadable (pending) sectors Jul 4 17:19:59 x8dtu smartd[8217]: Device: /dev/ada3, 1 Currently unreadable (pending) sectors Jul 4 18:19:58 x8dtu […]| Dan Langille's Other Diary
This is a follow up to Adding in a hot-spare for zfs on FreeBSD from two months ago. The replacement for the returned drive has arrived and after sitting for many weeks on my coffee table, it is installed in r730-03. Here it is, as found in /var/log/messages – this host is a Dell R730 which has drive cages allowing me to insert the drive without powering off the host. Jun 30 16:17:15 […]| Dan Langille's Other Diary
Solve ZFS DKMS compilation failures on small VPS with memory errors. Step-by-step guide to fix gcc killed signal with swap solution.| Pieter Bakker
Couple of months ago I got my sister a MacBook Air with the new M4 chip, and she was happier than ever. However, I was the happiest, as she gave me her old MacBook Pro 2015.| Antranig Vartanian
Understanding ZFS-ZED Service Failures on NixOS| Cloud Alchemist
A Case Study in Scaling for High Volume CT Data| Transparency.dev Community Blog
A simple approach to prevent ZFS pools from getting stuck due to a lack of free space.| IT Notes
Hardware failures are inevitable, but ZFS fault management ensures your data stays protected with automated responses.| Klara Systems
Klara recommends performing routine servicing tasks on your lightly- to moderately-tasked Open Source FreeBSD NAS every 4 to 8 weeks.| Klara Systems
The slides, the video, and the text behind my presentation at EuroBSDCon 2024 - 'Why and how we're migrating many of our servers from Linux to the BSDs.| IT Notes
Introduction --- Update April 2023: It has been fairly quiet since the announcement of this feature. The Github PR about this feature is rather stale and people are wondering what the status is and what the plans are. Meanwhile, FreeBSD has announced In February 2023 that they suspect to integrate RAIDZ …| Louwrentius
One day, I lost two virtual machines on our DR environment after a storage vMotion. Further investigation uncovered that any storage vMotion of a virtual machine residing on our DR storage array would corrupt the virtual machine's disks. I could easily restore the affected virtual machines from backup and once …| Louwrentius
When building your own DIY home NAS, it is important that you simulate and test drive failures before you put your important data on it. It makes sense to know what to do in case a drive needs to be replaced. I also recommend putting a substantial amount of data …| Louwrentius
I think the HP Proliant Microserver Gen8 is a very interesting little box if you want to build your own ZFS-based NAS. The benchmarks I've performed seem to confirm this. The Microserver Gen8 has nice features such as: iLO (KVM over IP with dedicated network interface) support for ECC memory …| Louwrentius
I'd like to argue that both ZFS and BTRFS both are incomplete file systems with their own drawbacks and that it may still be a long way off before we have something truly great. Both ZFS and BTRFS are two heroic feats of engineering, created by people who are probably …| Louwrentius
On February 2011, I posted an article about my motivations why I did not use ZFS as a file system for my 18 TB NAS. You have to understand that at the time, I believe the arguments in the article were relevant, but much has changed since then, and I …| Louwrentius
If something goes wrong with my zpool, I'd like to be notified by email. On Linux using MDADM, the MDADM daemon took care of that. With the release of ZoL 0.6.3, a brand new 'ZFS Event Daemon' or ZED has been introduced. I could not find much information …| Louwrentius
This article is no longer relevant. After a fresh net-install of Debian Wheezy, I was unable to compile the ZFS for Linux kernel module. I've installed apt-get install build-essential but that wasn't enough. The apt-get install debian-zfs command would just hang. I noticed a 'configure' process and I killed it …| Louwrentius
In this blogpost I argue why it's strongly recommended to use ZFS with ECC memory when building a NAS. I would argue that if you do not use ECC memory, it's reasonable to also forgo on ZFS altogether and use any (legacy) file system that suits your needs. Why ZFS …| Louwrentius
--- Update 2014-8-23: I was testing with ashift for my new NAS. The ashift=9 write performance deteriorated from 1.1 GB/s to 830 MB/s with just 16 TB of data on the pool. Also I noticed that resilvering was very slow. This is why I decided to abandon …| Louwrentius
Here are some notes on creating a basic ZFS file system on Linux, using ZFS on Linux. I'm documenting the scenario where I just want to create a file system that can tollerate at least a single drive failure and can be shared over NFS. Identify the drives you want …| Louwrentius
ZFS is a modern file system designed by Sun Microsystems, targeted at enterprise environments. Many features of ZFS also appeal to home NAS builders and with good reason. But not all features are relevant or necessary for home use. I believe that most home users building their own NAS, are …| Louwrentius
I'm performing some FIO random read 4k I/O benchmarks on a ZFS file system. So since I didn't trust the numbers I got, I wanted to know how many of the IOPs I got were due to cache hits rather than disk hits. This is why I wrote a …| Louwrentius
I really like ZFS because with current data sets, I do believe that data corruption may start becoming an issue. The thing is that the license under which ZFS is released does not permit it to be used in the Linux kernel. That's quite unfortunate, but there is hope. There …| Louwrentius
Here's a post I just entered on the Nexenta/gnusolaris Beginners Forum that has some good info about ZFS. Apparently the formatting got eaten on the mailing list so I'm reposting it here:| Ops Monkey
(Mirroring this for posterity in case Svens ever forgets to pay his yahoo hosting bill :-) )| Ops Monkey
This week I've played around with Nexenta. This is a neat operating system. It is basically a "distribution" of OpenSolaris, using debian-like package management system, built on the Ubuntu "Dapper Drake" release. Fortunately, it just happened to support all of the pieces of hardware in the test system, a Supermicro H8DAR-T, with Broadcom ethernet and a Marvell88 based onboard SATA2 controller.| Ops Monkey
Run zrepl on TrueNAS in a way that survives reboots and OS updates| Alan Norbauer
First a little context: At Faradai we had been using PostgreSQL on the ext4 file system for a number of years until it recently became evident that the constantly growing database size (over 5 TB at the time) would start to cause issues in general performance, cloud costs, and the ability to have reliable backups. We reviewed a number of alternative file systems, and chose ZFS mainly for its following three features…| Lack of Imagination
Experience a journey from Kubernetes to FreeBSD, achieving major cost savings and efficiency through real-world optimization.| IT Notes
This article describes how I got a NixOS install working with multiple encrypted| astrid.tech
| The Grumpy Troll: The Grumpy Troll
Experience the love from iXsystems and Klara with the Valentine's gift to TrueNAS Community - Fast Dedup technology.| iXsystems, Inc. - Enterprise Storage & Servers
You probably want to take advantage of the data integrity checking offered by Btrfs. Btrfs calculates checksums for all data written to disk. These checksums are used to verify the data hasn’t been unduly altered. While data is verified every time it is read, what about the data that isn’t read often? How long may bit rot go unnoticed in that case? That’s the crux of this blog post which will explain how to best preserve your data on Btrfs and detect corruption early.| JWillikers
Background| Binary Impulse
You have probably tried Stack Exchange and reddit at this point, so what do you have to lose?| ./techtipsy
If you are using Proxmox, ZFS is one of the best local storage options. You can easily backup your lxc containers using Borg.| IT Notes