Sometimes we want better-than-firewall security for things. For instance:| changelog.complete.org
Sometimes we want better-than-firewall security for things. For instance: An industrial control system for a municipal water-treatment plant should never have data come in or out Or, a variant of the industrial control system: it should only permit telemetry and monitoring data out, and nothing else in or out A system dedicated to keeping your GPG private keys secure should only have material to sign (or decrypt) come in, and signatures (or decrypted data) go out A system keeping your tax rec...| www.complete.org
Like the process described in Encrypting Filespooler Jobs with GPG, Filespooler can handle packets Encrypted with Age (Encryption). Age may be easier than GnuPG in a number of cases, particularly because it can use a person’s existing SSH keypairs for encryption. This process is fundamentally quite similar to handling encryption with GnuPG: we’ll be adding an encryption command in the pipe after prepare, and adding a decoder to the queue processing commands.| www.complete.org
Purpose of Append-Only Queues In the Using Filespooler over Syncthing example, we synced the entire b64queue directory. This is unnecessary. It would be better to sync only the jobs subdirectory of it, to prevent it looking like a valid queue for processing on the sender. Implications of Append-Only Queues The Filespooler Reference discusses append-only queues. An append-only queue will allow commands that append (fspl queue-write) and commands that inspect (for instance, fspl queue-ls) to su...| www.complete.org
Filespooler is designed to work well in automated situations, including when started from cron or systemd. It is a fairly standard program in that way. I’ll discuss a few thoughts here that may help you architect your system. Generally, the command you would schedule is fspl queue-process so that’s the assumption here. Output By default, Filespooler produces no output itself on success. The commands it executes may, and Filespooler may also when error conditions occur.| www.complete.org
This page is intended to describe how to run Debian’s backports on a Raspberry Pi running Raspberry Pi OS (Raspbian). This page is not relevant if you are directly running Debian on a Raspberry Pi. It is only for those running the default Raspberry Pi OS. Backports is Debian’s way of building newer packages for its stable releases. I intend this page specifically to help people run the Debian packages for NNCP and Yggdrasil, both of which are maintained by me, John Goerzen.| www.complete.org
This page gives you references to software by John Goerzen. I once tried to list every software project I made significant contributions to on this page. By January 2006, I had more than 40 items listed on this page, and it was missing quite a few. So instead of trying to provide a full list here, I will instead try to provide you with links to find my software yourself.| www.complete.org
Age is a public-key encryption system, similar in certain concepts to GnuPG (GPG), but simpler because it isn’t focused on building a web of trust. Homepage: https://github.com/FiloSottile/age Conveniently, Age can use not just its own keypairs, but also a person’s SSH keypairs for encryption and decryption. This is nice because many systems already have authenticated SSH keypairs for each other. Links to this note Encrypting Filespooler Jobs With Age Like the process described in Encrypt...| www.complete.org
In my writing about dar, I recently made that point that dar is a filesystem differ and patcher. We can exploit this property to do something really cool: build an Asynchronous rsync. What does that mean? rsync is a tool that has been in may *nix admins’ toolboxes for years. Typically used over ssh, rsync will compare the state of a local directory tree (or file) to the state of a remote tree, and efficiently make the remote match the local (or vice-versa).| www.complete.org
I loaded up this title with buzzwords. The basic idea is that IM systems shouldn’t have to only use the Internet. Why not let them be carried across LoRa radios, USB sticks, local Wifi networks, and yes, the Internet? I’ll first discuss how, and then why. How do set it up I’ve talked about most of the pieces here already: Delta Chat, which is an IM app that uses mail servers (SMTP and IMAP) as transport, and OpenPGP encryption for security.| www.complete.org
dar is a Backup and archiving tool. You can think of it as as more modern tar. It supports both streaming and random-access modes, supports correct incrementals (unlike GNU tar’s incremental mode), Encryption, various forms of compression, even integrated rdiff deltas. It is an ideal tool for backups over Asynchronous Communication for systems that don’t or can’t run ZFS. Homepage: http://dar.linux.free.fr/ I have written a lot about Dar: I first wrote about Dar in my 2008 roundup of ba...| www.complete.org
Sometimes with Filespooler, you may wish for your queue processing to effectively re-queue your jobs into other queues. Examples may be: Parallel processing Distributing jobs to various workers There are two approaches to this: Directly reusing the job file in the destination. To do this, you would hard link the job file into the destination queue’s jobs directory. See Parallel Processing of Filespooler Queues for an example of this approach. Alternatively, you could pipe data to fspl prepa...| www.complete.org
A distributed version control system. With gitsync-nncp, can be used over Asynchronous Communication. Links to this note Gitsync-Nncp gitsync-nncp is a tool for using Asynchronous Communication tools such as NNCP or Filespooler, or even (with some more work) Syncthing to synchronize git repositories. Many-to-One With Filespooler Since Filespooler is an ordered queue processor by default, it normally insists on a tight mapping between the sequence numbers in job files and execution order in a ...| www.complete.org
gitsync-nncp is a tool for using Asynchronous Communication tools such as NNCP or Filespooler, or even (with some more work) Syncthing to synchronize git repositories. Homepage: https://github.com/jgoerzen/gitsync-nncp The homepage contains information on using gitsync-nncp over NNCP. You can also see: gitsync-nncp over Filespooler Links to this note John Goerzen’s Software This page gives you references to software by John Goerzen. Introduction to Filespooler It seems that lately I’ve wr...| www.complete.org
GnuPG (also known by its command name, gpg) is a tool primarily for public key Encryption and cryptographic authentication. Homepage: https://www.gnupg.org There is a lot of material about GnuPG out there, so I won’t be duplicating it all here. But, I do have some more unique contributions for those familiar with GnuPG: There are alternatives to gpg/gpgv that may be useful in some cases: Age (Encryption) for encryption and signify for authentication.| www.complete.org
Keeping your data safe in the event of a disaster or compromise is important. That’s why we back up. Here is some information on backups: Backing up every few minutes with simplesnap on ZFS For hosting backups offsite, my 2021 Roundup of Unique Data/Storage Hosting Options can be useful. How and Why to use Airgapped Backups (see also Airgap) Links to this note Dar dar is a Backup and archiving tool.| www.complete.org
“OK,” you’re probably thinking. “John, you talk a lot about things like Gopher and personal radios, and now you want to talk about building a reliable network out of… USB drives?” Well, yes. In fact, I’ve already done it. What is sneakernet? Normally, “sneakernet” is a sort of tongue-in-cheek reference to using disconnected storage to transport data or messages. By “disconnect storage” I mean anything like CD-ROMs, hard drives, SD cards, USB drives, and so forth.| www.complete.org