This is a new major release of marginalia search, mostly leaning toward the technical side. Emphasis has been on ensuring the search engine has the technical capabilities to serve more types of queries, especially longer queries which it previously did not handle very well. Effort has also been put toward making sure it’s possible to install and run outside of docker. There is still some work to be done to streamline the installation process, but we’re getting there.| www.marginalia.nu
This is a major new release of the search engine software, corresponding to nearly four months of changes. In these months, the state of the code hasn’t been stable enough for a new release, but it’s now been brought to a stable point. Release Highlights: The installation procedure has been cleaned up. It’s now possible to run the search engine in a white label/bare-bones mode, without any of the Marginalia Search branding or logic.| www.marginalia.nu on marginalia.nu
This is a mostly technical release. It takes the index from 106M to 164M documents. Zero Downtime Upgrades and halved memory consumption The initial focus of the release was to address the sometimes lengthy downtimes that have plagued the project when loading a new index. There is a somewhat lengthy write-up about this here; but the short version is that this was very successful and a drastic optimization, removed not only the needed downtime, but added neat new features and slashed the RAM r...| www.marginalia.nu
This release mainly aims to improve the operational side of the search engine, with an emphasis of automating tedious manual processes and optimizing crawling and data processing to use fewer resources. Conventionally I try to link to relevant commits in these notes, but some of the changes were so sweeping and protracted it was hard to narrow it down to individual commits; in those cases I’ll link to the relevant code instead.| www.marginalia.nu
New Features Generator keywords To provide additional ways of selecting search results, a synthetic keyword has been added for the tag. This is basically a vanity tag that is used by some HTML generators to advertise themselves, and it’s also common for hand-edited HTML to include this tag with a string like “vim” or “myself”, as a wink to human readers of the code. The generator keywords have the form generator:value.| www.marginalia.nu
This is primarily a bugfix release that primarily addresses some issues with a metadata corruption that was introduced in the previous release. New Features File keywords To provide more tools for navigating the web, the converter now generates synthetic keywords for documents that link to files on the same server based on their file ending. If the file contains a link such as Download then he document will be tagged with the keyword file:zip as well as file:archive.| www.marginalia.nu