development-notes

Pre-history

This site was a Ghost blog, migrated from a Wordpress site. Given the small amount of content, as well as the interminable maintenance needed to put some words on a webpage using a Modern™ Content Management Tool™, I struck out for some kind of static site generator.


What was I looking for:


In the end, I wrote my own site generator in Factor, which is not the simplest approach, but I had fun building it and it meets my needs exactly.

First glimmerings: Pollen

A colleague pointed me to Pollen—The Book is a Program, and I was enrapt. I'd used https://www.sphinx-doc.org/en/master/ to generate documentation, and both ReStructuredText and the constructs needed to wrangle a book were too complex for my taste.


Pollen allows you to drop into a real programming language at any point, and has several nice conventions making it easy to publish a book-like document to the web or as a PDF.


One hurdle: Pollen is written in Racket, which is a fancy language in the Scheme family.


Undeterred, I wrote a converter in Racket to extract my posts from Ghost's SQLite database:


While Pollen is an elegant tool, I had a really hard time doing the things I wanted on my site, like "make a page that lists everything tagged with this tag."


Organize and curate your words, not your blog software

Pollen forced me to reevaluate my content. Did I really need posts going back 10 years that were no longer relevant? Instead of trying to fix every post, I decided to curate my content.


Despite the emergence of blogging, not everything needs to be a blog. My articles can be read in any order—so why force them into a date-based hierarchy? Instead, I grouped them into a few topics: Factor, programming, system administration.


I edited past entries & threw out writing that was no longer relevant. Moving from one CMS to another (for example, from Wordpress Ghost) my default a mindset was "everything that I wrote should be kept around forever no matter what."


"Retain everything" is an easy trap to fall into, leading to export, import, export, import ad nauseum between blogging systems without ever considering if the posts written are worth keeping. Trimming old & irrelevant content made it easier to contemplate building an entirely new site.


Reduce complexity, increase speed


To my dismay, Pollen was slow. Even on my small site (I've got about 40 pages, none more than a couple thousand words) it could take up to 10+ seconds to build. This may have been my misapplication of the tool, but I also have a short attention span. Slow will not do—that's why I avoided Jekyll and similar tools. (Hugo was also out, because it's impenetrable and I couldn't build a simple site within a couple hours of tinkering).


That said, a document system should be able to reflexively examine the content and generate hierarchies, etc. Pollen can do this, but I have no sense of how to extend it in Racket along the lines I had in mind.


I still love the idea of "the book is a program", but I disagree that Lisp is the most elegant way to attack this problem. The family of languages that spark joy for me are the concatenative languages—Forth & Factor in particular—which diverged from Lisp very early in the history of programming. Yet despite different evolutionary paths, Forth-like languages retain a Lisp-like expressiveness and conciseness.


Factor programs are, after all, words separated by whitespace. Articles are also words separated by whitespace, however, prose shouldn't as a rule be executed as a program.


Enter the snail


Just benchmarked the site build:

IN: listener: build-fred-site

Running time: 1.178203043 seconds


Which is just right. Snail takes a simplistic view of the world, and considers its job to be primarily copying files and sometimes rendering a template. Written in Factor, it lends itself towards interactive development and debugging.


Snail uses very simple include|exclude globs to filter down a tree of pages to render. When you build the site, it writes all output files into build/.


For each file:


It's also smart & recreates the directory tree so files get copied like:


Snail doesn't do "templates", per sé. Instead, there is a "shell" for each file extension. Shells wrap around a page, and are ideal for wrapping page content with pretty HTML + CSS. Shells only apply to .f files. Here's how Snail finds the shell for a given document:


  1. Examine the current path for ◊shell.html.f.
  2. If ◊shell.html.f exists, render the contents of about.html wherever the ◊yield word is used in the shell.
  3. Otherwise, go up one level and look for ◊shell.html.f, up to the root of the site.

This meets three simple goals: first, to make it easy to style a given file type. Second, there is no need to copy the template to multiple places, nor specify in any way. Third, it is completely optional: the default "shell" is an empty file that only uses the ◊yield word; that is, it passes through any without alteration.


Snail does not (yet):


Snail is extremely small, and Factor is extraordinarily interactive & flexible. Adding new features and fixing bugs is usually a matter of minutes.


Writing in Bear, and in HTML

Spoiler: I have cheated a bit. This document was written in & exported from Bear. When I want to put down words, I want to start typing on whatever device is nearest, preferably in a nice distraction-free environment.


The problem has never been "I have the wrong blog software." The problem has always been "how do I develop the habit of writing & organizing my thoughts."


Bear excels as a tool for both of those. It runs in my pocket, and it never gets in my way. It has a nice HTML export to boot.


Rather than try to integrate Bear as a step in the process, Bear is the process. Tagging something as ◊site guarantees it will get exported when I next build the site. Bear drops a ◊site folder, Snail analyzes it and finds HTML pages (as well as file & image attachments) and then copies them to the build/ output. Pages in ◊site can link to any other pages, and to each other. Pages outside ◊site can link to pages inside Bear's hierarchy.


There is a little bit of fragility inherent in the export, as file names are based on the title of the post. Bear does encode a unique ID into each file, however, and that is enough of a lever for future snail enhancement (link to doc by ID, redirect docs that changed names).


The future

I want to be able to draw on these web pages using my iPad. There are reasonable Javascript event hooks for this, but that will be built as a separate layer.

gato.svg


#◊site