Hello, Midway

I have more ideas for projects than I'll ever get time to build properly, even with the accelerated pace of AI code generation. It's easy to get to a point barely beyond a functioning code sketch, when the next step involves making it available to others for a reality check. That's where I've found the friction increases and you might need a server, a domain, and a(nother) monthly bill.

So I built myself a place to throw ideas at the wall and see what sticks: midway.

The Elixir language has held my attention for a few years now, as a nice middle way that is sufficient itself for many things and also helps to glue and orchestrate more specialised computation. The BEAM virtual machine it runs on is like an operating system for your code, and I thought why not take that notion more seriously.

When discussing programs on this VM, the word process means a green thread, with its own stack and heap, strictly isolated from other processes so the only interface to it is via message passing of copied immutable values. The word application is similarly overloaded to mean a process that serves as the root of a tree of processes within the VM. There can be many such applications in one VM, sharing the runtime and various services like pubsub, registries and worker pools. The applications only know about each other in so far as you tell them, or if you have them introspect the runtime.

The name "midway" is a carnival term - the strip of a fair where sideshows stand. A row of tents, each with something different inside, all part of the one carnival. That's basically the architecture: one Elixir release, one small VM, hosting this blog and whatever experiments I'm currently poking at, each one an application.

How it's put together

It's an umbrella. The blog and every experiment are ordinary Phoenix apps, each with its own data directory, database and subdomain. A shared app, commons, holds the shared plumbing they all lean on - nothing very specialised lives there, just the stuff that would otherwise be copy-pasted. It starts first and does some setup.

There is a related pattern is known as a poncho, where one app includes another by listing it as a code dependency. The two patterns can be seen as complementary.

The blog you're reading is an application called ballyhoo that follows the umbrella pattern, sitting next to commons within the same repository as midway. The first experiment, WebMacha is a standalone Elixir Phoenix app that, when configured to run in poncho mode, relies on the commons like the blog does.

The BEAM VM could terminate TLS, however there are sound reasons to leave that to other infrastructure so that capability remains on the shelf. One endpoint process routes each request to the right app by hostname, in the one VM.

Something I actually care about: an experiment that falls over doesn't necessarily take the blog down with it. Each one runs as a :temporary application - if it crashes, it restarts with backoff; if it keeps crashing, it just gets switched off and the rest of the site carries on like nothing happened.

# mix.exs: the blog must stay up; an experiment may fall over.
midway: [
  applications: [commons: :permanent, ballyhoo: :permanent] ++ experiments
]

And if I want to take something further, it's not stuck here - lifting an experiment out into its own standalone Phoenix project, data included, is meant to be easy peasy, not a rewrite.

Why bother?

Cost was the original motivation, but on fly.io you can ship experiments independently, they'll scale to zero at quiet times and cost very little. As things progressed I just wanted to learn more about this aspect of the BEAM VM, and it could prove more useful in other environments.