Oh Luna, my Luna

Luna: What is it?

Okay, let’s address the elephant in the room: Luna is vaporware. No implementation exists, outside of an incredibly barebones (and buggy) syntax highlighter so I can make this blog.

So, this blog is really just a little taste. I’m not trying to “sell” anything, both monetarily or socially. Language design just tickles my brain, and now that I’m doing inpatient chemo (long story, another blog post eventually), I certainly have the time. These ideas have been bouncing around my head for at least half a decade; it’s time to put pen to paper.

A Nibble

import std.io;
import std.json;

const exitCode = ['success' => 0, 'usage' => 64];

const main = fn () use (io, argv): int! => {
  let arguments = args();                        // argv is a capability
  die("usage: ${arguments[0]} <config.json>") if (arguments.count() != 2);

  var fd = openFile(arguments[1] as path);      // file!: error propogates out of main
  defer close(&fd);

  let config = fromJson(readAll(fd) as json);    // table!: bad JSON propagates too

  match (config['server']) {
    ['host' => h: string, 'port' => p: int] => println("serving on $h:$p"),
    ['port' => p: int]                      => println("serving on localhost:$p"),
    undefined                               => die('config has no server section'),
    _                                       => die('unrecognized server shape'),
  };

  return exitCode.success;
};

I’ll explain what this is doing at a high-level:

• importing some std modules.
• Running main, which uses the io and argv capabilities and can throw.
• Opening some files and parsing them as json.
• Matching the structure and typing of the result table, at the same time.

Whoa, that’s a lot going on. I just threw random words at you. But before we begin to address this pile of source code, we have to understand why I made the decisions I made.

Programming Languages: a Taxonomy

There exist four distinct kinds of programming languages:

Okay, I’ll admit: this table is rage-bait. I didn’t even include some programming languages, on purpose.

But, it does lend some key insights:

• Extremely restrictive languages are both hard to read and write. The read overhead comes from the syntactic and semantic cost. The write overhead comes from refactoring; even simple refactors in Rust can result in massive code changes.
• The less restrictive we make a language, the easier it is to read and write… to a degree. Eventually reading becomes impossible at an application level. There’s far too much to keep in your head. And now you have to live in the debugger and employ maximum TDD.
• Don’t use C or C++, they’re basically just bad. I love C++, but I would never write anything serious in it today.

Ahh, but you see, there’s a second axis we haven’t considered: check time

The Check Time Axis

All of these languages have type guarantees, to an extent. But where that checking happens matters. When we check more at compile-time, we get a chance to catch and correct errors before the program ever runs. But at run-time, we’re just guessing until something breaks. Unit tests help, sure, but now you’re ballooning your unit tests not with logic checks, but with type checks. And we’ve seen how this ends up: you get high coverage, because you’re running code and checking types, but nothing actually works because no logic is tested.

So, there’s a fundamental tension: when we make a language less restrictive, we can make it easier to read and write. But if we make a language too restrictive, we roll back around: it becomes hard to read, and hard to write.

The Secret Third Axis - Performance

This third axis is arguably the least important. Performance is merely a threshold, not a goal. But, it’s still a consideration, especially in complex backend applications where performance directly translates to cost. And, this is just one small part of the performance story. Performance is complicated, and it’s a whole-stack concern, from database to frontend. Simply choosing a highly static language like C# isn’t enough to get adequate performance.

Where Does Luna Fit in?

Luna is a statically-typed, compiled language. But, it’s not restrictive in the absolute. Luna aims to make augmenting and shaping data the main concern of the application. Types and such are just a means to an end. In my opinion, the shape of data and how it flows through the application is the real gem of programming.

That’s the heart of it, but not the whole story. A few more ideas give Luna its shape: protocols, a flexible type system, and the one I’m most eager to show off, capabilities. We’ll build up to each.

Luna aims to be simple, but never obtuse. Often simplicity becomes an excuse for poor ergonomics. Take C, for instance - even implementing a generic sort function in C is annoying, with void * and all the performance pitfalls that come with it.

The core of Luna is one data structure - the table. Tables can be numerically indexed, or string-key indexed. They’re hashmaps, they’re iterable, and they describe not just what data is, but the shape it should be in. Combined with match, we get comprehensive type checking and shape matching. This is in stark contrast to nominally typed languages, where names are king. Here, shape is king.

But that, too, can get annoying. Nobody wants to check the shape and type of things over and over again. Enter protocols. Protocols are how we define what data might do, what it’s intended to be used for. We can enforce, with an iron fist, what must be present in a table, and we can carefully control how it’s used. Then, we only need to apply protocols wherever they make sense. In contrast to, say, Rust traits, protocols may be applied at runtime, what I call dynamic application.

This trait-like system lends us all the guarantees of languages like Rust, without the complexity and mental overhead of inheritance.

Capabilities and Secrets

One concept I really wanted to lean into is capabilities. Capabilities describe what functions can do, but never how. They’re simple, data-less tags that we can use. We define them like const revealEnvVars = capability;. Then, we can have some arbitrary function that looks like const readEnvVars = fn () use (revealEnvVars) {};.

What makes capabilities unique and powerful is that they’re not simple captures. No, the capabilities of a function are assessed at call-time. A capability isn’t baggage a closure records when it’s created and hauls around; it’s checked against the call itself. So you can pass a function wherever you like and never smuggle a capability along with it - the check happens where the function is called, not where it was born. That’s exactly why they’re hard to bolt onto other languages, and why they’re so expressive: we can grant permissions that are difficult or impossible to pin down elsewhere. Want a function which can read DB credentials but not execute queries? Sure! Want a function that can’t use (io)? Sure! Want a function that can read stacktraces? Why not!

This ties in nicely with another primitive Luna provides - secrets. Secrets just wrap arbitrary data and intercept printing. They keep secrets, well, secret. And we can tie a specific type of secret to a specific capability, so only some functions can read them. This gives us a lot of functionality for free - we can create command pipelines, but keep the arguments hidden, which is especially important if they’re environment variables. We can execute queries, but keep DB credentials out of logs.

But where capabilities really shine is comptime. Luna has comptime, the idea that you can run an arbitrary function at compile-time. But it’s gated by capabilities. We have all the structural tools to ensure that a comptime function cannot access IO, or make HTTP requests, or do FFI. There’s no sandbox per se, because we don’t need it. The language itself has the tools to make leakage structurally impossible. We can’t use (io) in a comptime function because we say so - that capability is restricted to runtime. This inverts how comptime usually works. We don’t need to declare functions comptime, although we can. Instead, we just call them that way: const myTable = comptime constructTable();. Because capabilities already tell the compiler what constructTable() can do, it can determine on its own whether the call is comptime-eligible.

Built-in Types

Luna also isn’t shy about primitive types. If something is common enough to earn ergonomic, first-class support, it gets it, rather than being shoehorned in behind existing semantics.

But wait, there’s more! regex? Sure, that’s a built-in type, and we get ergonomic syntax with //. command? Sure, why not? And with capabilities, we can even separate the act of constructing shell commands from actually executing them. Construct a shell command at compile-time? Sure, but you can’t execute it there! string? Yes, and it’s UTF-8. bytes? Absolutely!

A Flexible Type System

Luna, like PHP, favors complex types over generics. It’s so much nicer to write const sum = fn (int|double num1, int|double num2); than to do the generic song and dance. And, of course, we can just declare new types: const numeric = int|double|decimal; But whoa - doesn’t this get crazy complex?

Actually no, because in Luna, the type universe is closed. We can never introduce new types at runtime, they’re all predetermined at compile-time. And Luna is statically typed, so we’re just one little hover in the IDE away from understanding what types some function operates on.

But what if we WANT type-erasure? Well, we can! any binds to, well, any type. fn without a signature binds to any non-errorable function. And fn! binds to any function, period. We then assess at runtime if it works, requiring manual type narrowing with as. We never coerce either; coercion is always explicit. Luna has no implicit truthiness, so even a conditional needs an explicit bool.

And, speaking of errors…

Errors: my Magnum Opus

There are two schools of thought about error handling in programming languages: exception semantics, and errors-as-values. Both suck for different reasons.

Exceptions hide complex control flow and can make programs harder to reason about. But errors-as-values force us to write the same tired manual unwinding semantics time and time again. It gets error-prone (ha), and people just stop trying altogether.

So with Luna, I thought, why not both?

const someFn = fn (num: int): int! {
    throw parseError() if (num == 0);
    return num;
};

What a strange signature, no? What is that int! doing there, what does that mean? Well, it means someFn can throw.

Now, we, the programmer, get to choose how to handle that. If we’re super deep in a call stack and in no position to handle it, then we just let it throw and follow standard exception semantics. But, if we want to handle it right then and there, we can.

// Standard exception unwinding
try {
    let result = someFn(0);
} catch (e: error) {
    match (@e) {
        parseError => die('Parse error occurred!'),
        _          => die('An unknown error occurred!')
    };
}

// OR we use errors as values
let result = try someFn(0);
let message = match (@result) {
    int   => 'Successfully parsed an int.',
    _     => 'Parse error occurred!',
};

The reason this works is two-fold: errors are checked, and errors are type-erased. Fully checked errors or exceptions are just too annoying to use in practice. What ends up happening is people abuse sentinel values like [] or '' or -1 to represent errors. And that sucks, right? Because now we need to read the function documentation, or worse, body, to determine what’s ACTUALLY an error. And all ergonomics are lost. No message, no stacktrace. Luna’s errors keep both - and here’s where capabilities sneak back in: actually reading that stacktrace is itself a capability. An error can propagate up the whole call stack, but only the frames you’ve blessed can crack it open and read the trace. Which is handy, since a stacktrace can leak plenty it shouldn’t.

So, we require that a function declares it can throw. And then, we employ - gasp - inheritance. Why? Because where data is all about shape, errors are all about kind, and a hierarchy captures ‘kind-of’ relationships perfectly. Errors naturally fall into a simple tree-like structure, where errors are subtypes of other errors. Maybe parseError <: runtimeError, and maybe intOverflow <: panic. And, on the topic of panics, we don’t need to declare them. Really just about any function can panic with something like outOfMemory, so declaring all potential panic points is just wasted noise.

Where’s Luna Now?

Luna is still in its very earliest infancy, and the spec is being refined. The spec and eventual implementation can be viewed at the Luna GitHub Repo. Once I’m happy with the specification, I’ll begin with an initial tree-walking interpreter implementation, which will serve as an oracle and the backbone of Test Driven Development for the compiler. The production compiler will be written in Go.