Gyuri's blog

Porting Lean to the ESP32-C3 RISC-V microcontroller

Wed, 31 Jul 2024 00:00:00 +0000

The goal of this project is to cross-compile Lean programs to the ESP32-C3. (You can find all the code in this repository.)

Lean is a dependently typed pure functional programming language and a proof assistant, though in this article I will mostly be using it just as a programming language. Its compiler can either emit LLVM IR or C code. I will be using the C output for simplicity, but using the LLVM IR instead should just involve setting up the right compiler flags.

The ESP32-C3 is a RISC-V microcontroller running at up to 160 MHz. Its main selling point is its integrated Wi-Fi peripheral, but I will not be using it for this project. It has 384 KiB of internal RAM, and I will be fitting everything into this including the program code, the heap and the stack. (Running code directly from flash memory would also be possible in order to save RAM, but it’s potentially slower than running from RAM.)

What’s the motivation behind this project? While the Lean compiler is pretty good, Lean programs will never be as fast or as small as programs written in C. In an environment where every byte and every CPU clock cycle matters, wouldn’t it make sense to write everything in C?

As far as microcontrollers go, the ESP32-C3 is actually a pretty beefy machine. 160 MHz and 384 KiB of RAM might seem tiny compared to the 3+ GHz and 8+ GB of RAM in your usual desktop machine, but it is significantly more than what e.g. the Commodore 64 or the original Game Boy had. Here is a table comparing the ESP32-C3 to some historical computers:

	CPU	RAM
Apollo Guidance Computer (1966)	16 bit 2 MHz	2 KiB
Commodore 64 (1982)	8 bit 1 MHz	64 KiB
Game Boy (1989)	8 bit 4 MHz	8 KiB + 8 KiB VRAM
ESP32-C3 (2021)	32 bit 160 MHz	384 KiB

Additionally, the success of projects like NodeMCU, MicroPython or Espruino show that running higher level languages on microcontrollers is very much feasible, and the developer effort saved by doing so can be worth it.

With that said, I will jump into describing the whole stack that goes into getting Lean running on the ESP32-C3. You can use the table of contents below to jump to a particular section that interests you, I would say ~80% of any section should still be understandable without having read the preceding ones.

Lean runtime: If you are interested in how the Lean runtime represents objects, and why its authors decided to use reference counting instead of a garbage collector for memory management, this is the section for you.
Toolchain: compiler and libraries: Instead of downloading the prebuilt toolchain provided by Espressif, I decided to compile and configure everything for myself to have more control. If you are curious about my adventures compiling my own libc and libc++, read here.
Booting and application initialization: Normally the boot loader provided by ESP-IDF handles setting up the hardware, but I decided to not use ESP-IDF. If you want to know how the boot chain and hardware setup works on the ESP32-C3, read this one.
Example application: RGB LED controller: Now that I have a working Lean environment on the ESP32-C3, I wanted to write a demo application for this blog post. If you want to see some Lean code (and maybe get inspired to try Lean, I would definitely recommend), and want to know how to blink an LED in a very complicated way (with dependent types!), read here.
Results: If you just want to see some clips of colorful flashing LEDs that are underpinned by all this work.

Lean runtime

In this section I will describe how Lean represents objects in memory, and how it uses reference counting to manage them. Note that I will assume a 32 bit architecture (as the ESP32-C3 has a 32 bit core), though Lean is more commonly used on 64 bit platforms.

Object representation

Lean objects are allocated on the heap, and referenced by a pointer. They consist of a header containing a reference count and some other information, followed by some arbitrary contents:

Unfortunately, this can be quite inefficient for small objects. For example, it does not make sense to allocate small integers or booleans on the heap. As an optimization, certain small objects are stored directly as a value representing themselves (instead of a pointer pointing to them). For example, the number 13 would be represented directly by the constant 0x1b:

But wait, how did 13 turn into 27? It turns out that in order to distinguish pointers from these so-called scalar values, the LSB (least significant bit) needs to be set to 1. The actual value is shifted left by one, so that the value n is encoded as (n << 1) | 1. This is how the value 13 is encoded:

Note that the LSB of pointers is guaranteed to be 0, since the objects in the heap are always aligned to 4 bytes. This way, scalars and pointers can be clearly distinguished based on their LSB.

Memory management

Most managed languages nowadays use some sort of tracing garbage collector to automatically clean up unused objects. Lean uses reference counting instead. The header of each heap allocated object contains a reference count. It gets incremented if there is a new reference to the object, and decremented when a reference is no longer needed. When the reference count reaches zero, the object is freed.

Lean opts to use reference counting for two major reasons. First, a big weakness of reference counting, the handling of cycles, is not an issue, since due to immutability it is guaranteed that the object graph is acyclic.

Second, it allows for an optimization that is particularly useful for languages where all values are immutable. Take the following Lean code:

def rep (n : Nat) (v : α) :=
match n with
| 0 => #[]
| n + 1 => Array.push (rep n v) v

This returns an array containing the value v n times. But since the array is immutable, it is created and destroyed n times, always with one extra element.

As a special optimization, when the Array.push builtin function receives an argument with a reference count of exactly 1, it can mutate the existing array instead of having to create a new one. Since the reference count guarantees that this mutation cannot be observed by any other part of the code, this does not violate functional purity.

You can read more about this specific optimization, and much more in this paper: [1]S. Ullrich and L. de Moura, “Counting immutable beans: Reference counting optimized for purely functional programming.” 2020. Available: https://arxiv.org/abs/1908.05647.

Getting the runtime to compile & run on the ESP32-C3

The runtime includes its own special allocator to speed up the allocation of small objects. It allocates 8 MiB segments from the system to then suballocate the objects. This is many times larger than the whole 384 KiB of RAM we have, so it definitely won’t cut it. The easiest solution is to just disable this special allocator, and rely on the malloc implementation being fast enough.

The next challenge is that the runtime is written in C++. If possible I would like to avoid having to deal with exceptions and vtables and other C++ features requiring special runtime considerations compared to C.

I went on a quest to slim down the runtime, and removed as many unnecessary things as possible:

There were some uses of C++ exceptions, but none of them were hard to remove.
I removed a bunch of unneeded features, like multithreading or debugging support.
Removed library functions assuming the presence of an OS, like file IO or process management.

What remains is mostly the core functionality used by compiled Lean programs:

Primitives for allocating objects and manipulating reference counts.
Optimized functions for calling closures.
Implementations for special-case types like strings, arrays, and natural number.
IO primitives, so things like IO.print work.

Toolchain: compiler and libraries

Overview

I wanted to have full control over the whole toolchain to be able to customize things if necessary, so I decided to build the standard C and C++ libraries for myself. I compiled everything with the -ffunction-sections flag, which allows garbage collection of individual functions at link time, reducing code size. The full tech-stack ended up being the following:

C/C++ compiler: Clang version 18.

Clang is a multi-target compiler, so unlike GCC, I don’t have to build a special RISC-V cross-compiler. I just pass the -target riscv32-none-elf -march=rv32imczicsr^{1Just rolls off the tongue, right? rv32 stands for “RISC-V 32 bit”, i is the base integer instruction set, m is the extension with multiplication and division instructions, c signifies support for compressed instruction encoding, and finally zicsr signals support for “Control and Status Register” (CSR) instructions.} flags and it works.
C standard library: picolibc.

Picolibc is specifically designed for small embedded systems with limited RAM. It includes a malloc implementation, which is needed for the Lean runtime. It also includes standard input/output functions, which are also used by the Lean runtime to implement IO.print. I hooked them up to the built-in USB serial peripheral on the ESP32-C3.
C++ standard library: libc++.

libc++ is developed under the umbrella of the LLVM project, and so is a natural choice to use together with clang.

C++ is needed to compile the Lean runtime. I think std::vector is the most important thing it needs from the C++ standard library. (Which uses C++’s new operator, which uses malloc under the hood. Fortunately all the C++ stuff I needed ended up just working relatively painlessly.)
Lean compiler: No custom modifications.
Lean runtime: My own slimmed down version, discussed above.
Lean standard library (Init): No modifications. (Though I do have a dirty hack to reduce its code size massively, see below.)
QEMU for testing: Espressif’s QEMU branch with support for the ESP32-C3.

The Nix package manager

If you ever tried to put together some software project with multiple customized dependencies that need to be compiled, you know how painful it can be. And even if you manage to get it working, and document how you did it, the chances of someone else getting it working on the first try (or even you yourself on another computer) is basically zero.

You did not forget to record all the custom build system flags you used, right? Also project X needs a one line modification to get it to compile in your environment, don’t forget about that. Also project Y needs an older version of dependency Z, so you have to temporarily downgrade it while compiling that one project, you recorded the exact version needed, right?

And even if you documented every step perfectly, some project is bound to depend on something in your environment you did not notice, but someone else might not have installed.

Also if you want to take your project and continue working on it let’s say 1 or 2 years later, chances are that something will have gotten broken by then. Even if you recorded the versions of ALL packages installed on your system, reproducing the exact same environment is impossible on most operating systems.

Nix is a package manager that builds packages in a reproducible manner. Every build is isolated from the rest of the world and only supplied with dependencies that were explicitly declared, so that it’s not possible to forget declaring a dependency. Every external input (e.g. source code downloaded from a git repository) must be pinned by its hash so that it cannot change implicitly.

If a nix package works today, it is pretty much guaranteed to continue working many years into the future without any changes.

Nix also makes it very easy to define one’s own packages using its package definition DSL called the nix language.

Because of these aforementioned advantages, I decided to use Nix for this project from the very start. This guarantees that anyone (either you, or future me many years from now) can just clone the project repository, and run nix build to compile the exact versions of everything I used for the project, including the final binary that runs on the ESP32-C3.

Targeting the ESP32-C3 with `nixpkgs`

The main package collection for Nix is called nixpkgs, which I will be using as a base to build my packages on.

If the description above made it seem like Nix is a magic bullet and everything is perfect and beautiful in the Nix ecosystem, I have some bad news, building software for embedded targets is not a very common (or commonly tested) use-case of nixpkgs, and some hacks will be needed. But importantly, these are reproducible hacks! Once they are working we can just shove them into a git repository and forget about them, they will keep working no matter how ugly they are.

With all that said, let’s see what it takes to target an embedded system with nixpkgs. nixpkgs has an stdenv (standard environment) for building packages that includes common dependencies like a C compiler and a libc. I want to instantiate a package set with an stdenv that comes with clang, picolibc, libc++, and preconfigured compiler flags which are needed to compile code for the ESP32-C3.

The most important compiler flag is the exact target processor architecture, which is configured as rv32imczicsr.^{1Just rolls off the tongue, right? rv32 stands for “RISC-V 32 bit”, i is the base integer instruction set, m is the extension with multiplication and division instructions, c signifies support for compressed instruction encoding, and finally zicsr signals support for “Control and Status Register” (CSR) instructions.}

The package definition for picolibc is fairly straightforward. To get libc++ to compile we have to apply some patches to nixpkgs: this consists of turning on some options that are only enabled for WebAssembly in stock nixpkgs (I guess embedded environments and WebAssembly have quite a bit in common), and adding some extra flags on top to turn off some more libc++ features (like locale support, which we definitely don’t need) to get it to compile.

Building the customized Lean runtime is moderately involved, the solution is graft it onto the original Lean source tree so that we can use the build system that’s already in place.

Lean likes to use its own C compiler wrapper to apply some extra flags, and it takes quite a bit of work to convince it that this is really rather not a good idea in this situation. (It tries to do things like link with stdc++ which is the GNU version of the standard C library which we don’t have, or turn on threading support.)

Compiling Lean’s Init library is fairly easy compared to everything else, though I don’t think compiling it separately from the rest of Lean is an intended use-case.

Finally, building the example app itself is not too hard either (though it still does take a moderately sized list of carefully crafted compiler flags), since all the hard work has already been done by setting up the stdenv and Lean.

Hacking on the project yourself

The project also contains a QEMU version compiled from Espressif’s repository, with support for the ESP32-C3, so you don’t even need a real device to try it out. Just run nix run .#qemu in the project root. You should see a message printed by the main function written in Lean.

For further information, see the README file.

Booting and application initialization

Memory layout

Here is how I decided to lay out the 384 KiB of RAM:

I decided to place the stack at the start so that a stack overflow can’t corrupt anything else (the stack grows downwards). I am not sure why placing it at the end of the RAM is the common practice, since that way it could silently corrupt the heap.

The code section (usually called .text by the compiler) is pretty self explanatory, the example application I will be showing later weighs in at around 64 KiB. One notable thing about the code section is that it needs to be accessed through a different address space, in somewhat of a Harvard architecture style. On the ESP32-C3 the range 0x40380000-0x403dffff is used to access the RAM for code execution, while 0x3fc80000-0x3fcdffff is used for regular data access. (So the above code section starts at 0x40388000 in this mapping.) Some linker script trickery is needed to make sure that all the addresses in the linked code are correct.

The data section contains read only constants (such as strings) and global variables with a defined value.

The bss section contains zero initialized global variables. (Don’t know what the name “bss” stands for though.) Forgetting to zero this actually caused me to spend a day debugging seemingly unrelated malloc issues, which only manifested on real hardware and not in QEMU. It turns out that malloc stores the head of the free list in a global variable which it expects to be NULL by default. In QEMU all RAM is zeroed by default, but on real hardware it could be full of random junk values. Adding memset(&_bss_start, 0, (&_bss_end - &_bss_start)); to zero the bss section fixed the issue. Oops.

Finally, the remaining space is designated as the heap. All we have to do in the linker script is to specify the __heap_start and __heap_end symbols, and from there picolibc’s malloc implementation knows that it can allocate memory in this range. In my example application around 284 KiB is left for the heap, which is plenty.

To run our program, the code and data sections need to be copied to their specific addresses, and then execution needs to begin from a specific entry address. This can be achieved by packaging our application in the firmware image format expected by the ROM boot loader.

Booting

Booting on the ESP32-C3 happens through the following steps:

On CPU Reset, the program counter is set to 0x40000000, and the CPU starts executing.
At 0x40000000 the chip’s internal ROM is located. The values on various GPIO pins can affect what happens, but by default, the ROM code loads the application code from the external SPI flash memory to RAM. The application in the specific firmware image format is expected to be located at the beginning of the flash.

The firmware image contains a number of segments to be loaded to specific addresses in RAM. The ROM code goes through them, and copies each segment to RAM.
Finally, the firmware image also contains an entry point address. The ROM code jumps to it, and from here we have full responsibility over execution.

Note that unfortunately the ROM code is not open source, but parts of it have been reverse engineered, for example for security analysis. The ROM and the Wi-Fi MAC are the only non-open source software parts in the ESP32 ecosystem that I am aware of.

The entry point of my application is named call_start_cpu0 (this could be any name, it’s specified in the linker script). Here is the entry point function (we could also be using picolibc’s crt0 instead, but I was too lazy to set it up):

void __attribute__((noreturn)) call_start_cpu0() {
 __asm__("la sp, _stack_end;");
 memset(&_bss_start, 0, (&_bss_end - &_bss_start));
 disable_wdt();
 main(0, 0);
 while (true)
 ;
}

The very first thing we do is load the stack pointer to the sp register so that we can call functions and have local variables, pretty important things for most programs. Next, we zero the bss section, which as I already mentioned, is pretty important. We also disable the default watchdog timers. Next, we call the main function, which is the entry point generated by the Lean compiler. In case main were to return, we enter a busy loop. (Trying to return here would probably lead to a stack underflow, followed by the CPU going off trying to execute some random code.)

Lean startup

The main function emitted by the Lean compiler looks something like this:

int main(int argc, char **argv) {
 lean_object *in;
 lean_object *res;
 lean_initialize_runtime_module();
 lean_set_panic_messages(false);
 res = initialize_Main(1 /* builtin */, lean_io_mk_world());
 lean_set_panic_messages(true);
 lean_io_mark_end_initialization();
 if (lean_io_result_is_ok(res)) {
 lean_dec_ref(res);
 lean_init_task_manager();
 res = _lean_main(lean_io_mk_world());
 }
 lean_finalize_task_manager();
 if (lean_io_result_is_ok(res)) {
 int ret = 0;
 lean_dec_ref(res);
 return ret;
 } else {
 lean_io_result_show_error(res);
 lean_dec_ref(res);
 return 1;
 }
}

Notable parts are:

The call to lean_initialize_runtime_module to do any initialization needed by the runtime. As I mentioned already I slimmed down the runtime a lot, so this doesn’t do too much.
initialize_Main initializes some global constants of the Main module. It also calls initialize_Init to initialize the standard library. The standard library initialization turned out to be an issue, since apparently Lean does not remove the initialization code for unused constants. The standard library is fairly big, and leaving its initialization as-is makes the code size of the application larger than the 384 KiB size of our whole RAM, so we clearly need to do something about this. My trick is override the initialize_Init function with a stub implementation that does nothing:
```
lean_object *initialize_Init(uint8_t builtin, lean_object *w) {
 return lean_io_result_mk_ok(lean_box(0));
}
```
As long as we carefully avoid using any constants from the standard library that would have needed initialization, this seems to just work. (Though it’s definitely not a safe solution.)
Finally, the call to _lean_main executes the main IO monad of our Lean program. This is what a hello world program in Lean would look like:
```
def main : IO Unit := do
IO.println "hello world"
```

Example application: RGB LED controller

The development board I have has a WS2812B RGB LED, so making it light up in various patterns was a pretty obvious choice for an example application.

First, I define an interface for arbitrary lighting patterns:

structure LightPattern where
State : Type
init : State
step : State → State
color : State → Color

State is the type of the internal state of the pattern. For example, for a simple blinking pattern it could be a Bool.
init is the initial state.
step is a function to compute the next state from the current one.
color is the function to get the current output color of the LED based on the state.

And yes, all the fields after State can refer to its value (a type) to define their own types, this is what dependent typing is all about.

To make this a little less abstract, let’s define a simple blinking pattern with an arbitrary color:

def LightPattern.blink (c : Color) : LightPattern := {
State := Bool
init := true
step := Bool.not
color := (if · then c else default)
}

State is a Bool, as I already mentioned previously. step is simply the boolean negation. color will be c when the state is true, and off otherwise. (default is defined as { red := 0, green := 0, blue := 0 }.) The syntax for defining color might be a little confusing, the · shorthand is used to define the function, which is equivalent to fun s => if s then c else default.

Let’s define a more advanced pattern, which cycles through the three color channels:

inductive ColorChannel := | Red | Green | Blue
def ColorChannel.toColor : ColorChannel → Color
| Red => { red := 0xFF, green := 0, blue := 0 }
| Green => { red := 0, green := 0xFF, blue := 0 }
| Blue => { red := 0, green := 0, blue := 0xFF }
def LightPattern.cycle : LightPattern := {
State := ColorChannel
init := .Red
step := fun
| .Red => .Green
| .Green => .Blue
| .Blue => .Red
color := ColorChannel.toColor
}

Next, let’s define a “light pattern transformer”, that takes an existing pattern and modifies it. In particular I want to be able to tune the speed of these patterns. I will be running the update loop at around a 1000 Hz, which would be much too fast to see the blinking pattern for instance. The following pattern will only run the step function of the underlying pattern p every n steps:

def LightPattern.every (n : UInt16) (p : LightPattern) : LightPattern := {
State := UInt16 × p.State,
init := (0, p.init),
step := fun (i, s) => if i == n then (0, p.step s) else (i + 1, s),
color := fun (_, s) => p.color s
}

Let’s unpack this:

We define the state to be a 16 bit counter^{2I use a 16 instead of a 32 bit counter since a 32 bit value would have to be allocated on the heap due to the way Lean’s object representation works, whereas a 16 bit value will be a scalar. It would be nice if Lean also had 31 bit integers like OCaml.}, plus whatever state p has. × denotes a product type, a.k.a. a pair.
init is zero and the initial state of p.
step is the core of the logic: when the counter reaches n, we call the step of the underlying p. Otherwise, we just increment the counter, and leave the state of p as-is.
color just outputs the color of p.

Finally, I wanted to also have a candle-like flicker effect. First, we need a (pseudo) random number generator for this (source):

abbrev Prng.State := UInt32
def Prng.step (state : Prng.State) : Prng.State := state + 0x9E3779B9
def Prng.get (z : Prng.State) : UInt32 :=
let z := z ^^^ (z >>> 16)
let z := z * 0x21F0AAAD
let z := z ^^^ (z >>> 15)
let z := z * 0x735A2D97
z ^^^ (z >>> 15)

The only state of our candle will be the 32 bit PRNG state. (Unfortunately, this will be allocated on the heap since it’s a 32 bit value.) Here is the code for the candle:

def LightPattern.flicker : LightPattern := {
State := Prng.State
init := 0
step := Prng.step
color := fun state =>
let random := Prng.get state
let brighness := 0x80 + random &&& 0x7F
{
red := brighness.toUInt8,
green := 150 * brighness / 255 |>.toUInt8,
blue := 50 * brighness / 255 |>.toUInt8,
}
}

All the candle-specific logic happens in the color function. We get the current value of the PRNG, calculate a brightness between 128 and 255 with it, and then mix up a warm white looking color while being very careful to avoid floating point maths. (Floating point has some issue I haven’t looked into yet and calculates bad values.)

Next, let’s define our application. We want to dynamically switch between patterns from a predefined array, which we parameterize our app over:

structure AppState (patterns : Array LightPattern) where
patternIndex : Fin patterns.size
state : (patterns.get patternIndex).State

patternIndex is the index of the currently playing light pattern, while state is the current state of the current light pattern, and its type depends on the current patternIndex.

Here is the rest of the app implementation, understanding it is an exercise left to the reader:

variable (patterns : Array LightPattern)
/-- Switch to the next pattern. -/
def AppState.nextPattern (s : AppState patterns) : AppState patterns :=
let patternIndex := s.patternIndex + Fin.ofNat' 1 (
match e : patterns.size with
| 0 => Fin.elim0 (e ▸ s.patternIndex)
| _ + 1 => (by omega))
{
patternIndex,
state := (patterns.get patternIndex).init,
}
/-- Update the state of the current pattern. -/
def AppState.step (s : AppState patterns) : AppState patterns :=
{ s with state := (patterns.get s.patternIndex).step s.state }
/-- Get the current output color. -/
def AppState.color (s : AppState patterns) : Color :=
(patterns.get s.patternIndex).color s.state
/-- Initialize a new app state. -/
def AppState.init (hPatterns : patterns.size > 0) : AppState patterns :=
let patternIndex := Fin.ofNat' 0 hPatterns
{
patternIndex,
state := (patterns.get patternIndex).init,
}

Next, we define our array of light patterns (assuming a 1000 Hz update frequency):

def patterns : Array LightPattern := #[
every 500 (blink white),
every 100 (blink white),
every 500 (blink red),
every 100 (blink red),
every 500 (blink green),
every 100 (blink green),
every 500 (blink blue),
every 100 (blink blue),
every 500 cycle,
every 100 cycle,
every 50 flicker,
breathe 1000 .Red,
breathe 1000 .Green,
breathe 1000 .Blue
]

And finally, we are ready to define our main loop:

def main : IO Unit := do
let mut appState : AppState patterns
:= AppState.init patterns (by simp [patterns])
let mut buttonPressedPrev := false
while true do
-- Read the button state, advance to the next pattern when pressed
let buttonPressed := !(← readGpio 9)
if !buttonPressedPrev && buttonPressed then
appState := appState.nextPattern
buttonPressedPrev := buttonPressed
-- Update the pattern state and output the current color to the LED
appState := appState.step
setLedState appState.color
-- TODO:
-- Use proper timer instead of just a wait, so the frequency does not
-- depend on the time it takes to do the update.
delayUs 1000

Wait, didn’t I say that Lean was a pure language? What are these mutable variables and while loop doing here? As it turns out it’s all syntax sugar, and a pretty advanced one at that. All the “impure” features are compiled down to pure monadic code by Lean. You can learn more by reading the relevant chapter of the Lean manual.

readGpio, setLedState, and delayUs are implemented in C:

@[extern "c_read_gpio"]
opaque readGpio : Fin 22 → IO Bool
@[extern "c_set_led_state"]
opaque setLedState : (@& Color) → IO Unit
@[extern "c_delay_us"]
opaque delayUs : Nat → IO Unit

For example, here is what the c_set_led_state function looks like:

lean_object *c_set_led_state(lean_object *state, lean_object *w) {
 uint32_t data = lean_ctor_get_uint8(state, 0) << 16 |
 lean_ctor_get_uint8(state, 1) << 8 |
 lean_ctor_get_uint8(state, 2);
 send_led_data(data);
 return lean_io_result_mk_ok(lean_box(0));
}

See the appendix below for how the send_led_data function works.

Results

Here are some clips of some LED patterns (in particular every 100 cycle, breathe 1000 .Blue, and every 50 flicker) on my Waveshare ESP32-C3-Zero development board:

Conclusions

Overall I am pleased with the results, beforehand I wasn’t even quite sure if running Lean in such a constrained environment would be possible. For now this is still very much just a proof-of-concept, though in the future I might try to build something actually useful with an ESP32 and Lean. Here is a list of ideas for how this project could be taken further:

Transition to first compiling everything to LLVM bitcode instead of directly to machine code, this would enable link-time optimization to further reduce code size, and might simplify the cross-compilation setup.
Use ESP-IDF, it already has a bunch of useful hardware abstractions, and could potentially make supporting other devices from the ESP32 family much easier.
Get threading support by putting FreeRTOS into the stack. (ESP-IDF already uses it.)
Get Wi-Fi working, would be pretty important for real-world applications.
Try upstreaming some of the needed modifications to Lean.
- Investigate if unused initialization code could somehow be detected and removed, this is definitely a big pain point.
- I think adding flags to make some features optional in the runtime (e.g. C++ exceptions, threads, and various OS interfaces) should be relatively uncontroversial.
- Make the build system in general behave a bit better when cross-compiling.
- Add 31 bit integers. (Maybe by special casing the Fin type?)
Debug why floating point operations are giving bad results.

Finally, if you are interested in any of the topics mentioned in this article, here are some useful resources:

Lean: I recommend the Functional Programming in Lean book, it’s a quite comprehensive introduction to programming in Lean. If you are by chance interested in using Lean as a proof assistant, take a look at the separate book titled Theorem Proving in Lean 4.
ESP32: If you want to program in a memory safe language for the ESP32 family of devices, I would recommend trying Rust, specifically the esp-rs project, which seems to have an active community around it.
Nix: Not going to lie, I find the resources in the Nix ecosystem to be pretty lacking if you want to do anything somewhat non-trivial. Be prepared to read a lot of nixpkgs source code if you want to do something on the level presented in this article. The best I can do is link you to the official learning resource collection on the NixOS website, but I cannot vouch for the quality of anything there.

Appendix: Bit banging the WS2812 protocol

The WS2812 family of addressable LEDs need to be driven with very particular timing constraints. A 1 is signaled with the data line being high for 800 ns and low for 450 ns, while a 0 is signaled by it being high for 400 ns and low for 850 ns. We need to transfer 24 bits like this, 8 for each color channel.

Using a 20 MHz CPU clock, a single clock cycle is 50 ns, meaning that the transfer of a single bit needs to take exactly 25 cycles.

Most instructions take 1 clock cycle on the ESP32-C3, except for branch instructions, which take 1 cycle if the branch is not taken, but take 3 if the branch is taken! (At least according to this article I found on the topic.) And be thankful that the ESP32-C3 does not have a branch predictor, so that even if the timings are somewhat complicated, they are at least deterministic.

With all that said, here is the working code that I ended up with:

__attribute__((noinline)) void send_led_data(uint32_t data) {
 const size_t gpio = 10;
 for (size_t i = 0; i < 24; ++i) {
 const uint32_t bit = data & (1 << (23 - i)); // shift + and: 2 cycles
 GPIO_OUT_W1TS_REG = 1 << gpio; // GPIO high: 1 cycle
 if (bit) {
 // if branch not taken: 1 cycles
 __asm__("nop; nop; nop; nop; nop;"
 "nop; nop; nop; nop; nop;"); // wait: 10 cycles
 GPIO_OUT_W1TC_REG = 1 << gpio; // GPIO low: 1 cycle
 __asm__("nop;"); // wait: 1 cycle
 // loop variable inc + jump taken: 4 cycles
 } else {
 // if branch taken: 3 cycles
 __asm__("nop; nop;"); // wait: 2 cycles
 GPIO_OUT_W1TC_REG = 1 << gpio; // GPIO low: 1 cycle
 __asm__("nop; nop; nop; nop; nop; nop; nop; nop; nop;"); // wait: 9 cycles
 // loop variable inc + jump not taken: 2 cycles
 }
 }
}

Writing it took some effort involving constantly looking at the disassembly to see what smart ways clang managed to come up with to helpfully “optimize” my code, and then accounting for them. E.g. you might notice that the jump back to the beginning of the loop takes a different number of cycles in the two branches of the if! It turns out that clang laid out the code in such a way that the else branch of the if is placed before the beginning of the loop, so execution just falls through into the beginning of the loop again without having to jump back. This is faster, as you can see, it saved 2 clock cycles for us…

This code will probably keep working up until I upgrade to the next version of clang, and it comes up with some other way to outsmart us.

Also you might notice that the clock cycles in an iteration above only add up to 20, and not 25. I had to take out a few nops, becuase the LED was not working with 25 for some reason. I still don’t know what the issue is, either there are still some clock cycles that I am not accounting for, or maybe the clock is slightly slower than 20 MHz for some reason. (I would need an oscilloscope or logic analyzer to debug this, which I don’t have.)

Overall, I would rate the experience of writing this function 0/10. If I ever need to write such timing sensitive code again I will make sure to do it in assembly, because that will probably cause much fewer headaches. Optimizing compilers are awesome, except in rare cases like this, when they are working actively against you.

Just rolls off the tongue, right? rv32 stands for “RISC-V 32 bit”, i is the base integer instruction set, m is the extension with multiplication and division instructions, c signifies support for compressed instruction encoding, and finally zicsr signals support for “Control and Status Register” (CSR) instructions. ↩︎ ↩︎
I use a 16 instead of a 32 bit counter since a 32 bit value would have to be allocated on the heap due to the way Lean’s object representation works, whereas a 16 bit value will be a scalar. It would be nice if Lean also had 31 bit integers like OCaml. ↩︎

Bibliography

[1]

S. Ullrich and L. de Moura, “Counting immutable beans: Reference counting optimized for purely functional programming.” 2020. Available: https://arxiv.org/abs/1908.05647

A formally verified R-tree implementation

Sun, 12 Nov 2023 00:00:00 +0000

Introduction

Bugs are ubiquitous in computer programming. Widely applied solutions exist to verify certain properties like memory or type safety. On the other hand, most programming languages only offer very rudimentary tools to enforce logical properties.

Proof assistants offer a potential solution, and significant portion of them today is based on the Curry–Howard isomorphism [1]H. B. Curry, J. R. Hindley, and J. P. Seldin, Eds., To H.B. Curry: Essays on combinatory logic, lambda calculus, and formalism. London ; New York: Academic Press, 1980.. The idea behind using the Curry–Howard isomorphism for theorem proving is that strongly typed functional programs can also be interpreted as proofs of mathematical theorems. As such, proof assistants based on the Curry–Howard isomorphism are just as powerful functional programming languages as they are proof assistants.

Coq is a proof assistant and a programming language based on Martin-Löf type theory. It allows arbitrary logical propositions to be stated and proved about programs.

Proving the correctness of algorithms in proof assistants is still very labor intensive, and an active research area. I present an implementation of the R-tree data structure and its correctness proof in Coq. (This is the first correctness proof of an R-tree specifically that I am aware of.) You can find the source code for this project here.

The reader is referred to my previous article for an introduction to type theory and its use for theorem proving, and this article for an introduction to the specific features of Coq.

R-trees

An R-tree [2]A. Guttman, “R-trees: A dynamic index structure for spatial searching,” in Proceedings of the 1984 ACM SIGMOD international conference on Management of data - SIGMOD ’84, Boston, Massachusetts: ACM Press, 1984, p. 47. doi: 10.1145/602259.602266. is a data structure that can efficiently query a set of axis-aligned bounding boxes (AABBs) in $\mathbb{R}^d$ that intersect a query box $\gamma$ .

The R-tree is a bounding volume hierarchy, meaning that each node in the tree has an associated volume, and each child is completely contained in its parent’s volume. In the case of the R-tree, all of the volumes are AABBs, and each non-leaf node’s volume is simply the minimum bounding rectangle of the volumes of its children.

The number of children of each non-leaf non-root node should be between $m$ and $M$ , which are freely chosen parameters of a particular R-tree. (In the example below $m=2$ and $M=4$ .) It’s the job of the insertion and deletion procedures to make sure that these bounds are respected. They also make sure that the tree is balanced. (That is, each leaf node is at the same height.)

In the example below, we are storing the rectangles $\{A,B,C,D,E,F,G,H,I\}$ in the tree, while $\{X,Y,Z\}$ and the root are inner nodes that were generated during the insertion procedure.

A visualization of the bounding boxes associated with the nodes in the tree, and a visualization of the tree hierarchy.

Note that in bounding volume hierarchies there is nothing preventing sibling nodes from overlapping. While the R-tree insertion procedure does try to minimize overlaps, in most cases it can’t completely eliminate them.

First, let’s look at how querying works using an example. We want to find all rectangles that intersect a query rectangle $\gamma$ :

First we check whether it intersects the root node, which it does in this case. Next, we look at the children of the root node. $\gamma$ intersects $X$ and $Z$ , but it does not intersect $Y$ . Since all descendants of $Y$ are completely contained within $Y$ , we can be sure that no descendant of $Y$ intersects $\gamma$ , and can completely skip searching its subtree. The same search procedure is now repeated recursively for $X$ and $Z$ . In the end, we find that the rectangles intersecting $\gamma$ were $C$ and $F$ . Here is a visualization of how we searched the tree:

In this particular case, the R-tree didn’t save us anything compared to brute force search: we had to do 11 intersection tests, while brute force would have only done 9. Gains are achieved when we can throw away large subtrees during search, which may or may not be the case even with larger trees, depending on their structure. Unfortunately, there is no complexity guarantee better than $O(n)$ for the plain R-tree.

Insertion

Let’s say we want to insert a new item $J$ into the tree:

The insertion algorithm will traverse the tree, and find that $Z$ is the best leaf node to put it in, since it already fully contains it. (The actual algorithm is to look for the leaf node that needs the minimum area enlargement to contain the new item. In this case, $Z$ needs zero enlargement, so it’s the best choice.) Let’s insert $J$ into $Z$ :

Now we encounter an issue with $Z$ : we said at the beginning that each node can have a maximum of $M=4$ children.

The insertion algorithm’s solution is to recursively split the nodes going up the tree until the constraint imposed by $M$ is satisfied again for all of the nodes.

In this case, splitting only $Z$ will be enough:

Visually, it’s obvious that partitioning $\{F,G,H,I,J\}$ into $\{G,H,I\}$ and $\{F,J\}$ is a “good” split. A good split should minimize the probability that some arbitrary future query rectangle $\gamma$ overlaps both. A heuristic algorithm tries to achieve this by minimizing the combined area of the two resulting rectangles. (There are actually multiple variations that can be used for this heuristic, I will go into the details later.)

Deletion

Deletion is conceptually easy after we have insertion. Delete the item, and if its parent has too few items, remove it as well. (And do this recursively all the way up the tree.)

Now we have a well-formed tree, but unfortunately we also removed a bunch of unrelated items.

The solution is to simply reinsert all of them using the insertion algorithm we already have. That’s it.

Complexity

Finally, just how fast are R-tree operations? Unfortunately we can’t say anything better than $O(n)$ ( $n$ being the number of items) for the worst case complexity (for any R-tree operation).

I have seen claims of $O(\log_M{n})$ average complexity for search, which does intuitively make sense if you assume that your data is uniformly distributed, and on average the search only continues down on a single path at each level.

Unfortunately, I haven’t seen any rigorous derivations of an average case complexity.

Specification

We are going to write an R-tree implementation in Coq, and prove it functionally correct. But, what does it mean for an implementation to be correct?

Correctness can only be established in relation to a specification, so we will have to come up with one. Fortunately in our case the specification is quite straightforward. (Well, straightforward compared to the proof at least. This exact specification would apply to any other spatial data structure as well.)

Informal specification

In this section I will lay out the specification without making everything perfectly rigorous, while in the next section I will properly formalize everything in Coq.

Let’s define bounding boxes to be hyperrectangles in $d$ dimensional Eucledian space, with intersection and all geometric concepts having their usual meanings. (Note that while we develop the specification here for arbitrary $d$ , my implementation later will be specialized to $d=2$ .)

We are going to denote the empty tree with $\varepsilon$ , and the multiset of all items contained in a tree $t$ with $\iota(t)$ . (Multiset because an item is allowed to be inserted multiple times.) We define three operations (functions) on our trees:

$\operatorname{insert}(t, i)$ denotes the tree $t$ with the additional item $i$ inserted.
$\operatorname{search}(t, q)$ denotes the multiset of items in the tree that intersect the bounding box $q$ .
$\operatorname{delete}(t, q, f)$ denotes the tree $t$ with the items removed that intersect $q$ , but do not satisfy the predicate $f$ . (This is the most general form of deletion that can efficiently be implemented on R-trees. Simpler variants (like only deleting a single item) can be trivially defined using this general form.)

First let’s state the trivial specification of the empty tree: $\iota(\varepsilon) = \empty$ . We are saying that the empty tree has no items.

Next up, insertion: $\iota(\operatorname{insert}(t, i)) = \{i\} + \iota(t)$ . The items in the tree with $i$ inserted are exactly the items of the original tree plus $i$ . ( $+$ instead of $\cup$ is important here, since we are dealing with multisets.)

Next, let’s specify search: $\operatorname{search}(t, q) = \{ i \in \iota(t) \mid q \cap i \neq \empty \}$ . Searching with a bounding box $q$ means returning all items that overlap with this box.

Finally, deletion: $\iota(\operatorname{delete}(t, q, f)) = \{ i \in \iota(t) \mid q \cap i = \empty \lor f(i) \}$ . We define deletion as a kind of filtering: we keep items that are either completely outside of $q$ , or which satisfy $f$ .

Formal specification in Coq

Multisets would be somewhat tricky to represent in Coq, so I will just use plain lists instead. Duplicate items are now represented simply by including them in the list multiple times.

Regular equality between lists would give wrong results though, since the items can be ordered differently. Instead, we will use the concept of permutations. Two lists are permutations of each other iff the multisets they represent are equal. (More plainly, if you can reorder the items of a list to get the other one, that means they have the same items.)

The Coq standard library already includes a the permutation relation, so we will just use it. For convenience, let’s denote the proposition that two lists $a$ and $b$ are permutations of each other with $a \sim b$ .

Let’s also tell Coq about this notation (you don’t need to understand the exact details of this definition, it just works as you would expect):

Notation "a ~ b" := (Permutation a b) (at level 70, no associativity).

The Coq standard library defines list concatenation as ++. negb stands for boolean negation, || for disjunction. bbox is the type of AABBs, and bbox_intersect is a function determining whether two bounding boxes intersect. filter is the usual filtering function from functional programming languages.

Here is the whole specification. I will break it down step-by-step afterwards:

Context {I : Type} {item_bbox : I -> bbox}.
Class Container := {
 T : Type;
 items : T -> list I;

 empty : T;
 insert : T -> I -> T;
 search : T -> bbox -> list I;
 delete : T -> bbox -> (I -> bool) -> T;

 empty_correct : items empty ~ [];
 insert_correct : forall t i,
 items (insert t i) ~ [i] ++ items t;
 search_correct : forall t q, search t q ~
 filter (fun i => bbox_intersect q (item_bbox i)) (items t);
 delete_correct : forall t q f, items (delete t q f) ~ filter
 (fun i => negb (bbox_intersect q (item_bbox i)) || f i) (items t);
}.

First we declare that there is some type I (the type of items), and a projection item_bbox that tells us the bounding box of any item.

The Container interface^{1Container is actually a typeclass, a feature popularized by Haskell. In Coq they are mostly equivalent to record types, which are just syntax sugar for inductive types.} then consists of the following entries:

T, the type of trees.
items, the function listing the items in a tree.
empty, the empty tree.
insert, search, and delete, our three operations on trees.
empty_correct, insert_correct, search_correct, and delete_correct, the proofs of the four correctness propositions.

The goal of the rest of this article is to provide an implementation of Container.

Techniques

This section introduces some general techniques I use in this project.

Extraction

The transpilation of a Coq program into some other non-dependently typed functional programming language is called extraction. The best supported extraction target is OCaml, so I will be focusing on that. (Note that extraction is not needed for the proofs themselves, but it is needed to run the verified implementations in a reasonable manner.)

The reason it’s called extraction is because we are erasing a lot of type information, and importantly all proofs.^{2While in theory there is no clear delineation between proofs and programs, in any reasonable Coq development all proofs will be in the Prop sort. Extraction erases everything in Prop.}

Let’s look at this simple example first:

Require Coq.extraction.ExtrOcamlBasic.
Definition double_list {A} (l : list A) := l ++ l.
Recursive Extraction double_list.

The first line tells Coq to configure common options for OCaml extraction. (E.g. to map common types, like list, to their native OCaml equivalents.) The word Recursive in the last line tells Coq to extract not just double_list, but also anything that it depends on. We get this as the result of the extraction:

(** val app : 'a1 list -> 'a1 list -> 'a1 list **)

let rec app l m =
 match l with
 | [] -> m
 | a :: l1 -> a :: (app l1 m)

(** val double_list : 'a1 list -> 'a1 list **)

let double_list l =
 app l l

This is mostly what we expected. app is also extracted beside double_list, since it depends on it.

Erasing proofs

Let’s take this implementation of the list head function:

Definition hd {A} (l : list A) {prf : length l > 0} :=
 match l return length l > 0 -> _ with
 | [] => fun prf => ltac:(simpl in prf; lia)
 | h :: _ => fun _ => h
 end prf.

It receives a proof that the list is non-empty, and with this it can prove the empty-list case impossible. Coq’s extraction does a wonderful job with this function:

(** val hd : 'a1 list -> 'a1 **)

let hd = function
| [] -> assert false (* absurd case *)
| h :: _ -> h

All of the complicated proof manipulation is erased, and the impossible case is marked with an assert false.

The main takeaway you should have here is that proof manipulation in Coq has no runtime performance impact, since all proofs are erased. And as long you trust the extractor, the proofs still ensure the correctness of the extracted program.

Dependent types

Now you might be thinking that sure, proofs can be erased, but Coq is dependently typed, while OCaml is not, what happens when we try to extract a dependently typed function? Here is an example (you are going to have to wait until the next section for an explanation of what Z is and how it extracts to a literal 1):

Definition good_luck_with_this_ocaml (b : bool)
 : if b then Z else bool
 := match b with true => 1 | false => false end.

And here is what it extracts to:

type __ = Obj.t

(** val good_luck_with_this_ocaml : bool -> __ **)

let good_luck_with_this_ocaml = function
| true -> Obj.magic 1
| false -> Obj.magic false

Obj.magic is OCaml’s name for an unsafe cast between two arbitrary types. Unlike in Java or other languages, there is no runtime type information in OCaml, so a bad unsafe cast leads to memory corruption. For this reason, its use is very much discouraged. But, in this instance, since Coq’s typechecker (the kernel) already checked the types, we can be pretty sure that these Obj.magic usages are correct.

In fact, the extractor could just use Obj.magic everywhere and ignore OCaml’s type system alltogether. Extracting to an untyped functional programming language like Scheme is actually much easier than to OCaml: just erase all of the types, and that’s it. In this case we are just using Scheme as a convenient implementation of lambda calculus.

Extracting the above function to Scheme yields no weird types, since there are no static types in Scheme:

(define good_luck_with_this_ocaml (lambda (b)
(match b
 ((True) `(Zpos ,`(XH)))
 ((False) `(False)))))

The extractor actually goes to great lengths to come up with some reasonable OCaml translation of the Coq types, and it only falls back to Obj.magic when absolutely necessary. You can read more about the extraction mechanism in the Coq reference manual and in [3]P. Letouzey, “A New Extraction for Coq,” in Types for Proofs and Programs, G. Goos, J. Hartmanis, J. Van Leeuwen, H. Geuvers, and F. Wiedijk, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2003, pp. 200–219. doi: 10.1007/3-540-39185-1_12..

Arithmetic

Arithmetic by default does not play very well with extraction. Let’s take this example:

Definition double (a : nat) := 2 * a.

This double function gets extracted to this OCaml code:

type nat =
| O
| S of nat

module Nat =
 struct
 (** val add : nat -> nat -> nat **)

 let rec add n m =
 match n with
 | O -> m
 | S p -> S (add p m)

 (** val mul : nat -> nat -> nat **)

 let rec mul n m =
 match n with
 | O -> O
 | S p -> add m (mul p m)
 end

(** val double : nat -> nat **)

let double a =
 Nat.mul (S (S O)) a

This is not… great. nat is good when proving mathematical theorems because of its conceptual simplicity, but storing a number $n$ using $O(n)$ storage is pretty bad. What’s wrong with OCaml’s built in integers and arithmetic operators? Why can’t we just extract double simply to let double a = 2 * a in OCaml?

The issue is that that such an extraction would technically be incorrect. OCaml’s integers are 63 bit, and can’t represent all integers. In everyday programming practice we usually just ignore such limits, as they are rarely exceeded.

A solution to the aforementioned problem is to use Z instead of nat for arithmetic. I will let the comments from the standard library explain how Z works:

(** [positive] is a datatype representing the strictly positive integers
 in a binary way. Starting from 1 (represented by [xH]), one can
 add a new least significant digit via [xO] (digit 0) or [xI] (digit 1).
 Numbers in [positive] will also be denoted using a decimal notation;
 e.g. [6%positive] will abbreviate [xO (xI xH)] *)

Inductive positive : Set :=
 | xI : positive -> positive
 | xO : positive -> positive
 | xH : positive.

(** [Z] is a datatype representing the integers in a binary way.
 An integer is either zero or a strictly positive number
 (coded as a [positive]) or a strictly negative number
 (whose opposite is stored as a [positive] value).
 Numbers in [Z] will also be denoted using a decimal notation;
 e.g. [(-6)%Z] will abbreviate [Zneg (xO (xI xH))] *)

Inductive Z : Set :=
 | Z0 : Z
 | Zpos : positive -> Z
 | Zneg : positive -> Z.

I won’t show the extraction results using Z as it would be even longer, but with Z we at least don’t lose asymptotic complexity with arithmetic.

Z is still quite inefficient though, as we are storing each bit of a number as a heap object.

For some applications where arithmetic is not critical for performance, this can be worth it in exchange for the correctness guarantees. (E.g. CompCert does this.)

If you want to go further and step into dangerous territory, you can tell Coq to extract Z to OCaml’s native integers. This:

Require Import Coq.extraction.ExtrOcamlZInt.
Definition double (a : Z) := 2 * a.

Now translates into this:

module Z =
 struct
 (** val mul : int -> int -> int **)

 let mul = ( * )
 end

(** val double : int -> int **)

let double a =
 Z.mul ((fun p->2*p) 1) a

Close enough, the OCaml compiler can optimize this to achieve glorious full native performance.^{3Well, the thing is… OCaml actually has 63 bit integers, because it uses the lowest bit for tagging. And unfortunately this does affect performance.} In return, we lose correctness, as overflow now becomes possible.

(For completeness I would also like to mention that Coq does have built-in support for 63 bit integers, which are in fact safe to directly extract to OCaml integers. With this you are stepping into the territory of modular arithmetic though, which can complicate some proofs.)

Transpiling to JavaScript

Chaining together the OCaml extraction process with an OCaml to JavaScript transpiler like js_of_ocaml or melange works surprisingly well to get Coq code running in the browser. js_of_ocaml is more mature, and focuses more on producing code with correct behavior, while melange focuses on producing readable JavaScript, that is as close to the original OCaml code as possible. For this reason, I will be using melange’s output in the following few examples.

Taking our previous good_luck_with_this_ocaml example, and transpiling the extracted OCaml with melange, we obtain perfectly legible JavaScript:

function good_luck_with_this_ocaml(param) {
 if (param) {
 return 1;
 } else {
 return false;
 }
}

The main reason I wanted to show some JavaScript output is to demonstrate why tail-recursion is useful. Here is an example tail-recursive function to sum a list of numbers:

Fixpoint sum (l : list Z) (acc : Z) :=
 match l with
 | [] => acc
 | h :: l' => sum l' (acc + h)
 end.

And here is the JavaScript it translates to:

function sum(_l, _acc) {
 while(true) {
 var acc = _acc;
 var l = _l;
 if (!l) {
 return acc;
 }
 _acc = acc + l.hd | 0;
 _l = l.tl;
 continue ;
 };
}

The variable names are a bit ugly, but most importantly, it got translated into a loop, without any potentially slow (or stack overflowing) recursive function calls.

The `sig` type

In this section we are going to take a look at the sig type, which can be used for adding pre- and postconditions to functions.

Let’s look at the definition of the sig (which stands for sigma) type from the standard library:

Inductive sig {A : Type} (P : A -> Prop) : Type :=
 exist (x : A) (_ : P x) : sig P.

This is a special kind of pair, where the first item is a value of some type A, while the second member is a proof of some predicate P defined on A. Basically, this lets us package together some value, and a proof about it.

Let’s look at an example:

Theorem five_gt_zero : 5 > 0. lia. Qed.

Definition a_positive_number
 : sig (fun n => n > 0) := exist _ 5 five_gt_zero.

sig (fun n => n > 0) is the type of pairs of a number, and a proof that this number is greater than zero. If we don’t want to be too pedantic with the language, we can simply say that sig (fun n => n > 0) is the type of positive numbers. (For this reason sig is also called a subset type.)

The standard library also defines a notation for sig:

Notation "{ x : A | P }" := (sig (A:=A) (fun x => P)).

This is inspired by the set builder notation, as the sig type is similar to sets defined in this way. $\{ n \in \mathbb{N} \mid n > 0 \}$ in set theory and $\{ n : \mathtt{nat} \mid n > 0 \}$ in Coq’s type theory basically represent the same idea: one is the set of positive numbers, while the other is the type of positive numbers.

The way we use sig for specifying functions is by taking in or returning sig types. A sig argument is basically a precondition, while a returned sig is like a postcondition. You are going to see such uses of sig in this section.

Coq’s `#[program]` attribute

The #[program]^{4Note that you will more often see the legacy Program form used elsewhere, but for the sake of consistency I will be using the attribute form #[program].} attribute is a little weird, it provides some loosely related additions to Coq’s term language. I use it extensively in this project, so I wanted to introduce it briefly. What does it do exactly? From the reference manual:

The goal of #[program] is to program as in a regular functional programming language whilst using as rich a specification as desired and proving that the code meets the specification using the whole Coq proof apparatus.

To this end, it enriches Coq’s term language with a couple new features:

Equality hypotheses are automatically injected into match expressions. (So e.g. in match n with 0 => ... | _ => ... end the first branch will have a proof of n = 0 in its context.)
Any value of T and { x : T | P } can be used interchangeably. This is very useful when writing functions with rich specifications: we can write our function (mostly) like in a regular programming language, and #[program] will generate any necessary proof obligations needed for the sig types.
Fixpoints can automatically be transformed to use a measure (or any well-founded relation) for termination checking with a simple annotation. (See this section for an example.)
Filling of holes is automatically attempted using a predefined “obligation tactic”, and a proof obligation is only generated if this tactic fails.

#[program] is not a magic bullet, but it still helps a lot when working with rich specifications, and I am using it for most non-trivial functions in my R-tree implementation.

Termination

In Coq, all functions have to terminate. If non-terminating functions were allowed, proving False would be trivial:

Fixpoint proof_of_false (n : nat) : False
 := proof_of_false (n + 1).

When trying to check the above in Coq, we get an error message:

Error: Recursive definition of proof_of_false is ill-formed.
In environment
proof_of_false : nat -> False
n : nat
Recursive call to proof_of_false has principal argument equal to
"(n + 1)%nat" instead of a subterm of "n".
Recursive definition is: "fun n : nat => proof_of_false (n + 1)%nat".

On the other hand, this (terminating) function is correctly accepted:

Fixpoint fib (n : nat) :=
 match n with
 | 0 => 0
 | S n' =>
 match n' with
 | 0 => 1
 | S n'' => fib n' + fib n''
 end
 end.

So given that the halting problem is undecidable and all, how does Coq magically know which functions terminate?

It doesn’t. It only accepts a small subset of terminating functions, whose termination can be proven using a simple algorithm. The above definition of fib is already a bit contrived to get it accepted by Coq’s termination checker.

Coq uses something called structural recursion. All recursive functions need to have one of their arguments designated as the decreasing argument. (Most of the time Coq automatically guesses which argument is decreasing, so we don’t have to specify it.)

It then checks whether in each recursive call, the decreasing argument gets structurally smaller. According to the Coq manual “the definition of being structurally smaller is a bit technical”, but you are welcome to dive into it if you are interested in the details. Intuitively, it means that the decreasing argument is always contained in (a subterm of) the previous value of the decreasing argument.

This is easy to see in the fib example above: both n' and n'' are obtained from n by pattern matching on it.

Now let’s examine why the straightforward way to write fib doesn’t get accepted by Coq:

Fixpoint fib (n : nat) :=
 match n with
 | 0 => 0
 | 1 => 1
 | _ => fib (n - 1) + fib (n - 2)
 end.

n - 1 and n - 2 are not structurally smaller than n, as far as Coq’s simple termination checking algorithm is concerned.

Unfortunately, a lot of useful recursive functions cannot be written in a structurally recursive form. I will discuss a solution in the next section.

Proving termination manually

A commonly used technique for going beyond structural recursion is using well-founded relations.

A relation $\prec$ is well-founded if it contains no infinite sequence of elements $x_0, x_1, x_2, \ldots$ such that $\forall n \in \mathbb{N}, x_{n+1} \prec x_n$ .

Intuitively, this means that the relation behaves like $<$ behaves on the natural numbers: you can’t have an infinite sequence of decreasing numbers. Eventually, you are going to hit a minimal element ( $0$ in the case of $\mathbb{N}$ ). (And indeed, $<$ is well-founded.)

If you don’t quite get well-foundedness yet, don’t worry: natural numbers are all I am going to use in the next example. Just $\mathbb{N}$ equipped with $<$ already gives rise to a very powerful primitive for proving termination: if I can measure the arguments of a recursive function with a natural number, all it takes to prove termination is proving that the measure decreases with every recursive call. (A “measure” in this case simply means any function that outputs a natural number.)

We will skip over all the stuff about proving relations well-founded in Coq, and how this actually lets us make Coq accept our functions as terminating. (Spoiler: it’s still structural recursion, but now it’s structural recursion on well-foundedness proofs. I recommend [4]A. Chlipala, Certified programming with dependent types: A pragmatic introduction to the Coq proof assistant. Cambridge, MA: The MIT Press, 2013. Available: http://adam.chlipala.net/cpdt/ as a good introduction to well-founded recursion in Coq if you would like to know more.)

We will use Coq’s #[program] attribute, which takes care of everything in the background and lets us use measures directly. So, let’s try again with the “straightforward” fib definition, but now equip it with a measure:

#[program] Fixpoint fib (n : nat) {measure n} :=
 match n with
 | 0 => 0
 | 1 => 1
 | _ => fib (n - 1) + fib (n - 2)
 end.

In this case we don’t even have to define a separate measure function, as conveniently n is its own measure. #[program] leaves us with two obligations to prove. By using the Next Obligation command, we can jump straight into proving it:

n : nat
fib : forall n0 : nat, n0 < n -> nat
H0 : 0 <> n
H : 1 <> n
___________________(1/1)
n - 1 < n

Well this is easy, we just have to prove that n - 1 < n. (And also notice how #[program] helpfully added the hypotheses that n is not equal to 0 or 1.) lia makes short work of this. The other obligation is exactly the same, except that we have to prove that n - 2 < n. The finished proofs are very simple:

Next Obligation. lia. Qed.
Next Obligation. lia. Qed.

And we are done! We defined a function and proved that it’s terminating using a measure, which is a much more powerful primitive than structural recursion.

Rewriting with permutations

In Coq, you can use equality proofs for rewriting terms. (I mean, what else would you be using them for?) Let’s say we have a proof state like this (a b c cut from the context for brevity):

ab : a = b
bc : b = c
___________________(1/1)
a = c

By using the rewrite ab tactic, we can rewrite occurrences of a to b in the goal:

ab : a = b
bc : b = c
___________________(1/1)
b = c

Now we could finish the proof either by using rewrite bc to obtain c = c and then appealing to reflexivity, or just by writing exact bc since b = c is already a hypothesis.

Now let’s say we have a very similar proof state, but instead of equality we have Permutation relations:

ab : a ~ b
bc : b ~ c
___________________(1/1)
a ~ c

Here we could of course just say that Permutation is transitive by definition: apply (perm_trans ab bc). But curiously, we can also use rewrite in this situation, just like in the previous example. After using rewrite bc, we get exactly what you would expect:

ab : a ~ b
bc : b ~ c
___________________(1/1)
b ~ c

So… why does this work? As it turns out, Permutation is an equivalence relation: it’s reflexive, symmetric, and transitive. A set (or in this case, a type) equipped with an equivalence relation is called a setoid in mathematics. Coq allows extending the rewrite tactic to work with any setoid, and not just regular equality. (The standard library already contains the relevant proofs about Permutation being an equivalence relation, so we can use rewrite with it.)

Rewriting with morphisms

Let’s say we have this proof state (where f is some arbitrary function):

ab : a = b
___________________(1/1)
f a = f b

This is again easy to prove just by using rewrite ab to rewrite a to b on the left side, and then appealing to reflexivity. Let’s try the same when we are using permutations:

ab : a ~ b
___________________(1/1)
f a = f b

Issuing rewrite ab here yields an error:

setoid rewrite failed: Unable to satisfy the following constraints:
UNDEFINED EVARS:
?X39==[T f a b ab |- Relation_Definitions.relation (list T)] (internal placeholder) {?r}
?X40==[T f a b ab (do_subrelation:=Morphisms.do_subrelation)
|- Morphisms.Proper (Morphisms.respectful (Permutation (A:=T)) ?r) f] (internal placeholder) {?p}
?X42==[T f a b ab |- Relation_Definitions.relation (list T)] (internal placeholder) {?r0}
?X43==[T f a b ab (do_subrelation:=Morphisms.do_subrelation)
|- Morphisms.Proper (Morphisms.respectful ?r (Morphisms.respectful ?r0 (Basics.flip Basics.impl))) eq]
(internal placeholder) {?p0}
?X44==[T f a b ab |- Morphisms.ProperProxy ?r0 (f b)] (internal placeholder) {?p1}
TYPECLASSES:?X39 ?X40 ?X42 ?X43 ?X44
SHELF:||
FUTURE GOALS STACK:?X44 ?X43 ?X42 ?X40 ?X39||

Uh-oh. What a clear and concise error message… But if we think about what we are actually trying to do, Coq is definitely right to fail here: f a = f b is not true in this case for an arbitrary f. For example if f is a function that checks whether a list is sorted, this equality is definitely false.

We somehow need to restrict f to being a function that satisfies the above equality. In the language of Coq’s setoid rewriting system, this is what we need to prove about f:

Proper ((@Permutation A) ==> Logic.eq) f

This basically means that if x ~ y, then f x = f y also holds. The ==> here denotes “morphisms that are both covariant and contravariant”. What does “morphism” mean in this context? What does “proper” mean? I am not sure exactly, but importantly they do exactly what we need them to do. You are welcome to dive into the relevant chapter of the Coq reference manual if would like to know more, godspeed.

What’s important is that with a proof of the above property in scope, the previous rewrite works now:

ab : a ~ b
___________________(1/1)
f a = f b

Trying rewrite ab here again does exactly what we wanted:

ab : a ~ b
___________________(1/1)
f b = f b

If you are interested in an actual example, I will be establishing a Proper instance for one of my functions in this section. To see the proof scripts utilizing permutation rewriting, you can check out the source code.

Proof automation

In general, I tried to follow the advice given in Adam Chlipala’s book, Certified Programming with Dependent Types [4]A. Chlipala, Certified programming with dependent types: A pragmatic introduction to the Coq proof assistant. Cambridge, MA: The MIT Press, 2013. Available: http://adam.chlipala.net/cpdt/:

In this book, I have been following an unusual style, where proofs are not considered finished until they are “fully automated,” in a certain sense. Each such theorem is proved by a single tactic.

My position is that any ideas that standard automation can find are not very big after all, and the real big ideas should be expressed through lemmas that are added as hints.

He advocates for a style where “important” ideas are expressed as separate lemmas, while individual proofs are automated to the furthest extent possible.

While I tried to follow this style, my Ltac skills are still only rudimentary. As such, I won’t discuss the details of any non-single-tactic proofs I made, since they probably don’t contain any important ideas, and are just a failure on my part to automate them sufficiently.

Similarly, since I only started getting my feet wet with more advanced proof automation, my Ltac scripts are still quite messy. I am not sure that showcasing them in detail would provide anything valuable, since there are probably better ways to achieve everything I am doing here. Nonetheless, I will show two tactics here as examples for what you can do with proof automation.

Reflection for use with `lia`

lia can only work with propositions and not booleans, so I made a tactic to turn boolean relations into their propositional equivalents:

Ltac tac_reflect _ :=
 match goal with
 (** normalize equality orders *)
 | H : true = ?x |- _ => symmetry in H
 | H : false = ?x |- _ => symmetry in H
 (** reflect leb as le *)
 | H : (?a <=? ?b) = true |- _ => apply leb_complete in H
 | H : (?a <=? ?b) = false |- _ => apply leb_iff_conv in H
 | H : (?a <? ?b) = true |- _ => apply Nat.ltb_lt in H
 | H : (?a <? ?b) = false |- _ => apply Nat.ltb_ge in H
 end.

This tactic can for example turn (x <? y) = false into x >= y. lia can only work with the second form, so this is quite useful.

I also have lemmas for turning boolean operators into propositions:

Theorem reflect_andb_true : forall (a b : bool),
 a && b = true <-> a = true /\ b = true.
Theorem reflect_andb_false : forall (a b : bool),
 a && b = false <-> a = false \/ b = false.

I haven’t integrated these with tac_reflect yet though.

(Note that reflection is a much more general technique that can be used in Coq, check out the relevant chapter in Software Foundations for an introduction. My relatively simplistic tactics were enough for this project though.)

Solving permutations

The following tactic solves goals of the form _ ~ _, where the two sides can be made equal just by rearranging them.

Ltac bring_to_front x :=
 repeat (
 try rewrite (app_assoc _ x);
 rewrite (Permutation_app_comm _ x);
 try rewrite <- (app_assoc x _)
 ).

Ltac solve_permutation :=
 solve [
 repeat rewrite <- app_assoc;
 rewrite <- ? Permutation_rev;
 repeat match goal with
 | |- ?x ++ _ ~ _ =>
 bring_to_front x;
 apply Permutation_app_head
 | |- ?x ~ ?x => reflexivity
 end].

solve_permutation expects both sides to be a sequence of terms separated by the ++ operator. It looks at the first term on the left hand side, and tries to find the same term and bring it to the front on the right hand side as well. bring_to_front works by repeatedly appealing to the associativity and commutativity of ++. (++ is not commutative with respect to regular equality of course, but it is with respect to permutation.)

If bring_to_front succeeds, the two head terms can be removed from both sides, and the procedure is repeated again. The loop terminates when both sides are equal, and the goal can be solved by simply appealing to the reflexivity of ~.

Implementation

Now that you have a basic overview of all the techniques that go into developing a verification project like this, we are ready to jump into discussing the actual implementation.

Axis-aligned bounding boxes

First, we have to develop a theory of the bounding boxes that the whole data structure is build upon.

We define bounding boxes as follows:

Definition interval : Type := Z * Z.
Definition bbox : Type := interval * interval.

An interval is a pair of integers, and a bounding box is simply a pair of intervals. (We could of course generalize to $d$ dimensions by defining a hyper-bounding box to be a vector of $d$ intervals, but for now I am satisfied with the $d=2$ special case.)

We define the following functions on intervals:

Definition interval_intersect : interval -> interval -> bool :=
 fun '(pair a_lo a_hi) '(pair b_lo b_hi) =>
 (a_lo <=? b_hi) && (b_lo <=? a_hi).
Definition interval_inside : interval -> interval -> bool :=
 fun '(pair a_lo a_hi) '(pair b_lo b_hi) =>
 (b_lo <=? a_lo) && (a_hi <=? b_hi).
Definition interval_mbr : interval -> interval -> interval :=
 fun '(pair a_lo a_hi) '(pair b_lo b_hi) =>
 (Z.min a_lo b_lo, Z.max a_hi b_hi).
Definition interval_measure : interval -> Z :=
 fun '(pair lo hi) => hi - lo.

interval_intersect: Decides whether two intervals intersect, that is, whether for two intervals $a$ and $b$ , $a \cap b \neq \empty$ .
interval_inside: Decides whether $a \sube b$ .
interval_mbr: Computes the minimum bounding interval of two intervals.
interval_measure: Computes the length of the interval.

Nothing fancy going on here, you would write these functions the same way in any other programming language. (We won’t even be proving them correct, e.g. that for any $a = [a_1, a_2]$ and $b = [b_1, b_2]$ , $a \cap b \neq \empty \iff a_1 \leq b_2 \land b_1 \leq a_2$ . We will just prove a few select properties which we need.)

Now that we have these definitions, extending them to bounding boxes is straightforward:

Definition bbox_intersect : bbox -> bbox -> bool :=
 fun '(pair ax ay) '(pair bx by_) =>
 interval_intersect ax bx && interval_intersect ay by_.
Definition bbox_inside : bbox -> bbox -> bool :=
 fun '(pair ax ay) '(pair bx by_) =>
 interval_inside ax bx && interval_inside ay by_.
Definition bbox_mbr : bbox -> bbox -> bbox :=
 fun '(pair ax ay) '(pair bx by_) =>
 (interval_mbr ax bx, interval_mbr ay by_).
Definition bbox_measure : bbox -> Z :=
 fun '(pair x y) => interval_measure x * interval_measure y.

I will be using $\sube$ and ${\lvert\,\cdot\,\rvert}$ to denote bbox_inside and bbox_measure respectively.

Next, here are the facts that we will need about bounding boxes:

Theorem bbox_mbr_comm (a b : bbox)
 : bbox_mbr a b = bbox_mbr b a.
Theorem bbox_mbr_assoc (a b c : bbox)
 : bbox_mbr (bbox_mbr a b) c = bbox_mbr a (bbox_mbr b c).
Theorem bbox_inside_refl (a : bbox)
 : bbox_inside a a = true.
Theorem bbox_inside_mbr (a b : bbox)
 : bbox_inside a (bbox_mbr a b) = true.
Theorem bbox_mbr_monotonic a b c
 : bbox_inside a b = true ->
 bbox_inside a (bbox_mbr b c) = true.
Theorem bbox_inside_not_intersect a b c
 : bbox_inside a b = true ->
 bbox_intersect c b = false ->
 bbox_intersect c a = false.

bbox_mbr_comm, bbox_mbr_assoc: The minimum bounding rectangle (MBR) operation is commutative and associative.
bbox_inside_refl: $\sube$ is reflexive on bounding boxes.
bbox_inside_mbr: Bounding boxes are inside their MBRs.
bbox_mbr_monotonic: If $a \sube b$ , and you take the MBR of $b$ with any other $c$ , $a$ will still be inside the resulting MBR.
bbox_inside_not_intersect: If $a \sube b$ , and there is a $c$ that does not intersect $b$ , then it also does not intersect $a$ . This is the essential property that will let us discard branches during tree search.

For these particular set of theorems, I was able to put together a single proof script to completely automate all of the proofs. This script can prove all of the theorems above:

Ltac t :=
 intros;
 repeat match goal with
 | H : interval |- _ => destruct H
 | H : bbox |- _ => destruct H
 end;
 simpl in *;
 repeat match goal with
 | |- (_, _) = (_, _) => f_equal
 end;
 reflect_lia;
 lia.

The idea is to completely break up the goal into a system of linear equations and simple logical connectives. For example, for bbox_inside_not_intersect, here is the proof state just before the final lia tactic:

z9, z10, z7, z8, z5, z6, z3, z4, z1, z2, z, z0 : Z
H : (z5 <= z9 /\ z10 <= z6) /\ z3 <= z7 /\ z8 <= z4
H0 : ~ ((z1 <= z6 /\ z5 <= z2) /\ z <= z4 /\ z3 <= z0)
___________________(1/1)
~ ((z1 <= z10 /\ z9 <= z2) /\ z <= z8 /\ z7 <= z0)

This form is already beyond the limit of easy human comprehension. (12 inequalities, 11 variables, various logical connectives interspersed.) Fortunately, a specialized automated solver like lia shines in a situation like this, and makes quick work of the proof.

Next, we extend the theory about minimum bounding rectangles to any list of boxes, not just two:

Definition mbr (l : list bbox) :=
 match l with
 | [] => ((0, 0), (0, 0))%Z
 | x :: l => fold_left bbox_mbr l x
 end.

The ((0, 0), (0, 0)) is just a dummy value for when the list is empty. It doesn’t really matter what we put here, in the theorems where it’s necessary we will specify that the list is non-empty anyway.

We extend the idea behind bbox_inside_mbr to our new mbr function as well:

Theorem mbr_inside l i : In i l -> bbox_inside i (mbr l) = true.

This should be quite intuitive, we are just saying that for any list of boxes, all of them are inside their minimum bounding rectangle.

I already talked about Coq’s generalized rewriting ability, and how we can use it to rewrite with the $\sim$ relation. We want to be able to do this rewriting in the argument of the mbr function as well, and essentially what we have to do is prove that the result of mbr does not depend on the order of its inputs.

As discussed in a previous section already, this is how we can express this in the language of Coq’s setoid rewriting system:

Instance Permutation_mbr
 : Proper (@Permutation bbox ==> Logic.eq) mbr.

The proof is quite straightforward, we are basically proving that mbr does not depend on the order of the boxes in its argument.

This should conclude everything we wanted to know about bounding boxes. To recap:

We defined what bounding boxes (bbox) are.
Defined functions on them like intersection, containment, and the minimum bounding rectangle.
We proved some basic theorems about these functions. We will be making heavy use of these in later proofs.

List utility functions

In this section we will define a few simple functions for manipulating lists which can’t be found in the standard library. The catch is that I have a few special requirements:

The functions should have strong specifications that we can use in later proofs.
They should also be tail recursive for runtime efficiency.

This is the first time we will be defining functions together with their own specifications (as opposed to proving separate theorems about them as we did in the previous section about bounding boxes), using the sig types I previously introduced.

Removing items from a list

Let’s look at the definition of the remove_ith function, which we will discuss afterwards:

#[program] Fixpoint remove_ith'
 {T : Type}
 (rev_front back : list T)
 (i : { i : nat | i < length back })
 {measure i}
 : { '(e, l) : T * list T | e :: l ~ rev_front ++ back } :=
 match back, i with
 | [], _ => _
 | (x :: back'), 0 => (x, rev_append rev_front back')
 | (x :: back'), (S i') => remove_ith' (x :: rev_front) back' i'
 end.

#[program] Definition remove_ith
 {T : Type}
 (l : list T)
 (i : { i : nat | i < length l })
 : { '(x, l') : T * list T | x :: l' ~ l } :=
 proj1_sig (remove_ith' [] l i).

remove_ith' follows an accumulator pattern to achieve tail recursion, while remove_ith is simply a wrapper around it with the initial accumulator values.

The specification for remove_ith states that it returns a pair $(x, l')$ which satisfies $x \mathop{::} l' \sim l$ . As you can see, the specification is now directly embedded in the function’s type, unlike in the previous section, where the bounding box functions and the theorems about them were separate.

Also notice how the parameter i : { i : nat | i < length l } is a subset type as well. It ensures that you don’t attempt to index the list out of bounds. In other programming languages handling simple constraints like this can be tricky, and no good solution exists really:

In an unsafe language like C you could just ignore the possibility of i being out of bounds, trusting every caller of your function to respect this implicit invariant.
You could check the value at runtime, and throw an exception or abort the whole program if it’s out of bounds.
You could use an optional type like option A, and return None when i is out of bounds.

In Coq, there is another unique way to deal with this problem. Let me repeat the relevant match expression from remove_ith':

 match back, i with
 | [], _ => _
 | (x :: back'), 0 => (x, rev_append rev_front back')
 | (x :: back'), (S i') => remove_ith' (x :: rev_front) back' i'
 end.

The first branch pattern [], _ matches when the back list is empty. The branch is simply a single underscore, which is a placeholder to be filled later. So, how is it filled?

In Coq, anywhere where a regular value is expected, we can also provide an impossibility proof (a proof of False) instead.^{5Actually this is not much of a special case at all. If we have a proof p of False, we can just pattern match on it like this: match p with end. This is a weird looking pattern match, which in fact has zero branches, since False has zero constructors. This pattern match can give you any value of your choosing. In Coq, ex falso quodlibet means you can not only prove anything, you can also get a value for anything.⁶} Here, this is quite easy: i carries a proof of i < length back, which in this case reduces to i < 0. lia can readily prove that this is a contradiction. (And it is part of the automation script that #[program] automatically applies, so the hole is automatically taken care of.)

I also defined remove_pair, which is similar except that it removes two items from a list. (Understanding its definition is an exercise left to the reader.)

Finding minimal items in a list

We start this section by declaring some common parameters:

Context {T M : Type} `{Ord_M : Ord M} (measure : T -> M).

T is the type of items, while M is the type of comparable values that the measure function (not to be confused with the measures we use for proving termination) produces from any T. Ord_M is an implementation of the total order for M, which importantly provides the leq comparator function. (We are over-generalizing here a bit, since we will only be substituting Z for M in later parts of the development, but this way we can ensure that we are only depending on Z being ordered, and not on any of its other properties.)

There is nothing fancy going on with get_min, we tail-recursively iterate the list and select the minimal element:

#[program] Fixpoint get_min'
 (i : nat)
 (l : list T)
 (mini : { mini | mini < i })
 (min : M)
 : { '(n, _) : nat * M | n < length l + i } :=
 match l with
 | [] => (mini, min)
 | x :: l' =>
 let mx := measure x in
 if leq mx min then
 get_min' (S i) l' i mx
 else
 get_min' (S i) l' mini min
 end.

#[program] Definition get_min
 (l : { l : list T | 0 < length l })
 : { '(n, _) : nat * M | n < length l }:=
 match l with
 | [] => _
 | x :: l' => get_min' 1 l' 0 (measure x)
 end.

Notice how we don’t actually prove get_min correct. In theory my implementation could be wrong and it might not return the minimal item. It turns out that we don’t actually need this for the R-tree correctness proof. (A theoretical bug existing in get_min cannot impact overall correctness, but it could degrade performance.)

I also needed a function to find minimal pairs of items in a list. The parameters are very similar, but now measure takes two Ts:

Context {T M : Type} `{Ord_M : Ord M} (measure : T -> T -> M).

Definition get_min_pair (l : { l : list T | 2 <= length l })
 : { '(ia, ib, _) : nat * nat * M | ia + ib + 1 < length l }.

The signature of get_min_pair is a bit more complicated since it has to handle two indices. The implementation is the straightforward quadratic brute force search.

Partitioning (for node splitting)

When explaining how R-tree insertion works, I glossed over what heuristic algorithm is used exactly to do the node partitioning. In this section we will cover the particular algorithm I used in detail.

The original R-tree paper [2]A. Guttman, “R-trees: A dynamic index structure for spatial searching,” in Proceedings of the 1984 ACM SIGMOD international conference on Management of data - SIGMOD ’84, Boston, Massachusetts: ACM Press, 1984, p. 47. doi: 10.1145/602259.602266. describes three different algorithms for doing node splitting:

The “Exhaustive Algorithm”, that simply tries all possible partitionings. This algorithm is exponential in complexity, but importantly it’s exponential in $M$ , so it can actually be a viable option when $M$ (which is a pre-defined constant for a whole R-tree) is small.
The “Quadratic-Cost Algorithm”, which is quadratic in $M$ .
The “Linear-Cost Algorithm”, which is linear in $M$ .

I chose to implement the quadratic algorithm as a good middle ground. It works roughly as follows:

Define the “dead-space” of two rectangles $a, b$ to be $\lvert \operatorname{mbr}(a, b) \rvert - \lvert a \rvert - \lvert b \rvert$ . This calculates the “wasted area” when grouping $a$ and $b$ together. In the first step choose two entries with the largest dead-space, and make them the initial seed entries of the two groups.
Repeatedly determine the next best entry to assign to a group, and assign it. The next best entry is the one which has the largest difference in area enlargement if assigned to the two groups. (The area enlargement an entry $e$ incurs on a group $g$ is simply $\lvert \operatorname{mbr}(g,e) \rvert - \lvert g \rvert$ .) Assign it to the group whose bounding rectangle has to be enlarged the least to accommodate the new entry.
If one of the groups has so few items that it needs all of the remaining entries to have at least $m$ items (as required for a well-formed R-tree), just assign all of the remaining entries to that group.

Picking the seeds can easily be implemented using get_min_pair:

#[program] Definition partition_pick_seeds
 (l : { l : list T | 2 <= length l })
 : { '(a, b, l') : T * T * list T | a :: b :: l' ~ l } :=
 let '(mini, _) := get_min_pair
 (fun a b => dead_space (item_bbox a) (item_bbox b)) l in
 remove_pair l mini (prf:=_).

The function for picking the next entry is not very difficult either using get_min:

#[program] Definition partition_pick_next
 (bbox1 bbox2 : bbox)
 (l : { l : list T | 1 <= length l })
 : { '(e, l') : T * list T | e :: l' ~ l } :=
 let neg_diff x :=
 let x := item_bbox x in
 Z.opp $ Z.abs (enlargement bbox1 x - enlargement bbox2 x) in
 let '(i, _) := get_min neg_diff l in
 remove_ith l i.

Assembling these together is a bit involved, you can look at the source code if you are interested in the details. The final signature of our partition function looks like this:

Definition partition (l : { l : list T | length l = M + 1 })
 : { '((la, mbra), (lb, mbrb)) : (list T * bbox) * (list T * bbox) |
 m <= length la <= M /\ m <= length lb <= M /\
 la ++ lb ~ l /\
 mbra = mbr (map item_bbox la) /\ mbrb = mbr (map item_bbox lb)
 }.

The postcondition says that:

both output groups have appropriate lengths,
the two groups together are a permutation of the input list,
and that the returned bounding boxes do indeed correspond to the returned groups.

The tree data structure

Now we need to define the type of trees. I initially did try to encode the needed invariants into the tree data structure itself, but I ran into issues with Coq’s positivity requirements, so the tree data structure itself is just like what you would see in non-dependent languages:

Inductive tree : Type :=
 | Leaf : bbox -> list item -> tree
 | Node : bbox -> list tree -> tree.

A leaf has items as its children, while inner nodes have other trees as their children.

Instead of encoding the invariants directly in tree, we will encode them as a separate inductive proposition called wf_tree (“wf” here stands for well-formed):

Inductive wf_tree : tree -> Prop :=
 | wf_leaf
 (b : bbox)
 (l : list item)
 (p : m <= length l <= M)
 (pmbr : b = mbr (map item_bbox l))
 : wf_tree (Leaf b l)
 | wf_node
 (b : bbox)
 (l : list tree)
 (p : m <= length l <= M)
 (pl : forall t, In t l -> wf_tree t)
 (pmbr : b = mbr (map tree_bbox l))
 : wf_tree (Node b l).

A leaf is well-formed if the number of its children is between the required bounds, and its bounding box is the minimum bounding rectangle of its children. Inner nodes have to satisfy the same properties, and additionally all of their children have to be well-formed as well.

We also define a property called wf_root, which is identical to wf_tree, except that the lower bounds for the number of children are relaxed. (Root nodes are a bit special, since we can’t guarantee that they will have at least m children.)

We will often need to deal with well-formed trees, and a pattern you will be seeing is a tree and a proof of its well-formedness packaged together into a single sig type: { t : tree | wf_tree t }.

We can now straightforwardly define tree_items (which we denoted with $\iota$ ):

Fixpoint tree_items (t : tree) :=
 match t with
 | Leaf _ items => items
 | Node _ nodes => flat_map tree_items nodes
 end.

We also prove an intuitively true, but important lemma, namely that all items in a well-formed tree are inside its bounding box (this is just a corollary of pmbr in wf_tree):

Definition Items_inside (t : tree) :=
 forall i, In i (tree_items t) ->
 bbox_inside (item_bbox i) (tree_bbox t) = true.

Lemma wf_tree_items_inside {t} (w : wf_tree t) : Items_inside t.

Last but not least, we need to define a child_tree relation, and prove that it’s well-founded (not to be confused with well-formed):

Definition child_tree (a b : tree) :=
 match b with
 | Leaf _ _ => False
 | Node _ nodes => In a nodes
 end.

Theorem wf_child_tree : well_founded child_tree.

The child_tree relation should be straightforward. well_founded comes from Coq’s standard library: we are proving that child_tree is a well-founded relation. (See this previous section for an explanation of well-foundedness.)

Again we are doing a lot of work just to prove something seemingly obvious: when we only recurse on child trees, eventually we have to get to the bottom, and we can’t get into an infinite loop!

Tree operations

Insertion

The core of the insert implementation looks like this:

#[program] Fixpoint insert'
 (t : { t : tree | wf_tree t })
 (i : item)
 {wf child_tree t}
 : { l |
 1 <= length l <= 2 /\
 (forall t, In t l -> wf_tree t) /\
 i :: tree_items t ~ flat_map tree_items l
 } :=
 match t with
 | Leaf mbr items =>
 if length items <? M then
 [Leaf (bbox_mbr (item_bbox i) mbr) (i :: items)]
 else
 let '((lx, mbrx), (ly, mbry)) := partition item_bbox (i :: items) in
 [Leaf mbrx lx; Leaf mbry ly]
 | Node _ nodes =>
 let '(idx, _) := get_min
 (fun n => enlargement (tree_bbox n) (item_bbox i)) nodes in
 let '(L, nodes) := remove_ith nodes idx in
 let nodes := insert' L i ++ nodes in
 if M <? length nodes then
 let '((lx, mbrx), (ly, mbry)) := partition tree_bbox nodes in
 [Node mbrx lx; Node mbry ly]
 else
 [Node (mbr (map tree_bbox nodes)) nodes]
 end.

We take a well-formed tree and an item to insert. We return either 1 or 2 well-formed trees, whose items are exactly the items of the original tree plus the inserted item.

The implementation uses partition to do the heavy lifting when splitting nodes, and is relatively straightforward. All the proofs that are returned by partition, get_min, and insert' itself in the recursive case can be used to assemble the proof for the specification of insert'.

Finally we wrap insert' with a separate function to handle the special cases with the root node, which will be our actual insert implementation:

#[program] Definition insert
 (t : { t : tree | wf_root t })
 (i : item)
 : { t' | wf_root t' /\ i :: tree_items t ~ tree_items t' }.

This signature is actually quite simple compared to some others we have seen. It says that insert takes a well-formed tree and an item, and it returns a well-formed tree, whose items are exactly the items of the original tree plus the new item.

Search

To search for items intersecting a query rectangle q, we traverse the whole tree and skip any subtrees that do not intersect q. The central lemma stating that this optimization is correct is called tree_item_intersect:

Lemma tree_item_intersect
 {q t i}
 (Hwf : wf_tree t)
 (Hin : In i (tree_items t))
 (Hint : bbox_intersect q (tree_bbox t) = false)
 : bbox_intersect q (item_bbox i) = false.

This lemma says that if q does not intersect the bounding box of a tree, then it cannot intersect the bounding box of any of its items either. This fact is a simple corollary of wf_tree_items_inside (all items of a tree are inside its bounding box) and bbox_inside_not_intersect.

Equipped with this lemma, we are ready to prove range_search correct:

#[program] Fixpoint range_search
 (t : { t | wf_root t })
 (q : bbox)
 {wf child_tree t}
 : { l : list item | l ~ filter
 (fun i => bbox_intersect q (item_bbox i)) (tree_items t) } :=
 match t with
 | Leaf _ items => filter (fun i => bbox_intersect q (item_bbox i)) items
 | Node mbr nodes =>
 let ns := filter
 (fun n => bbox_intersect q (tree_bbox (proj1_sig n)))
 (forall_distr (P:=(fun n => In n nodes)) nodes _) in
 flat_map (fun n => range_search (exist _ (proj1_sig n) _) q) ns
 end.

The specification is quite straightforward: range_search returns all items of the tree that intersect q. The implementation is quite straightforward, except for a couple extra things (forall_distr, proj1_sig, and exist) needed to shuffle around the correctness proof. (The proof script following the above definition is not long either, around 20 lines.)

Deletion

I define deletion as a kind of filtering: we are filtering items intersecting a bounding box q with a predicate f.

Context (f : item -> bool) (q : bbox).

For use in the specification of delete, we define f' to determine which items are actually kept: items either not intersecting q, or items which satisfy f.

Let f' i := negb (bbox_intersect q (item_bbox i)) || f i.

The specification of delete is then quite straightforward:

Definition delete (t : { t | wf_root t })
 : { t' | wf_root t' /\ (tree_items t' ~ filter f' (tree_items t)) }.

The items of the returned tree are just the items of the original tree filtered with f'.

To recap, deletion works by removing items that need to be removed, and also dropping any nodes that would become invalid as a result. At the end, we reinsert all items removed due to this “collateral damage”. To do this, we need a little helper function to insert a list of items into a tree:

#[program] Fixpoint insert_many
 (t : { t | wf_root t })
 (items : list item)
 {measure (length items)}
 : { t' | wf_root t' /\ items ++ tree_items t ~ tree_items t' } :=
 match items with
 | [] => t
 | i :: items' => insert_many (insert t i) items'
 end.

We can’t just do fold_left insert items t because we also need a correctness proof for insert_many.

The structure of the implementation is similar to that of insertion, with a tail recursive delete' function operating on the inner nodes, and a wrapper called delete handling special cases for the root.

In delete' we return an option { t | wf_tree t } * list tree pair. We either return the updated subtree or nothing (if the whole subtree is just gone as the result of the deletion), and a list of subtrees whose items need to be reinserted at the end.

The specification became quite verbose, so I pulled it out into a separate definition:

Definition delete_R (t : tree)
 : option { t | wf_tree t } * list tree -> Prop := fun '(t', N) =>
 match t' with
 | None => []
 | Some t' => tree_items (proj1_sig t')
 end ++ flat_map tree_items N ~ filter f' (tree_items t).

This basically says that the set of items in the returned trees (the optional one and the list that needs to be reinserted) is the same as the items in the original tree filtered with f'.

With delete_R, the signature of delete' becomes very simple:

Fixpoint delete' (t : { t | wf_tree t })
 : sig (delete_R (proj1_sig t)).

I won’t show the implementation here, it’s a mess, here is the source code if you are interested.

You can check out the final delete implementation here. It handles some special cases for the root, and at the end it uses insert_many to reinsert the needed items.

Putting everything together

Believe it or not, the hard part is over! Remember the Container interface from the section on specifications? All we have to do now is assemble all of our work into an implementation of this interface.

First, let’s define some parameters:

Context {I : Type} {item_bbox : I -> bbox}.

And some constants:

Let m := 4.
Let M := 8.
Let pm1 : 2 <= m := ltac:(lia).
Let pm2 : 2 * m <= M := ltac:(lia).

(We could make m and M parameters as well, but for the sake of simplicity I will leave them as constants for now.)

Let’s define the type of our container to be the well-formed tree:

Definition T := { data : tree I | wf_root m M I item_bbox data }.

Then we have to define some wrapper functions which operate on T (this basically amounts to plugging in m and M, and shuffling around some proofs):

Definition empty :=
 exist _ (empty I) (wf_empty m M pm1 pm2 I item_bbox).
Definition insert t i :=
 let '(exist _ data' (conj wf' _))
 := insert m M pm1 pm2 I item_bbox t i in
 (exist _ data' wf').
Definition search t q :=
 let '(exist _ res _)
 := (range_search m M pm1 pm2 I item_bbox t q) in
 res.
Definition delete t q f :=
 let '(exist _ data' (conj wf' _))
 := delete m M pm1 pm2 I item_bbox f q t in
 (exist _ data' wf').

Next, we have to prove the necessary theorems about these wrapper functions, exactly the ones that Container expects (again, this is very easy and just amounts to plugging in the proofs that the wrapped functions already return about themselves):

Ltac t :=
 intros;
 unfold all_items, insert, delete, search;
 repeat (match goal with
 | |- context[let '(exist _ _ _) := ?x in _] => destruct x
 | prf : _ /\ _ |- _ => destruct prf
 | prf : _ ~ _ |- _ => simpl; simpl in prf; now rewrite prf
 end).

Theorem empty_correct
 : all_items empty ~ [].
Proof. reflexivity. Qed.

Theorem insert_correct
 : forall t i, all_items (insert t i) ~ [i] ++ all_items t.
Proof. t. Qed.

Theorem delete_correct
 : forall t q f, all_items (delete t q f) ~ filter
 (fun i => negb (bbox_intersect q (item_bbox i)) || f i) (all_items t).
Proof. t. Qed.

Theorem search_correct
 : forall t q, search t q ~ filter
 (fun i => bbox_intersect q (item_bbox i)) (all_items t).
Proof. t. Qed.

And now we have exactly what’s needed to define an implementation of Container:

Instance rtree_container : Container := {
 T := T;
 items := all_items;

 empty := empty;
 insert := insert;
 search := search;
 delete := delete;

 empty_correct := empty_correct;
 insert_correct := insert_correct;
 search_correct := search_correct;
 delete_correct := delete_correct;
}.

And that’s it. rtree_container is now an R-tree implementation that carries the proof of its own functional correctness with it.

Benchmarks

In a famous quote Donald Knuth once wrote:

Beware of bugs in the above code; I have only proved it correct, not tried it.

So, let’s actually run the code. While performance was not the focus of this project, I still wanted to know how a program that has been directly extracted from a verified Coq implementation performed. Here are the results, comparing it to various other implementations:


my C implementation	0.05 ms
rbush (JavaScript) [5]V. Agafonkin, “RBush,” GitHub repository. Available: https://github.com/mourner/rbush	0.07 ms
brute force in C	0.60 ms
my JavaScript implementation	0.74 ms
brute force (JavaScript)	1.15 ms
verified-rtree (OCaml)	1.26 ms
verified-rtree (js_of_ocaml)	2.65 ms
ocaml-rtree [6]M. Eriksen and P. Ferris, “Ocaml-rtree,” GitHub repository. Available: https://github.com/geocaml/ocaml-rtree	3.11 ms
brute force (OCaml)	4.93 ms

Querying was measured with $N = 2 \cdot 10^5$ boxes, $k = 214$ results from the query, and $M = 8$ maximum branching factor. Results are averaged over 10 runs. Both Z and nat were extracted to OCaml integers for performance, I talked about the caveats here. Language implementations used: OCaml 5.1.0, Node.js 18.17.1, and GCC 12.3.0.

My naive unoptimized C implementation is the fastest, I am only mildly surprised by this. The big surprise in this benchmark is rbush [5]V. Agafonkin, “RBush,” GitHub repository. Available: https://github.com/mourner/rbush, written in JavaScript, which is basically on par with the C implementation in terms of speed. Shout out to whoever optimized it this well.

Next up is brute force (linear search) in C, not too surprised with this one either, it is well known that on modern CPUs simple linear algorithms which utilize cache prefetching and other hardware optimizations often outperform smarter algorithms.

The other results seem to suggest that in general JavaScript outperforms OCaml, which is quite surprising to me. Even brute force search in JavaScript is better than anything in OCaml.

My verified R-tree performs quite well in OCaml (it beats ocaml-rtree by a significant margin), but not so well in JavaScript (it’s outperformed by brute force search).

The worst one is ocaml-rtree, the only one it beats is brute force search in OCaml.

Conclusion? A lot of these results are surprising to me. If you discover something wrong with my testing setup, please let me know. It would also be interesting to dig into why some implementations are performing unexpectedly badly (like my R-tree compiled to JavaScript, or ocaml-rtree), and why some (like rbush) are performing so well.

The most important takeaway though in the context of this article is that formal verification doesn’t incur performance overhead. As this benchmark demonstrates, there are R-tree implementations out there (ocaml-rtree) that are conclusively beaten by my verified implementation in terms of performance.

Conclusions

On the topic of writing dependently typed programs in Coq, I have mixed feelings. When reading Adam Chlipala’s book [4]A. Chlipala, Certified programming with dependent types: A pragmatic introduction to the Coq proof assistant. Cambridge, MA: The MIT Press, 2013. Available: http://adam.chlipala.net/cpdt/, I got the feeling that dependently typed programming is something that he recommends you do in Coq. I unfortunately ran into obstacles when trying to scale it beyond the simple examples showcased in the book, manually inserting convoy patterns and inline proofs everywhere is simply prohibitive. The #[program] facility in Coq helps a bit with these issues (and I used #[program] extensively for this project), but #[program] has many issues and isn’t developed anymore as far as I can tell.

I still need to get better at writing proofs in Coq, I feel like too many things were done just by trial-and-error instead of advance planning. Still, this is by far the largest proof development I have done to date, and I learned a lot during the course of this project. If you made it this far, I hope you learned something new as well, and I managed to get you interested in using formal methods for improving software correctness.

Container is actually a typeclass, a feature popularized by Haskell. In Coq they are mostly equivalent to record types, which are just syntax sugar for inductive types. ↩︎
While in theory there is no clear delineation between proofs and programs, in any reasonable Coq development all proofs will be in the Prop sort. Extraction erases everything in Prop. ↩︎
Well, the thing is… OCaml actually has 63 bit integers, because it uses the lowest bit for tagging. And unfortunately this does affect performance. ↩︎
Note that you will more often see the legacy Program form used elsewhere, but for the sake of consistency I will be using the attribute form #[program]. ↩︎
Actually this is not much of a special case at all. If we have a proof p of False, we can just pattern match on it like this: match p with end. This is a weird looking pattern match, which in fact has zero branches, since False has zero constructors. This pattern match can give you any value of your choosing. In Coq, ex falso quodlibet means you can not only prove anything, you can also get a value for anything.^{6Actually, it is kind of a special case, since pattern matching on a type of sort Prop to get something other than Prop is usually not allowed. As a special case, it is allowed for types with exactly zero or one constructors.} ↩︎
Actually, it is kind of a special case, since pattern matching on a type of sort Prop to get something other than Prop is usually not allowed. As a special case, it is allowed for types with exactly zero or one constructors. ↩︎

Bibliography

[1]

H. B. Curry, J. R. Hindley, and J. P. Seldin, Eds., To H.B. Curry: Essays on combinatory logic, lambda calculus, and formalism. London ; New York: Academic Press, 1980.

[2]

A. Guttman, “R-trees: A dynamic index structure for spatial searching,” in Proceedings of the 1984 ACM SIGMOD international conference on Management of data - SIGMOD ’84, Boston, Massachusetts: ACM Press, 1984, p. 47. doi: 10.1145/602259.602266.

[3]

P. Letouzey, “A New Extraction for Coq,” in Types for Proofs and Programs, G. Goos, J. Hartmanis, J. Van Leeuwen, H. Geuvers, and F. Wiedijk, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2003, pp. 200–219. doi: 10.1007/3-540-39185-1_12.

[4]

A. Chlipala, Certified programming with dependent types: A pragmatic introduction to the Coq proof assistant. Cambridge, MA: The MIT Press, 2013. Available: http://adam.chlipala.net/cpdt/

[5]

V. Agafonkin, “RBush,” GitHub repository. Available: https://github.com/mourner/rbush

[6]

M. Eriksen and P. Ferris, “Ocaml-rtree,” GitHub repository. Available: https://github.com/geocaml/ocaml-rtree

Theorem proving in Coq

Sun, 18 Jun 2023 00:00:00 +0000

Introduction

Coq is a proof assistant that lets you state and prove propositions in a kind of constructive logic, the Calculus of Inductive Constructions. The main reason for using such a system is because proofs written in it can be efficiently machine checked.

Coq is built around the Coq kernel, the component responsible for checking the validity of proofs. The kernel is deliberately kept small, and carefully checked, since a bug in its implementation would mean that incorrect proofs could be accepted.

No other part of Coq can compromise the correctness of proofs that the kernel checks. In particular, the tactics that we will use in the main section of this article to construct proofs are not part of the kernel.

This is the most important concept about Coq, and worth reiterating: it doesn’t matter how we construct the proofs. We can use the built-in tactic language. We could write our own tactics. We could even ask ChatGPT to write the proofs for us. Doesn’t matter. If the kernel says that a proof is correct, it’s irrelevant where it came from.

The first section in this article will go over the basic syntax of Coq. This is mostly useful to bring you up to speed if you are already familiar with other dependently typed functional languages. (I cover most of the knowledge I am building on here regarding dependent types in my previous article.)

On the surface Coq tries to mimic regular mathematical practice with its syntax, and you can get pretty far with proving theorems in it without actually understanding the underlying type theory. Don’t expect to understand everything though, beneath the surface it relies on a system fundamentally different to ZFC, the set theoretic axiomatic system most of mathematics is built upon. Any non-trivial usage of Coq requires a proper understanding of its foundations.

In the theorem proving section, I will go over a very basic proof step-by-step. I will not attempt to explain any proofs in detail beyond this first example though. Coq’s interactive proofs are very hard to understand without stepping through it with an IDE yourself and seeing the proof state at each step. (An IDE plugin is just a useful way to display the proof state of Coq in real time, while you are editing. Plugins exist for most popular text editors like Emacs, Vim, or VS Code. You can also use Coq’s interactive command line interface, though it’s not very convenient. Coq also ships with its own IDE called CoqIDE, but I would recommend just going with the plugin for the editor you are already using.)

The last section will showcase some more advanced proofs to show how they can be heavily “automated” in Coq.

In summary, the goal of this article is to give you a very basic overview of Coq, and hopefully get you interested in learning it for yourself. At the end I summarize my tips for getting into and learning Coq.

Syntax

Global definitions can be created with the Definition command:

Definition one : nat := 1.

Each Coq top level command (called “The Vernacular”) is terminated with a period. (Definition is just one of the possible commands, we will see a few others too throughout this article.)

Functions are defined using the fun keyword, the identity function is denoted as fun x => x.

A convenient syntax is provided for function parameters when defining functions, making these two definitions equivalent:

Definition f : nat -> nat -> nat := fun x => fun y => x + y.
Definition f' (x y : nat) : nat := x + y.

(Notice the use of the apostrophe in an identifier. This is allowed so that names can more closely reflect the common mathematical practice of denoting closely related things with apostrophes.)

Coq also features very powerful type inference, we could have also written the previous definition without any type annotations at all:

Definition f'' x y := x + y.

Coq knows that the + operator takes nats, and can infer everything from there.

Dependent function types are denoted with the forall keyword. (Remember that on the logical side of the Curry–Howard isomorphism dependent function types mean universal quantification, hence the “forall” name.)

Definition g : forall (T : Type), T -> T := fun _ => fun x => x.

The above definition looks quite clunky, fortunately the simplified definition syntax works for dependent function types as well:

Definition g' (T : Type) (x : T) := x.

Notations

Coq has a powerful extensible parser. Let’s look at the definition of + from the standard library:

Notation "x + y" := (plus x y) (at level 50, left associativity).

(The actual definition is more complicated, but that doesn’t concern us right now.)

This should be quite straightforward: we ask Coq to translate the x + y infix notation to the plus x y function application.

Some additional information is provided to the parser between the second set of parentheses:

left associativity tells the parser that this notation should be left associative. (Does not really matter for +, but could matter for other operators.)
at level 50 sets the precedence level for this notation. Lower precedence levels bind more “tightly”. For instance, the * multiplication operator has level 40, so that the expression a + b * c parses as you would expect, and not as (a + b) * c that would be dictated just by left associativity.

One thing that might surprise you is that the -> function type constructor is just a notation as well:

Notation "A -> B" := (forall (_ : A), B) (at level 99, right associativity).

This should be fairly straightforward: non-dependent function types are simply a special case of dependent function types.

Theorem proving

We will start with proving some simple tautologies, considered the “Hello World” of proof assistants.

Let’s suppose we have two arbitrary propositions A and B. We can declare them in Coq with the Parameters command:

Parameters (A B : Prop).

Prop is the type of propositions in Coq. It is very similar to Type, but there are a few subtle differences. For now just remember that we will use Prop for propositions instead of Type.

The tautology we want to prove is this one:

A -> B -> A /\ B

/\ means conjunction of course, and it is defined as the and type constructor. The proposition then says that $A$ and $B$ implies $A \land B$ .

We could prove it like this:

Definition prove_and : A -> B -> A /\ B :=
 fun prf_a prf_b => conj prf_a prf_b.

While this is a perfectly fine proof, directly writing out the proof term is inconvenient and quickly becomes infeasible for larger proofs. Let’s look at the Coq way of proving things:

Theorem prove_and : A -> B -> A /\ B.
Proof.
 intros prf_a prf_b.
 split.
 - exact prf_a.
 - exact prf_b.
Qed.

Let’s break this down.

Theorem is the same as Definition^{1The Theorem command also makes terms “opaque”, meaning that the definition cannot be unfolded. This is useful for proofs since once the proof is finished, we only care about its existence and not the actual proof term.}, except instead of providing a value straight away, Coq enters its special “interactive proof mode” to help us interactively construct one. (Proof. is a no-op, it just helps visually separate the interactive proof script.)

And here is where the magic of interactive proof construction comes in: using an appropriate IDE, you can see see the “proof state” while writing the proof. Just after entering Proof., this is what the proof state looks like:

1 goal
___________________(1/1)
A -> B -> A /\ B

The goal (what needs to be proven) is shown below the line, while hypotheses are above the line. Currently there are no hypotheses, but we can introduce some with the intros tactic. The intros prf_a prf_b. tactic introduces the 2 hypotheses, namely the proof of A and the proof of B.

1 goal
prf_a : A
prf_b : B
___________________(1/1)
A /\ B

The hypotheses are now shown above the line. (We named them prf_a and prf_b with the intros tactic.) We have to prove A /\ B now. We will use the split tactic, which lets us prove the two sides of the conjunction separately. The state after split looks like this:

2 goals
prf_a : A
prf_b : B
___________________(1/2)
A
___________________(2/2)
B

Now we have two goals. Coq conveniently lets us focus goals with “bullet points”. After the first - bullet, the proof state looks like this:

1 goal
prf_a : A
prf_b : B
___________________(1/1)
A

The goal is to prove A. At this point this is trivial, since we do have a proof of A as a hypothesis. We can tell Coq which hypothesis we want with the exact tactic. After exact prf_a., Coq will tell us this:

This subproof is complete, but there are some unfocused goals:
___________________(1/1)
B

We completed one branch of the proof, but Coq tells us that to finish the whole thing we have to take care of the other branch as well. We can again go to the next goal with the - bullet:

1 goal
prf_a : A
prf_b : B
___________________(1/1)
B

The situation is very similar to the first branch. We have to prove B, and we have it as a hypothesis, so just use it. After the exact prf_b. tactic, Coq will show the following:

No more goals.

This means that we proved everything, and now the proof is complete. Again, the whole proof looks like this:

Theorem prove_and : A -> B -> A /\ B.
Proof.
 intros prf_a prf_b.
 split.
 - exact prf_a.
 - exact prf_b.
Qed.

The Qed. command terminates the proof, and sends the constructed proof term to the kernel to be verified.

Proof automation

Here is a proof of the commutativity of addition:

Lemma plus_n_O n : n = n + 0.
Proof.
 induction n.
 - reflexivity.
 - simpl. rewrite <- IHn. reflexivity.
Qed.

Lemma plus_n_Sm n m : S (n + m) = n + S m.
Proof.
 induction n.
 - reflexivity.
 - simpl. rewrite IHn. reflexivity.
Qed.

Theorem plus_comm a b : a + b = b + a.
Proof.
 induction a.
 - rewrite <- plus_n_O. reflexivity.
 - rewrite <- plus_n_Sm. rewrite <- IHa. reflexivity.
Qed.

You may wonder, if it takes this much just to prove that $a + b = b + a$ , how much can it take to prove something actually useful?

Here is a shorter proof of the above theorem:

Require Import Lia.

Theorem plus_comm a b : a + b = b + a.
Proof.
 lia.
Qed.

lia is a tactic for linear integer arithmetic. Proving plus_comm is just a trivial case for it. (It relies on existing proofs for the properties of the standard mathematical operators.) Let’s look at a more complex case:

Theorem binomial a b : (a + b) * (a + b) = a*a + 2*a*b + b*b.
Proof.
 lia.
Qed.

This is not even completely linear, but lia can handle some simple non-linear cases. If you still can’t quite appreciate how much heavy lifting lia is doing here for us, let me show you my handwritten proof that just appeals to the basic properties of addition and multiplication:

Theorem binomial a b : (a + b) * (a + b) = a*a + 2*a*b + b*b.
Proof.
 rewrite mul_add_distr_r.
 rewrite mul_add_distr_l.
 rewrite mul_add_distr_l.
 rewrite (mul_comm b a).
 rewrite add_assoc.
 rewrite <- (mul_1_l (a * b)).
 rewrite <- add_assoc.
 rewrite <- add_assoc.
 rewrite (add_assoc (1 * _)).
 rewrite <- mul_add_distr_r.
 change (1 + 1) with 2.
 rewrite add_assoc.
 rewrite mul_assoc.
 reflexivity.
Qed.

You probably learn to do the above multiplication in high school, but might not appreciate just how much goes into it if you rigorously write out every step.

Writing your own automation tactics

lia is a generally useful automation tactic based on well established algorithms, implemented in OCaml^{2Coq is written in OCaml, so the easiest way to write plugins is in OCaml.} for performance. You probably don’t want to start with writing your own tactics in OCaml though.

A language called Ltac ships by default with Coq for writing tactics. (In fact, all of the interactive proofs you have seen so far were written in Ltac, they just weren’t too complex, mostly consisting of simple sequences of tactic invocations.)

Ltac is a domain specific language that specifically evolved to be used with Coq. It is a dynamically typed, Turing-complete scripting language that features built-in backtracking for proof search. The language is not great by modern standards, it has a lot of inconsistencies stemming from its organic evolution alongside Coq. Efforts are underway to replace it, and alternative tactic languages exist as well. Nonetheless, Ltac is still the most widely used tactic language today for Coq, and probably the best starting point for learning proof automation.

Let’s use some logical tautologies as examples again:

Theorem taut1 A B C : (A -> B -> C) -> (A -> B) -> A -> C.
Proof.
 intros abc ab a.
 specialize (ab a).
 specialize (abc a).
 specialize (abc ab).
 exact abc.
Qed.

The proof is quite straightforward. Just apply hypotheses to implications until we reach C.

Let’s say we also need to prove a slight variant of taut1:

Theorem taut2 A B C : (B -> C) -> (A -> B) -> A -> C.
Proof.
 intros bc ab a.
 specialize (ab a).
 specialize (bc ab).
 exact bc.
Qed.

We can just copy-paste our previous proof script, and modify it a bit. Not too bad, right?

Well, imagine if we had to do a 100 proofs like this. All with very slight variations. Something like this can easily happen in real-world proof developments. It’s not uncommon to encounter a lot of very similar goals (usually as a result of some combinatorial explosion) that need to be proven. We clearly need some more scalable approach.

This is where proof automation comes in. We will implement our intuition (“just apply hypotheses to implications until we reach the goal”) in Ltac:

Ltac my_tauto :=
 intros;
 repeat (match goal with
 | x : ?H |- ?H => exact x
 | x : ?H, p : ?H -> _ |- _ => specialize (p x)
 end).

Let’s break this down. First, intros will introduce as many hypotheses as it can. (In this case their names are auto-generated by Coq.)

Next, we will use the very common repeat and match pattern. It will repeatedly choose a branch and execute it, until either the goal is solved or it can’t progress anymore.

The heart of this script are the two match patterns. Patterns on the left side of the turnstile (|-) match hypotheses, while the pattern on the right side matches the goal. Underscores are wildcards, while names prefixed with question marks are unification variables. (For example, the pattern ?A matches any proposition, while ?A -> ?B matches any implication. Any unification variables we don’t care about can just be replaced with underscores: _ -> _ matches implications as well, we just can’t extract the matched propositions afterwards.)

The first branch looks for any hypothesis directly proving the goal. This is achieved by using the unification variable ?H to match the goal, and also a hypothesis. If we find such a hypothesis, we can just directly prove the goal, and we are done. (We name this hypothesis x here. Of course the pattern can match a hypothesis with any name, and Coq will allow us to refer to the hypothesis with the name x in this branch.)

The second branch looks for any hypothesis proving ?H, and any implication with an antecedent ?H. (We don’t care about the goal in this case.) If it finds a pair like this, it specializes the implication to obtain its consequent as a hypothesis.

This simple algorithm makes short work of our two previous example tautologies:

Theorem taut1' A B C : (A -> B -> C) -> (A -> B) -> A -> C.
Proof.
 my_tauto.
Qed.

Theorem taut2' A B C : (B -> C) -> (A -> B) -> A -> C.
Proof.
 my_tauto.
Qed.

I hope this section was able to give you a glimpse of just how powerful proof automation can be.

Learning resources

I hope this short introduction gave you a good picture of what Coq is capable of, and maybe got you interested in learning more. Here a few resources I would recommend:

Volume 1 of Software Foundations: A very good introduction to Coq, building up to quite advanced levels. I would recommend this as a starting point.
Coq Reference Manual: The best resource to look up a specific detail about Coq. Are you interested in the exact backtracking semantics of Ltac, or maybe learning about $\eta$ -expansion? This is the right place.
Certified Programming with Dependent Types: A book about dependently typed programming and advanced proof automation in Coq. It shows off some cool ideas, and I think it’s good for getting some inspiration, but it is in no way a comprehensive guide for any particular technique.

The Coq website also has a long list of Coq related resources, but I have only read the ones I listed above, so I can’t vouch for the quality of any others.

The Theorem command also makes terms “opaque”, meaning that the definition cannot be unfolded. This is useful for proofs since once the proof is finished, we only care about its existence and not the actual proof term. ↩︎
Coq is written in OCaml, so the easiest way to write plugins is in OCaml. ↩︎

Introduction to dependent types and the Curry-Howard isomorphism

Thu, 20 Oct 2022 00:00:00 +0000

This article is meant to be a quick introduction to dependently typed programming, and theorem proving through the Curry–Howard isomorphism.

I assume basic knowledge of programming in statically typed imperative languages, as well as familiarity with some basic concepts from first order classical logic (quantifiers, implication, conjunction, disjunction). I will be using Idris for the examples, but I will explain the syntax and all necessary functional programming concept as we go. Still, this article covers a lot of material in a short amount of text. Take a break if you need space to process the new concepts, and use the table of contents to navigate.

Note that often an explanation for a new code sample is only offered afterwards, so make sure to read on. Sometimes it takes multiple paragraphs just to explain every aspect of a single example.

Motivating examples

Let’s look at a regular function in C++.

int id(int x) { return x; }

This is one of the simplest functions imaginable, the identity function:

$\mathrm{id} : \mathtt{int} \to \mathtt{int} \\ \mathrm{id}(x) = x$

The domain and codomain of this function is fixed. If we wanted an identity function for any other imaginable type out there, we would have to redefine it for that type.

In C++, we have templates to address this issue.

template<typename T>
T id(T x) { return x; }

This is what is usually called a polymorphic identity function. When calling id, you have to supply two arguments: a type for the parameter T, and a value of type T for the parameter x. The two parameters are quite different though: one is a type, and the other is a value. They are also clearly separated when calling the function:

id<int>(1)
id<char>('x')

Okay, so even though the syntax is a bit awkward, we can take types as arguments. Next, what about returning them?

template<typename A, typename B>
struct pair {
 A fst;
 B snd;
};

If you squint at this hard enough, it is actually a function taking two type arguments, and returning the type of pairs constructed from them. Again, I am deliberately using a language where it is awkward to express these type level concepts.

Next, what about a function that takes a value, and returns a type? Here is an attempt:

template<bool b>
using f = char;

template<true>
using f = double;

f<false> returns char, and f<true> returns double. This is almost what we are looking for, but something is not quite right. The issue with this example is that b is not truly a value, it can only be a compile time constant. In particular, you can’t do this:

auto g(bool b) -> f<b> {
 return b ? 1.0 : 'x';
}

Let’s examine what this function is trying to do. It takes a boolean, and depending on its value it either returns a char or a double. Such a function is called dependently typed, since its return type depends on the actual value supplied to it. The obvious issue here is that b might only be known at runtime (it could be input from the user for instance), so it’s impossible to know the type of this function at compile time. Allowing something like this would clearly break static typing, right?

Pure functional programming

For the rest of this article I will be using Idris, but the ideas should be applicable for any dependently typed functional language.

So, how do functional programming languages differ from the more common imperative languages? The most important difference is that functions are much more like mathematical functions. That is:

They have exactly one input and one output.
They can’t have side effects. When you “call” $\sin(x)$ in mathematics, it can’t mutate global variables, or affect anything else.

The notation $f : A \to B$ is used in mathematics to denote a function with domain $A$ and codomain $B$ . The same notation is used in Idris, but unlike in regular set theory, the domain and codomain are types instead of sets.

f : A -> B

Here is how you would define the function $f(x) = x + 1$ in Idris:

f : Int -> Int
f x = x + 1

Note the lack of parenthesis. As functional languages are all about functions, the notation around them is simplified as much as possible. Function application also omits parenthesis, so $f(1)$ would simply be written as f 1.

A very important concept in functional programming is that functions are first class values. (First class in this context means that functions are like any other value, they can be received and returned by functions.) The above definition is just syntactic sugar for defining f as the anonymous function value \x => x + 1:

f : Int -> Int
f = \x => x + 1

In mathematics we don’t usually define functions over functions, but in functional programming this is perfectly normal:

plus : Int -> (Int -> Int)
plus = \x => (\y => x + y)

In this example, we return a function from a function. This technique is called currying, and it is the most common way to define multi argument functions. The terse application syntax makes using such functions natural: plus 1 2. The meaning of this is to apply 1 to the plus function, then apply 2 to the function returned by plus 1.

Syntactic sugar also allows us to simplify the definition of plus. We can remove the parenthesis from the type as well, since the -> type constructor is right associative:

plus : Int -> Int -> Int
plus x y = x + y

One thing you might have glossed over so far is that I haven’t actually described what x + y means. It just seems natural, everybody knows what addition is, right?

In fact, in Idris, operators are just regular functions.^{1There are some exceptions, notably -> is built in.} x + y is syntactic sugar for (+) x y. Just remember this for now, later we will be defining some of our own infix operators as well.

At this point you may wonder how any actually useful programming gets done in a pure language. Let’s say, how do you print something? That certainly counts as a side effect.

Various techniques can be used to emulate imperative constructs while preserving functional purity, but this topic is not really relevant to this article, so I won’t go into any details. Look for any decent Haskell introductory material if you are interested in learning more.

First class types

So functions are first class. A much more unique feature of Idris is that types are also first class. You can manipulate types just like you would any other values.

Despite this, static typechecking is fully retained. The details on how such dependently typed programs can be typechecked and compiled are somewhat complicated, but they are well studied, and backed by solid formal mathematics. Let’s set aside our worries about the implementation for now, and explore what first class types can do. First consider our familiar identity function over Int:

id : Int -> Int
id x = x

We can extend it to be polymorphic by taking another parameter of type Type, which is the type of types:^{2And if you are perhaps wondering what is the type of Type, you are in for a ride. Fortunately though most of the time you can just ignore the complicated theory and pretend that Type : Type.}

id : (a : Type) -> a -> a
id _ x = x

An interesting new piece of syntax here is that we can name parameters in the function type. (In general, any time you see a colon, it means that the thing on its left hand side has the type on its right hand side.) Naming a is necessary since we want to refer to it in the rest of the type. This new id function has two parameters, but we don’t need to use the type parameter in the body, so we can write _ to just ignore it.

With this new definition, id Int would be equivalent to our previous id definition. We can of course now use this new definition with any other type as well:

id Int 1
id Char 'x'

Notice how the types are just regular arguments, not distinguished in any special way.

The above example is good for introducing the concept of first class types, but writing out the types every time we use id would be a bit verbose. In Idris we can mark some parameters as implicit, and they will be inferred at the call site instead of having to be explicitly provided:

id : {a : Type} -> a -> a
id x = x

With this definition we can just write id 1 and id 'x', and Idris will figure out what a should be.

We can go even one step further. This definition is equivalent to the previous one:

id : a -> a
id x = x

Since using implicit parameters is so common, Idris provides special syntactic sugar for them. In particular, any lowercase names in types will automatically get turned into implicit parameters.

We ended the motivating examples section with attempting to write a function that returned different types based on the value of the input. Let’s now write such a function in Idris:

g : (b : Bool) -> if b then Double else Char
g b = if b then 1.0 else 'x'

Notice how the same if construct is used at the type and the value level: types are first class, and there isn’t really any difference between types and values anymore.

Of course we could have easily written such a function in any dynamically typed language. The novel thing here is that the Idris typechecker is perfectly on board with the above definition, and will ensure that any usage of g is correct: the type of g True is Double, and the type of g False is Char.

One final piece of syntax I want to introduce before we move on is pattern matching definitions. We also could have written g like this:

g : (b : Bool) -> if b then Double else Char
g True = 1.0
g False = 'x'

We can pattern match on any inductive type in such a manner. So… what are inductive types exactly?

Inductive types

The only method we have seen so far for constructing types was ->, the function type constructor. Idris also supports inductive type definitions, a very powerful mechanism for creating our own types.

Algebraic data types

Let’s start with a subset of inductive types commonly referred to as algebraic data types. It turns out that Bool is not built into Idris, but defined as an algebraic data type:

data Bool = False | True

The names don’t matter, we could also define our own special isomorphic variant of Bool if we wanted to:

data Light = Off | On

Next, let’s extend this type a bit:

data Color = Red | Green | Blue
data Light = Off | On Color

Color is straightforward, it has 3 possible values instead of 2. This new Light type is also similar to the previous one, but we are attaching some extra data to the On state, namely its color. There are 4 possible values of this type: Off, On Red, On Green, and On Blue. On is called a data constructor, and it takes a single argument of type Color to construct a value of the type Light. (Off is also called a constructor, it just takes no arguments.)

data Intensity = Low | High
data Light = On Color Intensity | Off

Here I added another parameter to the On constructor. This Light type has 7 possible values, and this is how we can calculate that:

$\underbrace{\overbrace{3}^{\mathtt{Color}} \times \overbrace{2}^{\mathtt{Intensity}}}_{\mathtt{On}} + \underbrace{1}_{\mathtt{Off}} = 7$

At this point the reason for calling these algebraic data types should become apparent. The number of values is the sum of the number of values for each constructor, and the number of values for each constructor is the product of the number of values of each of its parameter. (You will often see types with these properties called sum types and product types respectively.)

Parameterized and recursive types

Parameterized types are supported as well:

data Prod a b = Pair a b
data Sum a b = Left a | Right b

Here we can see the concept of algebraic data types distilled into the notion of sums and products. Prod is not the type of pairs by itself, it is a type constructor taking two parameters: Prod Bool Bool is the type of Pair True False, and Prod Int Char is the type of Pair 1 'x'.

Note that I am glossing over a technical detail here, which is that we also have to supply the type arguments to the data constructors, but Idris automatically makes these parameters implicit.

For the next example, you need to be familiar with how the Peano axioms define natural numbers:

$0$ is a natural number.
For any natural number $n$ , $S(n)$ is also a natural number.

$S$ is called the successor function. $S(0)$ is 1, $S(S(S(0)))$ is 3, and so on.

We can encode natural numbers with a recursively defined type:

data Nat = Z | S Nat

Zero is represented as Z, one as S Z, three as S S S Z, just like in the Peano axioms. Idris in fact provides syntactic sugar for the built in Nat type, so that instead of S S S Z we can simply write 3.

At this point it’s important to note that Nat should not be confused with Int, even though the numeric notation can be used for both. Nat is the type of natural numbers $\mathbb{N}$ , its cardinality is $\aleph_0$ . Int is the type of machine integers, usually $[-2^{63}, 2^{63} - 1]$ , with cardinality $2^{64}$ .

Also I would suggest not really thinking about cardinalities for anything beyond simple algebraic types. Infinities in set theory are quite complicated, for instance the cardinality of Nat -> Nat should be $\aleph_0^{\aleph_0}$ . You can run into issues if you define types as sets in regular set theory, so just remember that types are not sets, they are defined by a completely separate system of axioms.

With that said, let’s get back to exploring inductive types. In this next example, we combine type parameters and recursion to define lists:^{3I have no idea why the list constructors are named Nil and Cons, but everyone names them like this. Just roll with it.}

data List a = Nil | Cons a (List a)

List Int is the type of the lists of integers. Nil is the empty list, Cons 1 Nil is a list with one element, Cons 1 (Cons 2 (Cons 3 Nil)) is a three element list containing 1, 2, and 3. (And again syntactic sugar comes to the rescue, the list notation [1, 2, 3] expands to constructors of the built in List type.)

Type indices

The syntax we have been using so far to define types is actually just the abbreviated syntax. The more verbose way to define List would be the following:

data List : Type -> Type where
 Nil : List a
 Cons : a -> List a -> List a

Here you can see the type of the type and data constructors fully written out. (Note that there is nothing special about the type parameter a here, it gets automatically inserted as an implicit parameter just like it would be when defining a function.)

When we write List Int it really is just a function application. When we write Nil the type parameter a is automatically inferred from the context. (Note that type and data constructors are not functions, but their types are the same, and they can be used in the exact same ways. The special thing about data constructors is that they can be deconstructed in pattern matching definitions.)

A question that arises is whether we have to parameterize the type constructor the same way in each data constructor? Inductive types in fact do allow for such heterogeneity. In this case these are usually called type indices instead of parameters, and the whole definition an indexed type family.^{4I don’t think Idris actually distinguishes type parameters and indices, but some languages like Coq do.}

With all that said, let’s consider the ubiquitous example when introducing type indices, the length indexed list. It is called Vect to differentiate it from the regular List.

data Vect : Nat -> Type -> Type where
 Nil : Vect 0 a
 Cons : a -> Vect n a -> Vect (n + 1) a

Let’s unpack this definition. The Vect type constructor has two parameters: a Nat that represents its length, and the type of the contained elements. So for instance Vect 3 Int is the type of 3 element Int vectors.

Let’s look at the constructors. Nil is quite straightforward: it says that the empty vector has length 0.

The magic mostly happens in the Cons constructor: it says that for any Vect n a, when you add one element, it will have length Vect (n + 1) a. (Note that both a and n are still just implicit parameters of Cons, even if the syntax makes it seem like they come out of thin air.)

So, how can such a type be used? We will look at an example in the next section.

Programming with dependent types

Indexed types allow for a level of expressiveness not seen outside dependently typed languages.

Consider the head function that returns the first item of a list:

head : List a -> a
head (Cons h _) = h
head Nil = ?

The Cons case is straightforward, but what should we do when the list is empty? In other languages, we would throw an exception, return a null value, or maybe an option type. Unfortunately none of these are very good solutions, and they make it harder to reason about our programs. What we would really want is a function that’s guaranteed to receive a non-empty list:

head : Vect (n + 1) a -> a
head (Cons x _) = x
head Nil impossible

In this example, we take a Vect with length n + 1 as the argument. This guarantees, at the type level, that it is not empty: $\forall n,\ n + 1 \gt 0$ . So, what do we write in the Nil case? We now know that it’s impossible. Idris can in fact handle this for us: by using the impossible keyword, Idris will go ahead and prove for itself that no n exists for which n + 1 = 0. (In more complicated cases we might have to provide our custom proof that a certain case is impossible.)

Except… there is a slight error. Idris won’t accept the above definition, and gives us a cryptic error message:

While processing left hand side of head. Can't solve constraint between: ?n [no locals in scope] and ?_ [no locals in scope].

Can you tell what’s the error based on this? Me neither. I think Idris is trying to be too smart for its own good here by attempting to unify some terms when pattern matching, and failing.

The fix is relatively simple in this case. Change occurrences of n + 1 to the more “natural” S n. While Idris tries to handle unification automatically, it can still fail even in simple cases like this. This is not Idris’ fault: unification is undecidable in general, and no algorithm can exist that covers every case. In this case, Idris can’t figure out by itself that n + 1 and S n are the same.

The main issue with dependently typed programming I have experienced so far is that you often can’t just write the code you would without strong types. The above failure is a contrived example, but issues like this also occur in practice in cases where you can’t just solve the issue with such simple tweaks.

Often you have to add explicit rewriting annotations, or even large hand written proofs to your code. This is not really surprising though: as you can express very complex properties with dependent types, a type checker that wouldn’t need annotations would essentially be a completely automatic theorem prover.

Using dependent types sparingly to enforce some key properties of your program can work amazingly. (The Idris compiler itself, written in Idris, does exactly this, and is able to encode some important guarantees in the types.) What definitely won’t work is putting any property you can think of indiscriminately in the types, and expecting to be able to program normally.

The Curry–Howard correspondence

data And a b = Conj a b
modus_ponens : And p (p -> q) -> q
modus_ponens (Conj prf impl) = impl prf

Look at the type of the suggestively named modus_ponens. It is supposed to mean $(p\ \wedge\ (p \rightarrow q)) \rightarrow q\text{,}$ that is, the proposition that if $p$ is true, and $p$ implies $q$ , then $q$ is true.

So… what am I talking about here? And is an inductive type, modus_ponens is just a function, p and q are types. What does this all have to do with logic?

As it turns out, quite a lot actually. In fact, the famous Curry–Howard isomorphism states a strong equivalence between programs and mathematical proofs. In this view, types are propositions, and programs are proofs.^{5Note however that we won’t get a classical logic here, but rather a kind of constructive logic, but we can ignore this technical detail for now.}

An important thing to note here is that this logical correspondence only applies to terminating programs. If you had an infinitely recursive function, the whole system would break down, and you would be able to prove any propositions.^{6Idris does not verify this by default, you have to provide the %default total directive to tell it to only accept functions for which it can prove termination. This built in checking works well for most cases.}

So, let’s go through the computational features we have learned about so far, and uncover their hidden logical meaning. (By the end, the meaning of the modus_ponens proof above should also become clear.)

First, as I already said, types correspond to propositions. p, q, p -> q, and And p (p -> q) -> q are all types, and can be interpreted as propositions as well. In turn, a value of a type corresponds to a proof of the proposition.

As a very simple example, let’s prove the proposition And a b. This is possible if we have a proof of a and a proof of b. Let’s create a function taking these proofs:

prove_and : a -> b -> And a b
prove_and prf_a prf_b = Conj prf_a prf_b

Once we have both of them, the proof of And a b can be simply constructed using the Conj data constructor we defined previously.

Actually, remember our Prod and Sum types from the previous section?

data Prod a b = Pair a b
data Sum a b = Left a | Right b

Our definition of And is exactly the same as Prod! Looking at it this way, a proof of a conjunction is really just a pair of proofs.

The logical meaning of Sum should also be obvious: it corresponds to disjunction. A proof of either a or b is enough to prove $a \vee b$ .

Next, as you might have already noticed, function types correspond to implications. Our modus_ponens proof above is actually superfluous, as modus ponens corresponds directly to function application: if you have prf_p : p and prf_pq : p -> q, then a proof of q is simply prf_pq prf_p.

As it turns out, inductive types can be used to encode many complex mathematical relations. Let’s take a look at our definition of the less than or equal to relation (note that S and Z are the constructors of the Nat type we discussed previously):

data (<=) : Nat -> Nat -> Type where
 LeN : n <= n
 LeS : n <= m -> n <= S m

Notice how we are defining the <= infix operator to be an inductive type. LeN defines the base case saying that $\forall n,\ n \leq n$ . LeS defines that given a proof of $n \leq m$ , $n \leq m + 1$ is also true. These two rules are sufficient to define the $\leq$ relation.

Let’s now state the proposition that $\forall n,\ 0 \leq n$ :

all_geq_z : (n : Nat) -> Z <= n

Notice that this is not a regular function type, but a dependent function type. Dependent function types correspond exactly to universal quantification.

Let’s now prove this proposition. The case where n is zero is quite simple, $0 \leq 0$ by definition.

all_geq_z Z = LeN

In the case where n is nonzero, we want to somehow construct a chain of LeS applications until we get down to zero. We can do this quite easily with a recursive function call:

all_geq_z (S n) = LeS (all_geq_z n)

What we just did is a proof by induction. We first proved our proposition for zero, then for any $n + 1$ by using the proof for $n$ .

As our final example, let’s define equality:^{7In reality Idris doesn’t accept this definition because equality is already defined by the standard library, and = is a reserved symbol.}

data (=) : a -> a -> Type where
 Refl : (x : a) -> x = x

The above definition might seem a bit strange at first though.

eq3 : 3 = 3
eq3 = Refl 3

Our equality works well for reflexive cases, but how could we ever prove two things equal that are not literally the same?

eq_2_1_plus_1 : 2 = 1 + 1
eq_2_1_plus_1 = Refl 2

Turns out Idris is perfectly content with this proof. So what’s going on? Can Idris just magically prove things equal?

plus_z : (n : Nat) -> n + 0 = n
plus_z n = Refl n

This proof won’t be accepted:

Can't solve constraint between: n and plus n 0.

What the typechecker actually is doing is called reduction. The + operator is just defined as the plus function. When faced with 1 + 1, Idris will apply the plus function and arrive at the answer 2. On the other hand, it can’t reduce n + 0 since it doesn’t know anything about n yet.

Let me now show what it actually takes to prove plus_z:

replace : (p : a -> Type) -> x = y -> p x -> p y
replace _ (Refl _) prf = prf

plus_z : (n : Nat) -> n + 0 = n
plus_z 0 = Refl 0
plus_z (S n) = replace (\x => S (n + 0) = S x) (plus_z n) (Refl (S (n + 0)))

That’s quite a mouthful, so let’s go through it step by step.

Let’s look at the definition of replace. It might not look too complex, but it’s actually a very powerful tool: it can replace arbitrary terms with each other as long as you have an equality proof between them. Suddenly our definition of equality becomes an immensely powerful tool for manipulating propositions, and we no longer have to rely just on reduction to unify terms for us. (Something similar called rewrite is actually built into Idris that can deduce p automatically, but I am deliberately showing here how it could be defined by hand.)

Using replace, we can prove plus_z by induction:

The 0 case is trivial.
In the S n case, we have to prove that S n + 0 = S n
- Idris’ reduction can still partially help us here: it reduces the above term to S (n + 0) = S n, by the definition of plus.
- We can trivially obtain a proof of S (n + 0) = S (n + 0), this is what Refl (S (n + 0)) stands for.
- We have plus_z n as a proof of n + 0 = n. We can use this to rewrite the right hand side of the above equality, and obtain S (n + 0) = S n, which is exactly what we wanted to prove.

Conclusion

We went over a lot here, building up from just introducing functional programming to writing mechanized proofs using dependent types. I really just went over the basics here, but these are the foundations modern type theory based proof assistants are built on. (Writing proofs like this by hand is not really practical though. Coq for example has a whole separate language just for automating the creation of proofs like these, so that most of the time you don’t even have to think about the details of what’s going on under the hood.)

If this topic has caught your interest, here is a list of resources I would recommend:

Software Foundations: A very good introduction to Coq, building up to quite advanced levels.
Certified Programming with Dependent Types: A book about dependently typed programming and advanced proof automation in Coq.
Functional Programming in Idris 2: A comprehensive introduction if you are interested in Idris specifically. (Idris does not have anything like Coq’s advanced proof automation capabilities though, it is more suited as a practical dependently typed programming language.)

There are some exceptions, notably -> is built in. ↩︎
And if you are perhaps wondering what is the type of Type, you are in for a ride. Fortunately though most of the time you can just ignore the complicated theory and pretend that Type : Type. ↩︎
I have no idea why the list constructors are named Nil and Cons, but everyone names them like this. Just roll with it. ↩︎
I don’t think Idris actually distinguishes type parameters and indices, but some languages like Coq do. ↩︎
Note however that we won’t get a classical logic here, but rather a kind of constructive logic, but we can ignore this technical detail for now. ↩︎
Idris does not verify this by default, you have to provide the %default total directive to tell it to only accept functions for which it can prove termination. This built in checking works well for most cases. ↩︎
In reality Idris doesn’t accept this definition because equality is already defined by the standard library, and = is a reserved symbol. ↩︎

My first impressions with OCaml

Sat, 12 Mar 2022 00:00:00 +0000

I currently am on a quest to find a generally useful functional language that I could experiment with on various projects.

While this article specifically is about OCaml, I have plans for trying out other languages as well, so first I will talk in general about my criteria for evaluating functional languages.

How am I evaluating functional languages

I don’t consider this list to be complete in any way, and expect it to be refined and extended as I learn more and try out new languages. Also, a large part of this can be applied to any software platform in general, not just functional programming languages.

Tooling

In general, a single tool for a purpose that’s “blessed” and endorsed by the ecosystem (or is at least a de facto standard) is a plus. Fragmentation of the ecosystem doesn’t really seems useful in this context.

Build system

Incremental compilation and good performance in general are nice to have. The build system also shouldn’t be overly difficult to learn and use.

Formatter

Ever since I tried prettier I have been hooked on the idea of not having to care at all about the formatting of my source code. I think having such a formatter is even more useful for functional languages, as deeply nested expressions can be especially tricky to format by hand.

Editor support

Type systems can be used to provide all kinds of useful editor features like smarter syntax highlighting, type annotations/queries, and context aware autocompletion. All of these are a plus for me.

Runtime environment

One feature I would value quite much that might sound surprising at first is the ability to target JavaScript/WASM. As it stands right now, the web is by far the largest and most commonly available software platform, despite all of its shortcomings. Being able to deploy my code onto it can be quite valuable.

For server use I don’t have any requirements, as with today’s container technology you can deploy pretty much anything anywhere.

Learnability

This criterion might even sound too obvious written out like this, but I want the language to be learnable with a reasonable amount of effort. By this I don’t mean that the language shouldn’t be complex, but there should be some resource, or collection of resources, that can give you a comprehensive understanding of the whole language.

Having a language specification is of course the best, but I am willing to compromise with a collection of other resources as well, as long as that’s a reasonable substitute for a specification. (For example, if I have a specific question about how some part of the language works, there should be a way to get an answer to that with a reasonable amount of effort.)

Functional purity

While purity is certainly useful, you eventually do have to interface with the real world, and have some side effects in one way or another.

I already use pure function as a generally useful tool in basically any language, and never felt the need for the language to enforce that. Nevertheless, my impression is that the methods for describing side effects in pure languages are quite mature too, and I wouldn’t mind using such a language either.

Type system

I would like a statically typed language. Not that I take any issue with dynamic typing, but I already have my default go-to language for that (Python), and various alternatives as well (JavaScript, Lua, even any kind of LISP).

The issue is that beyond a certain level of complexity, you are bound to make mistakes in a dynamic language, and a type system can effectively alleviate large classes of them. While I agree that static typing can also be a source of mental overhead, and it’s sometimes more convenient to just program without strong types, I want the option to ask the compiler for help with checking my work whenever I feel like I need it. I am just a human after all, and bound to eventually make mistakes.

I am open to trying various type systems, though some features I probably want are algebraic data types, and some method of generalizing over types. But stronger features like higher kinded types, or even dependent types are a plus too.

My first impressions with OCaml

So with all that said, I will give you my first impressions of OCaml.

It has a pretty mature community, a long history, and there are large companies (Facebook^{1Yes it’s “Meta” I know…} for instance) using it and helping with the development of the ecosystem.

The package management seems quite good, opam is the de facto standard package manager. I really like that the opam repository is curated, and the compiler developers actually use it as a giant test suite for the compiler. Submitting a package requires a pull request in opam-repository, so this way every package publication is reviewed. The compiler, package manager, package repository, and the build system all live under the same GitHub organization.

The formatter, ocamlformat, is a relatively new addition to the ecosystem, but it seems to work very well, I haven’t had any problems with it.

The editor support seems good as well. I only tried the plugin for VS Code for ease of installation, but I assume that it would work well with any LSP capable editor.

As for actually running your programs, OCaml compiles natively to most mainstream processor architectures. There are also multiple solutions for compiling it to JavaScript, namely js_of_ocaml and ReScript. While js_of_ocaml is intended for OCaml developers to compile their code to JavaScript, and integrates with the ecosystem, ReScript is targeted more at JavaScript developers seeking better type safety, and actually comes with an alternative JavaScript-like syntax for OCaml, but still has support for the regular syntax. (Note that actually js_of_ocaml also supports the ReScript syntax.) The ReScript syntax can be useful for using React as it supports JSX.

While I read from multiple sources that the language was lacking good learning materials, I had very good experience with the Real World OCaml book. There is a sort of language reference available as well, but the exact type checking and inference rules seem to be scattered in various research papers and university course notes. Nevertheless, because of the academic background, I have confidence in the soundness of the type system.

The language is not pure, you can have side effects, mutable values, and global variables. The language makes it explicit which values are mutable, but not which functions do and do not have side effects.

The type system is okay, you get amazing type inference, GADTs^{2Generalized algebraic data types}, but no higher kinded types (though you can supposedly emulate them with modules). The module system is powerful, but also pretty weird, there is a distinction between first-class and non-first-class modules, with explicit conversions between them, an artifact of first-class modules having been added later into the language. One annoying thing is that the language calls parameterized modules functors, which don’t really have anything to do with actual functors^{3I of course mean functors from category theory. For some reason programming languages really like naming random things functors, C++ and Prolog are also guilty of this.}.

Metaprogramming is done with the so-called ppx system, which can apply arbitrary AST level transformations to the source code. This sounds quite powerful, but the system is used for some core functionality like generating comparison functions and pretty printers, and it actually has some pretty major flaws.

First of all, the AST is apparently not very stable between compiler versions, and in practice ppx derivers have to be updated with each compiler version. The barrier to entry to writing them is also very high, requiring understanding of some compiler internals. I also had issues with composing extension nodes (using one inside of another), which is again pretty annoying given that they provide some core functionality.

There supposedly is a debugger for the language, but I could not get it working based on the official docs.

By far the largest issue I have with the language so far is the difficulty of debugging. It’s hard to insert random print statements to inspect arbitrary values, since there is no default pretty printer for values. Setting them up is non-trivial, given that the ppx derivers are non-recursive, so you have to annotate each type definition separately. The issue is exacerbated by the standard library’s (well actually there are multiple standard libraries, I was using base) lack of pretty printing definitions. All of these issues together make printf debugging needlessly difficult.

In summary, my largest complaints with the language are lack of good debugging methods, and lack of higher kinded types. But as there does not seem to be a fundamental obstacle to solving either of these (as I said earlier, higher kinded types can in theory be emulated with modules), I might consider revisiting OCaml sometime in the future.

Yes it’s “Meta” I know… ↩︎
Generalized algebraic data types ↩︎
I of course mean functors from category theory. For some reason programming languages really like naming random things functors, C++ and Prolog are also guilty of this. ↩︎

Gyuri's blog

Porting Lean to the ESP32-C3 RISC-V microcontroller

Table Of Contents

Lean runtime

Object representation

Memory management

Getting the runtime to compile & run on the ESP32-C3

Toolchain: compiler and libraries

Overview

The Nix package manager

Targeting the ESP32-C3 with nixpkgs

Hacking on the project yourself

Booting and application initialization

Memory layout

Booting

Lean startup

Example application: RGB LED controller

Results

Conclusions

Appendix: Bit banging the WS2812 protocol

Bibliography

A formally verified R-tree implementation

Table Of Contents

Introduction

R-trees

Insertion

Deletion

Complexity

Specification

Informal specification

Formal specification in Coq

Techniques

Extraction

Erasing proofs

Dependent types

Arithmetic

Transpiling to JavaScript

The sig type

Coq’s #[program] attribute

Termination

Proving termination manually

Rewriting with permutations

Rewriting with morphisms

Proof automation

Reflection for use with lia

Solving permutations

Implementation

Axis-aligned bounding boxes

List utility functions

Removing items from a list

Finding minimal items in a list

Partitioning (for node splitting)

The tree data structure

Tree operations

Insertion

Search

Deletion

Putting everything together

Benchmarks

Conclusions

Bibliography

Theorem proving in Coq

Introduction

Syntax

Notations

Theorem proving

Proof automation

Writing your own automation tactics

Learning resources

Introduction to dependent types and the Curry-Howard isomorphism

Table Of Contents

Motivating examples

Pure functional programming

First class types

Inductive types

Algebraic data types

Parameterized and recursive types

Type indices

Programming with dependent types

The Curry–Howard correspondence

Targeting the ESP32-C3 with `nixpkgs`

The `sig` type

Coq’s `#[program]` attribute

Reflection for use with `lia`