This is a story of a really poorly executed Moho mission. Getting to Moho is hard. If you try do it with a direct Hohmann transfer, then the $\Delta v$ map tells us you need a whopping $16,670m/s$ of $\Delta v$. That’s a big rocket and I don’t want to spend any money. So I’m going to send this rocket instead:

The Rocket

According to the VAB, the rocket has $9640m/s$ of $\Delta v$ broken down as follows

• $883 + 2039 = 2935 m/s$ for the lifter.
• $1937 + 1916 = 3853 m/s$ for the transfer stage
• $2866 m/s$ for the lander. The lander actually has $1909m/s$ of $\Delta v$ on board, but the VAB shows a higher number because there is no kerbal in the chair.

All in all, to get to Moho and back, we have three FL-T400 fuel tanks and a terrier, which gives $3,853 m/s$ of $\Delta v$. This is a little short of the $11,530m/s$ that we need according to the map. I’m going to go anyway and see what happens.

Aside: Gravity assists

To get to Moho, I’m going to use a lot of gravity assists. When you pass by a planet, it bends your trajectory. Bending your trajectory changes your velocity. Normally you would have to fire your engines to change your velocity. Therefore passing by a planet gives us $\Delta v$ for free.

Most tutorials for gravity assists online say to “just play around with maneuver nodes and you’ll get better at it” but I don’t find this answer very satisfying. Instead I have distilled the following 6 rules to help me plan what assists to use before I take off:

1. You enter a sphere of influence with the same speed you exit
2. If you do two assists in the same location, you will enter the second assist with the velocity you exited the first
3. The greater the mass of the body, the more it can bend your trajectory in a single assist
4. The greater the relative speed, the more you can get out of assists
5. High inclinations and oblique angles means greater relative speed
6. Fly to where you want the encounter to be, rather than where the planet is.

Getting to Eve

Firstly, I take off from Kerbin. With a decent ascent, I am left with $95m/s$ of $\Delta v$ on the main stage. I don’t wait for an Eve transfer window. Instead I want to use rule #6

6. Fly to where you want the encounter to be, rather than where the planet is

I want to use Eve to perform an inclination change, and to do that, Eve needs to be at the ascending or descending node of Moho.

To get an Eve encounter, I create one maneuver node where my orbit intersects Eve’s, and a dummy maneuver node where I use the “+/- orbit” buttons to search forward a few orbits for close encounters. This means I can make a small burn to get an encounter anywhere along Eve’s orbit. When doing precise burns such as this one, it helps to set the thrust limiter on the engine to 1%.

When I intercept Eve, I have a larger orbit. That means I will be going faster than Eve. To get to Moho, I want to go slower than Eve. Basically I want to perform a U turn around Eve so that all of my forward speed is converted into backwards speed.

I burn $887m/s$ of $\Delta v$ close to Eve to get a Moho Encounter. In tital, I use $2354 m/s$ to get a Moho encounter from Low Kerbin orbit. This is a saving of $1876 m/s$ compared to a direct transfer.

Eve to Moho

Once I have a Moho intercept, I want to kill as much relative velocity as possible. On the surface this seems impossible due to rules #1 and #2 which imply that no matter how many assists I do, I will always end up with the same relative velocity.

1. You enter a sphere of influence with the same speed you exit 2. If you do two assists in the same location, you will enter the second assist with the velocity you exited the first

I can use a Moho assist to lower my apoapsis, but this also causes the orbit to become more oblique. The net result is that my relative velocity is unchanged, and I am back to square one.

Luckily for me, I can use rule #5 in reverse:

5. High inclinations and oblique angles means greater relative speed

If high inclinations and oblique angles mean larger relative speed, then low inclinations and narrow angles mean low relative speed. I can perform deep space prograde burns at apoapsis to reduce the angle of intercept, and reduce my relative velocity.

Therefore, I can alternate assists and deep space burns to descend to Moho efficiently. I double dip and use the deep space burns to get Moho intercepts, so I don’t waste much fuel. I have to do three or four of these maneuver to get my relative speed down to something acceptable.

Using a lander

When I get to Moho, I can send down a small lander instead of landing the whole ship. This is a basic technique that you can use anywhere to give a good $\Delta v$ saving.

Additionally, it takes 310 m/s of $\Delta v$ to circularise my orbit once I capture into Moho. If I detach the lander in an elliptical orbit, I can save another chunk of mothership fuel at the expense of fuel in the lander. Again, this is a good choice to save $\Delta v$.

Rendezvous in an elliptical orbit is a bit harder, but not impossible. I can use the same technique we use to get an encounter with a planet. One maneuver node where you want the intercept, and another maneuver node to search a few orbits ahead.

This is made more difficult by the fact that the lander doesn’t have enough fuel. I need $(310 + 870) * 2 = 2360 m/s$ of $\Delta v$ but my lander has $1909m/s$ of $\Delta v$ onboard. I’m going to have to use the EVA jetpack to make orbit and rendezvous, which means no maneuver nodes.

I just make the rendezvous with fumes in the EVA jetpack tank. I had to use some of the mothership fuel to make the rendezvous, but that’s OK. Next time, I know to bring extra EVA jetpack fuel.

Getting Back

Moho assists can be used to get back, I simply apply the Moho descent technique in reverse: burn retrograde at solar apoapsis to get a Moho encounter, and then use the encounter to raise my apoapsis. Repeat as many times is necessary to get the orbit to the level of Eve.

Once I have an Eve encounter, I can use that to assist to raise my orbit to the level of Kerbin. In some cases, I’ve needed two Eve assists to do this, but in this run, I managed it with only one assist. Once I have a Kerbin encounter, I can aerobrake down to the surface, as long as I have heat protection

As long as I have heat protection

I get a Kerbin encounter with $209m/s$ of $\Delta v$ to spare. This is plenty if your heat shilds work, which they didn’t. The RC-001S sticks out to the side and overheats, destroying the whole craft. Instead, I aerobrake at the lowest altitude possible using the terrier as a heat shield.

After spending all of my $\Delta v$ slowing down, it’s still not enough, so I get out and push the craft to get a Kerbin capture. After multiple aerobraking passes, I finally touch down at last. After 30 years in the command pod, Valentina is finally home.

Post Mortem

With any terrible mission, we should reflect on what we can do better and improve it for next time

Bring extra EVA fuel

For dubious missions like this one, it’s always a good idea to bring extra EVA canisters. This gives you an extra $300m/s$ of $\Delta v$ to work with in the jetpack at the expense of not having a personal parachute. If you still want a parachute for safety, put it in the command module inventory and equip it before re-entry in case you need to bail out.

Note: if you do ever have to bail out, don’t forget to take your science with you!

Use larger heat shields

The RC-001S extends past the outside fo the 1.25m heat shield, making it effectively worthless. Testing reveals that a 1.875m heat shield protects the core sufficiently for re-entry. Alternatively, you can drop the probe core with the terrier engine.

In general, using heat shields with the exact diameter of the part it protects is dicey, since you are unprotected if the craft tilts to one side for any reason. Command pods are an exception, since they are conical in shape.

Unpowered Eve assist

To get to Moho, I used a powered Eve assist, but ignored rules #4 and $5:

4. The greater the relative speed, the more you can get out of assists 5. High inclinations and oblique angles means greater relative speed

If I increase the angle between my orbit and Eve’s, I give myself more relative velocity to work with and I can use an unpowered assist instead. This costs me about $200m/s$ of $\Delta v$ during my escape burn from Kerbin, but saves the $890m/s$ burn at Eve.

Overall, I got a Moho encounter from low Kerbin orbit in $1524m/s$. This is a saving of $2706m/s$ compared to a direct Hohmann transfer and $830m/s$ compared to a powered assist.

Summary

Using gravity assists, a lander, and the EVA jetpack, we save a lot of delta V getting to Moho. This allows us to send a really cheap rocket there and back. A summary of the techniques used are as follows:

As with anything in KSP, poorly planned missions sometimes still work out anyway. Nothing that EVA jetpack fuel, gravity assists, and engines-as-heat-shields can’t solve.

Summary

• Inheritance can be used to override methods.
• This could be used to memoize a recursive function for example.
• Dynamic dispatch can be desugared to continuation passing: a functional concept.
• This style of continuation passing can be denoted as “Open Recursion” because the recursive calls are left open to be overridden.

Memoization Using Inheritance

Using inheritance, you can call a base method from a derived class:

class Base {
    a() {
        console.log('Base Method A');
    }
}

class Derived extends Base {
    b() {
        console.log('Derived Method B');
        super.a();
    }
}

const derived = new Derived();
derived.b();

It’s also possible to do the opposite and call a derived method from a base class:

abstract class Base {
    a() {
        console.log('Base Method A');
        this.b();
    }

    abstract b(): void
}

class Derived extends Base {
    b() {
        console.log('Derived Method B');
    }
}

const derived = new Derived();
derived.a();

An interesting result is that the base and derived methods can call each other recursively:

class Base {
    method() {
        console.log('Base Method');
        this.method();
    }
}

class Derived extends Base {
    method() {
        console.log('Derived Method');
        super.method();
    }
}

const derived = new Derived();
derived.method(); // crashes or runs indefinitely

Say I have a recursive function that’s particularly slow to execute, for example a naive implementation of the Fibonacci sequence:

fib(n: number): number {
    return n === 0 || n === 1 ? 1 : fib(n - 2) + fib(n - 1);
}

In the tradition of object orientated programming we can put that function into a class:

class Fib {
    call(n: number): number {
        return n === 0 || n === 1 ? 1 : this.call(n - 2) + this.call(n - 1);
    }
}

const fib = new Fib();
console.log(fib.call(0));
console.log(fib.call(10));
// console.log(fib.call(100)); // will not terminate in any reasonable amount of time

One method to make this a lot faster is to memoize the function. Using inheritance, we can override the function and memoize it using a derived class:

class MemoFib extends Fib {
    store: Map<number, number> = new Map();

    call(n: number): number {
        const storedResult = this.store.get(n);
        if (storedResult !== undefined) {
            return storedResult;
        }

        const result = super.call(n);
        this.store.set(n, result);
        return result;
    }
}

const memoFib = new MemoFib();
console.log(memoFib.call(0));
console.log(memoFib.call(10));
console.log(memoFib.call(100)); // runs almost instantaneously

We can also reverse the inheritance hierarchy so the function inherits from memoize. This will have slightly different performance characteristics, but allow us to memoize any function we like:

class Memoize {
    store: Map<number, number> = new Map();

    call(n: number): number {
        const storedResult = this.store.get(n);
        if (storedResult !== undefined) {
            return storedResult;
        }

        const result = this.call(n);
        this.store.set(n, result);
        return result;
    }
}

class MemoFib extends Memoize {
    call(n: number): number {
        return n === 0 || n === 1 ? 1 : super.call(n - 2) + super.call(n - 1);
    }
}

const memoFib = new MemoFib();
console.log(memoFib.call(0));
console.log(memoFib.call(10));
console.log(memoFib.call(100));

We could stop here, however, this method is limited. If Memoize inherits Fib, the memoize logic is limited to the Fibonacci function, and we have to replicate it for every function we want to memoize. If Fib inherits Memoize, then we can’t swap out the kind of memoization we want to use.

Overriding methods like this is called “Open Recursion”

To see this, we need to make some changes to our code. In OOP, a method call obj.method(arg) desugars to method(obj, arg). We can desugar the Fib class that way:

interface Fib {
    call(self: Fib, n: number): number;
}

const newFib = (): Fib => ({
    call: (self: Fib, n: number) => {
        return n === 0 || n === 1 ? 1 : self.call(self, n - 2) + self.call(self, n - 1);
    }
});

The same can be done with the MemoFib class:

interface MemoFib extends Fib {
    superClass: Fib;
    store: Map<number, number>;
}

const newMemoFib = (): MemoFib => ({
    superClass: newFib(),
    store: new Map(),
    call: (self: MemoFib, n: number): number => {
        const storedResult = self.store.get(n);
        if (storedResult !== undefined) {
            return storedResult;
        }

        const result = self.superClass.call(self, n);
        self.store.set(n, result);
        return result;
    }
});

const memoFib = newMemoFib();
console.log(memoFib.call(memoFib, 0));
console.log(memoFib.call(memoFib, 10));
console.log(memoFib.call(memoFib, 100));

We can perform some simplifications to the code:

• The type Fib can be simplified to a closure
• The wrapper function newFib can be removed, all we care about is the contents
• The only pert of MemoFib we want to be publicly available is call, so we can return a closure instead of an object
• superClass can be removed since it’s always going to be the same

After applying these simplifications we get the following;

type Fib = (self: Fib, n: number) => number;

const fib = (self: Fib, n: number) => {
    return n === 0 || n === 1 ? 1 : self(self, n - 2) + self(self, n - 1);
}

const newMemoFib = (): Fib => {
    const store = new Map();
    return (self: Fib, n: number): number => {
        const storedResult = store.get(n);
        if (storedResult !== undefined) {
            return storedResult;
        }

        const result = fib(self, n);
        store.set(n, result);
        return result;
    }
};

const memoFib = newMemoFib();
console.log(memoFib(memoFib, 0));
console.log(memoFib(memoFib, 10));
console.log(memoFib(memoFib, 100));

Passing a function to another function like this is denoted “continuation passing” in functional languages, which essentially means passing a callback function as an argument.

We can perform a further simplification. Closures have access to themselves if we give them a name, so we shouldn’t need to pass self to self. We can remove the self argument from Fib that way:

type Fib = (n: number) => number;

const fib = (self: Fib, n: number) => {
    return n === 0 || n === 1 ? 1 : self(n - 2) + self(n - 1);
}

const newMemoFib = (): Fib => {
    const store = new Map();
    const memoized = (n: number): number => {
        const storedResult = store.get(n);
        if (storedResult !== undefined) {
            return storedResult;
        }

        const result = fib(memoized, n);
        store.set(n, result);
        return result;
    }
    return memoized;
};

const memoFib = newMemoFib();
console.log(memoFib(0));
console.log(memoFib(10));
console.log(memoFib(100));

newMemoFib can memoize anything if we pass it as an argument. We can also make a generic version of Fib over any argument or return value:

type Callable<A, R> = (arg: A) => R
type Memoizable<A, R> = (self: Callable<A, R>, arg: A) => R;

const fib: Memoizable<number, number> = (self, n) => {
    return n === 0 || n === 1 ? 1 : self(n - 2) + self(n - 1);
}

const factorial: Memoizable<number, number> = (self, n) => {
    return n === 0 ? 1 : self(n - 1) * n;
}

const newMemo = <A, R>(baseClass: Memoizable<A, R>): Callable<A, R> => {
    const store = new Map<A, R>();
    const memoized = (arg: A): R => {
        const storedResult = store.get(arg);
        if (storedResult !== undefined) {
            return storedResult;
        }

        const result = baseClass(memoized, arg);
        store.set(arg, result);
        return result;
    }
    return memoized;
};

const memoFib = newMemo(fib);
console.log(memoFib(100));

const memoFact = newMemo(factorial);
console.log(memoFact(20));

We see that newMemo is just a function that takes something that is memoizable and turns it into something callable. If I realize that my factorial function doesn’t perform any better with memoization, I can swap out newMemo with something else as long as it turns my memoizable function into something callable:

const newRecursive = <A, R>(baseClass: Memoizable<A, R>): Callable<A, R> => {
    const recursive = (arg: A): R => baseClass(recursive, arg);
    return recursive;
};

const memoFact = newRecursive(factorial);
console.log(memoFact(20));

Excellent. I can memoize any function that is memoizable, and I can swap out the kind of recursion I use. I no longer have any of the limitations of the inheritance version.

If you look at the type signature of Memoizable versus Callable, we can now understand the difference between open and closed recursion. Memoizable functions are “Open” and Callable functions are “Closed”. This is because Memoizable functions call themselves by argument, and Callable functions call themselves by name. Open recursion allows the caller to override the recursive call with whatever we want. In our case, we could override the recursive call with memoization.

Conclusion

It turns out maybe object orientated and functional programming have something in common. When methods are overridden in an inheritance hierarchy, this desugars to continuation passing. Functional programs that use continuation passing can be converted to objects using inheritance, and inheritance can be converted to open recursion.

This post was originally posted on Andrew’s Notepad

Summary

• Curry’s paradox allows proofs of any statement in some languages that have the ability to construct a self referential sentence.
• This paradox allows a proof of any statement, no matter how absurd.
• Some statically typed programming languages allow for basic logical statements to be made and subsequently proven. Typescript is one of these languages.

The Premise

If this sentence is true, then Santa Claus is real.

This statement seems innocuous enough, but it exposes a fatal flaw in some logical systems. If it is possible for the above sentence to exist as a statement, then it follows that Santa Claus is real. In fact it doesn’t matter what the second half of this sentence is. You could prove that the sky is purple, unicorns outnumber humans ten to one, or that ants wear hats. This paradox is called “Curry’s paradox” after the famous mathematician Haskell Curry.

So how does the paradox work? Consider a more normal example of a sentence of the form “if A then B”. For example consider the following sentence:

If you add salt and water together, you get salt water

In order to prove a statement like this true, first we assume the first part of the sentence has already happened and then see if the second part came true. In this example, we assume that we have already mixed salt and water together (the first part of the sentence). Then we see if the second part came true. Since we have salt water, we can safely say that the sentence is true.

Cool, let’s get back to the original example

If this sentence is true, then oranges are green

…Close enough. To prove this statement, first we assume the first bit (the sentence is true), then we see if the second part came true as a result. We have assumed that the sentence is true, so that would mean …oranges are grey. Cool, we proved oranges are grey if the sentence is true, so that means the sentence is true.

So we have proven that the sentence is true by following normal logical rules. But if the sentence really is true, then oranges are grey.

Curry-Howard

Curry’s paradox exists in a lot of places including english, and naive set theory. In addition to this we can even write it in a lot of programming languages. To do this, we need to understand the connection between say a formal language used in mathematics, and a programming language. The connection is referred to as the Curry-Howard correspondence and it basically states that types are predicates and implementations are proofs. Note that the Curry in “Curry’s paradox” and “Curry-Howard” are in fact the same person!

In typescript, we are a bit limited, but there are some easy statements that we can write using just types. A type is the same as a definition or an axiom in mathematics.

// Define salt, pepper, meat, potatoes, and ducks.
type Salt = 'salt'
type Pepper = 'pepper'
type Meat = 'meat'
type Potatoes = 'potatoes'
type Duck = 'duck'

// Define Santa Claus as never existing
type SantaClaus = never;

// A type for A and B
type And<A, B> = {
    theFirstPart: A;
    theSecondPart: B;
}

// To get a christmas quacker, we need Santa Claus and a duck
type CristmasQuacker = And<SantaClaus, Duck>;

// A type for A or B
type Or<A, B> = 
    { theFirstOption: A } |
    { theSecondOption: B };

// To get seasoning we need salt or pepper
type Seasoning = Or<Salt, Pepper>

// To get a meal, we need seasoning, and either meat and potatoes
type Meal = And<Seasoning, Or<Meat, Potatoes>>;

We have shown that we can write basic statements which contain And and Or. We can also write statements that contain “if” in them. The logical symbol for “if” is the right arrow (→), which corresponds to the fat arrow (=>) of JavaScript:

// If both A and B exist, then A exists
type First = <A, B>(both: And<A, B>) => A;

// If both A and B exists, then B exists
type Second = <A, B>(both: And<A, B>) => B;

This is all well and good, but these are just definitions. Take First as an example. First is a sentence that states that if both A and B exist, then A exists. However, First is not a proof of that statement, it is just that statement itself.

In order to prove something to be true, we have to create an instance of that type. Here are some examples:

const saltExists: Salt = 'salt';
const pepperExists: Pepper = 'pepper';

const seasoningExists: Seasoning = {
    theSecondOption: 'pepper'
};

const mealsExist: Meal = {
    theFirstPart: seasoningExists,
    theSecondPart: { theFirstOption: 'meat' }
}

// A exists because the first part of A and B is A.
const firstIsTrue: First = (both) => both.theFirstPart;

// B exists because the second part of A and B is B.
const secondIsTrue: Second = (both) => both.theSecondPart;

Based on our definitions of salt, pepper, meat and potatoes, we have proven the existence of seasoning, an meals! In addition to that we have proven our First and Second predicates from above.

Falsehood

Proving something to be true is all well and good, but what if we want to prove something false? Luckily TypeScript provides us with a type which can never be created called never. This type is sometimes referred to as void, false, or bottom. The mathematical symbol for never is this upside down T: ⊥. We can use “A → ⊥” (if A then false) to mean “A does not exist”:

// A does not exist
type Not<A> = (false_statement: A) => never;

Now we have everything to prove things are not true:

// Santa Claus does not exist because we defined him not to exist
const santaClausIsntReal: Not<SantaClaus> =
    (santaClaus) => santaClaus;

// Christmas Quackers do not exist because because the first ingredient
// of a Christmas Quacker (Santa Claus) does not exist.
const christmasQuackersArentReal: Not<ChristmasQuacker> =
    (christmasQuacker) => christmasQuacker.theFirstPart;

We can even perform other proofs using Not. For example, what about the principle of explosion which states that given a contradiction we can prove anything:

// A contradiction is when something is both true
// and false at the same time
type Contradiction<T> = And<T, Not<T>>;

// Lemma: given a contradiction, we can prove anything
type PrincipleOfExplosion =
    <T, U>(contra: Contradiction<T>) => U;

// ...and here's a proof:
const principleOfExplosionIsTrue: PrincipleOfExplosion =
    (contra) => contra.theSecondPart(contra.theFirstPart)

Exciting! We can prove basic mathematical theorems using only TypeScript. We now have everything we need to move onto our paradox.

Curry’s paradox

We need some way to express the statement “If this sentence is true, then Santa Claus is real” as a type. To begin, we will give the sentence a name, say CurrysParadox.

type CurrysParadox = ??

This way we can rewrite our sentence to say “If CurrysParadox is true, then Santa Claus is real”. We know how to write a sentence with “if” in it, and we know how to write “Santa Claus is real”, so we can now fill in the blank:

type CurrysParadox =
    (paradox: CurrysParadox) => SantaClaus;

To prove that the statement CurrysParadox is true, we need to create an instance of it:

const currysParadoxIsTrue: CurrysParadox =
    (paradox) => ??

We are given an instance of CurrysParadox an argument, and we want to return an instance of SantaClaus. CurrysParadox is a function that returns a SantaClaus, so if we can call it, then we can get SantaClaus as a result. We can call paradox by passing itself as an argument:

const currysParadoxIsTrue: CurrysParadox =
    (paradox) => paradox(paradox);

Now that we know that CurrysParadox is true, we need to get an instance of SantaClaus to complete the proof. Same as above, currysParadoxIsTrue is an instance of CurrysParadox. If we can call currysParadoxIsTrue, then we will get SantaClaus as a result. We can call currysParadoxIsTrue by passing itself as an argument:

const santaClausIsReal: SantaClaus =
    currysParadoxIsTrue(currysParadoxIsTrue);

This type checks, so once this code runs we will have an instance of SantaClaus. We have finally proven that Santa Claus exists using nothing but TypeScript!

The absurdity continues. From the principle of explosion it is possible to prove anything if you have an untrue statement first:

const anythingIsTrue: <T>() => T = santaClausIsReal;

Uh, What?

Hold on a second. When we were defining our types, we said SantaClaus was equal to never. When we run this program, what actually gets stored in the variable we named santaClausIsReal? Is it a number? Is it a function? Didi we actually manage to store Santa Claus in a variable? Running the program gives us the answer:

Uncaught RangeError: Maximum call stack size exceeded

So the code fails with a stack overflow. This is strange considering that we didn’t use any recursion in our entire program. But all is well. We never actually assign anything to the variable santaClausIsReal because the program fails before the function returns. Unfortunately, we don’t get to see Santa after all. As consolation, we are able to live in a world where the rules of mathematics still make sense.

So what happened?

If we write currysParadoxIsTrue inline for santaClausIsReal, and shorten paradox to p we get the following statement:

(p => p(p))((p: CurrysParadox) => p(p));

The astute functional programmer will realize that this is in fact Omega from lambda calculus. It is a famous example of a statement, that when executed, produces a perfect copy of itself. This explains the stack overflow: the function body produces a copy of itself, and then calls the copy repeatedly until the stack runs out of space.

So how do we reconcile the fact that TypeScript allowed us to write these absurd kinds of results in the first place? Is TypeScript broken? Well, no. In order to facilitate writing programs with loops and other non-obvious exit conditions, TypeScript allows us to write non-terminating programs. When we give a type to a function, we declare that when it returns, the value will have the specified type. If it never returns, then we never break any of the typing rules.

Conclusion

Unfortunately we don’t get to prove Santa Claus is real, but we do get an interesting insight into the parallels between mathematics and the type checker. Many languages allows us to write basic logical statements as types, and prove them using an implementation. However, this has it’s limitations. The proofs of statements are only applicable when it is known ahead of time that the program will terminate. In other cases, the language is susceptible to contradiction.

This post was originally posted on Andrew’s Notepad

Summary

• Switch statements over basic enums have lots of disadvantages
• Many of the limitations can be overcome by using either dynamic dispatch, or the visitor pattern depending on your use case
• In languages that support sum types natively, consider using those instead of the visitor pattern

Introduction

I have seen several articles recently that take the position that switch statements are an anti-pattern in regards to best practice object-orientated programming. In a way, the sentiment is usually fine, but I think that these articles are a bit incomplete and don’t tell the full story. Let’s take a lesson from functional programming to find when using a switch is OK.

I’ll use TypeScript in this article as it supports all of the necessary language features and is usually pretty easy to read.

The argument against switch statements

In basic usage switch statements are usually pretty limited, and are used to match against integers, and sometimes strings. Imagine we have two functions draw and area which can either take a unit square (side length 1) or a unit circle (radius 1). Using switch statements the code would look something like this:

enum Shape {
    UnitSquare,
    UnitCircle
}

const area = (shape: Shape) => {
    switch (shape) {
        case Shape.UnitSquare:
            return 1;
        case Shape.UnitCircle:
            return Math.PI;
        default:
            throw new Error("not a shape");
    }
}

const draw = (shape: Shape) => {
    switch (shape) {
        case Shape.UnitSquare:
            console.log("drawing a unit square");
            return;
        case Shape.UnitCircle:
            console.log("drawing a unit circle");
            return;
        default:
            throw new Error("not a shape");
    }
}

// usage
const doStuff = (shape: Shape) => {
    console.log(area(shape));
    draw(shape);
}
doStuff(Shape.UnitSquare);
doStuff(Shape.UnitCircle);

There are a number of problems with this code:

• Enums are just integers so you can’t set the side length of the square or the radius of the circle
• If you add another shape, you have to search through all of your code to find switch statements that don’t cover every case. If you miss one you could get runtime errors.

There are probably more reasons why this code is not the best. I will begrudgingly admit it that it has one benefit: as it stands, the code is dead simple to read. There’s no magic and it’s laid out in such a way that it’s very clear what it does.

At the moment, we have two functions that both accept two variants each. We could swap this around and have two variants that have two functions each:

interface Shape {
    area: () => number;
    draw: () => void;
}

class Circle {
    private radius: number;

    constructor (radius: number) {
        this.radius = radius;
    }

    area() {
        return Math.PI * this.radius * this.radius;
    }

    draw() {
        console.log("drawing a circle");
    }
}

class Rectangle {
    private width: number;
    private height: number;

    constructor (width: number, height: number) {
        this.width = width;
        this.height = height;
    }

    area() {
        return this.width * this.height;
    }

    draw() {
        console.log("drawing a rectangle");
    }
}

//usage
const doStuff = (shape: Shape) => {
    console.log(shape.area());
    shape.draw();
}

doStuff(new Circle(2));
doStuff(new Rectangle(2, 5));

Instead of having an enum, we provide an interface. The Circle and Rectangle classes implement that interface. This solves both of the original stated aims:

• The Circle has a radius, the Rectangle now has a height and width
• We can add new types of shapes easily without having to modify switch statements everywhere

So everything is good and we can call it a day right?

Visitors

Before I can head home safe in the knowledge that the second form is the best way to write that particular code I have to ask a question about the future of my program: Am I more likely to add new methods to the Shape interface, or am I more likely to add new types?. For the rest of this article, I will assume that you’re more likely to add new behavior than new types. If this is not your use case, then you can stop reading, use an interface as shown above, and say good riddance to the switch. If you are unsure, then keep reading, you might learn something new.

It’s easy to add more types to an interface: simply define a new class and implement the interface. You don’t have to modify any existing code, you can just add new functionality. However, if you decided that you wanted to add a new method to the Shape interface, then every class that implements Shape needs to be modified.

Compare this to the first example of code where we used a switch statement. If we want to add a new type, then we have to modify all of our switch statements. However if we want to add a new method, then we can just write a new function without modifying any of our existing code.

The first example has the complete opposite problem than the second example does. However, to tackle some of the other limitations of the first example, an OOP enthusiast would probably recognize that we can use the visitor pattern instead:

interface ShapeVisitor<T> {
    circle: (circle: Circle) => T;
    rectangle: (rectangle: Rectangle) => T;
}

interface Shape {
    visit: <T>(visitor: ShapeVisitor<T>) => T;
}

class Circle {
    radius: number;

    constructor (radius: number) {
        this.radius = radius;
    }

    visit <T>(visitor: ShapeVisitor<T>) {
        return visitor.circle(this);
    }
}

class Rectangle {
    width: number;
    height: number;

    constructor (width: number, height: number) {
        this.width = width;
        this.height = height;
    }

    visit <T>(visitor: ShapeVisitor<T>) {
        return visitor.rectangle(this);
    }
}

const area: ShapeVisitor<number> = {
    circle: (circle) => Math.PI * circle.radius * circle.radius,
    rectangle: (rectangle) => rectangle.width * rectangle.height
}

const draw: ShapeVisitor<void> = {
    circle: (_circle) => console.log("drawing a circle"),
    rectangle: (_rectangle) => console.log("drawing a rectangle")
}

// usage

const doStuff = (shape: Shape) => {
    console.log(shape.visit(area));
    shape.visit(draw);
}

doStuff(new Circle(2));
doStuff(new Rectangle(2, 5));

The visitor pattern is quite complicated, but it does the job.

• The circle has a radius, and the rectangle has side lengths
• New functionality can be added without modifying existing code
• (Bonus benefit!) If we add a new variant (say we add a triangle), then the compiler will point out all the visitors where we need to change our code and won’t compile until we have fixed it.

The the visitor pattern is actually a page taken out of the functional programming book: it is the church encoding for sum types in lambda calculus in disguise. That sounds like gibberish, but what that means for us humans is that we can write the visitor example with much more brevity. Note that I do not condone the use of church encoding in useful programs, but want to include it as a demonstration of how unreadable it is:

interface ShapeVisitor<T> {
    circle: (radius: number) => T;
    rectangle: (width: number, height: number) => T;
}

interface Shape {
    <T>(visitor: ShapeVisitor<T>): T;
}

const Circle = (radius: number): Shape => 
    ({ circle }) => circle(radius);
const Rectangle = (width: number, height: number): Shape => 
    ({ rectangle }) => rectangle(width, height);

const area: ShapeVisitor<number> = {
    circle: (radius) => Math.PI * radius * radius,
    rectangle: (width, height) => width * height
}

const draw: ShapeVisitor<void> = {
    circle: () => console.log("drawing a circle"),
    rectangle: () => console.log("drawing a rectangle")
}

// usage

const doStuff = (shape: Shape) => {
    console.log(shape(area));
    shape(draw);
}

doStuff(Circle(2));
doStuff(Rectangle(2, 5));

In this example we ditch classes, and use closures instead. This has all of the advantages of the visitor pattern, but is even more unreadable. Church encoding is not a great way to program.

Sum types

In the above examples we mentioned “sum types”. A sum type is something that can be one thing or another. For example, boolean is a sum type of true and false because it can either be true or false. Shape is a sum type too: it can either be a Circle or a Rectangle.

We used the “Church encoding” for sum types in the above example. The Church encoding is one method you can use when your language doesn’t support sum types natively. Luckily for us, TypeScript does support sum types, so we should prefer that:

type Shape = Circle | Rectangle

interface Circle {
    variant: 'circle';
    radius: number;
}
const Circle = (radius: number): Circle => {
    return { variant: 'circle', radius };
}

interface Rectangle {
    variant: 'rectangle';
    width: number;
    height: number;
}
const Rectangle = (width: number, height: number): Rectangle => {
    return { variant: 'rectangle', width, height };
}

const unreachable = (_x: never) => {};

const area = (shape: Shape) => {
    switch (shape.variant) {
        case 'circle':
            return Math.PI * shape.radius * shape.radius;
        case 'rectangle':
            return shape.width * shape.height;
        default:
            return unreachable(shape);
    }
}

const draw = (shape: Shape) => {
    switch (shape.variant) {
        case 'circle':
            console.log("drawing a rectangle");
            return;
        case 'rectangle':
            console.log("drawing a circle");
            return;
        default:
            return unreachable(shape);
    }
}

// usage

const doStuff = (shape: Shape) => {
    console.log(area(shape));
    draw(shape);
}

doStuff(Circle(2));
doStuff(Rectangle(2, 5));

The funny unreachable function is used as a static assertion. The program will not compile if there is a valid code path that reaches the unreachable function. We no longer need to throw an error in the switch branches because the compiler can statically determine that the default branch is never executed. This means that if you added a new variant (like a triangle) the program won’t compile until you fix all of the relevant switch statements.

Conclusion

We have come full circle. We took a long journey but we are back at a switch statement. The above code essentially has all the benefits of the original visitor example, because semantically they describe the same thing.

• We started off with a switch on an enum and found it had lots of disadvantages
• We could fix those disadvantages by using the visitor pattern
• The visitor pattern is a way to describe sum types using Church encoding in languages that don’t support them natively
• In languages that support sum types, consider using them directly

The differences between the first and last examples are:

• The switch statement matches on the variants of a sum type rather than a basic enum
• The switch statement is exhaustive due to the unreachable function

So if your language supports sum types and you think you’re more likely to add new functionality to existing classes than you are to add a new types, consider using switch.

Parting Note

In languages that don’t support sum types natively, don’t be afraid to use a visitor if you were thinking about using a switch statement. After all, it is a well established design pattern. If you find yourself needing to branch based on type, or if you need to use downcasting, then consider using a visitor as an alternative.

This post was originally posted on Andrew’s Notepad

• “Don’t repeat yourself” (DRY) is a widely accepted programming principle, but it has several limitations
• An alternative derivation of DRY is presented, that aims to alleviate some of these limitations
• Programs are made up of functions, and to make functions maximally reusable they should
  • Have the most lenient preconditions
  • Have the most strict postconditions
  • Perform the smallest amount of non-trivial work

If you think this article is too long, you can skip to Should I refactor my code? below

Don’t Don’t Repeat yourself

In software engineering, “clean code” is a sought after ideal. Code that is “clean” is easy to read, understand and maintain. Unfortunately, what clean code looks like is often disputed. Some commonly accepted software engineering best practices have lost sight of their original goals, or do not represent a complete picture. I believe Don’t Repeat Yourself (DRY) may be one of these principles.

DRY is a widespread and generally well accepted best practice in software engineering. The main stated advantage to DRY is that code does not need to be modified in multiple locations. If code has been de-duplicated, then bugfixes and new features only need to be added once. In addition, code only needs to be tested once rather than multiple times. The end result is that code is less error prone and faster to write.

Unfortunately, it is also well known that the overzealous application of DRY can lead to poor abstractions, and code that is difficult to modify. If DRY were to be taken to the extreme, then it’s aim is just to increase the entropy of written code and turn it into code golf. In some instances this is a good thing, for example: obfuscation, or minified JavaScript, but in most cases, code written in this style is obviously detrimental. This indicates that DRY has some limitations as a principle.

This article will explore some of the limitations of DRY and provide an alternative perspective that does not have the same limitations.

Limitations of DRY

One of the oft discussed limitations of DRY is that it can lead to poor abstractions. Heuristic solutions like the “rule of three” exist to alleviate this problem, where only code that is duplicated three or more times is worthwhile refactoring. This is not at all perfect. For example, the fourth replication may require an additional parameterization that the first three applications did not.

Often ignored in this discussion, plain, non generic functions are the basic unit of code reuse, and relatively often these functions only have a single call site. Following a strict “rule of three” application, it would follow that a function with a single call site should instead be written inline, and only factored out once it has been written three times. Therefore there appear to exist some worthwhile abstractions, even if they do not reduce the repetition of code:

// Large functions that have comments in them are
// considered good candidates for refactoring
function bigFunc() {
    // Do part A
    console.log("Did part A");
    // Do Part B
    console.log("Did part B");
    // Do Part C
    console.log("Did part C");
}

// It is recommended to split such a function into
// smaller parts. Cited reasoning includes:
//  - Easier testing
//  - Self documenting code
//  - Reusability

function doPartA() {
    console.log("Did part A");
}

function doPartB() {
    console.log("Did part B");
}

function doPartC() {
    console.log("Did part C");
}

function newBigFunc() {
    doPartA();
    doPartB();
    doPartC();
}

Another argument against applying too much DRY is that overly generic code can be more difficult to understand and maintain. I’m a bit skeptical of how significant this really is, as ‘Difficulty’ is a very subjective concept and would vary from person to person, but I suppose there may be some truth to the claim that this negatively affects software development in some measurable capacity.

In addition, I dispute the assertion that using DRY is necessarily faster to write. Copy and paste is quick, and I suspect that the vast majority of developer time is not spent actually typing in the first instance. In the case of modification, a global find and replace works for actually writing the code (and for copying over any relevant tests).

Obviously copy and paste, and global find and replace are not the recommended tools for refactoring a program, but the concept here is that writing the code itself is not the bottleneck.

To prove me wrong (or right) in this regard, someone would have to measure the time it takes to write out duplicated code versus non duplicated code. Additionally, they would need to show that there was a significant difference between them whilst proving that the difference wasn’t just due to chance. I doubt that such a study will be done for some time.

From First Principles

Considering how widespread the principle of DRY is, there is little empirical evidence to say to what extent it improves code quality. The promotion of DRY is largely based on the subjective experiences of software engineers over the course of their work. As such it is unknown exactly how effective it really is.

Certainly, DRY is not an end goal. End goals include things such as reducing time to market, reducing cost, or increasing developer productivity. The quality of code can also play a part; it could be measured by the frequency and severity of bugs that affect users.

Since the empirical evidence is not of sufficient quality, the next best thing would be to start with some small, but fundamental assumptions about how code works, and then derive what good code should look like from those assumptions. Note that I will not discuss how writing code one way will fix errors that writing code in another way exposes, nor will I discuss lessons that I have learned writing software. The conclusions drawn in this article will be drawn without the need to have experienced real world code.

I will assume that a developer never deliberately introduces bugs into their programs, and would choose to write a program with no bugs if at all possible. This is actually quite an ambitious assumption. Some bugs are just not worth fixing under the scrutiny of a cost-benefit analysis; bugs that are extremely mild in nature, or are rare enough that they are never experienced by an end user might not be worth going through the trouble to fix. However, I will persist with this assumption for the purposes of this article.

In addition, I will assume that the harder it is to prove a program correct, the more likely it contains errors. I define a program as ‘hard’ to prove correct if that program requires a lot of symbols in some theorem proving language to ensure correctness. Conversely, a program that requires few symbols is therefore ‘easy’ to prove correct. I note then that the number of symbols required for a proof is proportional to the number of errors you would find along the way.

There's an implicit assumption that during the proving process, bugs can be fixed without adding or removing symbols. In this way it makes sense to talk about an incorrect program whilst also talking about proving it correct.

In the biggest logical leap of this article, I will mention something something Curry-Howard, and state without much proof at all that the number of symbols required to prove a program correct is proportional to the number of symbols in the code itself. This is a huge assumption, and probably not at all correct in a lot of instances. However, it does give us a convenient proxy on the number of errors that are in a given piece of code: if the code is longer, or is more complex, then it has more errors.

Note that in a functional language with dependent types, where the Curry-Howard isomorphism can actually apply, the proof of a function's correctness and it's implementation is literally the same code. So saying that the length of the proof is the length of the code is indeed a reasonable assumption to make.

So at least one way to reduce the number of errors in code will be to reduce the number of symbols in the code itself. This sounds a lot like DRY. Note that code that is terse does not fit the bill for having less errors, as it still contains the same number of symbols. In this way, naming, styling, and indentation are ignored.

My Recommendation

Programs regardless of paradigm are made out of functions. Well, maybe not in some cases, but they must be made of something. And in order for a discussion about code reuse to be meaningful there must be something in a programming language that can be reused. I’ll use “function” to mean the smallest reusable part of a program

Perhaps to the chagrin of an Object Orientated programmer, I will specifically not talk about objects being reusable. Objects can have multiple methods or members, so if an object is reusable, then that means that there is something smaller that is reusable too.

So if we consider a function to be the smallest reusable part of a program, then in order for our program to have the maximum reuse, any given function should:

• be able to be reused as much as possible, and
• it should be as useful as possible, and
• it should be used as much as possible.

In order to achieve these goals respectively:

• A function should have the smallest possible set of preconditions
• A function should have the largest set of postconditions
• A function should perform the smallest non trivial amount of work

The Smallest Possible Set of Preconditions

A maximally reusable function should be able to be called wherever it is applicable. Functions can only be called when the caller can fulfill the preconditions of the function. If the caller cannot fulfill the preconditions of a function, but calls the function anyway, then this is an error.

I’ll provide a short and not at all rigorous proof that maximally reusable functions have minimal preconditions using contradiction. Assume that there exists a maximally reusable function f which specifies a precondition that is unnecessary. That means that there exist potentially valid calling contexts where the function cannot be called, because the precondition cannot be satisfied. f can be made more reusable by removing the unnecessary precondition. Because f is already maximally reusable, this is a contradiction.

In fact, this must be the only thing that determines if a function is able to be used in as many places as possible. The only other places that f could potentially be used cannot satisfy necessary preconditions. Since it is already known that the function f has a minimal preconditions, any call to f in these locations is invalid.

In other words a function with minimal preconditions can be used anywhere it would be valid to call such a function, and the only places it cannot be called are all invalid anyway.

Example 1: Generics

Say you have a function that takes the maximum of two integers. I use Haskell and Rust, because in a lot of cases, you can specify the preconditions as part of the function type signature.

-- Haskell
maxInt :: Int -> Int -> Int
maxInt a b = if a > b then a else b

// Rust
fn max_int (a: i32, b: i32) -> i32 {
    if a > b { a } else { b }
}

In this example, the precondition that the numbers must be integers is unnecessary. The only preconditions that are truly required are that both arguments have the same type, and that they can be compared to each other. We can specify that both arguments have the same type with polymorphism, and specify that they can be compared using a type constraint:

maxAny :: (Ord a) => a -> a -> a
maxAny a b = if a > b then a else b

fn max_any<T: Ord> (a: T, b: T) -> T {
    if a > b { a } else { b }
}

The second examples are more reusable. This should be fairly obvious considering the second example is generic, where the first example is not. Not to say that I advocate making absolutely everything as generic as possible, only that more generic functions are more reusable. More specifically, the reason that generic functions are more reusable is because they specify a smaller set of preconditions.

	Recommendation: Use generics to make code more reusable.

Although the above reasoning is agnostic regarding the programming paradigm used, those who are familiar with Object Orientated design idioms may find this familiar. Reducing the preconditions by making a function more generic corresponds to the ‘I’ and ‘D’ of SOLID. The ‘Interface Segregation’ principle recommends directly reducing the interface size, whilst the ‘Dependency Inversion’ principle recommends using interfaces in the first instance.

Those that are familiar with functional programming may see that the second example has been universally quantified. If the type signature of the above example can be thought of as a predicate in first order logic, then a universally quantified predicate is a stronger statement than an unquantified one.

	Recommendation: Use best practice for your preferred programming paradigm.

Example 2: Unnecessary Input

Although making functions more generic will make them more reusable, it does not cover every use case. Most notably passing too much data into a function limits it’s reuse. I use TypeScript in this example as it is usually easy to understand:

// Typescript
class Person {
    private name: string;
    private phoneNumber: string;
    private age: number;

    constructor(name: string, phoneNumber: string, age: number) {
        this.name = name;
        this.phoneNumber = phoneNumber;
        this.age = age;
    }

    textMessage(message: string) {
        console.log(`Message sent to ${this.phoneNumber}`);
        console.log(`The message was: "${message}"`);
    }
}

const person = new Person("John Doe", "0491 570 110", 35);
person.textMessage("Save on a new car with out promotional deal!");

In this case, the textMessage method takes a Person object as this, and a string representing the message. The textMessage function only uses the phoneNumber property of Person. Requiring a Person is an unnecessary precondition. The textMessage function can be made more reusable by removing that precondition:

function textMessage(phoneNumber: string, message: string) {
    console.log(`Message sent to ${phoneNumber}`);
    console.log(`The message was: "${message}"`);
}
textMessage("0491 570 110", "Save on a new car with out promotional deal!")

It should be possible to send a text message to any valid phone number, not just phone numbers that have been assigned to a particular person. It should also be possible to send a text message if the name or age of the person is unknown.

	Recommendation: Pass the minimum required data into a function.

The larger the class is, the more unnecessary data will be passed to any given method. For example: if a class is responsible for two separate items, then methods which regard only the first item, or only the second will be passed unnecessary state as part of this.

Note that this ties back in with the ‘S’ of SOLID. The ‘Single Responsibility’ principle recommends breaking classes with many responsibilities into smaller classes.

	Recommendation: Keep classes and data structures small.

Other Examples

• Functions that require more arguments than are necessary.
• Non static methods that can be made static.
• Making a function async when it doesn’t need to be.
• Using mutable references when immutable references would suffice.

The Largest Set of Postconditions

In order for a function to be maximally reusable, it must be useful in as many places as possible. Each potential call site may place requirements on the result of the function. Therefore, it would make sense that the more strict the postconditions, the more places the function can satisfy the requirements of the call sites.

Example 1: Return the Parent Class

In Object Orientated programming, a parent class can be used wherever a child class can be used, but not the other way around. To make the function as reusable as possible, return the parent class:

For functional programs, returning a base class would be the equivalent of returning an existentially quantified data type.

// Typescript
class Child {
    childFunc() {
        console.log("I'm a Child");
    }
}

class Parent extends Child {
    parentFunc() {
        console.log("I'm a Parent");
    }
}

function factoryChild (): Child {
    return new Parent();
}

function factoryParent (): Parent {
    return new Parent();
}

const p1: Parent = factoryParent();
// Error: Child does not implement parentFunc
// const p2: Parent = factoryChild();

const c1: Child = factoryParent();
const c2: Child = factoryChild();

Note that it is often recommended elsewhere to use abstract factories which return interfaces, or virtual base classes instead of concretions. Returning the parent like I recommend in this case is therefore discouraged. If this makes you uncomfortable, you can always write a wrapper function to turn a concretion into an abstraction. This can’t be done the other way around:

// Convert a concretion to an abstraction
function factoryChild2 (): Child {
    return factoryParent();
}

// Cannot be done in reverse
// function factoryParent2 (): Parent {
//     return factoryChild();
// }

Alternatively if the factory function could return multiple concretions, then the return type has to be an abstraction, so the above code doesn’t even apply:

class Parent2 extends Child {
    parent2Func() {
        console.log("I'm the second Parent");
    }
}

// Can't return a Parent type here. Returning
// a Child is the most minimal postcondition.
function abstractFactoryChild(): Child {
    if (Math.random() > 0.5) {
        return new Parent();
    } else {
        return new Parent2();
    }
}

	Recommendation: Return the most specific type possible.

Example 2: Validating Arguments

In the general case, throwing exceptions is a normal part of programming. Some things are just not in our control, so we just do the best we can, handle it and clean up the mess. If a function validates arguments and returns early on failure, then this indicates that the preconditions are too lenient:

// Typescript
function mayHaveBadArgument(argument: number | null): string {
    if (argument === null) {
        throw "Bad argument";
    }
    
    return argument.toString();
}

Perhaps this is too obvious an example and would not appear in the real world, I suspect something like the following is written more commonly:

function mayHaveBadArgument(argument: number | null): string | null {
    if (argument === null) {
        return null;
    }
    
    return argument.toString();
}

In this case, null is an invalid value, so why accept it as an argument at all? Maybe it would be better to just take a number and return a string, and make it up to the caller to check for the precondition.

function printNumber(argument: number): string {
    return argument.toString();
}

Note that this is a bit controversial for two reasons. Firstly it gives stricter preconditions and stricter postconditions, when we would prefer stricter postconditions and more lenient preconditions. Secondly, argument checking code might end up being duplicated at a lot of call sites.

Both of these problems can be fixed by creating a second function that only validates the arguments. Note here, that in this toy example it doesn’t really provide any benefits, but the benefit may become more obvious in larger examples.

function printNumberChecked(argument: number | null): string | null {
    if (argument === null) {
        return null;
    }
    
    return printNumber(argument);
}

On one hand, the callers of printNumber do not have to check the return value for null. On the other hand, callers using printNumberChecked do not have to check argument for null. If argument is known not to be null, then use printNumber, otherwise, use printNumberChecked.

	Recommendation: Don't write a function that compromises lenient preconditions for strict postconditions or vice versa. Write two functions instead.

This is the first recommendation that states that something is specifically a bad abstraction. Not every abstraction will be a good one, so there should be rules for things that specifically shouldn’t be done.

Advanced Usage for Functional Programs

Note that for most languages printNumberChecked may not be that good an abstraction. It’s not reusable enough and does some fairly trivial checking. I wouldn’t stress if this appears in your code a lot. A functional programmer may notice that T | null is a functor, and printNumberChecked is a poorly written version of map or fmap specialized for printNumber:

-- Haskell
printNumber :: Int -> String
printNumber = show

printNumberChecked :: (Functor f) => f Int -> f String
printNumberChecked = fmap printNumber

// Typescript
function mapOptional<T, R>(func: (arg: T) => R): (arg: T | null) => R | null {
    return (arg: T | null) => {
        if (arg === null) {
            return null;
        }

        return func(arg);
    }
}

const printNumberChecked = mapOptional(printNumber);
console.assert(printNumberChecked(null) === null);
console.assert(printNumberChecked(5) === '5');

	Recommendation: Don't use a function with side effects when a pure function will do.

Note that this is generally not idiomatic code except for functional languages. I wouldn’t use this unless you, and all of you co-contributors are on the same page.

Perform the Smallest Amount of Non-trivial Work

In order for a program to actually realize the benefits of reusable code, the program must actually reuse it’s components. Whilst the above principles will help determine what abstractions are good to take, this principle determines how they should be applied.

If a function provides no benefit over writing the code inline, then I would consider it a trivial amount of work. That means trivial functions do not improve the ergonomics, prevent errors or improve any other measurable (or subjective) element of code.

	Recommendation: Don't write trivial abstractions.

A function does not perform the smallest possible non-trivial work if it can be broken down into at least two smaller non-trivial pieces. For the sake of argument, say that a non-trivial function called f could be refactored into two smaller non-trivial functions called a and b that could be recombined to form f. f must either contain duplicates of a and b inside it’s body, or it must be defined in terms of a and b.

Assume f is defined in terms of a and b. a and b would have smaller preconditions than f, so they would be able to be used in more places. If f is defined in terms of a and b, it either contains the the smallest amount of non trivial work to combine a and b, or it can be broken down further, in which case we do so and re-examine the components.

Either way, a function either contains duplicate code, contains preconditions that are too strict, or it performs the smallest amount of non-trivial work. In other words, large functions can be broken down into pieces to make them more reusable.

Examples

I won’t talk much more about this topic as there appears to be a broad consensus that smaller composable functions are better than larger ones. I’ll link to NASA’s ten rules for safety critical software, which states that functions written in C should not have more than a single page’s worth of code in them.

	Recommendation: Break large functions into smaller ones.

Should I Refactor my Code?

• I have two similar functions, should I refactor them into one?
  • If you can combine the two functions without weakening the postconditions, or strengthening the preconditions: then go ahead. If the function is too big, consider splitting it into parts.
• I have two identical functions, should I refactor them into one?
  • Yes.
• I have a function that I think could be refactored based on the above principles but it isn’t duplicated anywhere, should I do it?
  • If you can weaken the preconditions, strengthen the postconditions, or if you think the function is too big, it should be OK to refactor. This assumes that any requirement could change anywhere in your program at any time.
• I have a function that I think might be big enough that it can be split into parts, but it’s borderline. Should I split it?
  • The triviality of a function is subjective. If you think that it improves ergonomics, makes the code easier to read, or reduces the possibility of errors compared to writing it inline, do it.
• I have a function that could be more generic, should I do it?
  • If you don’t weaken the postconditions in doing so, or make the function too big, then it should be fine. Caveat: sometimes the function already does it’s job and you don’t need it to be generic. Then it might not be worth the effort.
• I have a function where the return type could be made more specific, should I do it?
  • If you don’t weaken the preconditions, or make the function too big, then go ahead.
• What about return type polymorphism? Doesn’t that make code more reusable while making the postconditions weaker?
  • Return type polymorphism is equivalent to passing an (often zero sized) type as an argument, therefore weakening the preconditions. The three principles do not state if this is a good or a bad thing. Consider having both a specialized variant, and a generic one for different circumstances.
• I have an object that might be too large, should I split it into two?
  • If at least one of the methods does not need access to the whole object, consider splitting it. Exception: delegation, getters/setters.
• Is XXX is a good abstraction?
  • If you can define the preconditions, postconditions, and invariants well, then it should be OK, otherwise: probably not.
• Is XXX object a good abstraction?
  • Examine the methods individually, and apply the principles above. If all of the methods individually are OK, then the whole object should be too. This assumes that you only access the object through its methods.

Conclusion

DRY is a programming principle that has many limitations. An often cited limitation is the capacity of DRY to create inefficient abstractions. Heuristic solutions have been adopted to try to find a good medium. This article presents a different way of deriving DRY from some basic assumptions. It also provides three principles for writing abstractions that aim to be reusable. This allows programmers to write code with no information about the calling context whilst providing some guarantees that the code can and will be reused as many times as possible.

This post was originally posted on Andrew’s Notepad

With these kinds of game jams, there is a restrictive time limit involved. In this case, 30 days. On day 0, you receive a theme in which to base your game upon: the theme was “Leaps And Bounds”. Generally there is not enough time to create a large, polished game given the limitations, and in my case, the game was largely unfinished. There was definitely enough gameplay for a few hours, but I had hopes for a bit more. Well, anyway, here’s how it went.

Brainstorming

For the first day or two, I tried to come up with ideas on how my game was going to work. I wanted to make a puzzle game of some description, and hopefully something that I would have fun playing as well. As the puzzle maker, it’s almost a given that you know the solution, but there are some problems which you can give yourself a blank palette, describe what you want, and then fill in the gaps. This gives the player (and me) a wide artistic license to solve the problems in their own way.

I had an idea for a tile based automation based on Befunge where the stack was part of the main memory. A basic language would need an instruction pointer and a stack pointer. In a game scenario, it would make sense that the instruction pointer and stack pointer would be physical entities. I ended up settling on cows, since I could reuse some assets from the last game jam I entered.

In addition, if I was going to create a computer, I needed to give it some limitations. If I made it too easy to write programs, then it would just be programming, however, if I add some artificial limitations, then it becomes a puzzle that needs to be solved.

Initial sprint

With only 30 days, I decided to try to remain with things that I already know, which would give me a predictable time limit to complete the game. I knew that I would not need a physics or collision engine, and decided to just create a game that works on a canvas in HTML with no framework. I thought that the game was minimal enough that learning additional frameworks or libraries would have simply wasted time, as the majority of the effort would be spent writing original code anyway.

In hindsight, this was a huge mistake, and I should’ve gone straight to Godot or a well established game engine. Just bundling the program at the end took almost half a day due to various reasons, but Godot would have instantly given me a cross platform binary, and a web version to boot!

For a few days, I used plain Javascript, but it became apparent that some things were going to eat up a lot of time. I would have spent too much time creating persistent data structures for the undo functionality for example. In addition, it became apparent that some of the data was not well suited to JavaScript’s weak typing. As much as JavaScript is quick to prototype, it is largely difficult to refactor.

Given my abilities as a programmer, I decided that it would be faster to switch to a language which has much stronger guarantees about the program at compile time, so that I can offload as much of my mental effort as possible onto the compiler. My personal experience with Rust is that it requires significantly less thinking while programming; the compiler just tells you what you’ve done wrong.

It took me a day and a half to rewrite what I had in Rust and compile to web assembly. I strongly believe that my productivity increased significantly after the change. In addition, the quality of my code was better than if I had written JavaScript. The reduced debugging ability of Rust on web assembly was largely offset by the fact that most bugs were caught before the program compiled. The bugs that did get past the compiler were easier to fix than JavaScript anyway.

The First Mistake

Theres a saying that if you only have a hammer, then everything starts looking like a nail. I’m no artist or sound designer. I write programs. The main mistake that I made was thinking that this game was going to be a technical exercise in programming. The reality of game design is far different.

Many modern game engines advertise that users can create significant portions of their game without writing any code at all. This is without a doubt a true statement. Unfortunately for me, I had (stupidly) decided not to use a game engine. I had vastly underestimated how much of a game was created by using the tools of an editor.

All of the luxuries that make your life easier with a fully featured game engine were not present. Every animation had to be coded by hand. Every graphic on the sprite sheet was manually selected. The level editor was hand coded. I even ended creating an ad-hoc component system and scene manager. All of this code was strictly unnecessary, and wasted time that I didn’t really have.

I suppose the other benefit to a game engine is having everything already set up for you. With Rust, I had to deal with the JavaScript foreign function interface. Using a foreign function interface is not great even at the best of times, so spending a lot of time getting trivial things to work was a bit stifling sometimes.

It’s not all a downside: some of the code I wrote would have been difficult to write correctly in a dynamic language (although I am of the opinion that all code is a little bit more difficult to write correctly in a dynamic language). However, in my opinion, 90% of code is usually “dumb” code, which just marshals around data from here to there, and 10% is “smart” code which does something interesting from a computer science standpoint. That means I’m only really worried about making 10% of my code more difficult to write. Godot has official support for many languages, so I’m sure that the benefits would have completely outweighed the downside.

In essence, I would have benefitted greatly by using an engine to create my game. Some of the smaller things that were easier in the past were more difficult. Therefore, some of the more important parts of the gameplay were missing; there simply wasn’t enough time to complete all of it.

The Second Mistake

Apart from not using an engine. My other error was to think of the game as a technical coding exercise rather than a piece of entertainment. Some of the greatest games are not technical masterpieces, and nobody gives wide acclaim to a game and attributes that exclusively to the quality of its code. In reality, if a game is programmed well, then nobody will know.

Unfortunately I’m not an artist or a musician. Therefore, these aspects were left on the back burner until the very end, an afterthought. In reality, I should have given these parts equal, or more attention than the code itself. If 90% of my code is just marshaling data, then maybe I could have eliminated some of it by using the built in engine capabilities, and spent the rest of my time creating assets that look and feel good.

If I could go back and do it again, I would probably create a series of assets which are more cohesive and atmospheric. Certainly making cows move around a paddock is not particularly dramatic, but I suspect that the graphics could have made a bit more sense.

That’s it!

That was exhausting! 30 days of game-making in your free time is hard work! For 30 days and no engine, I’m quite happy with the result (even if it is incomplete). I made myself a puzzle game that even I had trouble with solving. If nobody else enjoys it, I still had fun, and learned a few things about the relevant technologies.

More importantly, I learned some things that aren’t purely technical: Use the right tools for the right job; manage your time effectively; don’t reinvent the wheel; and if you’re making a game, make it fun to play!

I hope you enjoyed this article, if you want to try the game in all of it’s unfinished glory, you can play it here. The real puzzles start in the second world. Any feedback is welcome.

This post was originally posted on Andrew’s Notepad

• A persistent data structure can easily keep track of previous versions of itself with little overhead.
• A regular data structure can be converted into a persistent one by replacing instances of Box with Rc, and replacing mutable dereferences with Rc::make_mut.
• The resulting structure is both more performant and uses less memory if you plan on performing lots of clones.

What is a persistent data structure?

Data structures are persistent when they can keep track of all of their old states. Every time a modification is made, a new structure is created instead of modifying the original data. Persistent data structures are often referred to as immutable, because the original data is left untouched.

A naive persistent version of any data structure is trivial to make: Just clone the entire structure every time a change is made, and then commit the relevant changes to the new version. That way the old version is kept. However, for anything larger than a trivially small structure, this is going to be problematic.

There is a better way to create persistent data structures. Instead of copying the entire structure each time a modification is made, only the parts that have been modified need to be copied. The rest of the data is shared between old and new versions. This can be performed through the use of clone-on-write pointers.

Rust provides three clone-on-write pointers in the standard library: Cow, Rc and Arc. Cow only sometimes owns it’s own data, so is not suitable for these kinds of data structure. In this article, Rc will be used, but Arc is equally valid.

Note that in Rust, instead of creating a new version every time a structure is modified, the persistent structures will have both a cheap clone function, and will share memory with older versions of themselves, but will otherwise share an API with their non-persistent counterparts.

Example 1: A singly linked list.

Let’s get stuck into into it. Consider a simple implementation of a non persistent singly linked list using Box. Mind you, that I don’t recommend using this kind of data structure for everyday use for well known reasons.

#[derive(Debug, Clone)]
pub enum ListBox<A> {
    Nil,
    Cons(A, Box<ListBox<A>>),
}
impl<A> ListBox<A> {
    pub fn new() -> Self {
        ListBox::Nil
    }
    pub fn cons(&mut self, elem: A) {
        let tail = std::mem::replace(self, ListBox::Nil);
        let mut list = ListBox::Cons(elem, Box::new(tail));
        std::mem::swap(self, &mut list)
    }
    pub fn uncons(&mut self) -> Option<A> {
        let list = std::mem::replace(self, ListBox::Nil);
        match list {
            ListBox::Nil => None,
            ListBox::Cons(elem, mut tail) => {
                std::mem::swap(self, &mut tail);
                Some(elem)
            }
        }
    }
}

Seems simple enough. new creates an empty list and cons adds an element to a pre-existing list. To read from the list, uncons removes the first element. Essentially, this list acts like a stack where cons is push and uncons is pop.

To test this list, a dummy structure is used which implements Clone. This can be used to track how many clones are performed at runtime:

#[derive(Debug)]
struct CloneTracker(u32);
impl Clone for CloneTracker {
    fn clone(&self) -> Self {
        println!("Cloning {}...", self.0);
        CloneTracker
    }
}

The list works as expected:

let mut list_box = ListBox::new();
for i in 0..10 {
    list_box.cons(CloneTracker(i));
}

// prints "Cloning x..." ten times
let _clone = list_box.clone();

list_box.cons(CloneTracker(20));
assert_eq!(list_box.uncons(), Some(CloneTracker(20)));

for i in (0..10).rev() {
    assert_eq!(list_box.uncons(), Some(CloneTracker(i)));
}

assert_eq!(list_box.uncons(), None);

The list is cloned to keep an older version of it for later. Every element in the list is cloned which is confirmed by the output produced by the CloneTracker.

To make the list persistent, two modifications are required: Firstly, change any Box to an Rc. Secondly, whenever a Box was mutably dereferenced in the old code, replace it with Rc::make_mut:

#[derive(Debug, Clone)]
pub enum List<A> {
    Nil,
    Cons(A, Rc<List<A>>),
}
impl<A: Clone> List<A> {
    pub fn new() -> Self {
        List::Nil
    }
    pub fn cons(&mut self, elem: A) {
        let tail = std::mem::replace(self, List::Nil);
        let mut list = List::Cons(elem, Rc::new(tail));
        std::mem::swap(self, &mut list)
    }
    pub fn uncons(&mut self) -> Option<A> {
        let list = std::mem::replace(self, List::Nil);
        match list {
            List::Nil => None,
            List::Cons(elem, mut tail) => {
                std::mem::swap(self, Rc::make_mut(&mut tail));
                Some(elem)
            }
        }
    }
}

To confirm that the list is correct, it is tested:

let mut list = List::new();
for i in 0..10 {
    list.cons(CloneTracker(i));
}

// prints "Cloning x..." once!
let _clone = list.clone();

list.cons(CloneTracker(20));
assert_eq!(list.uncons(), Some(CloneTracker(20)));

for i in (0..10).rev() {
    // prints "Cloning i..."
    assert_eq!(list.uncons(), Some(CloneTracker(i)));
}

assert_eq!(list.uncons(), None);

This works almost identically to the previous example. The difference is that the clones are performed at different times. In the first example The clones are performed at the clone call. In the second example, the clones are delayed until the list is actually modified. You can confirm that this is the case by commenting out the lines in question.

It turns out that the original list and its clone share the majority of the data. During a modification, only the data that needs to be modified is cloned, and the rest remains shared. In essence, each list pretends that it holds a clone of all of its own data, but in reality, the clone operation only occurs when the data is written.

Example 2: A tree.

Next consider a simple implementation of a tree using Box. For brevity, this will not be a balanced tree. Again, this implementation is overly simplified and I would not recommend using this structure in practice:

#[derive(Debug)]
pub enum TreeBox<A> {
    Leaf,
    Node(Box<TreeBox<A>>, A, Box<TreeBox<A>>),
}
impl<A: Ord> TreeBox<A> {
    pub fn new() -> Self {
        TreeBox::Leaf
    }
    pub fn singleton(value: A) -> Self {
        TreeBox::Node(Box::new(TreeBox::Leaf), value, Box::new(TreeBox::Leaf))
    }
    pub fn insert(&mut self, input: A) {
        match *self {
            TreeBox::Leaf => *self = TreeBox::singleton(input),
            TreeBox::Node(ref mut left, ref value, ref mut right) => {
                if &input < value {
                    left.insert(input);
                } else if &input > value {
                    right.insert(input);
                }
            }
        }
    }
    pub fn find (&self, elem: &A) -> bool {
        match *self {
            TreeBox::Leaf => false,
            TreeBox::Node(ref left, ref value, ref right) => {
                if elem < value {
                    left.find(elem)
                } else if elem > value {
                    right.find(elem)
                } else {
                    true
                }
            }
        }
    }
}

new creates an empty tree and singleton creates a tree with a single element. insert is used to build a tree by inserting elements, and find will return true if the element exists in the tree. Most of the implementation of a tree is omitted, but you get the idea.

The tree can be tested to ensure that it works correctly:

extern crate rand;
use rand::seq::SliceRandom;

let mut tree = TreeBox::new();

// even numbers only.
let mut numbers: Vec<u32> = (0..50).map(|x| x * 2).collect();
numbers.shuffle(&mut rand::thread_rng());

for num in numbers.clone() {
    tree.insert(CloneTracker(num));
}

// prints "Cloning x..." 50 times.
let _clone = tree.clone();

tree.insert(CloneTracker(47));
tree.insert(CloneTracker(15));

for num in numbers {
    assert_eq!(tree.find(&CloneTracker(num)), true);
}
assert_eq!(tree.find(&CloneTracker(47)), true);
assert_eq!(tree.find(&CloneTracker(15)), true);

Note that when the clone operation is performed, all of the nodes are cloned. This behaves exactly as you would normally expect. To store an old version of the structure, twice the memory of the original structure is required.

Again, if we make the same modifications: replacing every instance of Box with Rc and using Rc::make_mut every time we want a mutable reference, the structure requires far less cloning:

pub enum Tree<A> {
    Leaf,
    Node(Rc<Tree<A>>, A, Rc<Tree<A>>),
}
impl<A: Ord + Clone> Tree<A> {
    pub fn new() -> Self {
        Tree::Leaf
    }
    pub fn singleton(value: A) -> Self {
        Tree::Node(Rc::new(Tree::Leaf), value, Rc::new(Tree::Leaf))
    }
    pub fn insert(&mut self, input: A) {
        match *self {
            Tree::Leaf => *self = Tree::singleton(input),
            Tree::Node(ref mut left, ref value, ref mut right) => {
                if &input < value {
                    Rc::make_mut(left).insert(input);
                } else if &input > value {
                    Rc::make_mut(right).insert(input);
                }
            }
        }
    }
    pub fn find (&self, elem: &A) -> bool {
        match *self {
            Tree::Leaf => false,
            Tree::Node(ref left, ref value, ref right) => {
                if elem < value {
                    left.find(elem)
                } else if elem > value {
                    right.find(elem)
                } else {
                    true
                }
            }
        }
    }
}

If the exact same test is performed as above, the results are markedly different. The difference only increases as the tree becomes larger.

let mut tree = Tree::new();

// even numbers only.
let mut numbers: Vec<u32> = (0..50).map(|x| x * 2).collect();
numbers.shuffle(&mut rand::thread_rng());

for num in numbers.clone() {
    tree.insert(CloneTracker(num));
}

// prints "Cloning x..." only once!
let _clone = tree.clone();

// prints "Cloning x..." a few times.
tree.insert(CloneTracker(47));
// prints "Cloning x..." a few times.
tree.insert(CloneTracker(15));

for num in numbers {
    assert_eq!(tree.find(&CloneTracker(num)), true);
}
assert_eq!(tree.find(&CloneTracker(47)), true);
assert_eq!(tree.find(&CloneTracker(15)), true);

During the clone call, only the root node is cloned. During the insertion, only the nodes that have been modified are cloned. This can be confirmed by commenting out the relevant sections of code. The rest of the data is shared between the old version and the new version.

Conclusion

Data structures that involve Box can be converted easily into persistent versions by replacing Box with Rc. These structures easily keep track of their previous state without much overhead. In Rust, this means that the structures have a minimal cost when performing a clone operation, and the clone is performed incrementally as the structure is modified. The performance is greater and the memory usage is smaller for persistent structures than their traditional counterparts if clones are expected often.

This post was originally posted on Andrew’s Notepad

• Child entities which are both mutable and shared can make any, or all of their parents invalid.
• In freeform shared mutability, we cannot cache any calculation that depends on a shared mutable entity.
• If we limit the scope of sharing to code we trust not to invalidate our cache, then we can cache results as much as we like.
• Limiting the scope of sharing implies that we have some kind of container entity: an Arena.

Interfaces, encapsulation and references.

First of all a disclaimer. I’m going to use the terms such as “interface” and “encapsulation”, and I’m going to define them a little bit differently to normal. It may be a bit strange, but hopefully it will make sense.

Instead of writing some code and trying to fix it, let’s try creating some good code first try. I won’t guarantee that it’s performant, but I’ll settle for (mostly) correct. To do that, we have to somehow prove that our code works. Generally speaking, we start off logical derivation with some definitions.

We imagine an entity A which is part of your program. A holds some data, which can either be in a valid state or invalid. If A ever manages to get into an invalid state and stay there, then we say that our program is broken. Sometimes we need to break A temporarily when we are transitioning from two valid states. To ensure that A doesn’t stay broken, we draw a box around it. Code inside the box can break A and code outside the box can’t. A is always valid outside the box. The barrier between the two areas, we’ll call the interface. To make it easy to understand, I’ll call the area inside the box the danger zone and code outside the box the safe zone.

Now I won’t discuss how to construct an interface, or what it should look like. It’s going to be different for every language. Safe to say that we can run any code we like in the safe zone and it will not break A, because that’s what we defined the interface to be.

Code from the safe zone can call code in the danger zone, but it must go through the interface. When the code from the danger zone returns, A must be valid. It must be valid because that’s how we defined our interface. In between, A could be made invalid temporarily, as long as when we return to the safe zone, we have a valid A.

A function that we can call from the safe zone interface but exists inside the danger zone, we will call a public function. A public function is allowed to assume that the entity is valid when it starts, and has a responsibility to ensure that the entity is valid when it exits.

So now we have some idea of what an interface is. Let’s talk about encapsulation. Let’s say that we want to make the danger zone as small as possible so that the amount of code that can break A is minimised. I’ll define encapsulation as an attempt to make the danger zone as small as reasonably possible.

If the danger zone extends out forever and encompasses our entire program, then we always have to worry if we might break A by accident when we write code. We don’t want to worry about breaking every other part of our program when we modify any other part. That’s too much thinking. If we make the danger zone small, then we only have to worry about breaking A when we’re inside it.

We’ll also introduce one more definition. The definition of a reference. Let’s say we have two entities, a parent and a child. If you can break the parent by modifying the child or making it invalid, we say that the parent holds a reference to the child. We’ll define a reference this way whether or not an entity holds a physical pointer to the other entity. I might get slack and say that an entity “knows about” another.

In this article we will only deal with the case where there are no circular references.

Shared mutability

Consider that we have two entities in out program A and B. A knows about B, but B doesn’t know about A. We’ll draw this like so:

B doesn’t know about A, so the interface for B could just be around B. The opposite is not true. A knows about B and could be made invalid if B were invalid. To encapsulate A, we must surround the danger zone for B as well.

We can now figure out the correct size for the interface for any entity using the following rules.

• The danger zone for A contains A itself.
• If an entity A knows about another entity B, then the danger zone for A includes the danger zone for B.
• If an entity A does not know about another entity B, then B is outside the danger zone for A, because B can’t make A invalid by our definition of a reference.

We have a problem with shared mutability. Consider two objects A and B which do not know about each other, but have a shared child called C that they both know about. The danger zone for A encompasses A and C, and the danger zone for B encompasses B and C. Because A and B don’t know about each other, they don’t belong in each other’s danger zones. If it were possible that B could cause A to break, then it would imply that A knows about B: a contradiction.

Now we can see this issue. The interfaces of A and B cross over. In order to modify C we must go into both the danger zones for A and B first. Potentially we have a situation where A could modify C and make B invalid, or vice versa. This problem only becomes worse if we have more parents. We will discuss two ways to manage this issue.

Example problem.

For the purposes of this article we will look at a series of entities called tasks. Each task has a duration (in days) and a name. A task can only be completed after all of its dependencies have been completed. A task with no dependencies starts straight away. For this article, We’ll use the following example:

Solution 1: Freeform shared mutability.

In safe rust, we can share ownership using Rc. This means we don’t have to worry about how long something is going to live for; it will automatically be dropped when nothing else references it. To have shared mutability in a single threaded context, we can use RefCell. If we combine the two we can generate arbitrary graph like structures which have shared mutability. I mention this implementation for completeness, and advise that with almost certainty a RefCell inside of an Rc is an anti-pattern.

We begin by defining our data type. It has a name, duration, and a list of dependencies. We wrap this all up inside our Rc, RefCell sandwich:

use std::cell::RefCell;
use std::rc::Rc;

#[derive(Clone, Eq, PartialEq, Debug, PartialOrd, Ord)]
pub struct InnerTask {
    name: &'static str,
    duration: u32,
    dependencies: Vec<Task>,
}
#[derive(Clone, Eq, PartialEq, Debug, PartialOrd, Ord)]
struct Task(Rc<RefCell<InnerTask>>);

We have seen from the previous part that we cannot have shared mutability and this idea of “internal consistency” simultaneously. This means that we are forced to calculate, on the fly any result that depends on children. We solve the danger zone problem by asserting that the parent isn’t allowed to care what the child’s value is, only that it has a value. This is important if we want to use a RefCell based implementation:

impl Task {
    fn new (name: &'static str, duration: u32, dependencies: Vec<Task>) -> Self {
        Task(Rc::new(RefCell::new(InnerTask {
            name: name,
            duration: duration,
            dependencies: dependencies,
        })))
    }
    fn set_duration (&self, duration: u32) {
        self.0.borrow_mut().duration = duration;
    } 
    fn start_time (&self) -> u32 {
        (&self.0.borrow().dependencies)
            .into_iter()
            .map(|dependency| dependency.end_time())
            .max()
            .unwrap_or(0)
    }
    fn end_time(&self) -> u32 {
        self.start_time() + self.0.borrow().duration
    }
}

The start time of a Task is the maximum of all of its children’s end times. The end time is just the start time plus the duration of the task.

As you can see, start_time accesses every dependency in turn to get a result. In our small example this will be trivial, but in more complicated graphs, we could perform an exponentially increasing number of calls. We are not allowed to cache the results, because some other part of our program could modify one of the other tasks, and make the start and end times invalid.

This is a simple implementation. I provide no methods to modify dependencies; with a RefCell inside an Rc, we have to be careful about creating circular references. We limit ourselves to only creating a Task using older Tasks.

fn main () {
    let lay_foundation = 
        Task::new("Lay Foundation", 10, vec![]);
    let build_walls = 
        Task::new("Build Walls", 5, vec![lay_foundation.clone()]);
    let build_roof = 
        Task::new("Build Roof", 11, vec![build_walls.clone()]);
    let paint_walls = 
        Task::new("Paint Walls", 2, vec![build_walls.clone()]);
    let furnish_house = 
        Task::new("Furnish House", 3, vec![build_roof.clone(), paint_walls.clone()]);

    println!("Days require to finish house: {}", furnish_house.end_time());
}

This works, but I don’t recommend it. It is up to the programmer to guarantee that we never cache the results of any of our calculations, however tempting it may be. In addition, this is asymptotically slow. The start and end time for lay_foundation has to be calculated multiple times. In more complicated examples, it may have to be calculated an exponential number of times.

Solution 2: An Arena.

The previous example has shared mutability, but no “internal consistency”. However, if we limit the spread of mutable references to entities that we trust not to break our structure, we can cache the results as much as we like. All the nodes themselves must be “trusted” since they could have a reference to any other node, and we can only give mutable references to “trusted” entities.

In order to create a place where only trusted entities can modify our nodes, we have to create an interface. Inside the interface, we have trusted code, and outside, we have everything else. We call the entity that holds the interface P.

We don’t allow P to hand out mutable references to individual nodes to the safe zone, because outside of P isn’t trusted not to break our structure by our definition of P. P must therefore have exclusive write access to all of the individual nodes. Considering that P is an entity with an interface, that has exclusive write access to its individual nodes, we can say P is an arena.

You might note that I don’t say that P an object. P could be anything. It could be a struct, and enum, a combination of objects or any other entity that could be defined. I only state that whatever P is, it happens to have two properties: all code inside P is trusted not to break it, and it has exclusive mutable access to all of its nodes.

Like the previous article, in order to solve our shared mutability problem, we get rid of the sharing part, and say that all mutable access is through public functions for P.

Note that in order to cache the results of calculations we must have an interface and its associated entity P, and we know that P must have exclusive write access to its members. An arena is not the “best” way of solving the problem of mutability with internal consistency, it’s the only way. Einstein said to make things “as simple as possible, but no simpler”. Although an arena may not be simple, it is the simplest way to write our code whilst remaining correct.

To implement this, we can create a Graph data structure which is just a collection of nodes with unique id’s. A GraphNode contains some data, and a set of incoming and outgoing edges.

use std::collections::{HashMap, HashSet};
use std::hash::Hash;

use uuid::Uuid;

#[derive(Debug, Clone, Eq, PartialEq)]
struct GraphNode<T> {
    data: T,
    incoming: HashSet<Uuid>,
    outgoing: HashSet<Uuid>,
}
impl<T> GraphNode<T> {
    fn new (data: T) -> Self {
        GraphNode {
            data: data,
            incoming: HashSet::new(),
            outgoing: HashSet::new(),
        }
    }
}

#[derive(Debug, Clone)]
pub struct Graph<T: Eq + Hash> (
    HashMap<Uuid, GraphNode<T>>
);

This is a very simple implementation of a graph. We have some basic functionality that I will brush over for the most part revolving around adding, deleting and querying nodes. There are more efficient implementations, but they are not relevant to this article:

impl<T: Eq + Hash> Graph<T> {
    pub fn new() -> Self {
        Graph(HashMap::new())
    }
    pub fn add_edge(&mut self, start: &Uuid, end: &Uuid) {
        self.0.get_mut(start).map(|node| {
            node.outgoing.insert(*end);
        });
        self.0.get_mut(end).map(|node| {
            node.incoming.insert(*start);
        });
    }
    pub fn remove_edge(&mut self, start: &Uuid, end: &Uuid) {
        self.0.get_mut(start).map(|node| {
            node.outgoing.remove(end);
        });
        self.0.get_mut(end).map(|node| {
            node.incoming.remove(start);
        });
    }
    pub fn remove_node(&mut self, node_id: &Uuid) -> T {
        let node = self.0.remove(node_id).expect("remove_node: invalid key");
        for start in node.incoming.iter() {
            self.0.get_mut(start).map(|start_node| {
                start_node.outgoing.remove(node_id);
            });
        }
        for end in node.outgoing.iter() {
            self.0.get_mut(end).map(|end_node| {
                end_node.incoming.remove(node_id);
            });
        }
        node.data
    }
    pub fn add_node(&mut self, node: T) -> Uuid {
        let key = Uuid::new_v4();
        self.0.insert(key, GraphNode::new(node));
        key
    }
    pub fn get(&self, key: &Uuid) -> &T {
        &self.0.get(key).expect("get: invalid key.").data
    }
    pub fn get_outgoing(&self, key: &Uuid) -> &HashSet<Uuid> {
        &self.0.get(key).expect("get_outgoing: invalid key.").outgoing
    }
    pub fn get_incoming(&self, key: &Uuid) -> &HashSet<Uuid> {
        &self.0.get(key).expect("get_incoming: invalid key.").incoming
    }
}

We implement our actual Task struct which contains name and duration fields and nothing else. One of the advantages of using an arena is that we can separate the data that represents relations between nodes from the nodes themselves. This means that a node can therefore uphold its own invariants that don’t depend on it being part of a larger structure:

#[derive(Clone, Eq, PartialEq, Hash, Debug, PartialOrd, Ord)]
pub struct Task {
    name: &'static str,
    duration: u32,
}
impl Task {
    fn new (name: &'static str, duration: u32) -> Self {
        Task {
            name: name,
            duration: duration,
        }
    }
}

Like the previous article, we create a “view” of the graph, that freezes it temporarily. This allows us to cache the results of calculations as much as we like. When we modify any of the nodes, or the structure of the graph, we have to discard the cache because it’s no longer valid. The view struct contains a cache of start times and end times:

struct GraphView<'a> {
    graph: &'a Graph<Task>,
    start_times: HashMap<Uuid, u32>,
    end_times: HashMap<Uuid, u32>,
}

If we wanted to cache the results we can use a little bit of boilerplate and write something like the following:

impl<'a> GraphView<'a> {
    fn new (graph: &'a Graph<Task>) -> Self {
        GraphView {
            graph: graph,
            start_times: HashMap::new(),
            end_times: HashMap::new(),
        }
    }
    fn end_time(&mut self, key: &Uuid) -> u32 {
        // Boilerplate
        if let Some(result) = self.end_times.get(key) {
            return result.clone();
        }
        
        // Actual query
        let result = self.graph.get(key).duration + self.start_time(key);

        // Boilerplate
        self.end_times.insert(key.clone(), result);
        result
    }
    fn start_time(&mut self, key: &Uuid) -> u32 {
        // Boilerplate
        if let Some(result) = self.start_times.get(key) {
            return result.clone();
        }

        // Actual query.
        let result = self.graph.get_incoming(key)
            .into_iter()
            .map(|key_out| self.end_time(key_out))
            .max()
            .unwrap_or(0);

        // Boilerplate
        self.start_times.insert(key.clone(), result);
        result
    }
}

The boilerplate simply checks if we have already calculated the value. If we have, we just return the value, otherwise, we calculate the value, and insert the result into the HashMap. This is actually just depth first search in disguise.

This allows us to write any kind of graph traversing query in a declarative manner. We can write what we want the result to be based on other nodes in the graph and not have to worry about the actual execution of the query.

Mind you that this implementation is asymptotically efficient for both read heavy and write heavy use cases. In read heavy implementations, we cache the results of intermediary calculations, and lazily perform the minimal number of steps required to produce a result. In write heavy loads, we discard the cache entirely, and only have to update the essential data in the individual nodes themselves, and nothing else.

We can use the graph like the following:

fn main() {
    let mut graph = Graph::new();

    let lay_foundation = graph.add_node(Task::new("Lay foundation", 1));
    let build_walls = graph.add_node(Task::new("Build walls", 2));
    graph.add_edge(&lay_foundation, &build_walls);

    let build_roof = graph.add_node(Task::new("Build roof", 4));
    graph.add_edge(&build_walls, &build_roof);

    let paint_walls = graph.add_node(Task::new("Paint walls", 8));
    graph.add_edge(&build_walls, &paint_walls);

    let furnish_house = graph.add_node(Task::new("Furnish house", 16));
    graph.add_edge(&paint_walls, &furnish_house);

    let mut view = GraphView::new(&graph);
    println!("Days require to finish house: {:?}", view.end_time(&furnish_house));
}

This solution is verbose, but has strong guarantees. We can extend this structure in multiple ways. If we need nodes of different types, we can have one collection of nodes for each type inside our Graph structure. If we need different kinds of connections between nodes, we can modify our GraphNode structure to have more fields than just incoming and outgoing. We can even detect cycles if we modify the view code slightly:

struct GraphView2<'a> {
    graph: &'a Graph<Task>,
    // Store an Option rather than the value itself.
    // None means a circular reference
    start_times: HashMap<Uuid, Option<u32>>,
    end_times: HashMap<Uuid, Option<u32>>,
}
impl<'a> GraphView2<'a> {
    fn new (graph: &'a Graph<Task>) -> Self {
        GraphView2 {
            graph: graph,
            start_times: HashMap::new(),
            end_times: HashMap::new(),
        }
    }
    fn end_time(&mut self, key: &Uuid) -> Option<u32> {
        if let Some(result) = self.end_times.get(key) {
            return result.clone();
        }
        // We assume that a node is part of a circular reference
        // until proved otherwise.
        self.end_times.insert(key.clone(), None);
        
        let result = self.start_time(key)
            .map(|time| time + self.graph.get(key).duration);

        self.end_times.insert(key.clone(), result);
        result
    }
    fn start_time(&mut self, key: &Uuid) -> Option<u32> {
        if let Some(result) = self.start_times.get(key) {
            return result.clone();
        }
        self.start_times.insert(key.clone(), None);

        let result = self.graph.get_incoming(key)
            .into_iter()
            .map(|key_out| self.end_time(key_out))
            .fold(Some(0), |max_time, end_time| Some(max_time?.max(end_time?)));

        self.start_times.insert(key.clone(), result);
        result
    }
}

In this example start_time and end_time return None if the nodes are part of a circular reference, so we can detect them and handle them accordingly.

Conclusion

I present two ways to construct code that has shared mutability in some kind of arbitrary acyclic graph structure in Rust. On one hand we have a freeform structure which involves Rc and RefCell. The disadvantage of this kind of structure is that we cannot cache results that depend on other entities. In addition, we have to be careful not to introduce reference cycles. It appears simple but has some complicated caveats.

On the other hand, we can create some kind of arena. I provide an example implementation: a graph. I can separate the data that represents relations between nodes from the data of the nodes themselves. The graph can be temporarily frozen to mutation so we can perform queries. It does not use RefCell, so it has stronger static guarantees regarding borrowing. It is asymptotically efficient for a variety of use cases. We can write declarative queries and have them efficiently executed. It can also be extended in a variety of manners. It is my opinion that this is the simplest “correct” implementation.

In the next article, I will discuss structures which contain reference cycles.

This post was originally posted on Andrew’s Notepad

• For the majority of code, shared mutability is usually not required.
• We cannot have sharing, mutability and “internal consistency”. A program that tries to have all three is provably incorrect.
• If we want sharing, and mutability but do not need “internal consistency”, we can use a file, a database handle, a mutex, or any other similar structure.
• If we need mutability, and “internal consistency” but do not need sharing, we can have all modifications go through a common ancestor.
• If we need sharing, and “internal consistency” but not mutability, we can freeze our data, or have persistent data structures.

The problem with shared mutability.

Safe Rust provides us with some guarantees: namely safe memory access, no undefined behavior, and no data races. In addition to this, safe Rust makes it difficult (but not impossible) to have memory leaks, mutate structures through immutable references, or create memory cycles.

I’ll start by saying that these things are made difficult to do because they are difficult to reason about. Safe rust introduces some resistance so that programmers are more likely to design their programs in such a way that they are more likely to be correct.

If we ignore IO or interior mutability for a moment, safe Rust has property that whenever you hold an immutable reference to an object, the holder of the reference doesn’t know (or doesn’t care) if other structures also hold a reference to it. If you wanted to, you could clone the data and it would make no difference to the program.

To illustrate this point, let’s draw a diagram where circles are objects, and an arrow means that an object “knows” about the other. We have objects A and B which are regular objects that don’t know about each other. We also have object C for “Child”. Both A and B know about C, but not necessarily the other way around. We won’t talk about the case where A and B know about each other, because then we would have a circular reference, which is beyond the scope of this article.

As long as you don’t try to change the child C, it doesn’t matter if you had your own copy or not. The program behaves exactly the same way if every object had their own copy of C, or if they all pointed to the same one.

However, this makes it difficult when you want to write a program with shared mutability. Specifically when you have an object with multiple owners, when one owner changes the data, we want the other owner to be able to see that the data has changed. We now require that the owners have a reference to the same object.

This presents us with a problem. If we need to ask B about something that depends on C, then we have to recalculate it every single time because it could have been changed by A. As far as B is concerned, C could be anything because B doesn’t know about A.

If this is your use case, and C is a database connection, a webpage, a file, a mutex, or some other kind of IO. then you don’t have to continue reading. You will have to refresh the webpage, or perform a new database request, or obtain the mutex lock if you want your data to be up to date (which I will refer to now as “internally consistent” or just “consistent”). If you don’t trust the data to be unmodified between requests you have no choice but to recalculate every time:

struct A<'c> {
    value: u32,
    // we consider the child volatile, so we have to check it every time.
    child: &'c std::cell::RefCell<C>
}
impl<'c> A<'c> {
    // we need to calculate this every time we query it.
    fn total (&self) -> u32 {
        self.value + self.child.borrow().value
    }
}

struct B<'c> {
    value: u32,
    child: &'c std::cell::RefCell<C>
}
impl<'c> B<'c> {
    // we need to calculate this every time we query it.
    fn difference (&self) -> u32 {
        self.value - self.child.borrow().value
    }
}

struct C {
    pub value: u32,
}

fn main () {
    let c = std::cell::RefCell::from(C {value: 10});
    let a = A {value: 15, child: &c};
    let b = B {value: 30, child: &c};

    println!("A's total: {}", a.total());               // 25
    println!("B's difference: {}", b.difference());     // 20

    c.borrow_mut().value = 5;

    println!("A's total: {}", a.total());               // 20
    println!("B's difference: {}", b.difference());     // 25
}

The “Internal Consistency Problem”

If this is not your use case, and you want your data to remain consistent without having to recalculate for C every single time, then we need to make some changes.

In this case, A and B are going to need to be notified when C changes, so they know that their old data is no longer valid. In order to do this, there must be some object, let’s say P for “Parent” which knows about A, B, and the common child C.

Any time we need to change the child C, we go though the parent P. That way P can tell both A and B that C has changed, and our data is internally consistent. We’re not allowed to modify C through A or B anymore, because A and B don’t know about each other.

We now realize that A and B don’t need to know about C. Every time C changes, P tells A and B all the necessary information about the change. A and B are then updated accordingly. In fact, A and B can’t know about C. If they knew about C, then we have the same problem that we started with, except now we have shared mutability between three objects not just two.

Now P is now the only object which has mutable access to C. We solved the problems of shared mutability by not having sharing. We had to make the changes above if we wanted our data to remain internally consistent. If we take this to the extreme, we can say that A, B and P are the same object. Below is an implementation:

struct P {
    a: u32,
    b: u32,
    c: u32,
    a_plus_c: u32,
    b_minus_c: u32,
}
impl P {
    fn new (a: u32, b: u32, c: u32) -> Self {
        P {
            a: a, b: b, c: c,
            a_plus_c: a + c,
            b_minus_c: b - c,
        }
    }
    fn set_c (&mut self, c: u32) {
        self.c = c;
        self.a_plus_c = self.a + self.c;
        self.b_minus_c = self.b - self.c;
    }
    // we don't need to recalculate each time.
    fn total(&self) -> u32 {
        self.a_plus_c
    }
    fn difference(&self) -> u32 {
        self.b_minus_c
    }
}

fn main () {
    let mut p = P::new(15, 30, 10);

    println!("A's total: {}", p.total());               // 25
    println!("B's difference: {}", p.difference());     // 20

    p.set_c(5);

    println!("A's total: {}", p.total());               // 20
    println!("B's difference: {}", p.difference());     // 25
}