So this post comes from that infinitely-recursive rabbit hole of wikipedia topics I’ve been falling down for the last few weeks, and you can probably expect a few more like this one; I’ll try to keep focused.

We begin (as do All Good Things) with the Untyped Lambda Calculus:

The System

The lambda calculus, invented by Alonzo Church, is a formal system for modeling functional computation and logic. It has an extraordinarily simple set of semantics, has no notion of data as a separate idea, and yet (as was proven later) is Turing complete. It serves as the skeleton and sinew of the Lisp and Scheme programming languages.

At its invention it seems that the system sat right at the crossroads of formal logic, mathematics and computational theory and could embrace all three fields. But it quickly became apparent that there were apparent flaws with the Lambda Calculus as a universal system.

The Paradox

In 1935 it was shown that the lambda calculus was logically inconsistent by the Kleene–Rosser paradox. The “paradox” is an apparently self-contradictory lambda expression, and goes like this:

We have a lambda calculus expression we will call 'k', which takes an argument, applies it to to itself, and negates (the ¬ symbol) the result (for some undefined, logical definition of “negate”).

k = λx.( ¬ (x x))

Okay, no problem. But look what happens when we apply the function 'k' to itself:

k k = λx.( ¬ (x x)) k
k k = ¬ (k k)

After we reduce the expression, we seem to be saying that (k k) is equal to the opposite of itself, a self-contradictory statement.

To address this apparent shortcoming of his system, Church went on to define a typed lambda calculus. This new system differentiated between function types and simple terms in order to disallow these kinds of problematic expressions.

But as with many paradoxes, this one was really an issue of perspective…

The Foundation

Church’s new “Simply Typed Lambda Calculus” had an interesting property that the untyped lambda calculus didn’t have: the normalization property.

This more restricted version of the Lambda Calculus was “strongly normalizing”, meaning that any expression could be reduced to some normal form (i.e. a form that cannot be reduced further) through some sequence of rewrite/reduction steps. In programming terms, this means that every expression in the typed lambda calculus is guaranteed to terminate.

That sounds pretty damn useful until you realize that this interesting property also makes Church’s new typed version of his system not Turing complete!

To prove this is so, consider this: imagine you encoded a program in the typed lambda calculus; it searched for (and halted) when it found an integer that was greater than two, and was not the sum of two primes.

Merely being able to encode such a function would imply the function halted, thus disproving Goldbach’s conjecture, one of the great open problems of mathematics. All without even needing to evaluate said expression! The function referred to is of course not expressible in the typed lambda calculus.

It turns out that an apparent logical contradiction was actual the essential secret to computation.

Embracing Inconsistency

When you think of the LC as a model of computation, rather than a framework for logical assertions, the Kleene–Rosser paradox becomes what we might call the “Kleene–Rosser useless function”.

Let’s go ahead and express a nearly identical paradox in haskell:

k :: (Num a) => a
k = negate k

Haskell has no notion of mutability, so what we are saying here is “k is the negation of itself”. No logical fallacies or great existential questions here, just an infinite loop (and not a very useful one)!

Let’s express another similar “paradox”, one we can actually use:

babble :: [ String ]
babble = “blah” : babble

How can a list be equal to itself with an extra element added? Doesn’t that imply a paradox? Well I know the runtime isn’t swayed by that argument, producing an infinite stream of “blah”s. We can even use this stream because of haskell’s lazy evaluation strategy.

But note: it isn’t laziness that lets us get away with defining paradoxes; it just allows us to make use of some more blatant paradoxical expressions without them blowing up in our faces as soon as we call them.

Conclusion and Further Thoughts

Interestingly, there are quite a number of non-turing complete programming languages (or language subsets) that have been created both for theoretical purposes and for practical use. Here are a few. They seem to owe their existence and usefulness to an environment of turing complete systems.

So does this mean that formal logic is Turing Incomplete? Let me know your thoughts, and please let me know if I harmed any knowledge in the making of this post.

Note: this is part of a series of posts is related to the “17×17 Challenge” posted by Bill Gasarch. The goal is to color cells of a 17 by 17 grid, using only four colors, such that no rectangle is formed from four cells of the same color.

This is something I’ve been wanting to play with for months now, but haven’t made the time to implement: I wondered if simulated annealing techniques could be effective in finding a complete grid covering.

We would simply start with four copies of the 74-color grid and swap their rows and columns around, within each color set, trying to cover all of the cells by the union of the four subsets. So all of the sets of cells of a single color are fundamentally simply different permutations of the same 74-cell rectangle-free Graph.

Here is an illustration of the concept. Our algorithm starts with all four colored sets of cell overlapping (the red cells are on top, covering the other three colors):

…then we swap two rows or two columns from a colored subset of cells, until the four colored subsets spread out covering as much of the grid as possible. Here we cover all but 21 cells:

A Brief Intro to Simulated Annealing

Simulated Annealing is a bit of an intimidating term (has its origins in metallurgy of all things), but the idea is simple: you start off with an initial solution (in my case, four sets of cells, all of which overlap) and then you mutate it randomly by choosing a random “neighbor” solution (in my case, a grid in which two rows or two columns of the same color are swapped).

You score these solutions so that you can compare a current solution with a proposed mutated solution, and here’s the key:

You begin by accepting nearly every mutation, whether it improves your solution or not. But as the procedure progresses you become more picky about “how much worse” a new solution can be from the previous one.

So in the classic Traveling Salesman Problem a neighbor solution would be a new route that goes from city C to city E, where in the last solution we went from city E to city C.

Threshold Acceptance

There are various ways of accomplishing the task of “ratcheting down” the computation over time. The traditional method (the method most matching the metallurgical metaphor, if you will) is to have a function that computes the probability of a proposed change being accepted. This probability function changes over time, such that as we progress it becomes more and more unlikely that a solution worse than the previous will be accepted.

Another method, and the one I chose to implement, seems to be referred to as Threshold Accepting; instead of having a probability function, you accept all mutations that are below a certain threshold of change from the previous solution. The threshold becomes more strict over time.

So in our case we might start off with a threshold of five, meaning that we will accept all changes that give us a worse solution of no more than 5 new un-colored cells. So for example: if swapping blue rows 5 and 12 create a new solution with 6 fewer colored (covered) cells from the previous solution (i.e. a worse solution by six), then it won’t be accepted by our meta-algorithm at this stage.

Notes on the implementation

The approach I tried to describe above was really attractive to me because it short-circuits the whole complex “rectangle-free” constraint entirely. We simply start with a known rectangle-free subset and see if we can make four copies fit together.

Also, because we swap two rows or columns at a time, we need only look at the local change of score produced by the swap and then add it to the global score to obtain the new global score.

The astronomical number (17!^2) of possible permutations of a single rectangle-free subset give me hope that this approach could work, but that is an open question as far as I know.

I wrote this first draft of the script in haskell (as usual), and was happy with how fast it is, thanks in large part to the vector library, which was a pleasure to work with. The code uses 4 pairs of Vectors (arrays of Ints), 8 in total, each of which represent a row or column ordering of one of the four colors.

We store the original 74-cell rectangle-free subset as a 2D Array (Int,Int) Bool. The pairs of Vectors act as references to row or column slices of this 2D array.

When we want to score a swap, we have to “render” all four colors of the pair of rows/columns in question using our 2D reference array. Then a swap consists of swapping two Ints in a single Vector. This is much more friendly on more poor laptop than if we were to try storing all four color subsets as 2D arrays and swapping them around!

You can download the code here, but mind it isn’t incredibly pretty.

Initial findings

The best I’ve done with this initial version is a grid with 19 empty cells:

It is obvious though that the heuristic needs a lot of tuning, and that in my case the “Threshold Accepting” approach isn’t working well. That is because it is far too granular: we get from complete disorder to a minimum in just a handful of threshold rounds.

Here is a quick graph overlaying two runs, one of which starts with a threshold (each threshold round bracketed in black) of 5, the other (in blue) starts with a threshold of 2. The Y-axis is the number of uncolored cells in the grid at the current solution:

You can see that both basically look like random Brownian walks during the course of a threshold round, and quickly drop to a lower energy level (better solution) as soon as the threshold is tightened. Most dramatic is the jump when we go from 2 to 1.

This looks like we’re being thrust into a local minima, which is the opposite of what we want here. But whether a more finely-tuned annealing schedule can do better is anyone’s guess.

What’s next?

It should be very simple to implement a more traditional annealing procedure that would ease us into lower “energy levels” without the harsh dives we get from using only a handful of threshold levels. So that is what is next on the agenda for this problem.

If this annealing business looks promising after I master the black art of “tuning the meta-heuristic”, it would be cool to come up with a single-coloring algorithm, capable of generating rectangle free colorings of 72,73, or 74 cells. We would then feed those solutions into our annealing framework, finding the best combinations of colorings via a genetic algorithm or the like.

But that leads to the question I’ve posed earlier: it is sometimes trivial to determine if two colorings are “unique” (i.e. by counting colored cells in rows and columns); but when it isn’t trivial, determining whether two colorings are “equivalent” seems to be a very difficult problem.

What a hole I’ve fallen into…

Note: this is part of a series of posts is related to the “17×17 Challenge” posted by Bill Gasarch. The goal is to color cells of a 17 by 17 grid, using only four colors, such that no rectangle is formed from four cells of the same color.

An almost silly amount of time passes between my new ideas on this problem and investigating/coding those ideas, but I’m trying to be better about helping my thoughts see the light of day before I get entirely bored of them.

So in that spirit, here are a few more thoughts on the 17×17 Problem investigated.

Will Graph Layout Algorithms show Symmetry?

Force-based algorithms tend to be good at creating symmetrical layouts of graphs. The idea is to translate a Graph into a system of springs (edges) and charged particles (nodes). You then run a simulation of this 2-dimensional sytem and the graph eventually should work itself out into some equilibrium state.

At that point hopefully you get a layout which is more clear and perhaps makes clear some important relationships of the Graph.

I used the graphviz software suite to do the force-directed layouts and wrote some ugly code to coax some rectangle-free single-colorings into the graphviz file format. I hoped that some of the single- and double-symmetrical relationships that I noticed would emerge in the process.

This worked perfectly for a small 5×5 cell single-coloring:

Since what we’re really looking for are symmetrical relationships between rows and columns rather than between cells, I’ve expressed the graph above as colored cell nodes connected to nodes representing the row and the column in which the cell resides.

I realized I could tighten up the graph by getting rid of the nodes for the cells themselves and just showing Column nodes connecting to Row nodes, where an edge exists between the two when a colored cell lies on their intersection. (note: the rows and columns here don’t correspond to the ones above):

Unfortunately this wasn’t effective for the 17×17 single-coloring:

I suppose there were too many nodes and the symmetrical relationships are too tenuous to emerge. That said, I’m no graphviz expert so it’s quite possible that this could work with the right black magic.

Investigating Weights of Colored and “Toxic” Cells

Note: I’ve started calling cells which have been made un-colorable (because they would form a rectangle of that color) “toxic”; so that’s how I’ll refer to them here.

I wanted to investigate the relationships between colored cells and the cells that they make toxic (and vice versa), so I coded up a Haskell script that parsed a CSV format grid and did the following:

1. For each colored cell, show the toxic cells that it has a part in making toxic
2. For each toxic cell, list all the triplets of colored cells that each make this cell toxic

What I was primarily looking for were some insights that could lead to an heuristic to guide some kind of ladder-climbing search algorithm. For example if all the colored cells

The first point wasn’t particularly interesting to me: in the 17×17 grid of 74 colored, each colored cell helped toxify between 31 and 40 un-colorable cells, which corresponded with the number of row/column neighbors that a colored cell had.

What was more interesting was when I looked at the graph from the perspective of the toxic (blank) cells.

Each toxic cell had 3, 4 or 5 unique triplets of colored cells making it toxic. Here you can see a kind of heatmap with the toxic cells in shades of gray, corresponding to how many different 3-groups of colored cells help make this toxic:

An Obvious Property of Toxic Cells

Looking at toxic cells helped me realize an interesting property of those cells:

There is a one-to-one correspondence between a toxic cell’s row neighbors and its column neighbors in rendering the cell in question toxic.

That is for a toxic cell T a colored cell in T’s column will team up with at most a single colored cell from T’s row (and vice versa) in toxifying our cell. If this weren’t the case, it would imply a rectangle on our grid (you can work it out on paper if you like).

To go further, this means that if T contains a colored cell in its row and its column, both of which don’t contribute to making T toxic, then T is essentially not doing it’s job as well as it can. That is we might be able to add a new colored cell to form a triplet with those two colored cells, adding a new colored cell to our graph and making T a more effective “rectangle sink”.

Here’s an illustration of two different toxic cells and their respective (and partially overlapping) sets of toxifying triplets. The second tab shows how we might “improve” one of the toxic cells:

Of course adding a colored cell to form a triplet with the two cells in T’s row, but not affecting T would very likely cause other currently colored cells to become toxic in the process.

Conclusion

I’m going to try code up an algorithm that attempts to produce an optimal single-coloring by trying to make toxic cells as effective as possible, e.g. by looking at where we can add a single colored cell and bridge several ineffective pairs at one time. I’ll also look at some other more conventional search strategies.

My next and hopefully last post on this junk should be of shorter and of more general compsci interest.

Problem A in the qualifying round of this year’s Google CodeJam contest was really clever. The problem used a classic made-for-TV gadget from the 80s: The Clapper™ (“snapper” in google’s version) and went like this:

The Snapper is a clever little device that, on one side, plugs its input plug into an output socket, and, on the other side, exposes an output socket for plugging in a light or other device.

When a Snapper is in the ON state and is receiving power from its input plug, then the device connected to its output socket is receiving power as well. When you snap your fingers — making a clicking sound — any Snapper receiving power at the time of the snap toggles between the ON and OFF states.

In hopes of destroying the universe by means of a singularity, I have purchased N Snapper devices and chained them together by plugging the first one into a power socket, the second one into the first one, and so on. The light is plugged into the Nth Snapper.

Initially, all the Snappers are in the OFF state, so only the first one is receiving power from the socket, and the light is off. I snap my fingers once, which toggles the first Snapper into the ON state and gives power to the second one. I snap my fingers again, which toggles both Snappers and then promptly cuts power off from the second one, leaving it in the ON state, but with no power. I snap my fingers the third time, which toggles the first Snapper again and gives power to the second one. Now both Snappers are in the ON state, and if my light is plugged into the second Snapper it will be on.

I keep doing this for hours. Will the light be on or off after I have snapped my fingers K times? The light is on if and only if it’s receiving power from the Snapper it’s plugged into.

It’s a great problem because we have a situation that is easily modeled with a computer program, and yet a naive solution will be far too inefficient to process the inputs that google provides (e.g. 10^8 claps).

Here is a direct-to-code translation of the functioning of a “chain of snappers” implemented in (very verbose) Haskell:

We can then look at a few iterations (snaps) and see how the chain changes. True is a “snapper” in the ON state:

*Main> take 4 $ iterate clapOnOff [False,False,False]
[[False,False,False],[False,False,True],[False,True,False],[False,True,True]]

A Solution

After working out a few iterations on paper I realized that the snappers follow the pattern of an ascending binary number count: 000, 001, 010, 011...

The lamp will be on when all of the snappers are 1s (i.e. in the ON state) which will occur every (2^number_of_snappers) snaps. Thus my solution was simply the following:

caseAlgorithm (n,k) = 
    if (((k+1) `mod` 2^n) == 0)
       then ON
       else OFF

More Code and Thoughts

Here is a more terse and better version of the Snapper simulation code I posted above, now simply called increment for incrementing a binary number:

We can convert our lists into pretty strings with…

And play with it like so:

*Main> map showBinary $ take 9 $ iterate increment  [False,False,False]
["000","001","010","011","100","101","110","111","000"]

I liked realizing that a binary count could be represented recursively and mechanically. It led me to look into how binary counter circuits are implemented in electronics (it turns out The Clapper™ is not generally involved), and got me thinking about cellular automata.

A binary counter could be implemented as Cellular Automata in which each bit is an Automaton with two states: an ON/OFF state and a POWERED/UNPOWERED state.

…although I think POWERED or UNPOWERED would have to be evaluated lazily, which might not fit the model.

Oh well, good luck to everyone doing the competition this year!

We will be using Lennart Augustsson’s numbers library which can be installed from hackage with cabal-install:

$> cabal install numbers

Consider two functions: the Prelude function sum and genericLength from the List library:

 genericLength :: (Num i) => [b] -> i
 sum :: (Num a) => [a] -> a

…two simple functions that have the potential to be very expensive, depending on the length of the list and the values of the elements.

Say for instance that all we want is to use:

The checkLengths function above has to count every element of every element of every sub-list in order to return a Bool value, and if any of those lists are infinite our program will never terminate:

…or so we might think!

Lazy Natural Numbers

It turns out we can solve our seemingly hopeless situation with a type signature:

And suddenly, as if by magic:

*LazyArithmetic> hopeless
True

What’s going on here? Well it turns out the problem is that Haskell’s built-in numeric types are boring, old-fashioned and (in some cases) just plain wrong. But let’s step behind the curtain and look at how the Data.Numbers library defines the Natural type internally:

 data Natural = Z | S Natural

We see that we’re using an abstract data type to represent non-negative integers, and it looks nearly identical to our list type []. The upshot of that is we gain, in our numeric type and arithmetic operations, the same type of laziness that lists afford us!

So in the above example, hopeless, we need only carry out the operations far enough to see that the result is > 2.

The numeric type in Data.List.Natural are actually known as Peano numbers. They are essentially just lists of units and, as you might imagine, are not the most efficient way to represent an integer.

But they are exactly what we want for this application. If we failed to realize that we could solve our problem with an appropriate numeric representation, we might try to rig up something like this:

…which works identically to checkLengths with Peano Numbers, but completely obfuscates the intention of the function. As usual laziness gives us expressive power rather than an efficiency boost.

Limitations, Haskell’s Numeric Type Classes, and the Numeric Prelude

The Data.Number.Natural library is convenient because it provides Num and Integral instances so we can use the the Natural type with any function that is polymorphic on those classes.

The problem is the Num instance is incomplete and error-prone: and that’s because a natural number type shouldn’t be an instance of Num! We can’t negate a Natural, which in turn makes subtraction dangerous, etc.

*LazyArithmetic> let x = 1 :: Natural; y = 2 in x - y
*** Exception: Natural: (-)

We might wish that haskell’s Numeric type classes were more expressive, allowing for more abstract numeric types. For example it might be more “haskelly” for the length function to be defined with the type:

 length :: Natural n => [a] -> n

After all we don’t use integers for boolean values as in C; we have the abstract type Bool. So why return an Int when a Natural type (or class) might be more expressive? The answer of course is that that gets really complicated really fast.

For one approach at making haskell’s numeric class hierarchy more expressive and flexible, check out the Numeric Prelude. It’s haddock documentation is very thorough and an interesting read, and you can see the sheer number of design decisions necessary for such an undertaking.

For some other approaches and links, see here.