LAMBDAPHONE » recursion

The Gift of Inconsistency

jberryman — Thu, 22 Jul 2010 23:35:23 +0000

Computer science is a fascinating and maddening thing. Even the most seemingly-esoteric topics turn out to be fundamental. Go out to the fringes of CS and you find yourself smack in the middle of Philosophy. You try to understand a single point and you suddenly find yourself embracing everything.

So this post comes from that infinitely-recursive rabbit hole of wikipedia topics I’ve been falling down for the last few weeks, and you can probably expect a few more like this one; I’ll try to keep focused.

We begin (as do All Good Things) with the Untyped Lambda Calculus:

The System

The lambda calculus, invented by Alonzo Church, is a formal system for modeling functional computation and logic. It has an extraordinarily simple set of semantics, has no notion of data as a separate idea, and yet (as was proven later) is Turing complete. It serves as the skeleton and sinew of the Lisp and Scheme programming languages.

At its invention it seems that the system sat right at the crossroads of formal logic, mathematics and computational theory and could embrace all three fields. But it quickly became apparent that there were apparent flaws with the Lambda Calculus as a universal system.

The Paradox

In 1935 it was shown that the lambda calculus was logically inconsistent by the Kleene–Rosser paradox. The “paradox” is an apparently self-contradictory lambda expression, and goes like this:

We have a lambda calculus expression we will call 'k', which takes an argument, applies it to to itself, and negates (the ¬ symbol) the result (for some undefined, logical definition of “negate”).

k = λx.( ¬ (x x))

Okay, no problem. But look what happens when we apply the function 'k' to itself:

k k = λx.( ¬ (x x)) k
k k = ¬ (k k)

After we reduce the expression, we seem to be saying that (k k) is equal to the opposite of itself, a self-contradictory statement.

To address this apparent shortcoming of his system, Church went on to define a typed lambda calculus. This new system differentiated between function types and simple terms in order to disallow these kinds of problematic expressions.

But as with many paradoxes, this one was really an issue of perspective…

The Foundation

Church’s new “Simply Typed Lambda Calculus” had an interesting property that the untyped lambda calculus didn’t have: the normalization property.

This more restricted version of the Lambda Calculus was “strongly normalizing”, meaning that any expression could be reduced to some normal form (i.e. a form that cannot be reduced further) through some sequence of rewrite/reduction steps. In programming terms, this means that every expression in the typed lambda calculus is guaranteed to terminate.

That sounds pretty damn useful until you realize that this interesting property also makes Church’s new typed version of his system not Turing complete!

To prove this is so, consider this: imagine you encoded a program in the typed lambda calculus; it searched for (and halted) when it found an integer that was greater than two, and was not the sum of two primes.

Merely being able to encode such a function would imply the function halted, thus disproving Goldbach’s conjecture, one of the great open problems of mathematics. All without even needing to evaluate said expression! The function referred to is of course not expressible in the typed lambda calculus.

It turns out that an apparent logical contradiction was actual the essential secret to computation.

Embracing Inconsistency

When you think of the LC as a model of computation, rather than a framework for logical assertions, the Kleene–Rosser paradox becomes what we might call the “Kleene–Rosser useless function”.

Let’s go ahead and express a nearly identical paradox in haskell:

k :: (Num a) => a
k = negate k

Haskell has no notion of mutability, so what we are saying here is “k is the negation of itself”. No logical fallacies or great existential questions here, just an infinite loop (and not a very useful one)!

Let’s express another similar “paradox”, one we can actually use:

babble :: [ String ]
babble = “blah” : babble

How can a list be equal to itself with an extra element added? Doesn’t that imply a paradox? Well I know the runtime isn’t swayed by that argument, producing an infinite stream of “blah”s. We can even use this stream because of haskell’s lazy evaluation strategy.

But note: it isn’t laziness that lets us get away with defining paradoxes; it just allows us to make use of some more blatant paradoxical expressions without them blowing up in our faces as soon as we call them.

Conclusion and Further Thoughts

Interestingly, there are quite a number of non-turing complete programming languages (or language subsets) that have been created both for theoretical purposes and for practical use. Here are a few. They seem to owe their existence and usefulness to an environment of turing complete systems.

So does this mean that formal logic is Turing Incomplete? Let me know your thoughts, and please let me know if I harmed any knowledge in the making of this post.

Code Jam 2010: Incrementing a Binary Counter

jberryman — Sun, 09 May 2010 21:36:44 +0000

The Problem

Problem A in the qualifying round of this year’s Google CodeJam contest was really clever. The problem used a classic made-for-TV gadget from the 80s: The Clapper™ (“snapper” in google’s version) and went like this:

The Snapper is a clever little device that, on one side, plugs its input plug into an output socket, and, on the other side, exposes an output socket for plugging in a light or other device.

When a Snapper is in the ON state and is receiving power from its input plug, then the device connected to its output socket is receiving power as well. When you snap your fingers — making a clicking sound — any Snapper receiving power at the time of the snap toggles between the ON and OFF states.

In hopes of destroying the universe by means of a singularity, I have purchased N Snapper devices and chained them together by plugging the first one into a power socket, the second one into the first one, and so on. The light is plugged into the Nth Snapper.

Initially, all the Snappers are in the OFF state, so only the first one is receiving power from the socket, and the light is off. I snap my fingers once, which toggles the first Snapper into the ON state and gives power to the second one. I snap my fingers again, which toggles both Snappers and then promptly cuts power off from the second one, leaving it in the ON state, but with no power. I snap my fingers the third time, which toggles the first Snapper again and gives power to the second one. Now both Snappers are in the ON state, and if my light is plugged into the second Snapper it will be on.

I keep doing this for hours. Will the light be on or off after I have snapped my fingers K times? The light is on if and only if it’s receiving power from the Snapper it’s plugged into.

It’s a great problem because we have a situation that is easily modeled with a computer program, and yet a naive solution will be far too inefficient to process the inputs that google provides (e.g. 10^8 claps).

Here is a direct-to-code translation of the functioning of a “chain of snappers” implemented in (very verbose) Haskell:

> module Main
>     where
>
> type ClapperChain = [Bool]
>
>
> clapOnOff :: ClapperChain -> ClapperChain
> clapOnOff []     = []
> clapOnOff (c:cs) =
>     if isPowered cs
>        then toggleState c : clapOnOff cs
>        else c : clapOnOff cs
>
> isPowered :: ClapperChain -> Bool
> isPowered []     = True -- at the outlet
> isPowered (c:cs) = inOnState c && isPowered cs
>
> inOnState = id
> toggleState = not

We can then look at a few iterations (snaps) and see how the chain changes. True is a “snapper” in the ON state:

*Main> take 4 $ iterate clapOnOff [False,False,False]
[[False,False,False],[False,False,True],[False,True,False],[False,True,True]]

A Solution

After working out a few iterations on paper I realized that the snappers follow the pattern of an ascending binary number count: 000, 001, 010, 011...

The lamp will be on when all of the snappers are 1s (i.e. in the ON state) which will occur every (2^number_of_snappers) snaps. Thus my solution was simply the following:

caseAlgorithm (n,k) =
    if (((k+1) `mod` 2^n) == 0)
       then ON
       else OFF

More Code and Thoughts

Here is a more terse and better version of the Snapper simulation code I posted above, now simply called increment for incrementing a binary number:

> increment :: ClapperChain -> ClapperChain
> increment []    = []  -- [] is the outlet
> increment chain@(c:cs)
>     | and cs    = map not chain
>     | otherwise = c : increment cs

We can convert our lists into pretty strings with…

> showBinary :: ClapperChain -> String
> showBinary = map (\b-> if b then '1' else '0')

And play with it like so:

*Main> map showBinary $ take 9 $ iterate increment  [False,False,False]
["000","001","010","011","100","101","110","111","000"]

I liked realizing that a binary count could be represented recursively and mechanically. It led me to look into how binary counter circuits are implemented in electronics (it turns out The Clapper™ is not generally involved), and got me thinking about cellular automata.

A binary counter could be implemented as Cellular Automata in which each bit is an Automaton with two states: an ON/OFF state and a POWERED/UNPOWERED state.

…although I think POWERED or UNPOWERED would have to be evaluated lazily, which might not fit the model.

Oh well, good luck to everyone doing the competition this year!

Enumerating all n-integer Sets Which Sum to an Integer x

jberryman — Thu, 30 Jul 2009 01:27:34 +0000

I came up with what I think is an elegant use of monadic code, while working on a program to compute solutions to the Postage Stamp Problem. The problem asks:

Consider that an envelope is able to hold a limited number of postage stamps. Then, consider the set of the values of the stamps — positive integers only. Then, what is the smallest total postage which *cannot* be put onto the envelope?

The algorithm below takes a number of stamps that can fit and a total postage to apply and returns a list of all the sets of different stamp values that can be combined to form the target postage sum:

> sumLists :: Int -> Int -> [[Int]]
> sumLists = f 0 where
>     f _ 1 n = return [n]
>     f i c n = do a  <- [i.. div n c]
>                  as <- f (i+a) (c-1) (n-a)
>                  return (a : as)

Thus the potential stamp combinations for a letter which can hold three stamps and costing 5 cents to mail would be:

*Main> sumLists 3 5
[[0,0,5],[0,1,4],[0,2,3],[1,1,3],[1,2,2]]

As you can see, the algorithm also produces ordered lists naturally!

But perhaps all we care about are unique stamp values that. If we want to adjust our goal such that the algorithm returns a list of all sets of unique stamp values sufficient to create the target postage, we need only adjust the return line:

> return (a : dropWhile (==a) as)

Similarly we could drop the zeros (which mean “no stamp” in our case) from the beginnings of each set if we desired. The above code have applications to many similar problems including Subset Sum, and Knapsack as well as other partitioning problems.

Fun with Lazy Arrays: the LZ77 Algorithm

jberryman — Thu, 18 Jun 2009 17:30:29 +0000

This is my third post investigating compression techniques related to the DEFLATE algorithm: the first on run-length encoding, and the second on simple Huffman Coding. This post models the LZ77 algorithm, the second of the two compression strategies used by DEFLATE, and in the process explores some interesting properties of Haskell’s basic Arrays.

IMPLEMENTATION:

> module LZ77
>     where

we will use GHC's "basic non-strict arrays" for this
experiment:

> import Data.Array

and use Ints to store the length of the entire decoded
message (needed to create our array):

> type Length = Int
> type Offset = Int

in place of the length in the standard length-offset pair
I've decided to use an Int representing the index of the
last element in the span relative to the element at
offset. Thus the pair encoding a span of only one element,
two elements back would be (0,2), rather than the more
traditional (1,2).

This simply makes more sense to me, especially since I want
to be able to encoded reversed sequences, as you will see.

> type Index = Int

and a few simple dataypes for our uncompressed and
compressed data respectively:

> type Decoded a = Array Index a
>
> data Encoded a =
>     Enc Length [ Either a (Index,Offset) ]

The decompress function works by traversing the encoded message,
keeping track of our array index position (since offsets are
relative to the current position), and building an Array lazily
from a list which we generate, in part by referencing elements
from the partially generated array itself.

So when we see a Right value we look up in the Array the elements
referenced by the length-offset, concat-ing that list with the
result of processing the rest of the encoded message.

If we hit [] in 'dec' we call an error because the stored value
for the length of the uncompressed message in the Encoded type
was longer than what the 'decompress' function could produce.

It's diffficult to describe, but I hope the code is clear:

> decompress :: Encoded a -> Decoded a
> decompress (Enc el es) = decoded
>     where decoded = listArray (0,el - 1) $ dec 0 es
>
>           dec  _     [] =
>               error "message is shorter than it should be"
>
>           dec n (Left x : xs) =
>               x : dec (n+1) xs
>
>           dec n (Right (iRel,off) : xs) =
>               let i1 = n  - off
>                   i2 = (if iN > i1 then succ else pred) i1
>                   iN = i1 + iRel
>
>                in [ decoded!i | i <- [i1, i2 .. iN] ] ++
>                   dec (n + 1 + abs iRel) xs

Some interesting things about the code above:

    1) we create an array from a list, which we build,
       in part, by looking up elements from the array
       we are in the process of building.

    2) we can compress a sequence of symbols which were
       seen previously but in reverse order, simply by
       storing a negative relative index in the
       (relative_index,offset) tuple. So, the string...

            "her racecar returns to race"

       might compress to:

            {her race(-5,2)turns to(4,19)}

       I'm not sure if this is useful in real compression,
       especially when it comes down to the binary encoding.

    3) even more interesting, we can use this same decoder
       function to decompress data that matches a sequence
       in parts of the array we haven't built yet! We simply
       use a negative offset in our tuple.

EXAMPLES AND CONCLUSION:

point (3) may or may not be something like the LZ78 algorithm,
which apparently works by encoding future data, but it is
defintely a cool thing to be able to do with arrays:

> coolArray = listArray (0,4)
>        [0,  coolArray!4 - 3,  2,  coolArray!1 + 2,  4]

Here's an example with great compression where the relative
index exceeds the offset:

> exceedsOffset = elems $ decompress $
>       Enc 25 [Left 'B', Left 'l', Left 'a', Left 'h', Left ' ',
>               Left 'b', Right (17,5), Left '!']

...and a code example for (2) above:

> reverseReference = elems $ decompress $
>       Enc 27 [Left 'h',Left 'e',Left 'r',Left ' ',Left 'r',
>               Left 'a',Left 'c',Left 'e',Right(-5,2),Left 't',
>               Left 'u',Left 'r',Left 'n',Left 's',Left ' ',
>               Left 't',Left 'o',Right(4,19)]

...and finally an example combining (2) and (3):

> reverseLookAhead = elems $ decompress $
>       Enc 5 [Left 1, Right (-1,-3), Left 3, Left 4]

I was surprised to discover these properties in Haskell's lazy
Arrays. hope they came as a surprise to a few others.

Run-length Encoding

jberryman — Tue, 26 May 2009 23:27:03 +0000

Just a quick implementation of a RLE algorithm for lists in haskell. We compress a list by converting “runs” of consecutive elements into a tuple of the form: (run_length, element).

import Data.List (group)
import Control.Arrow

runLengthEnc :: Eq a => [a] -> [(Int,a)]
runLengthEnc = map (length &&& head) . group

decode :: [(Int, a)] -> [a]
decode = concatMap (uncurry replicate)

If the &&& combinator looks foreign to you, check out David R. Maciver’s very enlightening blog about Arrow functions.

I’m always curious to see how naive-looking functions like the above compare in performance to a from-scratch implementation with explicit recursion, so I came up with the following:

runLengthEnc' :: Eq a => [a] -> [(Int,a)]
runLengthEnc' [] = []
runLengthEnc' (a:as) = run 1 a as
    where run n x [] = [(n,x)]
          run n x (x2:xss) | x == x2   = run (n+1) x xss
                           | otherwise = (n,x) : run 1 x2 xss

I tested both functions on a list of 100,000 random 1s and 0s and found the explicit version to be only marginally better performing, completing the list in 49 ticks & 130Mb, compared to 54 ticks & 139 Mb for the one-liner!