If you aren't familiar with Applicative Functors, they are somewhere in between the familiar and simple Functor class and the Monad class; an abstraction for function application with a context. Check out here for more.

Away from the computer I got out a pad of paper and went about trying to figure out how the < *> method was defined over functions. Here are the types:

The polymorphic type:
(< *>) :: (Applicative f) => f (a -> b) -> f a -> f b

Substitute (x -> ...) for f to get the type for the functions instance:
(< *>) :: (x -> a -> b) -> (x -> a) -> (x -> b)

You can try defining the instance from the type definition; there seems to be only one sensical way. When I did it I was struck by something: the definition for the function above is the same as the S combinator in the SKI combinator calculus!

At that point I started looking for other connections to combinatory logic, and quickly noticed that the class method pure was essentially the K combinator!. Here is the actual instance definition:

instance Applicative ((->) a) where
        pure = const
        (< *>) f g x = f x (g x)

For those not familiar, the SK(I) Combinator Calculus is a formal system for modelling computation, much like the lambda calculus; one can express any computable function using only the three (actually only S and K are necessary) symbols and parentheses. I've written about and implemented the combinator calculus, and you can read more on the wikipedia page.

Exploring the Connection

It's pretty astounding to me that the only two methods required to define an Applicative instance turn out to be sufficient (the details of haskell's type system aside) to define a complete combinator base, capable of expressing any computable function.

At first I wondered whether this connection was the brainchild of some clever haskeller who defined the (->) instance, and therefore somewhat artificial. But it turns out that the definitions for pure and (< *>) fall out quit naturally from the type definitions. So naturally in fact that we can use djinn to automatically derive their definitions!:

Djinn> ? (< *>) :: (x -> a -> b) -> (x -> a) -> (x -> b)
(< *>) :: (x -> a -> b) -> (x -> a) -> x -> b
(< *>) a b c = a c (b c)
Djinn> ? pure :: a -> (x -> a)
pure :: a -> x -> a
pure a _ = a

Observe how the function definitions above could have been pulled right from an explanation of the S and K combinators.

Now let's play with this in GHCi. We'll first define our S and K combinators as the Applicative methods (giving type hints to keep the typechecker happy), then define the I combinator in terms of S and K, then we'll express a Church numeral:

Prelude> :m + Control.Applicative
Prelude Control.Applicative> let s = (< *>) :: (x -> b -> c) -> (x -> b) -> (x -> c)
Prelude Control.Applicative> let k = pure :: a -> (x -> a)
Prelude Control.Applicative> let i = s k k
Prelude Control.Applicative>
Prelude Control.Applicative> let two = s (s (k s) k) i
Prelude Control.Applicative>
Prelude Control.Applicative> two ('x':) []
"xx"

In the last last line above we sort of "render" the church numeral by applying it to a function and value that will construct a list; remember that church numerals represent integers as n-fold function composition; in the lambda calculus the numerical represenatations are more straightforward: two = \f x-> f (f x), but the idea is the same.

Conclusion

It makes a lot of sense that a general type class meant to aid programming in an "applicative" style would have some similarity with the combinator calculus (in which even data is function application), but I thought it was pretty exciting to see such a fundamental computer science concept seemingly emerge naturally out of a library designed to solve a very high-level problem.

I have no idea whether these observations are novel or old and obvious, but would love to hear from anyone who has any thoughts on this.

EDIT: After taking a closer look at Conor McBride and Ross Paterson's paper on Applicative Functor's, I see now that he labels his ((->) a) instance methods with "S" and "K", but does not explore the subject further. So I'm leaning towards obvious.

Thanks for reading :)

Here I show how we can use Haskell’s type classes and other language features to model the Combinator Calculus. The goals of this module were:

1. don’t force any one evaluation strategy
2. allow users to define new combinators without exposing the implementation
3. allow new combinators to be defined in terms of other already-defined combinators.
4. discourage creation of non-sensical combinator expressions, or possible mis-use of the module
5. hide existential types and anything else exotic

If you want to play with it, you can do a:

$> darcs get http://coder.bsimmons.name/code/CombinatorCalculus

A Quick Combinator Calculus Primer

A Combinatory Logic expression consists of a sequence of combinator terms and sub-expressions. Combinator terms are like functions which consume some arguments (themselves combinators or sub-expressions), and return a particular re-arrangement of those arguments. When no term can consume enough arguments to be “evaluated” any further, the expression is said to be in normal form.

The SKI Combinator Calculus is a system that uses only three combinators (‘S’, ‘K’ and ‘I’) and forms a Turing Complete system. This means we can express any computation with expressions that look like SK(SSS)I(S(KS)K)I. It can even server as the basis of a turing complete programming language!

It’s a really fascinating topic, and you can get a great overview from wikipedia’s excellent Combinatory Logic article.

The Design of the Module

All terms in combinatory logic take some arguments and form a new expression from them, but different combinators transform their arguments in different ways.

We can express this in Haskell by creating a typeclass with methods that describe the behavior of a combinator. Then we can make a type (corresponding to a specific Combinator term) an instance of our class to define its behavior.

The Combinator Class

Here is how we define our Combinator class:

 -- a Class for combinators. Its methods represent a combinator's behavior on
 -- its arguments:
class (Show c)=> Combinator c where
    takesArgs :: c -> Int

    applyArgs :: c -> [Term] -> Expr
    
     -- HIDDEN METHOD: EXPORTED AS A FUNCTION:
     -- if a combinator takes 0 args, we assume it can be evaluated and we
     -- say it is not in Normal Form:
    isNormal :: c -> Bool
    isNormal = (> 0) . takesArgs
    
     -- HIDDEN METHOD: user defined combinators are placed in a black box:
    toTerm :: c -> Term
    toTerm = Term

The first two methods are the only ones exported to users of the module to define an instance of Combinator. takesArgs is used to indicate how many arguments this particular combinator requires before it can be evaluated.

applyArgs actually performs the transformation of those arguments; it is given a list of terms (of length equal to `takesArgs`) and returns a new expression (type Expr).

The third method is used to check if an expression is in normal form. It is exported as a regular function, meaning that users can use it in their code but cannot use it in their own instance declarations. Instead the default value would apply.

The fourth method `toTerm` is used internally to encapsulate some Combinator class item in a `Term` data type. We use this when composing combinator expressions, in order to preserve the tree structure of the expression. (see below for more on this)

The Term and Expr types

The module defines two new datatypes, both of which are instances of `Combinator`.

`Expr` is a newtype wrapper around Seq Term, but we don’t export the implementation so we can change this if we don’t want to use Sequences:

newtype Expr = Expr { combSeq :: Seq Term }

It simply represents a CL expression as a sequence of terms. Making it an instance of `Combinator` expresses the notion that an expression can be thought of as a combinator term whenever the need arises. In fact the distinction is simply one of evaluation strategy.

`Term` is a wrapper for Combinator terms and sub-expressions. It is mutually-recursive with `Expr` and the two together form the tree structure of a combinator expression:

data Term = forall c. Combinator c => Term { unbox :: c }
          | SubExpr   { subExpr :: Expr }
          | NamedExpr { subExpr :: Expr,
                        name    :: String }

All three constructors are hidden from the user. `SubExpr` simply holds an `Expr` type and represents a parenthesized sub-expression within the top level combinator sequence. `NamedExpr` is identical except that it also contains a String, which will be returned when `show` is called on that constructor. Here is the `Show` instance for `Term`:

instance Show Term where
    show (NamedExpr _ n) = n
    show (SubExpr e)     = "("++ show e ++")"
    show (Term t)        = show t

The remaining constructor is the most interesting: the `Term` constructor houses an existentially-quantified type and can hold any type value of the `Combinator` class.

Notice that the type variable does not appear on the left side of the = in the type declaration. What this means in practical terms is that the `Term` constructor becomes a “black box”; once a value is inside, the only things we can “do with it” are to apply polymorphic Combinator class functions and methods.

And that’s why we have the internal `toTerm` method; it allows us to wrap sub-expressions in the `SubExpr` constructor, preserving the tree structure of an expression as we compose it. Without it we would either need to export multiple functions for composing a term with an expression, an expression with an expression, a term with a term, and an expression with a term. Ugly!

Exported functions

Our types and classes allow us to do some neat stuff. As mentioned above, we can compose expressions using a single haskell infix function. Since all the nice ASCII combinators were taken (and since I’m the only one using this) I decided to go with a Unicode symbol, the middle dot, for composition:

 -- operator for composing combinator expressions. This is the Middle Dot,
 -- unicode character 00B7. It is easily inserted in vim using the digraph:
 --       Ctrl-K '.' 'M'
infixl 7 ·

…and the implementation:

 -- for composing Combinator expressions:
(·) :: (Combinator c1,Combinator c2)=> c1 -> c2 -> Expr
c1 · (toTerm->t2) = 
     case toTerm c1 of                                                                
          (SubExpr e) -> e `appendTerms` singleton t2                                
           -- named expression or bare Term start a new sub-expression:              
          t1          -> Expr (singleton t1 |> t2)

We also have the ability to define new combinators in terms of other combinators using the `named` function. Here is an example showing how `I` can be defined in terms of `S` and `K` and used to form a Combinator Calculus equivalent of the Church Numeral 2:

60 two = S·(S·(K·S)·K)·cI
61     where cI = (S · K · K)  `named` "I"

Playing with it

The evaluation function exported in the module was written fairly hastily, but it is good enough to let us play with evaluating expressions.

I’ve run out of time on this post, but you can grab the darcs repo above and run the ‘main’ function in `Examples.hs` and check out the source for more info. I’ll maybe show some cool examples in a later post.

Thanks for reading!

I came up with this variation of the Move-to-Front transform which doesn’t require that there be a known alphabet of symbols. Instead it builds up the alphabet as it goes along and encounters a new symbol. I’m sure that this is a known algorithm, as it’s a fairly obvious variation, but I don’t know what the proper name for it is, so I’m calling it an Adaptive Move-to-Front.

I’m very interested in hearing if this is similar or identical to any other algorithms, so if anyone has any insights, please comment!

Differences from the Standard MTF Transform:

The traditional MTF algorithm, as I understand it, uses some known ordered alphabet of symbols (e.g. the bytes from 0-255, or the letters a-z), so presumably, one need not store this alphabet along with the encoded/compressed data.

With the algorithm I describe here though, the final permutation of the alphabet list, output by the encoder, must be retained to decode the message. The encoded message is identical to the output of the traditional MTF transform, except where a symbol is encoded which has not appeared previously.

Here we have zipped the output of the standard MTF (the fst) with the adaptive version (the snd) to make it easy to compare elements:

Message: “the rrrrain in sssspain falls maaiinly on the plain”

Output (standard MTF, adaptive):

[(116,0),(105,1),(103,2),(35,3),(115,4),(0,0),(0,0),(0,0),(101,5),(107,6),(112,7),(4,4),(2,2),(2,2),(2,2),(116,8),(0,0),(0,0),(0,0),(115,9),(5,5),(5,5),(5,5),(5,5),(109,10),(4,4),(113,11),(0,0),(7,7),(4,4),(114,12),(4,4),(0,0),(7,7),(0,0),(7,7),(6,6),(121,13),(6,6),(116,14),(4,4),(2,2),(14,14),(14,14),(14,14),(3,3),(13,13),(8,8),(10,10),(10,10),(8,8)]

The standard algorithm was working with the ASCII alphabet, so each new symbol is located somewhere far back into the alphabet; it is then moved to the front. In the adaptive version, when we see a new symbol, it is automatically made the next in our running alphabet and then of course moved to the front.

This means that with the adaptive version, for any given stretch from the beginning of the encoded message, we are using the minimal number of different index integers to encode the stream, always choosing the next greatest integer when we require a new one. Here is the output of the adaptive version alone:

Encoded phrase:

[0,1,2,3,4,0,0,0,5,6,7,4,2,2,2,8,0,0,0,9,5,5,5,5,10,4,
11,0,7,4,12,4,0,7,0,7,6,13,6,14,4,2,14,14,14,3,13,8,
10,10,8]

Final permutation of minimal dictionary list:
"nialp ehtoymsfr"

What these means for actual compression schemes on a binary level, I have no idea, but I would like to explore this topic much more. Without further ado, here is some code.

The Encoder:

This “adaptive move-to-front” encoder works in the same way as the standard algorithm, except that we start out with an empty alphabet list and build it up as we see new elements not yet in our list.

Essentially if, while searching for an element, we hit the empty list [], we pretend that the element we were looking for was at the end of the list.

This is exactly what happens in the traditional algorithm: when we try to encode a symbol that we haven’t yet encountered, we venture into a new part of the alphabet list that we haven’t touched yet.

The Decoder

Our decoder requires that we pass it the final permutation of the symbol list, along with the encoded list of indexes. It then simply does the reverse of the encoding function.

To decode we follow these steps:

1. Return the head of the list of symbols
2. Re-insert the symbol into the index location specified by the head of the encoded/index list
3. Move on to the next encoded index, using the new list of symbols. go to (1)

The Data Declaration:

To understand a monad you look at it’s datatype and then at the definition for bind (>>=). Most monad tutorials start by showing you the data declaration of a State s a in passing, as if it needed no explanation:

newtype State s a = State { runState :: s -> (a, s) }

But this does need explanation! This is crazy stuff and nothing like what we’ve seen before in the list monad or the Maybe monad:

1. The constructor State holds a function, not just a simple value like Maybe’s Just. This looks weird.
2. Furthermore there is an accessor function runState with a weirdly imperative-sounding name.
3. Finally, there are two free variables on the left side, not just one.

Yikes! Let’s try to get our head on straight and figure this out:

First of all the State monad is just an abstraction for a function that takes a state and returns an intermediate value and some new state value. To formalize this abstraction in haskell, we wrap the function in the newtype State allowing us to define a Monad instance.

Stepping back from the abstract and conceptual, what we have is the State constructor acting as a container for a function :: s -> (a,s), while the definition for bind just provides a mechanism for “composing” a function state -> (val,state) within the State wrapper.

Just as you can chain together functions using (.) as in (+1) . (*3) . head :: (Num a) => [a] -> a, the state monad gives you (>>=) to chain together functions that look essentially like :: a -> s -> (a,s) into a single function :: s -> (a,s).

Let’s bring the discussion back to actual code and try to make sure we understand those three points of weirdness outlined above. Here’s a stupid example of a function that can be “contained” in our state type:

-- look at our counter and return "foo" or "bar"
-- along with the incremented counter:
fromStoAandS :: Int -> (String,Int)
fromStoAandS c | c `mod` 5 == 0 = ("foo",c+1)
               | otherwise      = ("bar",c+1)

If we just wrap that in a State constructor, we’re in the State monad:

stateIntString :: State Int String
stateIntString = State fromStoAandS

But what about runState? All that does of course is give us the “contents” of our State constructor: i.e. a single function :: s -> (a,s). It could have been named stateFunction but someone thought it would be really clever to be able to write things like:

runState stateIntString 1

See, all we’ve done there is used runState to take our function (fromStoAandS) out of the State wrapper; it is then applied it to it’s initial state (1). We would do this runState business after building up our composed function with (>>=), mapM, etc.

That leaves point 3 unanswered. Let’s start exploring the instance declaration for State.

The Instance Declaration

We’ll start with the first line:

instance Monad (State s) where

We create a Monad instance for (State s) not State. You can think of this as a partially-applied type, which is equivalent to a partially applied function:

(State) < ==> (+)
(State s) < ==> (1+)
(State s a) < == (1+2)

So (State s) is the m in our m a. This means the type of our state will remain the same as we compose our function with (>>=), whereas the intermediate values (the as) may well change type as they move through the chain.

Before we move on to the meat of the instance declaration, I’d like to get your mind calibrated to look at the definitions for return and (>>=):

Whenever you see m a, as in

return :: (Monad m) => a -> m a

…remember that m a is actually

State s a

…and when you remember (State s a), think

(s -> (s,a)).

So in your mind, m a becomes function :: s -> (a,s) everywhere you see it. Just forget about the silly State wrapper (the compiler does)!

The definition of return and Bind

Let’s wet our feet with the definition for return:

    return a = State $ \s -> (a, s)

All return does is take some value a and make a function that takes a state value and returns (value, state value). If we ignore the whole State wrapping business, then return is just (,) :: a -> b -> (a, b)

Now recall the definition of bind:

(>>=) :: (Monad m) => m a -> (a -> m b) -> m b

Which in our case is:

(>>=) ::  State s a        -> 
          (a -> State s b) -> 
          State s b

And which is just a silly abstraction for the super special function composition that’s going on, which looks like:

(>>=) ::  (s -> (a,s))       -> 
          (a -> s -> (b,s))  -> 
          (s -> (b,s))

So on the left hand side of (>>=) is a function that takes some initial state and produces a (value,new_state). On the right hand side is a function that takes that value and that new_state and generates it’s own (new_value, newer_state). The job of bind is simply to combine those two functions into one bigger function from the initial state to (new_value,newer_state), just like the simple function composition operator (.) :: (b -> c) -> (a -> b) -> a -> c

At this point, we can show you bind’s definition:

    m >>= k  = State $ \s -> let
        (a, s') = runState m s
        in runState (k a) s'

You can work through that on your own, keeping in mind that we’re doing function composition here. The main thing to remember is that the s at the top, right after State, won’t actually be bound to a value until we unwrap the function with runState and pass it the initial state value, at which point we can evaluate the entire chain.

A Final Note About The State Monad with do Notation

State is often used like this:

stateFunction :: State [a] ()
stateFunction = do x <- pop
                   pop
                   push x

Remember that the functions above are desugaring to m >>= \a-> f... or if there is no left arrow on the previous line: m >>= \_-> f... That a in there is an intermediate value, the fst in the tuple. The push function might look like:

push :: State [a] ()
push a = State $ \as -> (() , a:as)

The function doesn’t return any meaningful a value, so we don’t bind it by using the < -. For more work with do notation and some fine pictures, see Bonus's post on something awful.

infixr 9 .:

(.:) :: (b -> c) -> (a -> a1 -> b) -> a -> a1 -> c
(.:) = (.)(.)(.)

This can be used to compose a string of functions with a binary function stuck on the end. For example:

lookupPlusOne :: (Ord k, Monad m, Num n) => k -> Map k n -> m n
lookupPlusOne = liftM (+1) .: lookup

(picked up from some folks on #haskell)