fragmentary ideas ䷿ intellectual what-nots ䷷ and haskell programming ䷴

UPDATE: Thanks to steve for pointing out that the scheme I describe here is essentially the same as Algorithm BSTW.

I came up with this variation of the Move-to-Front transform which doesn’t require that there be a known alphabet of symbols. Instead it builds up the alphabet as it goes along and encounters a new symbol. I’m sure that this is a known algorithm, as it’s a fairly obvious variation, but I don’t know what the proper name for it is, so I’m calling it an Adaptive Move-to-Front.

I’m very interested in hearing if this is similar or identical to any other algorithms, so if anyone has any insights, please comment!

Differences from the Standard MTF Transform:

The traditional MTF algorithm, as I understand it, uses some known ordered alphabet of symbols (e.g. the bytes from 0-255, or the letters a-z), so presumably, one need not store this alphabet along with the encoded/compressed data.

With the algorithm I describe here though, the final permutation of the alphabet list, output by the encoder, must be retained to decode the message. The encoded message is identical to the output of the traditional MTF transform, except where a symbol is encoded which has not appeared previously.

Here we have zipped the output of the standard MTF (the fst) with the adaptive version (the snd) to make it easy to compare elements:

Message: “the rrrrain in sssspain falls maaiinly on the plain”

Output (standard MTF, adaptive):

[(116,0),(105,1),(103,2),(35,3),(115,4),(0,0),(0,0),(0,0),(101,5),(107,6),(112,7),(4,4),(2,2),(2,2),(2,2),(116,8),(0,0),(0,0),(0,0),(115,9),(5,5),(5,5),(5,5),(5,5),(109,10),(4,4),(113,11),(0,0),(7,7),(4,4),(114,12),(4,4),(0,0),(7,7),(0,0),(7,7),(6,6),(121,13),(6,6),(116,14),(4,4),(2,2),(14,14),(14,14),(14,14),(3,3),(13,13),(8,8),(10,10),(10,10),(8,8)]

The standard algorithm was working with the ASCII alphabet, so each new symbol is located somewhere far back into the alphabet; it is then moved to the front. In the adaptive version, when we see a new symbol, it is automatically made the next in our running alphabet and then of course moved to the front.

This means that with the adaptive version, for any given stretch from the beginning of the encoded message, we are using the minimal number of different index integers to encode the stream, always choosing the next greatest integer when we require a new one. Here is the output of the adaptive version alone:

Encoded phrase:

[0,1,2,3,4,0,0,0,5,6,7,4,2,2,2,8,0,0,0,9,5,5,5,5,10,4,
11,0,7,4,12,4,0,7,0,7,6,13,6,14,4,2,14,14,14,3,13,8,
10,10,8]

Final permutation of minimal dictionary list:
"nialp ehtoymsfr"

What these means for actual compression schemes on a binary level, I have no idea, but I would like to explore this topic much more. Without further ado, here is some code.

The Encoder:

This “adaptive move-to-front” encoder works in the same way as the standard algorithm, except that we start out with an empty alphabet list and build it up as we see new elements not yet in our list.

Essentially if, while searching for an element, we hit the empty list [], we pretend that the element we were looking for was at the end of the list.

This is exactly what happens in the traditional algorithm: when we try to encode a symbol that we haven’t yet encountered, we venture into a new part of the alphabet list that we haven’t touched yet.

The Decoder

Our decoder requires that we pass it the final permutation of the symbol list, along with the encoded list of indexes. It then simply does the reverse of the encoding function.

To decode we follow these steps:

1. Return the head of the list of symbols
2. Re-insert the symbol into the index location specified by the head of the encoded/index list
3. Move on to the next encoded index, using the new list of symbols. go to (1)

I’ve written this brief tutorial on haskell’s State monad to help bridge some of the elusive gaps that I encountered in other explanations I’ve read, and to try to cut through all the sticky abstraction. This is written for someone who has a good understanding of the Maybe and List monads, but has gotten stuck trying to understand State. I hope it’s helpful!

The Data Declaration:

To understand a monad you look at it’s datatype and then at the definition for bind (>>=). Most monad tutorials start by showing you the data declaration of a State s a in passing, as if it needed no explanation:

newtype State s a = State { runState :: s -> (a, s) }

But this does need explanation! This is crazy stuff and nothing like what we’ve seen before in the list monad or the Maybe monad:

1. The constructor State holds a function, not just a simple value like Maybe’s Just. This looks weird.
2. Furthermore there is an accessor function runState with a weirdly imperative-sounding name.
3. Finally, there are two free variables on the left side, not just one.

Yikes! Let’s try to get our head on straight and figure this out:

First of all the State monad is just an abstraction for a function that takes a state and returns an intermediate value and some new state value. To formalize this abstraction in haskell, we wrap the function in the newtype State allowing us to define a Monad instance.

Stepping back from the abstract and conceptual, what we have is the State constructor acting as a container for a function :: s -> (a,s), while the definition for bind just provides a mechanism for “composing” a function state -> (val,state) within the State wrapper.

Just as you can chain together functions using (.) as in (+1) . (*3) . head :: (Num a) => [a] -> a, the state monad gives you (>>=) to chain together functions that look essentially like :: a -> s -> (a,s) into a single function :: s -> (a,s).

Let’s bring the discussion back to actual code and try to make sure we understand those three points of weirdness outlined above. Here’s a stupid example of a function that can be “contained” in our state type:

-- look at our counter and return "foo" or "bar"
-- along with the incremented counter:
fromStoAandS :: Int -> (String,Int)
fromStoAandS c | c `mod` 5 == 0 = ("foo",c+1)
               | otherwise      = ("bar",c+1)

If we just wrap that in a State constructor, we’re in the State monad:

stateIntString :: State Int String
stateIntString = State fromStoAandS

But what about runState? All that does of course is give us the “contents” of our State constructor: i.e. a single function :: s -> (a,s). It could have been named stateFunction but someone thought it would be really clever to be able to write things like:

runState stateIntString 1

See, all we’ve done there is used runState to take our function (fromStoAandS) out of the State wrapper; it is then applied it to it’s initial state (1). We would do this runState business after building up our composed function with (>>=), mapM, etc.

That leaves point 3 unanswered. Let’s start exploring the instance declaration for State.

The Instance Declaration

We’ll start with the first line:

instance Monad (State s) where

We create a Monad instance for (State s) not State. You can think of this as a partially-applied type, which is equivalent to a partially applied function:

(State) <==> (+)
(State s) <==> (1+)
(State s a) <== (1+2)

So (State s) is the m in our m a. This means the type of our state will remain the same as we compose our function with (>>=), whereas the intermediate values (the as) may well change type as they move through the chain.

Before we move on to the meat of the instance declaration, I'd like to get your mind calibrated to look at the definitions for return and (>>=):

Whenever you see m a, as in

return :: (Monad m) => a -> m a

...remember that m a is actually

State s a

...and when you remember (State s a), think

(s -> (s,a)).

So in your mind, m a becomes function :: s -> (a,s) everywhere you see it. Just forget about the silly State wrapper (the compiler does)!

The definition of return and Bind

Let's wet out feet with the definition for return:

    return a = State $ \s -> (a, s)

All return does is take some value a and make a function that takes a state value and returns (value, state value). If we ignore the whole State wrapping business, then return is just (,) :: a -> b -> (a, b)

Now recall the definition of bind:

(>>=) :: (Monad m) => m a -> (a -> m b) -> m b

Which in our case is:

(>>=) ::  State s a        -> 
          (a -> State s b) -> 
          State s b

And which is just a silly abstraction for the super special function composition that's going on, which looks like:

(>>=) ::  (s -> (a,s))       -> 
          (a -> s -> (a,s))  -> 
          (s -> (b,s))

So on the left hand side of (>>=) is a function that takes some initial state and produces a (value,new_state). On the right hand side is a function that takes that value and that new_state and generates it's own (new_value, newer_state). The job of bind is simply to combine those two functions into one bigger function from the initial state to (new_value,newer_state), just like the simple function composition operator (.) :: (b -> c) -> (a -> b) -> a -> c

At this point, we can show you bind's definition:

    m >>= k  = State $ \s -> let
        (a, s') = runState m s
        in runState (k a) s'

You can work through that on your own, keeping in mind that we're doing function composition here. The main thing to remember is that the s at the top, right after State, won't actually be bound to a value until we unwrap the function with runState and pass it the initial state value, at which point we can evaluate the entire chain.

A Final Note About The State Monad with do Notation

State is often used like this:

stateFunction :: State [a] ()
stateFunction = do x <- pop
                   pop
                   push x

Remember that the functions above are desugaring to m >>= \a-> f... or if there is no left arrow on the previous line: m >>= \_-> f... That a in there is an intermediate value, the fst in the tuple. The push function might look like:

push :: State [a] ()
push a = State $ \as -> (() , a:as)

The function doesn't return any meaningful a value, so we don't bind it by using the <-. For more work with do notation and some fine pictures, see Bonus's post on something awful.

infixr 9 .:

(.:) :: (b -> c) -> (a -> a1 -> b) -> a -> a1 -> c
(.:) = (.)(.)(.)

This can be used to compose a string of functions with a binary function stuck on the end. For example:

lookupPlusOne :: (Ord k, Monad m, Num n) => k -> Map k n -> m n
lookupPlusOne = liftM (+1) .: lookup

(picked up from some folks on #haskell)