• Distribute - a drop-in replacement for setuptools
• pip - a command-line Python package management utility
• virtualenv - allows you to create a sandbox environment to install packages into without having to install to your global Python site-packages folder. This presents a solution for conflicts between applications that require different versions of the same package.

The distribute documentation has a nice info graphic that illustrates the current direction:

The current state of Python packaging

Here’s another one, also from the distribute documentation; slightly subtler:

Python packaging public service announcement info graphic

For information on how to create a Python package, see the Hitchhiker’s Guide to Packaging.

Insertion sort is a simple iterative, incremental sorting algorithm.  Although its running time grows quadratically, it is relatively efficient and simple to implement, making it a good choice for sorting small data sets.

Description

It is designed around two loops:

• An outer loop that traverses up from the second element in the input array to the last. It places each element, or key, into sorted position as it goes.
• An inner loop that iterates down through the already sorted elements towards the first element of the input. It shifts elements upwards until the current key’s position is found. As soon as the correct position is found it can stop working, so it does not have to visit every element in the sorted sub-array every time.

At the start of each iteration of the outer loop, the elements in the sub-array a[0..j-1] will be in sorted order (the loop invariant).

Here is an example implementation written the Go language:

/*
 Sort an array of integers, in place, using the insertion sort algorithm.
 */
func InsertionSort(a []int) {
    // Iterate up from the second element to the last
    for j := 1; j < len(a); j++ {
        key := a[j]
        i := j - 1

        // Iterate down through the sorted subset [j-1..0], shifting
        // elements upwards until key's position is found
        for i >= 0 && a[i] > key {
            a[i+1] = a[i]
            i--
        }

        // Insert key into its position in the sorted subset
        a[i+1] = key
    }
}

You can get this from my gosort repository on GitHub.

Characteristics

Insertion sort has the following characteristics:

Uses

Because of its simplicity and relatively small constant factors, it can be used to handle the base case in composite divide and conquer algorithms, when the number of elements is small (Mehlhorn suggests n <= 10). The Go language’s sort implementation provides a real example of this: insertion sort is used within the quick sort implementation when the number of elements is less than 7 (see lines 181 to 183).

Because it processes its input serially, piece-by-piece, it can be used to sort data directly from a stream (online).

Further reading

More information can be found on this algorithm in the following sources:

• Section 5.1 of Data Structures and Algorithms by Mehlhorn and Sanders
• Section 2.2 of Introduction to Algorithms by Cormen et al
• And good old Wikipedia: Insertion Sort

1. Feed your creativity. To positively enhance your creativity, seek unfamiliar experiences. Actively break habits and patterns. New experiences provide new concepts to mix up with your existing knowledge, which leads to new ideas. Amazingly, it was even shown that something as mundane as deliberately putting a sandwich together in a different order than usual, contributes to this effect.
2. Play with things. Whenever you learn or experience something new, spend some time brainstorming things you could use it for. Be creative with it. It was amazing watching volunteers in the experiments brainstorming things that a brick could be used for other than building a house. In fact, I used to do this with my guitar; whenever I learned a new soloing technique or chord I used to toy with it until I had several new ideas based on the original. This digestive process helped me learn and create.
3. Busting mental blocks. When you are “stuck in a rut” stop what you are doing, but don’t do nothing. Instead, switch to some other undemanding task. When you return to the original task later you should be in a more creative state. The programme featured an experiment where 3 people’s creativity was tested (I think it was the brainstorming things you could do with a brick moment). Then one was told to stop and do nothing, the second was given a simple pleasurable task (arranging Lego bricks by colour), and the third was given something hard to do (can’t remember what this task was, maybe building something with the Lego). Surprisingly, the person who returned to the brick-brainstorming and had the most new ideas, was the person who had been given the simple, pleasurable and non-taxing task.
4. Check your assumptions. One of the biggest difficulties in problem solving is the fact that our brains sometimes make huge sweeping assumptions that may not really be a part of the problem. On the programme this was demonstrated by an experiment where a dollar bill was pinned under a large upside-down pyramid. The participants were challenged to remove the bill without moving or lifting the pyramid. They were stumped for quite a while until someone had a flash of inspiration that there was actually no requirement to keep the bill intact; the bill could be removed by burning it. Because the item to be moved was perceived as having value, the idea of destroying it was intuitively rejected even though the problem statement didn’t require this.

Background

The Go Gopher by Renée French

Go is just 3 years old and only recently achieved its first stable milestone release, version 1.0.0, in March 2012. It has been designed and built internally at Google by Rob Pike and Ken Thompson (both known for the Plan9 operating system, UTF-8 and Unix), Robert Griesemer (who worked on Strongtalk and the Hotspot JVM), and others. Their remit was to design a language that could scale to cope with the huge number of interconnected systems, users, engineers, and lines of code that Google needs to deal with day to day.

So what does it look like then?

It is a statically typed, garbage collected language that sits at the level somewhere between C/C++ and Java/C#, and compiles down to platform native code.

Although it has a C-like syntax, it also includes high-level features like built-in dictionary/map support similar to scripting languages like JavaScript or Python, and function closures like JavaScript.

Here’s an example, taken from Rob Pike’s “Go Concurrency Patterns” talk, that demonstrates several important features:

package main

import (
    "fmt"
    "math/rand"
    "time"
)

func main() {
    // A function returning a channel
    c := boring("boring!")

    for i := 0; i < 5; i++ {
        // Read a string from the channel and print it
        fmt.Printf("You say: %q\n", <-c)
    }

    fmt.Println("You're boring; I'm leaving.")
}

// A generator function that returns a receive-only channel of strings.
func boring(msg string) <-chan string {
    // Create the channel of strings
    c := make(chan string)

    // Define an anonymous function and run it as a goroutine
    go func() {
        for i := 0; ; i++ {
            // Send a message to the channel
            c <- fmt.Sprintf("%s %d", msg, i)
            time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond)
        }
    }()

    // Return the channel to the caller
    return c
}

I’ll leave it to Rob to explain what’s going on here in detail.  Admittedly, this may be a little challenging to understand if you are not already familiar with the basics of the language.  For a quick step-by-step introduction take the tour.

Probably the most important feature is its support for parallelism and concurrency.  For this, Go’s designers refer to ideas from a 1978 paper (really has it take us the length of my entire lifetime to get here?) by Tony Hoare called “Communicating Sequential Processes”. These are realised in Go by goroutines, which are concurrent processes that communicate with each other by sending messages over channels. The idea is, by coordinating through message passing, rather than shared memory, it enables lock-free programming.

What I like about it so far

I have only been investigating it for a few weeks, and haven’t built anything significant with it yet. However, there are already a number of features that appeal to me.  I had intended to keep this list small, but to hell with it, there are a lot of things I like about it already  here we go:

• It has a C-like syntax, but its terser. Semi-colons are not required, braces around for and if  statements are dropped, and if variables are assigned where they are declared you can drop the type specification.
• Capitalisation of names is used to control public/private access, which eliminates a lot of extra syntax.
• It has borrowed some conveniences from Python: array slices, a built-in map type, and functions can return multiple values.
• It has first class functions, which are full closures.  Enabling a tonne of great programming paradigms that are in common use in languages like JavaScript and Ruby.
• It is sort-of object oriented, but more like the way C is, in that objects are declared as structures with methods defined outside of the structure declaration. It does not support inheritance.  Instead it provides a mixin-like style of extension, known as embedding, where new types can be created by mixing together existing types or interfaces to produce a type which is a union of the features of the types it combines.
• It supports the concept of interfaces like C++/Java/C#, but with additional flexibility because it follows a sort of static version of the duck typing idiom: any object can implement an interface simply by defining functions that match the signatures declared in that interface.  There is no need to explicitly declare that an object implements an interface.  The benefit this brings is that it makes it possible to write polymorphic code without requiring the individual types involved to have been pre-designed for use in this way.
• You can create new types from anything, even an int, and can then add methods to your new type.
• It has a well thought out declaration syntax, with intent of straightening out the complicated C declaration rules to produce something that is easier to parse by eye or machine.
• They fixed the switch statement (hurray!) so that cases break by default, and enhanced it so that any type can be the switch condition.  Plus, there is a switch with no condition, which has an implicit condition that defaults to true, and provides a compact replacements for long if-else chains.
• It has a logically named set of basic types, including <stdint.h> style explicit precision versions: bool, int, int8, int16, int32, int64, uint, uint8, and so on.
• It has a built-in unicode string type.
• It supports pointers for those times when you need to pass by reference instead of passing by value.  However, pointer arithmetic is not supported.
• It compiles to platform native binary formats. So no messing around with byte code interpretors or class paths required to run an application. In fact, I had a quick look at what DLLs a simple ‘hello world’ style Go binary links with on Windows: only kernel32.dll and winmm.dll for a multimedia timer API. On Linux there are no dynamically linked dependencies at all. So the resulting binaries don’t require the user to install a Go runtime, which makes them refreshingly portable.
• It has a garbage collector, so you don’t have to worry about the large majority of memory leak issues that you have to in C.

Oh, it also has a cute mascot called the ‘Gopher’ [see the image above]

Toolchain and workflow

What is also excellent about Go is that the tool-chain has not been left an after-thought.

It comes bundled with more than just a compiler, so there is no need to mess around with makefiles!!  This makes it very easy to get up and running.

The main way of building Go source code is by using the go front-end command-line tool. It has a simple Git/Subversion inspired interface, with sub-commands for building, installing, testing, formatting, and various other tasks.

It also has a run sub-command that builds and runs in one step.  Impressively, because the compiler is fast enough, this means that Go can be used almost as if it is a scripting language. A simple idea really, but a nice touch that makes it so easy to code up small experiments and small tasks. I have often created shell scripts to do the same in C when trying to hack out a small idea quickly.

As far as I can tell at the moment, there are two main schemes for source organisation:

• All project sources in one directory – for example the go command sources are organised like this (see $(GOROOT)/src/cmd/go in your installation). Typically, twenty to thirty source files all in one directory along with any scripts and README files etc.
• Sources organised into a directory hierarchy of packages called a Go workspace.  Each package sub-directory may be a separate project or source repository.

Interestingly, it appears there is no middle ground here: if you take the first scheme and try to organise some of your code into packages in sub-directories, the compiler just cannot find the nested sources. Granted, there may be a way around this but I need to dig deeper (please leave a comment if you know this is wrong). Anyway, for more on this topic see: here.

What types of tasks is it good for?

At the moment it appears to be targeted at writing network and system services. But, I guess this may expand in time. Here’s a big list of projects written in Go.

SoundCloud are using it to write back-end systems.

Further reading

Here are some other interesting sources of information:

• The project home page http://golang.org/.
• The Go Tour – an interactive online tutorial
• The Design FAQ
• Effective Go
• Calling Go from Python via JSON-RPC by Bruce Eckel

• Clarity: it has a pretty terse syntax that absorbs ideas from C style languages and scripting languages like Python. Also, because it is a garbage-collected language there is no extra boiler-plate code required to handle memory management. Overall I think it will make the intent of the algorithms clear and easy to understand.
• Familiarity: its syntax is close enough to C that it should be easy to follow for anyone who knows a language in the C family.
• Modernity: it is a very new language, only about 3 years old.  Consequently, it realizes contemporary language design ideas such as: a safe version of the switch statement (not that this is a new idea, I know), function closures, mixin style type extension and so on.  Plus it includes a great tool-chain that allows you to get up and running quite quickly.  Of course, this also means I get to learn a new language as part of this exercise, which adds additional motivational value for me

I have started drafting a more detailed intro to Go in another blog post too, because I want to talk about some of the things I like about it so far.

So, the series didn’t happen. Why? Well, when I set out to do this I was looking to invent reasons to write.  I have since discovered that I have plenty to write about, too much in fact, and having to write a series to an artificial deadline is too restrictive, and a practical impossibility with pressures from work and responsibilities at home.

So what am I going to do instead? I will publish some posts on algorithms because I have started drafting some already.  They will probably not be published in a particularly logical order; just when they are ready.  I also intend to write freely about whatever I feel like writing about.

Original Post:

Following on from my previous post, I have now worked out a plan for how I will work through this series.

I want to complete a meaningful first set of articles within the first few months of this year, then be able to stop, evaluate what I have achieved and plan what to focus on next.  So as a start, I have decided to limit my focus to sorting and searching algorithms, and basic data structures.

Also, instead of working through these topics in order, I am going to try to cover two algorithms from each topic, each month. I feel it is important not to be stuck on one topic area for too long.  So, by doing it this way I can progress both topics in parallel, and by sticking with just two topics I will still be able to complete something meaningful in a sensible amount of time.

So here is the initial plan:

January 2013

• Algorithmic Complexity Analysis – a short note on this.
• Insertion Sort
• Merge Sort
• Stack
• Queue

February 2013

• Heap Sort
• Quick Sort
• Linked List
• Hash Tables

March 2013

• Counting Sort
• Radix Sort
• Binary Search Tree
• Red-Black Tree

April 2013

• Binary Search
• Bucket Sort
• Graphs

So as you can see, basic core stuff. There are deadlines at the end of each month, and a final deadline on the last day of April. I have created a public Trello board to track my work: https://trello.com/b/MEnlz31y.

As my primary guide I will be using Introduction to Algorithms by T. Cormen et al.  But, I will also be referring to other sources to find alternative ideas if useful.

A big question I have here is: what programming language to use for the examples? Or even, whether to use a real language at all, e.g. should I use pseudo-code? Being a bit of a software generalist, there are several languages I would potentially choose.

Well, pseudo-code can immediately be crossed off of the list because it cannot be compiled and tested. I don’t really want to use a pure functional language such as Erlang, Haskell or Clojure, because they still don’t appear to be as widely used as imperative, procedural or object-oriented languages, and although interesting, may be rather too specialist for this endeavour.

The other choices really fall into to three camps for me:

1. Simple, terse scripting languages – such as Python or Ruby – that are close to pseudo-code in terms of readability, and do not require large amounts of tiresome boiler plate structural code so the algorithms themselves can be communicated as clearly as possible.  (Note, I couldn’t bring myself to include JavaScript in this category because I just feel it is a step closer to C or Java than it is to pseudo-code).
2. Well known, or general software engineering languages – such as C, C++, Java, JavaScript or C#.  These are the common denominators; the languages a very large number of people know or can at least read.
3. New generation languages – such as Go, CoffeeScript or Scala.  Using any of these would motivate me (because I like learning new things), and potentially make the resulting work more interesting and unique.

So how to decide? Clarity, universality or modernity?  I need to think about this.

Next step: make a plan.

class HelloWorld(object):
    def __init__(self, name):
        self.name = name.title()

    def sayHello(self):
        print "Hello from {0}!".format(self.name)

hello = HelloWorld("python")
hello.sayHello()

How about some Erlang?

-module(hello).
-export([hello_world/0]).

hello_world() -> io:fwrite("hello from Erlang\n").

Ruby?

class HelloWorld
   def initialize(name)
      @name = name.capitalize
   end
   def sayHi
      puts "Hello #{@name}!"
   end
end

hello = HelloWorld.new("World")
hello.sayHi

How about some over-the-top OO style C?

#include <stdio.h>
#include <string.h>

const size_t HELLO_WORLD_NAME_MAX = 32;

typedef struct
{
    char name[HELLO_WORLD_NAME_MAX];
} HelloWorld;

void helloworld_init(HelloWorld *self,
                     const char *name,
                     size_t name_length)
{
    self->name[0] = '\0';

    if (NULL != name &&
        name_length < HELLO_WORLD_NAME_MAX)
    {
        strncpy(self->name, name, name_length);
        self->name[name_length] = '\0';
    }
}

void helloworld_say_hi(HelloWorld *self)
{
    printf("Hello from %s!", self->name);
}

int main(int argc, char* argv[])
{
    const char *name = "C";
    HelloWorld hello;

    helloworld_init(&hello, name, strlen(name));
    helloworld_say_hi(&hello);

    return 0;
}

exec 1>/dev/null

This redirects the file stream to the /dev/null device which silently swallows the output.

This is useful if for instance you are writing a cron job which you want to have log it’s steps while you are developing or debugging it, but don’t want any output except the stderr once it’s running.

1) Print predefined macros:

gcc -dM -E - </dev/null | sort | less

2) Print predefined macros and macros introduced by a specific header

echo "#include <stdlib.h>" > foo.h
gcc -dM -E foo.h | sort | less

Reference for -dM is in the cpp man page.