Despite some harsh words about Visual Studio 11, I’m finding that it makes my heart go pitter-pat every time I use it. Why? Because this early release is finally incorporating a decent set of long-awaited C++11 features. In this article I’ll show you how a little thing like a lambda can make a big difference in your coding style.

Microsoft and C++ – We Have History

Microsoft has a cyclic relationship with C++. In the early MFC days, the love was there big time – you had access to most of the system API using C++. However, around the turn of the millennium, Microsoft came under the Rasputin-like influence of Anders Hejlsberg and his beloved offspring, C#. Now it appears that maybe the pendulum is swinging back a bit, and C++ is no longer viewed as an afterthought. Great news.

Although Visual Studio 11 is a developer’s preview, Microsoft is saying that it is production ready – you can use this to create programs that are ready for release. In addition to touting a complete implementation of the C++11 standard library, an impressive list of language features have been turned on as well. (N.B. the path ahead is still long and arduous.)

Modern C++

Before even using Visual Studio C++ 11 to test a single line of code, I really appreciated reading Welcome Back to C++ (Modern C++), a manifesto that includes the following bullet points:

Modern C++ emphasizes:

• Stack-based scope instead of heap or static global scope.
• Auto type inference instead of explicit type names.
• Smart pointers instead of raw pointers.
• std::string and std::wstring types instead of raw char[] arrays.
• Standard template library (STL) containers—for example, vector, list, and map—instead of raw arrays or custom containers.
• STL algorithms instead of manually coded ones.
• Exceptions, to report and handle error conditions.
• Inline lambda functions instead of small functions implemented separately.

I feel that all of these changes result in safer code that is easier to read and maintain, without giving up the type-safety and efficiency that we love so much. Fully implementing these features either leans heavily on C++11 or requires it outright.

A Simple Example Using Naive C++98

It’s interesting to watch the evolution of code from C++ 98 to C++11 and see how it affects your code. You’ll see that the transformation can make it look like you are literally using a new programming language.

In this simple program, I’m taking a Scrabble rack of tiles and whipping through the Scrabble dictionary to find matches. Since it is a one-time call, I’m not storing the words, just doing a quick online comparison. In C++ 98, my code might have looked like this:

void find_matches( std::string rack, const std::string &filename )
{
    std::sort( rack.begin(), rack.end() );
    std::ifstream sowpods( filename.c_str() );
    std::string word;
    while ( sowpods >> word ) {
        std::string sorted = word;
        std::sort( sorted.begin(), sorted.end() );
        if ( sorted == rack )
            std::cout << word << " ";
    }
}

int main(int argc, char* argv[])
{
    find_matches( "etaionsr", "sowpods.txt" );
    return 0;
}

This works properly and I get what looks like correct output:

anoestri arsonite notaries notarise rosinate senorita

Classes Good, Templates Better

As people started to get more comfortable with templates and iterators, algorithms like this were commonly rewritten to take an range of iterators as input – much as the standard library algorithm functions do. This meant changing the function to a template function, but it did make it a lot more flexible. I could now call the function to operate on data from a file, just as before, but I can also now use any other container, or even an array as input:

template<typename ITERATOR>
void find_matches( std::string rack, ITERATOR ii, ITERATOR jj )
{
    std::sort( rack.begin(), rack.end() );
    for ( ; ii != jj ; ii++ ) {
        std::string sorted = *ii;
        std::sort( sorted.begin(), sorted.end() );
        if ( sorted == rack )
            std::cout << *ii << " ";
    }
}

int main(int argc, char* argv[])
{
    std::ifstream sowpods( "sowpods.txt" );
    find_matches( "etaionsr",
                  std::istream_iterator<std::string>( sowpods ),
                  std::istream_iterator<std::string>() );
    return 0;
}

More or less the same number of lines of code, but it is now generic.

Of course, just like with OOP, you need to take some care with template programming. Generic programming can’t be beat when it makes sense, but programmers have a particularly strong susceptibility to pro-innovation bias.

Using the Algorithms Library

Again, prodded by changing styles in the C++ world, my next step is to use a standard library algorithm to do the work. We’re told over and over that turning to the algorithms library allows you to use code that has been optimized to the n-th degree by the clever library teams.

In order to make this work, I have to call an algorithm with a predicate functor, seen below as class sorted_not_equal. Note also that I can’t use the logical function for this, which would be copy_if(). Why not? The committee forgot to put it in back in 1998, 2003, and 2005, a mistake that was fortunately remedied in C++11. So I have to use the inverse function, remove_copy_if(), and invert the logical sense of my functor:

class sorted_not_equal {
    std::string str;
public :
    sorted_not_equal( const std::string & test )
    {
        str = test;
        sort( str.begin(), str.end() );
    }
    bool operator()( std::string test )
    {
        sort( test.begin(), test.end() );
        return ( str != test );
    }
};

template<typename INPUT_ITERATOR, typename OUTPUT_ITERATOR>
void find_matches( std::string rack,
                   INPUT_ITERATOR ii,
                   INPUT_ITERATOR jj,
                   OUTPUT_ITERATOR kk )
{
    std::sort( rack.begin(), rack.end() );
    std::remove_copy_if( ii, jj, kk, sorted_not_equal( rack ) );
}

int main(int argc, char* argv[])
{
    std::ifstream sowpods( "sowpods.txt" );
    find_matches( "etaionsr",
                  std::istream_iterator<std::string>( sowpods ),
                  std::istream_iterator<std::string>(),
                  std::ostream_iterator<std::string>(std::cout,  "\n" ) );
    return 0;
}

Functors Not So Hot

So this new approach is supposed to soup up my code by taking advantage of the algorithms that come with the standard library. But if you look around at the code people have been writing for the past 10 years, you’ll find that this style is pretty common in textbooks and magazine articles, but no so much in the real world.

Why not? Well, it’s pretty obvious. The generic algorithms in the library need lots of predicate glue to make them useful, and the work to create these predicates is just a pain. My code is almost twice as long, and the functionality that took two lines of code earlier is now bloated into a complete class definition. It pollutes my namespace, takes up a lot of space, and has to be defined somewhere distant from where it is actually used. Not a win.

This is obviously a problem when you look at the history of Linux, C, and C++. An entire family of technologies and infrastructure was developed with the implicit goal of reducing the number of keystrokes programmers had to enter. (I’m kidding, but only somewhat.) Functors are a step in the wrong direction.

Lambda to the Rescue

So it is with much relief that C++11 delivers lambdas, which allow us to write short sweet predicates exactly where we need them, as shown in this C++11 version of the example:

template<typename INPUT_ITERATOR, typename OUTPUT_ITERATOR>
void find_matches( std::string rack,
                   INPUT_ITERATOR ii,
                   INPUT_ITERATOR jj,
                   OUTPUT_ITERATOR kk )
{
    std::sort( rack.begin(), rack.end() );
    std::copy_if( ii, jj, kk,
                  [&rack](std::string str) ->bool
                  {
                      std::sort( str.begin(), str.end() );
                      return rack == str;
                  }
                );
}

int main(int argc, char* argv[])
{
    std::ifstream sowpods( "sowpods.txt" );
    find_matches( "etaionsr",
                  std::istream_iterator<std::string>( sowpods ),
                  std::istream_iterator<std::string>(),
                  std::ostream_iterator<std::string>(std::cout,  "\n" ) );
	return 0;
}

Yes, I now have to get used to a new syntax for writing lambda functions – I think that was unavoidable. But my lambda function is short, it is quite easy to see exactly what it is doing, and it replaces a gangly and awkward functor class.

Best of all, I use the lambda exactly where I need it – as the predicate parameter to an algorithm used in the standard library. Locality rules.

Using Lambdas

Visual C++ 11 provides a great framework for experimenting with lambdas, as they are supporting the 1.1 definition that was ratified as part of the standard. If you want the gory details, I believe the working group’s proposal has essentially the same wording that went into the standard. For a detailed tutorial, Herb Sutter’s talk can’t be beat.

When I updated my LZW reference code to use the latest C++ features, I abstracted my input and output functions using templates. Data was read and written using the iostreams paradigm, which requires simple classes that implement just a few functions. Would I have been better off using the iterator paradigm instead? The C++ algorithms library favors that method of processing data, and it can be both elegant and powerful. Which of the two paradigms is the right one for data compression?

The Conflict

General purpose data compression routines tend to be used on binary streams of data, either from files or in-memory objects. So what is the best general paradigm for input and output when compressing data?

You might analyze this problem by imagining that you need to write a binary copy routine.

template<class INPUT_ITERATOR, class OUTPUT_ITERATOR>
void bcopy( INPUT_ITERATOR input, INPUT_ITERATOR eof, OUTPUT_ITERATOR output )
{
    while ( input != eof )
        *output++ = *input++;
}

This routine is particularly nice when you are performing a simple copy using pointers to memory – the generated code should be really efficient.

However, the iterator paradigm doesn’t work quite as well when you want to perform a binary copy of data in a file. I can make use of iterators that almost do the job:

 std::ifstream in( "input.txt", std::ios_base::binary );
 std::ofstream out("output.txt", std::ios_base::binary );
 bcopy( std::istream_iterator(in),
        std::istream_iterator(),
	std::ostream_iterator(out) );

But the bad news is that both istream_iterator and ostream_iterator use the insertion and extraction operators, which are really meant for whitespace-delimited textual data, not binary data. The copy routine shown here will not make a binary byte-for-byte copy of the input file.

So when using files, the stream approach seems to be the way to go:

template<class INPUT_STREAM, class OUTPUT_STREAM>
void bcopy( INPUT_STREAM in, OUTPUT_STREAM out )
{
    char c;
    while ( in.get(c) )
        out.put(c);
}

If my files have been opened using the iostream classes, you can use this binary copy function without having to write any glue code – they already support the get and put methods, so this works right out of the box.

My Choice

If I’ve made up my mind that my data compression routine is going to use one of these two paradigms, it means I am going to have to write some glue code. If I choose the iterator-based approach, I need the equivalent of istream_iterator and ostream_iterator for binary files – and these aren’t in the standard library. If I choose the stream-based approach, I need efficient put() and get() members for blocks of memory. In some cases basic_stringstream might do the job, but not in all cases.

After dithering around with various solutions, I tentatively opted for the stream paradigm. I found the implementation for various sources of data to be fairly simple, and the interface is easy to understand. I don’t know if it’s the perfect choice, and I’ll keep experimenting, but for now it works for me. My abstraction of the LZW code still needs a lot of work, so it’s always possible I could rethink this at a later date.

I’d like to hear your thoughts – is there an obvious right answer to this question?

Microsoft has never been a slacker in the C++ department – they’ve always worked hard to provide a top-notch, compliant product. Visual Studio 10 supports their current incarnation, and for the most part it is up to their usual standards. It’s a great development environment, and I am a dedicated user, but I have to give Microsoft a demerit in one area: their C++11 hash containers have some serious performance problems – so much that the Debug versions of the containers may well be unusable in your application.

Background

I first noticed the problem with unordered_map when I was working on the the code for my updated LZW article. I found that when running in the debugger, my program would hang after exiting the compression routine. A little debugging showed that the destructor for my hash table was taking a long time to run. And by a long time, I mean it was approaching an hour!.

This seemed pretty crazy. Destroying a hash table wouldn’t seem to be a complicated task. I decided to see if I could come up with a reasonable benchmark. I wrote a test program that does a simple word frequency count. As a starter data set, I used the first one million white space delimited words in the 2010 CIA factbook, as published by Project Gutenberg. This data set yields 74,208 unique tokens.

I wrote a simple test rig that I used to test the word count program using four different containers:

• unordered_map indexed by std::string
• unordered_map indexed by std::string *
• map indexed by std::string
• map indexed by std::string *

The reason for testing with std::string * was to reduce the cost of copying strings into the hash table as it was filled, and then to reduce the cost of destroying those strings when the table was destroyed.

I ran tests against map expecting to see a pretty big difference in performance. Because map is normally implemented using a balanced binary tree structure, it has O(log(N)) performance on insertions. A sparsely populated hash table can have O(1) performance. By using fairly large data sets, I expected to see a big difference between the two.

I tried to eliminate a few obvious sources for error in my test function – and I used a template function so that I could use the same code on all the different container types:

template<class CONTAINER, class DATA>
void test( const DATA &data, const char *test_name )
{
  std::cout << "Testing container: " << test_name << std::endl;

#ifdef _DEBUG
  const int passes = 2;
#else
  const int passes = 10;
#endif
  double fill_times = 0;
  double delete_times = 0;
  size_t entries;
  for ( int i = 0 ; i < passes ; i++ ) {
    CONTAINER *container = new CONTAINER();
    std::cout << "Filling... " << std::flush;
    clock_t t0 = clock();
    for ( auto ii = data.begin() ; ii != data.end() ; ii++ )
      (*container)[*ii]++;
    double span = double(clock() - t0)/CLOCKS_PER_SEC;
    fill_times += span;
    entries = container->size();
    std::cout << " " << span << " Deleting... " << std::flush;
    t0 = clock();
    delete container;
    span = double(clock() - t0)/CLOCKS_PER_SEC;
    delete_times += span;
    std::cout << span << " " << std::endl;
  }
  std::cout << "Entries: " << entries
            << ", Fill time: " << (fill_times/passes)
            << ", Delete time: " << (delete_times/passes)
            << std::endl;
}

I didn’t go overboard when it came to instrumenting this problem, I just used the timing functions built into the C++ library. On my Windows and Linux test systems, the values of CLOCKS_PER_SEC are both high enough that I’m not worried about granularity issues.

The First Results

I ran my test program in Visual C++ Release mode, using all the standard settings for a console application. For purposes of comparison, I ran the same program using g++ 4.6.1 on the same computer, booted up under Linux. For the set of 1,000,000 tokens, the results are shown below:

There are a few interesting points to take away from these tests:

• Microsoft’s compiler is taking an exceptionally long time to destroy hashed containers – one order of magnitude greater than it took to create it, and two orders of magnitude greater than it takes g++ to do the same task.
• It doesn’t look like constructing and destroying the strings is a big factor. Both compilers have roughly the same performance with both std::string and std::string *. Microsoft’s behavior is counterintuitive, as it takes longer to construct and destroy containers using the pointer.
• The GNU compiler appears to be able to run through this exercise notably faster.

The time it takes to destroy the table is a concern – having a C++ program hang for over 3 seconds to destroy a modestly large data structure is a serious concern – particularly when the same task completes in a few milliseconds with g++.

The Pathological Results

These concerns are nothing compared to what I see when running in debug mode. Setting my Visual Studio project to Debug mode, then running the same test, yields the results shown here:

Those numbers are hard to believe. Destroying a hash table takes one millisecond when using g++. In VC++ 10, it takes almost ten minutes!

Worse, we suddenly see that hashed containers are slower than the containers built on red-black trees. Again, this just doesn’t make sense.

The big problem with these numbers is that it means the debug mode of the compiler is effectively unusable for a lot of tasks. Regardless of how much testing it does, when it is this slow, it is just not useful.

A Workaround

I didn’t invest the time to try debugging Microsoft’s library, so I don’t really know where the time is being spent. I did try a few things to speed things up, and I found one technique that helps a lot. Before including any Microsoft header files, try entering this single line in your source:

#define ITERATOR_DEBUG_LEVEL 0

With this definition in place, the delete times return to ball park of the times seen when running in release mode. Of course, you give up some debugging. I believe that an explanation of what this macro does might be found here.

In the final analysis, I think Microsoft has some serious work to to do here. The performance of their hashed containers, and to some lesser extent, the pre-C++11 associative containers, needs some serious examination. If the library is going to run this much slower than the competition, I need a good explanation why.

Source

HashTest.cpp

For most of the thirty-some years that I’ve been paid to program, I’ve worked with Assembly language, C, or C++. There have been some diversions to Java, which is pretty painless for a C++ programmer, and always the odd short job or two in anything from Perl to Haskell.

But for the last three months or so I’ve been working full time on a short-timeline project that is using PHP as a general-purpose scripting language. So I’ve been working in this unfamiliar language at a fairly frantic pace, and I have to say that PHP, like Perl, is a strange place for a C programmer.

The Normal Problems

Any tutorial in a scripting language like PHP or Perl always makes a big deal out of Type Juggling, meaning that the type of a variable is inferred from the context in which it is used. In a simple example, a string literal can behave like an integer without any fuss, and vice versa:

{appliance:~} php -r '
> $a = 25;
> printf( strlen($a) . " " );
> printf( 15 + $a );
> printf( "\n" );'
2 40

Maybe it’s just me, but this aspect of PHP doesn’t seem to cause too much trouble. It may be because C and C++ already do implicit data conversions in a lot of situations; PHP just turns the dial up a notch or two.

What does give me a lot of trouble (and this is not exactly news) is that PHP doesn’t require declaration of variables, being perfectly content to create them on first use.

There’s of course nothing wrong with this, but as a C programmer, I have not developed any defensive skills in this area. A PHP programmer is much less likely to enter the following code, because he or she will be a lot more careful about entering variable names:

{appliance:~} php -r '
>$first = "Mark";
>$last = "Nelson";
>$name = $fist . " " . $last;
>print( "$name\n" );'
 Nelson

The good news on variable declaration is that I can jack up PHP’s error reporting so that these mistakes are flagged at runtime. Increased error reporting will also catch another error that plagues C++ programmers:

{appliance:~}php -r '
>$version = "1.2.3";
>print( version . "\n" );'
version

Yes, string literals are just as good without quotes as they are with.

This Week’s Puzzle

After a few months I start feeling a bit of a false sense of confidence in my understanding of the language, and naturally, I got some serious comeuppance this week, demonstrated by the following code:

{cslinux1:~} php -r '
>class base {
>  private $secret="42";
>  public function print_secret() {
>    printf( "$this->secret\n" );
>  }
>};
>class derived extends base {};
>$foo = new derived();
>$foo->secret="43";
>printf( $foo->secret . " " );
>$foo->print_secret();'
43 42

As a C++ programmer, I fully expect the attempt to modify $foo->secret to generate a runtime error. $secret is a private member, and as such I shouldn’t have access to it via the derived class. But the runtime happily makes the assignment.

When I actually print the value, I see that the base class and derived class print different values for $secret. What has happened here is that the assignment actually created a new data member in the derived class with the name $secret.

So what exactly is happening here? The reason for the difference in behavior is that PHP uses the private and protected modifiers to define not just accessiblity, but visibility as well. The $secret member in the base class is not only inaccessible outside of the base class, it is invisible as well.

That invisibility is the key to the behavior. When I try to assign a value to $this->secret, the runtime looks to see if the object of class derived already has a member called $secret. Since the member in the base class is invisible, the conclusion is that there is no member with that name, and as a result, a new variable is created on the fly: derived::$secret.

This can’t be fixed by simply amping up the error level, unfortunately, and it definitely is a problem for C++ or Java programmers. It doesn’t just cause me a lot of confusion, it just seems wrong.

Of course, there is plenty to like about developing in PHP, and it is definitely the right tool for this project. The speed bumps just make the process a lot more interesting. The concept of visibility set me back a few hours, but I’m ready to move on – until I hit the next bit conceptual roadblock.

Back in the day the Sapir-Whorf hypothesis was all the rage in the study of linguistics. With apologies to those who actually work in the field, I’ll crudely summarize it as the idea that the language you speak both constrains and influences how you think. The idea says that if your language only has one word for snow, for example, you will actually have a hard time seeing any difference between light powder and crunchy ice pack.

Sapir-Whorf was seen as completely discredited back when I learned about it, and while Linguistic Relativity has enjoyed a slight comeback with a weakly restated set of hypotheses, it seems fairly certain that human thought is by no means confined to a cage built out of vocabulary and grammar.

Our field has long had its own Sapir in E.W. Dijkstra, whoe spelled it out with money quotes like these:

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.
The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offence.

Closer to today, we have the famous rant against C++ from Linus Torvalds, who feels that a programmer who uses C++ is going to wreck any project he or she touches:

I’ve come to the conclusion that any programmer that would prefer the project to be in C++ over C is likely a programmer that I really *would* prefer to piss off, so that he doesn’t come and screw up any project I’m involved with.

C++ leads to really really bad design choices.

Dogmatism à la Torvalds

At the large corporate entity that pays my bills, we have a slogan on the back of our badges: No Technology Religion. It might not be easy to live up to this, but yes, I try.

To me, this admonishment means two things:

1. Try to objectively choose the best tool for the job
2. Don’t let your preferences in tools dictate the way the job should be done

Linus is clearly saying in his rant that anyone who programs in C++ is guilty of breaking both of these rules.

I disagree. I think there are times when C++ is clearly the right tool for the job, and that you can arrive at this conclusion fairly objectively. I think Linus is clearly fogged in by his particular Technology Religion.

A Simple Example

As the final assignment for my C/C++ programming class last semester, I asked my students to implement a simple token counting program in C. The goal was to reproduce the behavior given by this C++ fragment:

map<string,int> counts;
string s;
while ( cin >> s )
    counts[s]++;
for ( auto ii = counts.begin() ; ii != counts.end() ; ii++ )
    cout << ii->second << " : " << ii->first << endl;

This particular program highlights a number of features of C++ that are not present in C:

• The versatile replacement for C arrays, vector<T>.
• The string class.
• Safe input using iostreams.
• Associative arrays as part of the standard library.

This program is quite easy to write in C++, and is basically complete. One could flesh it out a bit with of error handling on the input stream, but that’s really not even necessary.

Do it in C

Rewriting this in C is a straightforward task with one big speed bump: the lack of any sort of associative array in the library. There are a number of ways to deal with this – I chose the following strategy:

• Read all the tokens into an array.
• Upon completion, sort the array
• Once the array is sorted, walk through it to get the count for each token.

While this algorithm uses more space than the C++ program, it probably takes up the same amount of time, assuming you don’t bump into one of the pathologically bad cases of qsort().

Since I asserted that this program is easier to write in C++ than C, it behooves me to give a list of reasons why.

• C I/O deficiencies. Reading strings in C is considerably more difficult due to the fact that the C I/O library doesn’t have a standard way to read strings of unbounded length. (Compiler-specific extensions can be used, but that raises other problems.) Your input code has to do a lot of checking for error cases, or you have to build your own string input functions.
• Memory management of C arrays is a very manual task. I have to allocate the original space for my array, take care that I reallocate if I exceed its length, and free the space when I am done.
• Memory management of C strings has exactly the same probelms.
• Sorting the array of strings is just a tiny bit more inconvenient with qsort(), and qsort() doesn’t give me the performance guarantees of the sort() function in the C++ library.

The C version of my function has more lines of code, and more bookkeeping tasks that need to be done manually. There are more opportunities to make mistakes.

A final reason I like the C++ version of the program better is that it lends itself well to working with other types. Any type that has insertion and extraction operators, and a comparison operator, can use that same code with just one declaration change. Turning it into a function template accomplishes the same thing with no code changes needed at all.

Some of My Best Friends are C Programmers

So am I a language bigot for preferring the C++ version of this code?

I hope not. For one thing, I can see that the C version of the program has some nice advantages:

• You can write this program using POSIX system calls for almost everything except memory allocation and sorting, resulting in an extremely small footprint.
• The C version of the program will be faster due to the use of low-level I/O. C++ iostreams get better all the time, but their layered approach will always be at a disadvantage when it comes to efficiency.

So for a program like this, the choice of language really comes down to context. If you believe in the 80/20 rule, you might think this code should be written in C++ if it is outside of the expensive core part of your program. With fewer lines of code you have fewer chances for error, and efficiency is probably not a big consideration.

If this is in a critical section of code that is executed frequently, you might decide C is your best choice. Make sure to put a little extra time into code review to ensure that the code is free of memory leaks and pointer errors, and you are in business.

Sic Temper Linus

So how does Linus’ rant hold up when looking at the C++ code shown at the top of the post? I would venture a guess that given the assignment, any decent C++ programmer would produce code similar to this. Linus says:

You invariably start using the “nice” library features of the language like STL and Boost and other total and utter crap, that may “help” you program, but causes infinite amounts of pain when they don’t work (and anybody who tells me that STL and especially Boost are stable and portable is just so full of BS that it’s not even funny

In this program I make good use of the standard library components that were once part of the STL. Having been part of the standard for over a decade, they work really well and have no portability or correctness issues in any compiler I am aware of. Saying that components like map and vector are problematic is just wrong.

inefficient abstracted programming models where two years down the road you notice that some abstraction wasn’t very efficient, but now all your code depends on all the nice object models around it, and you cannot fix it without rewriting your app.

Well, despite the fact that C++ has made it impossible for me to do so, I managed to write the program without using any abstractions – no new classes, no interfaces. Basically just straight procedural C code that happens to employ a few useful classes.

And at least with the people I work with, I think this is the rule rather than the exception.

In other words, the only way to do good, efficient, and system-level and portable C++ ends up to limit yourself to all the things that are basically available in C.

When C has container classes, a string class, typesafe I/O, and the programmer’s gift from the gods, RAII, then this statement will be true. For now, it is bollocks.

Before modern C++ was available, I probably would have stuck with a simple pipeline to accomplish this task:

tr [:blank:] '\n'  | grep -v "^$" | sort | uniq -c

The fact that I can do the same thing just as easily in a compiled language gives me some flexiblity. I think I can appreciate that fact without being a bigot.

Can you?

A couple of weeks back I took issue with a statement in the New York Times Magazine:

If you have a million coin flips, it’s almost certain that somewhere in those coin flips there will be 20 heads in a row.

My blog post took the Times to task for printing what seemed to be an obviously bogus statement. Since a run of 20 heads is roughly a one-in-a-million occurence, a basic feel for probability should tell you that trying to do this a million times is not going to be a certainty – fairly far from it.

Naturally, I thought to back up my argument with some hard facts, and I came up with a calculation that showed that the chances of this happening were actually around 60%. Unfortunately, I made a pretty basic mistake in calculating the probability, as was demonstrated to me by correspondent Andy Langowitz. My intuition about the likelihood of success was on the money, but my estimate was a bit high.

The story of how I corrected my error in order to get the correct number, around 37.9%, is a good lesson in careful examination of probability rules, with a small detour into the land of the bignum.

Independent Probabilities

When trying to develop a formula like this one, it is often easier to work from the inverse. Instead of calculating how likely it is for 20 heads to occur at any point in the sequence of a million tosses, I thought to instead calculate the probability that it wouldn’t happen. We can then take that number and subtract it from one to get the desired result.

The chance of n heads in a row occurring is 1/2ⁿ, so the inverse probability is (2ⁿ-1)/2ⁿ. If we multiply that probability once for all 999,981 possible occurences of a streak of 20 heads, it seemed to me that I would be in business. Doing this is a simple enough calculation, and the result was the 60% figure. That figure felt like it was in the ballpark to me, and I left it that.

Mr. Langowitz, however, was smart enough to actually test the theory on a smaller set of numbers. Let’s apply this theory to find out how likely we are to throw two heads in a row in four tries.

The chance of two heads in a row is 1/4, so my formula would give a result of 1 – (3/4)³, or a result of 37/64 – a little better than 50% chance of it occuring. But probability is nothing but the art of enumerating and counting, and we can do just that to check the result. The 16 equally possible outcome of four tosses are:

HHHH HHHT HHTH HHTT
HTHH HTHT HTTH HTTT
THHH THHT THTH THTT
TTHH TTHT TTTH TTTT

Look through those outcomes and see how many have two consecutive heads. It turns out to be exactly 8, meaning that my calculation of 37/64 is just flat wrong.

Locating the Mistake

My mistake is a pretty common one among probability neophytes. I assumed that probability at each step in the sequence was identical, because the sequences were completely independent. It’s easy enough to think that if you don’t examine problem carefully.

What actually happens is that when we examine the possibility of an unsuccessful run of heads at toss i, we slightly bias the outcome at toss i+1. A demonstration will show exactly how this happens.

For the sample problem of a sequence of two heads out of four tosses, we can first examine the chance of a negative outcome starting at toss 1. There are just four possible outcomes that have two tosses starting at position 1:

HH HT TH TT

And only one of these tosses yielded two heads in a row, so the probability of not seeing two heads after two tosses is 3/4.

But now when we look at the sequence of tosses starting at position two, we have to throw out the outcomes where we had two heads at toss one – we’ve already seen two heads, so we can’t continue flipping coins in those outcomes. So our universe of possible outcomes is now a bit different:

HTH HTT
THH THT
TTH TTT

Instead of eight outcomes, we have six. And if we look at the first toss seen in position two, instead of having an even distribution of heads and tails, you can see that sample is biased: only two have a head in position two, while four have tails. So the chances of not seeing two heads starting at position two increases to 5/6. Note that this change in probability occurs because we have selected only those outcomes without a streak of two heads at position one.

Likewise, when we look at the possible outcomes for streaks starting at position three, we get a different probability again. Because we have to throw out one sequence in the previous test, the universe of possible outcomes is now limited to:

HTHH HTHT
HTTH HTTT
THTH THTT
TTHH TTHT
TTTH TTTT

So now we have just ten possible outcomes, and two of those will produce the desired outcome, meaning the probability has changed to 4/5.

So what is the probability of all three possible positions not containing a streak? That would be (3/4)*(5/6)*(4/5) which reduces nicely to 1/2, the correct answer.

Finding the General Solution

So let’s generalize the question at hand: what is the probability of seeing k consecutive heads when a fair coin is tossed n times? The previous section showed that we can work it out by hand for small numbers of tosses, but it should be clear that if are going to toss a coin a million times, the universe of possible outcomes is going to get unwieldy. We need a general description of the problem in order to solve it for any values of k and n.

For many probability problems, finding a solution is simply a way of figuring out how to count things, and coin tosses indeed appear to be just such a problem. Let’s try to see if we can count the number of times a sequence of k heads will appear at a given toss.

To start to work out the solution to the problem, I will set k to a value of three – in other words, we will be trying to see what is the probability of seeing three consecutive heads is at toss i, given that there have not been three heads at an earlier toss. To calculate the probability, we need to know two things. First, we need to know all the possible outcomes in our universe of samples at toss i. In the previous section, with a value of k=2, we saw the the number of outcomes for tosses 1, 2, 3, and 4, was 2, 4, 6, and 10.

After determining the number of outcomes, we then need to determine how many of those outcomes were successes. If we defined success as being the number of outcomes in which a k heads in a row appear at position i, the values from the previous section would have been 0, 1, 1, 2.

Counting the Successes

I’ll start with the more difficult problem: counting the number of times k heads appear at toss i. To start with, we have the degenerate cases where i is less than k. In all of those tosses, we know that the number of successful outcomes is going to be zero, because there have not been enough tosses to achieve success yet.

If we work our way forwards with the example of k=3, our first four tosses end up giving us three sets of outcomes:

H T

HHTH HHTT
HTHH HTHT
HTTH HTTT
THHH THHT
THTH THTT
TTHH TTHT
TTTH TTTT

Note that when we get to toss 3, there is just one successful outcome. Likewise, in toss 4, there is just one successful outcome.

It may not be immediately obvious, but we can in fact always tell how many successes we will achieve at toss i+k after we have enumerated all the possible outcomes at toss i. The number is defined as the number of outcomes at toss i that end in a tail.

The logic behind this is straightforward: in order to have a success at position i+k, we need to generate a sequence of k heads, starting at toss i+1. If we have an outcome that currently ends in a tail, it will generate 2^k outcomes in the next k tosses, and precisely one and only one of these will have k consecutive heads. None of these outcomes will result in an sequence of k heads before toss i+k, because they currently terminate in a tail, so all of the outcomes generated from that outcome at position i will be included in the outcomes seen at toss i+k.

Likewise, none of the outcomes at position i that currently end in a head are going to be able to contribute to a success at toss i+k. Any streak of k heads that follows a terminating head at toss i will result in a run of k heads before we reach toss i+k.

Looking at our outcomes for k=3, we can see that at toss 1 we have one outcome ending in a tail, so at toss 4 we will have one success. At toss 2 we have two outcomes ending in a tail, so at toss five we will have two successes. And we have the special case of toss 0 – we have one sequence starting at toss 0 that generates a sequence of k heads at toss k. Although there were no tails tossed at position 0, any sequence that starts there doesn’t have any preceding heads tosses either, so it is as if there was a single outcome at toss 0 with a value of tails.

So each toss that ends in a tail acts as the root of a successful outcome at a future position. This is good information, but in order to turn this in to a formula we need to be able to compute the number of outcomes ending in tails at toss i – we don’t want to have to enumerate all the outcomes in order to get there. I’ll refer to these special outcomes as anchors, as they form the anchor of a future outcome.

Counting the Anchors

The number of anchor outcomes at each position starts out as a nice number while i is less than or equal to k: 2^i. But after toss k, successful outcomes start being removed from the sample set and the formula no longer holds. For k=3, the anchor count starting at toss1 is: 1, 2, 4, 7, 13, 24.

It turns out that the anchor at position i does more than just generate a success at toss i+k. It is also responsible for generating new anchor outcomes at tosses i+1, i+2, …, i+k-1.

Looking at an example for k=3 should clarify this. Our lone anchor outcome at toss 1 is the sequence T. We know that this anchor will create a new successful outcome at toss 4: THHH. But it also creates new anchors at all intermediate tosses: TT, THT, and THHT.

This observation holds true for the generation of all new anchors, and with a little work we can turn this into a usable recurrence. If each anchor at toss i is going to create a new anchor at tosses i+1 through i+k-1, we can calculate the number of anchors at toss i using this formula:

anchors(i)=anchors(i-1) + anchors(i-2).

Yes, that is the Fibonacci sequence. For values of k higher than 2, the formula is the n-step Fibonacci sequence. The well-known Fibonacci sequence recursively adds the previous two values to get the current value. The n-step Fibonacci sequence adds the previous n values in order to get the current value. (For the rest of this article, the n-step Fibonacci number will be referred to as fib_n(i).)

In the standard Fibonacci recurrence definition, we define a base value of fib(1) = 1, and fib(x) = 0 for all x less than 1. Our anchor count is skewed by one, since our base value at toss 0 is 1. As a result, the the anchor count at toss i is equal to fib_k(i+1). And from our observation of the link between anchors and successful outcomes, we can then observe that the number of successful outcomes at toss i is equal to fib_k(i+1-k).

Counting the Outcomes

Knowing the number of successful outcomes at toss i only gets us halfway to knowing the actual probability of seeing a sequence of k heads at that point. To get the full probability, we need to know the number of outcomes as well.

Fortunately, this calculation is nearly trivial. In the previous section we saw that the number of anchors at toss i is equal to fib_k(i+1). Each anchor at toss 1 and greater is simply a sequence of tosses that ends in a tail. And for every sequence ending in a tail, there is a corresponding sequence that is identical except for one key change: it ends in a head instead of a tail. So the number of outcomes at toss i is twice the number of anchors, or 2*fib_k(i+1).

The Absence of a Streak

Now that we can compute the probability of seeing a streak of k heads at toss i, we need to do just a bit more work to see the what the odds are of seeing that streak at any time in a sequence of n tosses. To get there, we need one more piece of information: the probability of not seeing a streak of k heads at toss i – in other words, the chances of a negative outcome.

It’s pretty easy to calculate that number – simply subtract the number of successful outcomes from the total number of outcomes. The sample sequences for values of k corresponding to 2, 3, and 4 are shown here:

In the third line of the figure I simply break out the value first term of the equation into its two component parts. Now, in the next line, I expand the value of fib_k(i+1) into its component parts, using the definition of the Fibonacci sequence. I keep that grouped in parentheses to make it clear that the new terms are the result of the expansion. Just as an example, if we were doing this expansion for a value of k equal to 3, we would expand fib₃(i+1) into the sum of fib₃(i), fib₃(i-1), and fib₃(i-2), using the identity of the n-step Fibonacci sequence.

Note that in line 3, the last term of the expanded Fibonacci sequence in parentheses is canceled by the last term overall, which was the number of successful outcomes. When we remove those two elements from the equation, line 5 has simplified into the identity of fib_k(i+2). So we now have a reasonable number we can use to determine the count of failures at a given toss.

Put it All Together

With all these formulas in hand, we have the tools to determine the final probability we’ve been working towards: the probability that a sequence of k heads will appear in a sequence of n tosses. To do this, we calculate the cumulative probability that the event does not occur, and subtract that value from 1. The result will be the answer to the question.

The first equation below shows the general approach we take to this problem – multiplying the probability of failure at each toss. Once we plug in the actual formula for the failure count at each point, and the formula for the total number of outcomes, we see that most of the terms cancel out. We are left with fib_k(n+2) on top, and 2ⁿ on the bottom. And that is the final formula that provides an answer to the question.

Unfortunately, my calculator is not really up to this. Even a desktop calculator that could handle arbitrary precision math won’t normally have a fib_k button.

If I had a copy of Mathematica, and knew how to use it, I think I could solve this with just a few lines of input. But I don’t, so I coded up a short solution in Java.

Java has two classes that enable me to solve this problem in relatively easy fashion: java.math.BigInteger and java.math.BigDecimal. BigInteger performs simple integer calculations with arbitrary precision, and BigDecimal supports floating point math.

My simple app, contained in Flipper.java, uses BigInteger to calculate fib_k(n+2) and 2ⁿ, then uses BigDecimal to divide the two numbers and subtract the result from 1. Despite fact that the two intermediate results are over 300,000 digits each, the program ran in a very reasonable amount of time, less than an hour. (Optimization of this program would be a very interesting exercise.)

The output of the program is shown here:

fib(20,1000002) = 614579313398524367786474463596 (301000 digits elided)
2^1000000 = 99006562292958982506979236163 (301000 digits elided)
Div = 0.379253961388950068663971868 (999980 digits elided)

So at last, we know the correct result. If you flip a coin a million times, you have a 38% chance of seeing 20 heads in a row. A long way from the certainty claimed by the New York Times, and a bit off from my initial 60% value.

Postscript

Working out the details of this problem was a very enjoyable piece of math. When I first started in on the problem, I hoped I would be able to find a reference that simply told me how to calculate the number, but I had no luck. As I worked through the problem, I ran into the existence of the n-step Fibonacci numbers, which I had never heard of. Once I found the reference to them on the Wolfram Alpha page (linked above), I saw that the page had a terse note describing this problem, but with no details. With luck the next person trying to understand this problem will be able to make sense of it by reading this page.

Exceptions are a necessary part of the C++ language, but for most programmers they are worse than worthless - they are unusable. When exceptions were first added to the language back in the days before standardization, they were seen as a brilliant improvement over the hideous setjmp/longjmp facility from ANSI C. Because exceptions unwind the stack when they are thrown, they call destructors and clean up all of your untidy messes as they go.

Tom Cargill

It didn't take long for somebody to notice that the emperor had no clothes. In 1994, Tom Cargill published a prescient article in the late C++ Report. Tom pointed out in detail just how difficult it was to write code that actually held up when exceptions were used. Scott Meyers flags this as one of his five Most Important C++ Non-Book Publications...Ever. (Incidentally, Tom is responsible for a piece of wisdom that sounds like a an offhand quip: the ninety-ninety rule. Any project manager who doesn't understand this rule all the way into his or her bones needs to quit. Now.)

The intervening years have not been kind to exceptions. The problems Tom foresaw in 1994 were real, and while they can be managed, writing exception-safe code is still fraught with peril. As a result, exceptions are basically unused by by most C++ programmers. After all, we have to deal with tough problems like writing safe multithreaded apps, understanding template metaprogramming, and dealing with the language's high-maintenance memory management. The last thing we need is to make use of a feature, that while useful, is basically impossible to use correctly.

Naturally, after this rather verbose introduction, you have probably guessed that I'm using this article to tell you about a simple little utility class I use to help me throw informative, easy to build exceptions in my C++ programs. Yes, exceptions are problematic, but there's one place I can use them with impunity: to cause a fatal error that aborts my program.

Fatal Errors Considered Exceptional

If you analyze Tom Cargill's article, or look into any of the additional work that has been done on exceptions up until today, you'll see that most of the nasty side effects are only a problem for a program that tries to keep running in a predictable and safe fashion while throwing and catching exceptions.

The side effects of code that isn't quite exception-safe include memory leaks, partially constructed objects, and invalid containers. All bad things. But the one place they usually don't matter to me is when things have degraded in my program to the point where I'm ready to throw in the towel anyway. That point is when I've encountered a fatal error and am aborting the program. At that point, I throw an exception, which generally percolates all the way up to main(), where it is caught, and error message is printed, and the program exits.

Handling fatal errors this way makes for much cleaner code. I generally assume that every method or function called in my program succeeds, and don't worry about creating special returns with error codes. I blithely march through call after call without checking results, secure in the knowledge that my code is working properly. I know this is the case, because if something went wrong, an exception would have been thrown, and my program would abort.

This error handling strategy is implemented entirely in main(), which generally follows this form in one of my programs:

There are a couple of interesting points to note about the error handler you see here. First, because I am catching std::exception, I will catch any errors in the standard library, such as bad_alloc, logic_error, runtime_error, etc. It's a good idea to have a top-level try/catch block for these items anyway, so my use of exceptions for fatal errors forces good hygiene practices.

Second, I'm using the what() method to give human-readable feedback on exactly what happened to send my program off the rails. This method is defined as virtual for the base class std::exception, so all derived classes will support it. There's no requirement that they populate this string with great prose, but we can at least expect implementors to provide something informative here.

My Exception Class

When I first started using exceptions for fatal errors, I kept things simple by just throwing instances of std::runtime_error when I wanted to abort my program. I could pass the constructor of this object a descriptive string as it was constructed, and that same string would be returned when what() was called in the exception handler. Typical usage might look like this:

This code ensures that if the user leaves off the necessary command line argument, the catch() block will print out a usage statement and return an error value to the invoking shell, which is just what we want. I manage to handle faulty input with just one line of code, don't have to set any error variables, no if/then/break clause to print an error and then exit. It's tidy and works well.

Of course, it wasn't long until I ran into situations where I wanted to provide the user a little more information with a fatal error - information that had to be formatted at runtime. I ended up writing a lot of code that looked like this:

This more or less worked, but it added a lot of lines to the code, and as a rule, the fewer lines, the less typing, the more I like it. In this case I have to construct a separate std::stringstream object to hold the formatted error message (or a character buffer if I choose to go old school and use vnsprintf() , a second line to do the formatting, and a third line to construct and throw the exception.

In addition to writing three lines instead of one, I now have to enclose the whole thing in brackets, because the clause following the if statement is multiple lines, which makes the whole thing require five lines of code instead of one!

A Better Way

I needed to find a better way to do this. I first dabbled with a class derived from std::exception that used vnsprintf() to format arguments, C-style. But there were a few disadvantages to this, the primary one being that it made it hard to take advantage of classes that have their own overrides to stream classes. In other words, if I've bothered to create a stream override so I can print the contents of class foo, it's easy to write that to a stream, but not so easy to insert it into a buffer being formatted with vnsprintf().

So I determined that I wanted to have a class that that let me rewrite the code shown above so that it all fits on one line, like this:

With this routine I'd just need to override the definition of what() and I would be in business.

But there was a fatal flaw in this problem: the absence of a copy constructor for std::stringstream. I had assumed that the copy constructor wouldn't be needed, as I was catching the exception object by reference. My thought was that the compiler would create a temporary object, pass it along to my catch clause, and all would be well.

No such luck. In this case, the compiler has the right to make a copy, and even if it doesn't make a copy, it has the right to insist on a copy constructor even if it is only considering the possibility of making a copy. Strike one.

This One Works

So I needed to make a version of my fatal_error class that doesn't try to call the copy constructor for std::stringstream. This is accomplished easily enough by using composition instead of inheritance:

This solves one problem, but creates another. Because fatal_error is no longer derived from std::stringstream, I've lost the overloaded operators that insert text into the object before it is thrown. In other words, my wished-for code shown above won't compile.

In this case, the solution is simple, I just add a template method to the class:

Now the template class takes care of routing the stream insertion output into the std::stringstream member I've incorporated into my class. Note also that the overloaded insertion operator returns a reference to the fatal_error object, allowing me to chain insertions.

Almost Done

There only two more issues to deal with in order for this class to be ready for use. First, I need to deal with the possibility of a copy constructor being called as this exception is thrown. Because I can't copy the std::stringstream object from the old object to the new, I have to save off its contents into a mutable std::string member called mWhat. The result looks like this:

Finally, I need a good version of what() that conforms to what is expected for std::exception. This is nice and easy:

Wrapped Up

With that, I have an exception that I can use to easily generate fatal error exceptions with as much data as I want. I only have to include a single header file in any code that uses the class, and I can create and throw the exception in a single line of code, which adheres to the C ideologoy of minimal typing.

Life is good.

Complete source code from the single file, fatal_error.h, is given below. This code should work in g++ 3.x code, and Visual Studio 2003 and 2005 programs. If you run into problems with different versions of various compilers, please let me know!

Dr. Dobb's Portal November, 2007 Article on DDJ site

As I've confessed in the past, I'm a sucker for word puzzles. My recent post on a Will Shortz puzzle from NPR Morning Edition ended up provoking a surprising amount of comment, much of it in the vein of Watch me solve it better, faster, and with more style using language XXX.

I certainly enjoyed watching other people solve the problem, and found their solutions instructive. As the XP crowd has figured out, we programmers spend too much time working on our own problems and not enough time watching how other people work. There's a lot to learn, both good and bad, from getting a peek inside another person's head.

Out of the Blue

Which brings me to the puzzle at the center of this article.

Beth Katz

In what at first seemed to be an incident completely unrelated to word play, I had a pleasant email exchange with Beth Katz, who was teaching a Data Structures class at Millersville University. I happened to look at Beth's current homework assignment for her class, and you can imagine my reaction when I saw the problem she had posted for her class:

We define word reduction as removing a single letter from a word while leaving the remaining letters in their original order so that the resulting sequence of characters is also a word. A good word can be reduced step-by-step until all that is left is a or i. Your program will answer the question: what is the biggest word in the given dictionary that can be reduced?

Beth gave a short example of a good word: planets:

planets
plants
pants
pant
ant
an
a

As you can see, you can remove one letter at a time, and each time you are left with a valid word one character shorter.

This makes for an interesting problem indeed. I've read that the average English speaker has a vocabulary of perhaps 15,000 to 20,000 words, but many reasonable word lists have upwards of 100,000 English words. How many of these words qualify as good words?

As I discussed in the previous word puzzle article, the highly evolved pattern matching facility in the human mind is often pretty good at solving these problems, and I think this is the case (in a limited way) for this particular problem. If I give you a word (like planets) above, I think you'd be able to find a possible reduction path quickly, subject to the limitations of your own vocabulary.

But the human mind is not so good at certain variations on the same problems. Asking you for the biggest word that fits this pattern presents you with an almost impossible task. Basically, it requires you to be able to iterate through the words you know, ordered by length, and test each one. Unless you are subject to some pretty incredible flashes of insight, I think you're going to need a computer for this.

Going Bottom Up

Maybe the feeling isn't universal among programmers, but when I look at a problem, my first instinct is usually to try a top-down approach. For this problem, a top-down approach would mean identifying the longest words in the dictionary, then attempting to decompose them into successively shorter words.

This approach will work, but a little mental analysis shows that it might be a little resource heavy. Imagine that you are decomposing a 10 letter word by taking away one letter at a time. In the worst case, you might find all 9 shorter words in the dictionary, and then you could find all 720 8 letter words, and so on. Although in the general case you might only find one or two matches, particularly at the long lengths, even the potential for factorial growth leaves some room for concern.

So I took a shot at a bottom-up approach instead. It isn't usually my first choice, but I think you'll see that in this case it yields a much more satisfactory and efficient solution to the problem.

The Inner Loop

For this program to succeed, it must terminate its processing with a container that holds the longest good words in the dictionary. For this particular problem, my choice of container is the C++ hash_set, which is non-standard but universally implemented.

My bottom-up approach means that I will fill in the hash_set container for words of length 1 first, then words of length 2, and so on. For reasons of efficiency, it works out better if I keep a separate hash_set for each word length, so the hash_set objects are actually stored in a map that is indexed on the word size:

std::map<size_t,hash_set<std::string> > good_words

To fill in the hash for size i, I need a loop that iterates over all words of size i, removing one character at a time and then testing to see if the result is a good word of size i-1. If it is, I add it to good_words[ i ]. When I'm done iterating over all words of that size, I have a hash_set that contains all good words of that size, and I can move up to the next larger size.

So if we're testing words of size i the innermost part of the loop will look like this:

In this inner part of the loop, we repeatedly remove a character from the word, testing to see if the resulting word appears in the list of words saved in the previous size hash set. If a match occurs, the word is inserted into the hash set for the current word size, and the loop breaks.

This bottom-up approach seems like the ideal way to build my lists of good words for a couple of reasons. First, I don't go to the expense of adding words to the hash sets unless they are good words. Second, determining that a given word is a good word only requires a test against the words at level i-1; I don't have to test for a complete path down to 'a' or 'i'.

Building The Input Data

In the previous section I mentioned that the innermost loop was going to be called as I iterated over all the dictionary words of a given size. So how do I get all the words of a given size?

The first thing that might occur to you is that you could read in all the words from the dictionary, then sort them by size. This approach would work, but given that the standard sort routines available to you in the C++ library are all going to work in O(n·lgn) time, it might get kind of expensive as the dictionary grows to hundreds of thousands of words.

The good news is that with this data set we're in position to take advantage of a linear sort. Yes, we can sort data and do substantially better than O(n·lgn) when we know that the data to be sorted is constrained to a small set of values.

In this case, I just create one linked list for each word size, and as I read the input file, I add each input word to the front of the appropriate linked list. This would be a true linear algorithm if I constrained the input size to a fixed number, say 25, but for convenience I actually store the lists in a map that looks like this:

std::map<size_t,std::list<std::string> > words_by_length

As a result the input code runs in close to linear time. The actual loop that reads in the data is nice and simple:

Once all the words are read in, I can access the list of words of a given size with a simple map lookup: words_by_length[ i ].

Word Lists

One final detail before I can compile my library - I need some lists of words! Lists of words are not hard to come by, although coming up with a suitable one for this exercise might take some work.

One of the first places to look is is on your local Linux system. My system has a list of words in /usr/share/dict/words, which is used by the spell checker application, and possibly by the password app. One one of my systems, this dictionary has a whopping 483,524 words, which means it is packed with obscure words. Just as an example, a typical 12-character good word and its derivation found using /usr/share/dict/words yields this head-scratching sequence:

abranchiate
branchiate
branchiae
branchia
branchi
branch
ranch
rach
ach
ch
h

Probably the first thing you want to do with that file is go through and remove all the single letter words except 'a' and 'i', but even so, you're going to be boggled by some of what you see.

Another good alternative are the collection of word lists distributed on Project Gutenberg as the Moby Word List. This includes a wide variety of lists of various sizes.

Beth Katz had several good dictionaries listed along with her homework assignment, including a short one called kids.dict that is nice and short, making it good for debugging runs.

Finally, Google searches for "word lists" will turn up many other good choices.

Wrapping it up

Once I had the bottom-up good word builder working with the file-based word list, I was ready to put it all together. The core of main() now looks like this:

Procedure read_words() simply reads all the words from the file into the map of hash_set containers called word_by_length, as described earlier.

Then build_up_words collects all the good words and stores them, organized by size, in the map of hash_set containers called good_words. The core of that routine was described earlier.

I show the results of the top two levels in print_good_words(), which is simple enough to read up on in the source code.

The Results

By now you are no doubt dying to see some results from the program. But first, to scope the size of the problem, here's the output from the program as the good_words hashes are populated on a run against SINGLE.TXT, a 354,985 word database:

Loading words from: SINGLE.TXT
Loaded 354985 words from SINGLE.TXT
Found 431 eligible words out of 431 total at length 2
Found 2075 eligible words out of 2092 total at length 3
Found 6213 eligible words out of 6758 total at length 4
Found 11322 eligible words out of 15047 total at length 5
Found 12495 eligible words out of 28473 total at length 6
Found 8939 eligible words out of 40094 total at length 7
Found 4295 eligible words out of 49528 total at length 8
Found 1210 eligible words out of 51216 total at length 9
Found 174 eligible words out of 43964 total at length 10
Found 20 eligible words out of 36082 total at length 11
Found 0 eligible words out of 28009 total at length 12

So that means about 13% of the words in this vocabulary were good words. I'm a little surprised that it's that high. To add some sanity to the mix, I removed all the single character words from SINGLE.TXT with the exception of 'a' and 'i', and the ratio went down to a more reasonable 8%.

You can also see, as you would expect, that the proportion of good words goes down at each level. At lengths 2, 3, and 4 nearly all words are good words, but by the time we get to length 11, we're down to less than one-tenth of one percent good.

Even with my modified version of SINGLE.TXT, you're bound to get plenty of esoteric words when working your way through the derivation of an 11 or 10 character good word. Of the 18 words of eleven characters, the derivation that works best with my vocabulary would be the following:

sparklingly
sparkingly
sparingly
springly
springy
spring
sprig
prig
pig
pi
i

With the more manageable scrabble.dict dictionary, containing 79,340 words, some of the first sequences that pop out include:

shopping hopping hoping oping ping pig pi i
breaches beaches baches aches aces ace ae a
marchese marches arches aches aces ace ae a
prawning pawning awning awing wing win in i
stablest stalest stales tales ales als as a
bravoing braving raving ravin rain ain in i
failings filings flings lings lins ins is i
relaters elaters elates elate late ate ae a
roadster roaster raster rater rate ate ae a
semioses semises seises seise seis sis is i
clambers lambers lamber lamer lame lam am a
claviers clavers lavers avers aves ave ae a
shrieves shrives shives hives hies his is i
stalkier talkier talker taker take tae ae a
statutes statues states tates ates ate ae a
swarming warming waring wring ring rin in i
brambled rambled ambled amble able ale ae a
stratous stratus status stats tats tas as a
paddlers paddles padles pales ales als as a
thirling tirling tiring iring ring rin in i
trucking trucing truing ruing ring rin in i
brawlier brawler bawler baler bale ale ae a
frilling filling filing fling ling lin in i
carouses arouses arouse arose arse are ae a

No doubt there are still plenty of obscure words here, but remember, this dictionary is probably composed of at least 50% words that aren't in your working vocabulary.

Efficiency

When run against a word list with 350K+ entries on my anemic notebook computer, it takes almost 10 seconds for the program to terminate, including display time. The vast majority of that time is spent checking words for goodness, which requires removing characters one at a time, then checking to see if their small descendants are in the word list.

Obviously, if you want to optimize this program for better performance, that's the place to do it. My guess would be that the std::string class member to erase characters from a word is probably far from optimal, and could be replaced by a hand-coded routine designed to do the same task with much greater speed.

Issues With Non-Standard Library Functions

Because hashed containers did not make it into the original C++ standard, there is a somewhat higher level of peril when using them. Problems ranging from syntactic inconsistency to lack of performance guarantees definitely make hash_set and hash_map second class citizens compared to the other standard containers. I saw a good example of this when I first started work on this article.

When solving this problem, the first thing that seemed obvious to me was that we were going to be storing references to dictionary words in hash tables. And I thought it might be interesting to see how well the C++ hash classes were going to be able to handle input data with hundreds of thousands of words.

I thought a good test program would be one that simply reads in the text file and adds it to a hash set:

Under Visual C++ .NET 2005 on a fairly slow laptop, I immediately saw a nasty problem with the Microsoft implementation of hash_set. Every time the counter hit a an even power of two, there was a bit of a pause. The pause grew longer and longer as the count grew larger, until by the time I was up to 128K words, it stretched out to many seconds.

Lesson learned. Hash table resizing can be expensive under some implementations of this non-standard class. Just resizing hash tables in any implementation can be difficult, but the additional requirements imposed on C++ library containers adds significantly to the work that must be done at this point.

I hoped that I would find a reserve() method or a constructor option that would let me preallocate a hash_set with perhaps 200K buckets, but this doesn't seem to be possible with Microsoft's implementation. The good news is that the hash_set replacement in TR1, unordered_set, will impose a requirement that conforming libraries allow for a bucket count as part of the container's constructor.

It turned out to not be too important, however. As I worked on the implementation of the algorithm, I drastically reduced the number of strings that were stored in any one hash, making this a moot point.

Source Code

You can download the project here, including project files for Visual Studio 2003 and 2005, and a simple Makefile for gcc. The code has been checked under gcc 3.4, but I make no claims that it will work with all later versions of the library.

References

Ward, Grady. "Moby Word Lists by Grady Ward - Project Gutenberg." Main Page - Gutenberg. 12 Nov. 2007 http://www.gutenberg.org/etext/3201.
"ISO/IEC JTC1/SC22/WG21 - The C++ Standards Committee." Open Standards. 13 Nov. 2007 http://www.open-std.org/jtc1/sc22/wg21/.

This article presents the algorithm that solves the Byzantine General’s Problem, as first described by Lamport, Pease, and Shostak in 1982 [1]. While Lamport’s algorithm is not particularly complex, programmers who aren’t used to working on distributed computation might find it difficult to implement. To accompany the explanation of the algorithm, I have included a C++ program designed for experimentation with the solution.

Introduction

The Byzantine General’s Problem is one of many in the field of agreement protocols. In 1982, Leslie Lamport described this problem in a paper written with Marshall Pease and Robert Shostak. Lamport framed his paper around a story problem after observing what he felt was an inordinate amount of attention received by Dijkstra’s Dining Philosophers problem [2].

This problem is built around an imaginary General who makes a decision to attack or retreat, and must communicate the decision to his lieutenants. A given number of these actors are traitors (possibly including the General.) Traitors cannot be relied upon to properly communicate orders; worse yet, they may actively alter messages in an attempt to subvert the process.

When we’re not dwelling in storybook land, the generals are collectively known as processes, the general who initiates the order is the source process, and the orders sent to the other processes are messages. Traitorous generals and lieutenants are faulty processes, and loyal generals and lieutenants are correct processes. The order to retreat or attack is a message with a single bit of information: a one or a zero.

In general, a solution to an agreement problem must pass three tests: termination, agreement, and validity. As applied to the Byzantine General’s problem, these three tests are:

1. A solution has to guarantee that all correct processes eventually reach a decision regarding the value of the order they have been given.
2. All correct processes have to decide on the same value of the order they have been given.
3. If the source process is a correct process, all processes have to decide on the value that was original given by the source process.

Note that one interesting side effect of this is that if the source process is faulty, all other processes still have to agree on the same value. It doesn’t matter what value they agree on, they simply all have to agree. So if the General is subversive, all lieutenants still have to come to a common, unanimous decision.

Difficulties

This agreement problem doesn’t lend itself to an easy naïve solution. Imagine, for example, that the source process is the only faulty process. It tells half the processes that the value of their order is zero, and the other half that their value is one.

After receiving the order from the source process, the remaining processes have to agree on a value that they will all decide on. The processes could quickly poll one another to see what value they received from the source process.

In this scenario, imagine the decision algorithm of a process which receives an initial message of zero from the source process, but sees that one of the other processes says that the correct value is one. Given the conflict, the process knows that either the source process is faulty, having given different values to two different peers, or the peer is faulty, and is lying about the value it received from the source process.

It’s fine to reach the conclusion that someone is lying, but making a final decision on who is the traitor seems to be an insurmountable problem. And in fact it can be proven that it is impossible to decide in some cases. The classic example used to show this is when there are only three processes: one source process and two peer processes.

In the two configurations shown in Figure 1 and Figure 2, the peer processes attempt to reach consensus by sending each other their proposed value after receiving it from the source process. In Figure 1, the source process (P₁) is faulty, sending two different values to the peers. In Figure 2, P₃ is faulty, sending an incorrect value to the peer.

You can see the difficulty P₂ faces in this situation. Regardless of which configuration it is in, the incoming data is the same. He has no way to distinguish between the two configurations, and no way to know which of the two other processes to trust.

Figure 1
The case in which the source process is faulty

Figure 2
The case in which P₃ is faulty

Under the rules of this approach, it is easy to show that regardless of how many processes are in the system, a subversive source process with one collaborator can cause half the processes to choose to attack, while half the processes elect to retreat, leading to maximum confusion.

The Lamport, Pease and Shostak Algorithm

In 1982, Lamport, Pease, and Shostak published a fairly simple solution to this problem. The algorithm assumes that there are n processes, with m faulty processes, where n > 3m. Thus, for a scenario such as that in Figure 1 and 2 with 1 faulty process, there would have to be a minimum of 4 processes in the system to come to agreement. (For the rest of this article, n will always refer to the count of processes, and m will always refer to the number of faulty processes.)

The definition of the algorithm in the original paper is short and succinct, but at least in my experience, is somewhat confusing for programmers without a lot of experience in distributed algorithms.

Lamport’s algorithm is a recursive definition, with a base case for m=0, and a recursive step for m > 0:

Lamport’s algorithm actually works in two stages. In the first step, the processes iterate through m+1 rounds of sending and receiving messages. In the second stage of the algorithm, each process takes all the information it has been given and uses it to come up with its decision.

I found this to be non-obvious without quite a bit of study, which is the reason for this article.

The First Stage

The first stage of the algorithm is simply one of data gathering. The algorithm defines m+1 rounds of messaging between all the processes.

In round 0, the General sends the order to all of its lieutenants. Having completed his work, the General now retires and stands by idly waiting for the remaining work to complete. Nobody sends any additional messages to the General, and the General won’t send any more messages.

In each of the remaining rounds, each lieutenant composes a batch of messages, each of which is a tuple containing a value and a path. The value is simply a 1 or a 0. The path is a string of process ids, <ID₁, ID₂, …, ID_n>. What the path means in this context is that in Round N, PID_N is saying that was told in round N-1 that P_IDN-1 was told by… P_ID1 that the command value was v. (This is very much like the classic party game in which a message is whispered from ear to ear through a chain of players, becoming slightly mangled along the way.) No path can contain a cycle. In other words, if ID1 is 1, no other ID in the string of process IDs will be a 1.

The message definition is easy in round 1. Each process broadcasts a message to all the other processes, including itself, but excluding the General, with the value it received from the General and its own process ID.

In subsequent rounds, things get more complicated. Each process takes all the messages it received from the previous round, appends its process ID where allowed, and sends those messages to all other processes, including itself. (The "where allowed" just means that the process skips any messages where adding its process ID to the list would create a cycle in the string of process IDs.)

For example, let’s suppose that in Round 0 that P₁, a faulty general told P₂, P₃, and P₄ that the command value was 0, and told P₅, P₆, and P₇ that the command value was 1. In round 1, the following messages would be sent:

In the previous table I showed the messages being sent to all six processes, which is fairly redundant, since the same messages are broadcast to all processes. For round 2, I’ll just show you the set of messages that each process sends to all of its neighbors.

The first message that P₂ is sending in round 2, {0,132}, is equivalent to saying "P₂ is telling you that in round 1 P₃ told it that in round 0 that P₁ (the General) told it that the value was 0". The five messages shown in P₂’s column in the table are sent to all six lieutenant processes, include itself.

It’s easy to see that as the number of processes increases, the number of messages being exchanged starts to go up rapidly. If there are N processes, each process sends N-1 messages in round 1, then (N-1)*(N-2) in round 2, (N-1)*(N-2)*(N-3) in round 3. That can add up to a lot of messages in a big system.

The Second Stage

While sending messages in each round, processes are also accumulating incoming messages. The messages are stored in a tree format, with each round of messages occupying one rank of the tree. Figure 3 shows the layout of the tree for a simple configuration with six processes, one of which can be faulty. Since m=1, there are just two rounds of messaging: the first, in which the general sends a value to each lieutenant process, and a second, in which each process broadcasts its value to all the other processes. Two rounds of messaging are equivalent to two ranks in the tree.

Each node in the tree has three elements: an input value, a path, and an output value. The input value and path are defined in the first stage of the algorithm - they are simply the messages received from the peer processes. The output value is left undetermined until the second stage of the algorithm, which I am defining here. Note that in the figure below, the output values are initially set to '?', indicating that they are presently unknown.

Figure 3
The Tree Layout for 5 processes with 1 faulty process

Once a process has completed building its tree, it is ready to decide on a value. It does this by working its way up from the leaves of the tree, calculating the majority value at each rank and assigning it to the rank above it. The output value at each level is the third item in the data structure attached to each node, and those values are all undefined during the information gathering stage.

Calculating the output values is a three step process:

1. Each leaf node in the tree (all values at rank m) copies its input value to the output value.
2. Starting at rank m-1 and working down to 0, the output value of each internal node is set to be the majority of the output values of all its children. In the event of a tie, an arbitrary tie-breaker is used to assign a default value. The same default value must be used by all processes.
3. When complete, the process has a decision value in the output of the sole node at rank 0.

In Figure 3, step 1 of the process assigns the initial values to the leaf nodes. In the next step, the majority value of { 1, 1, 1, 0, 0 } is evaluated and returns a value of 1, which is assigned to the output value in rank 0. Because that is the top rank, the process is done, and P₁ decides on a value of 1.

Every lieutenant value in a given exercise will have the same paths for all its nodes, and in this case, since only the General is faulty, we know that all lieutenants will have the same input values on all its leaves. As a result, all processes will agree on the same value, 1, which fulfills the agreement property.

A More Complicated Example

Getting a good understanding of the algorithm really requires walking through an example that has at least three ranks. (Examples on the web, mostly extracted from lecture notes, nearly always have the simple two-rank example.) For this example, consider an example with n=7 and m=2. We’ll continue with the convention that the General is P₁, and instead of having a faulty general, we’ll have P₆ and P₇ be faulty processes. After the initial three rounds of information exchange, we have the three-ranked tree shown in Figure 4:

Figure 4
A tree with n=7, m=2, and faulty processes P₆ and P₇

You’ll see that at rank 1, the values from path 17 and 16 are both set to X. In the first round the two faulty processes communicated possibly false values to all other processes, and may have arbitrarily changed the values sent to different processes in order to skew the results.

As a result of those bad values in rank 1, we see their frequent occurrence in rank 2. The incorrect values show up not only in direct messages from the faulty processes, but also in any message from a correct process that includes a faulty process earlier in its path.

All in all, at the leaf nodes, we have 18 deceptive values at the leaf nodes, and only 12 accurate messages that trace their way all the way back to the general through nothing but correct processes. Obviously, if we just voted on the majority of the messages we had received, we would be susceptible to falling for the wrong value.

Fortunately, the layout of the tree guarantees that we will actually get a correct value. In Figure 4, the roll up of the output values hasn’t occurred yet, so every node has a question mark in the output value. In Figure 5, the output values are shown. The leaf rank has the output values set to the input values, with X used to indicate unknown values from faulty processes.

When the leaf rank is rolled up to the second rank, the nodes with paths 12, 13, 14, and 15 all have clear majority values of 0 for their output values, with 16 and 17 set to X, as their values are uncertain.

The final roll up to the top rank successfully sets the output value to 0, as four of the inputs are set to 0 and only 2 are set to X. Mission accomplished. And because of the way this was calculated, we know that the correct result will be achieved regardless of what deceptive values are sent by the two faulty processes.

Figure 5
The tree after calculating the output values

The Sample Code

I’ve included a simple C++ program that implements this algorithm, with extensive internal documentation. It has a Process class that is used to send and receive messages, as well as to roll up the decision tree. A Traits class is used to define the number of processes, the number of faulty processes, the source process, and what values the faulty processes send in various rounds.

To help with visualization, the program will output the tree for a given process in the format used by dot, part of the free Graphviz program. You can then use dot to create a nice picture of the output graph – all the figures in this article were created that way. I find that using the SVG format for the output produces very readable results.

As supplied, the program is set for values of n=7 and m=2. Good exercises to perform while experimenting with it include:

• Attempt to invalidate the program or the algorithm by getting incorrect results with some particular combination of faulty messages.
• Add a third faulty process and show that it is relatively easy to get invalid output when n=7 and m=2.
• Reduce n to 6 and show that it is relatively easy to get invalid output with two faulty processes.
• Move up to m=3 and n=10. Experiment with various combinations of faulty Generals and lieutenants and see if you can create incorrect results.

Note

Most implementations of this algorithm include logic designed to deal with the case in which a faulty process fails to send a message. This is equivalent to simply having the faulty process send an arbitrary value, so I don’t treat it as a separate case.

Source Code

source.zip, which contains:

• main.cpp
• README.TXT
• VS2003/byzantine.sln
• VS2003/byzantine.vcproj
• VS2005/byzantine.sln
• VS2005/byzantine.vcproj

References

[1] The Byzantine Generals Problem (with Marshall Pease and Robert Shostak)
ACM Transactions on Programming Languages and Systems 4, 3 (July 1982), 382-401.
http://research.microsoft.com/users/lamport/pubs/byz.pdf

[2] The Writings of Leslie Lamport: http://research.microsoft.com/users/lamport/pubs/pubs.html#byz

Lynch, Nancy A. Distributed Algorithms. San Francisco, CA: Morgan Kaufmann, 1997. ISBN: 1558603484.

Graphviz – Graph Visualization Software. http://www.graphviz.org/

CS 6378: Advanced Operating Systems Section 081 Notes on Agreement Protocols (lecture notes by Neeraj Mittal, University of Texas at Dallas) http://www.utdallas.edu/~neerajm/cs6378su07/agreement.pdf

Update 01-November-2007
Distributed Computing With Malicious Processors w/o Crypto or Private Channels, YouTube Video of Google TechTalk, Valerie King, Department of Computer Science, University of Victoria, Victoria, BC, Canada

Dr. Dobb's Portal April, 2007 Article on DDJ site

I'm an inveterate fan of wordplay of all sorts - puzzles, anagrams, crosswords. I've been known online by my anagrammatic name, SnorkelMan, all the way back to the ancient days of the text mode BBS. My continual hectoring of the staff at the Dallas Morning News over errors in their print version of the New York Times crossword puzzle led them to finally just give me the job of proofreading it. I spend way too much time on the crosswords and other puzzles, both online and in print. In other words, I'm a sucker for a good word puzzle.

Figure 1 - Dallas Morning News Article

My sense of wordplay was naturally piqued this weekend when I heard the latest weekly puzzle challenge from Will Shortz on NPR Weekend Edition. The challenge, from contributor David Edelheit, read as follows:

Take the names of two U.S. States, mix them all together, then rearrange the letters to form the names of two other U.S. States. What states are these?

As sometimes happens, when I heard this puzzle, and the answer didn't click immediately, my first thought was "I could write a program to solve this faster than I can figure it out myself."

That's a treacherous thought for a puzzler, because it immediately diverts that little thread in the back of your mind that is supposed to be solving the puzzle, instead putting it on the task of writing the program.

But it turned out to be an interesting problem in efficiency, and so I'm glad I went down that path.

First Pass

Most of my work these days is in C++, and while C++ doesn't have the world's best string manipulation facilities, I thought it had enough to do the job on this puzzle. Figuring that the problem was small enough to solve via brute force, I decided that the general course of the program would be to work my way through all 50*49/2 combinations of states, and test them against all 48*47/2 remaining combinations. That's just a little more than a million operations, which ought to be child's play. Thus, the basic program loop was going to look like this:

Still thinking brute force, I was looking for the simplest way to store the data so that it would be easy to compare, and turned to std::multiset. I knew that if I stored all the characters from states i and j in one std::multiset object, and all the characters from states m and n in another, I could quickly compare one against the other with a simple equality operator.

So in the above loop, I inserted these lines after the first two for statements:

(Note that I had snagged the names of the states from one of the first sites in a Google search, and inserted them into an array of character pointers called states.)

I inserted a similar definition for label2 inside the second set of two for statements, which means all I had left to do was a simple comparison of label1 against label2:

I did a quick compile, tested the code, and sure enough, the contents of the multisets were indeed sorted concatentations of the letters of each pair of states. Time to run!

My first disappointment was seeing that, while the program was indeed running properly, it was going slow enough that it looked like it was going to take a sizable fraction of an hour to make it through the entire alphabet. I could just wait, but in this case I decided I could optimize faster than it would take to wait for the first results.

Second Pass

It's pretty obvious that calculating the concatenation of state m and n in the innermost loop is full of wasted cycles, since it is repeatedly calculating the same state values. I knew it was inefficient, but I didn't think it was going to matter too much.

Since it turned out that it did matter, I decided to precalculate all the values before entering the four-deep nested comparison loop, with code like this:

Then I didn't have to do any computation in my main loop, I just had to modify the comparison line in the innermost loop:

This modified program did indeed speed things up considerably, bringing the run time down from a fraction of an hour to just a few seconds, even with a bit of progress tracing turned on.

Third and Final Pass

Although I could have stopped here, I thought it might be interesting to see how expensive the use of an associative container like multiset was compared to something simpler. I replaced the setup code so that it stored the data in an std::vector instead of a multiset, on the theory that the comparison operator would run much faster on a vector. I had to add a call to sort the data after inserting it in the vector, which would be executed 50 times, but nowhere near as many times as the comparison operator.

The results were more or less as I expected. When run under Windows with default Release optimization, the vector version of the program ran about twice as fast as the multiset version. When compiled with g++ 3.3.5 with -O2, I saw roughly the same ratio of execution speeds.

No Spoilers

I'm not going to spoil the puzzle for you by giving away the answer. Let's just say that it is a good word puzzle, and if you manage to arrive at the answer you'll see why. If you don't manage to solve it in your head, you can download the source, compile it, and get there by brute force, just like I did.

usa.cpp

But I don't think it will count as a spoiler if I tell you that you don't need a computer program to solve this problem.