In my early years of learning about data compression, the book Text Compression by Timothy Bell, John Cleary, and Ian Witten was my resource of first resort. I was in Christchurch, New Zealand, recently and had a chance to meet Tim for the first time in person. Tim teaches at the University of Canterbury in Christchurch, and is Deputy Head of the Computer Science and Software Engineering department. I got a chance to ask him about his work in data compression as well as one of his new areas of interest, Computer Science education.

TB: I’ve sometimes wonder about this myself… during the early days of computing and especially personal computers, it took some time for the latest technology to reach us “down under”, so perhaps we were motivated to get more out of what we had rather than wait some months for a larger disk or new memory to arrive from overseas. When the Internet arrived we started with a very small pipe, so a good compression algorithm could do the equivalent to laying a second cable from NZ to the US – who can resist getting something for free?

MN: Since you wrote Text Compression back in the early 90s, I’d say the biggest development in lossless compression has been the Burrows-Wheeler transform. Is lossless text compression basically done? Are we left with just incremental improvements as processor resources increase?

TB: That seems to be the case; the only big improvements we’ve seen have turned out to be frauds — we even had one in NZ recently, where a Nelson man raised NZ$5.3 million for an impressive sounding method; he was convicted of fraud last year. The main indicator we have that we’re running out of steam (apart from a lack of new discoveries) is Shannon’s experiments on predicting text which gave a bound in the order of 1 bit per character for English text, and current methods are approaching this. Of course, there’s plenty of room for dealing with new kinds of data (for example, bioinformatics deals with massive amounts of data that we’re still trying to understand) and for finding better data structures and algorithms for performing the compression and decompression. Lossy compression is a whole different story…

A Change In Focus

MN: It looks like you are now dedicating a large amount of your time to establishing computer science as part of the basic curriculum in high school education, for students in the 15-18 age range. In many ways, this is as much a bureaucratic problem as an academic one. What motivated you to take it on?

TB: It’s been a problem that we’ve complained about for decades, and it’s been getting worse and worse as computing in schools has focussed increasingly on using computers and not preparing students to be developers. A lot of this can be attributed to bureaucracy – it’s hard to explain to government officials that putting word processors in every classroom isn’t the same as building a computationally literate society. As a result of some strategic lobbying done by others, a small window of opportunity opened for me to be on a group to advise our Ministry of Education, just over 3 years ago. The group managed to convince the officials that something useful could be done, and then we had to work very quickly to come up with a concrete proposal before the enthusiasm died down. This has happened rapidly; the advisory group first met in November 2008, and Computer Science started being taught in schools in February 2011.

MN: What have you been able to accomplish in New Zealand so far?

TB: Computer science (including programming, but also topics the involve understanding the importance of things like algorithms, HCI, programming languages and even compression) is currently available as part of computing courses for two of the three final years of our main high school graduation qualification, with all three years being covered from 2013. After that we would expect some of the introductory material to start filtering down to earlier classes, and for wider offerings as teachers become more confident in the subject. One of the biggest challenges has been preparing teachers, few of whom have significant experience in Computer Science. Many have embraced it enthusiastically, and the universities and others have done a lot of work to help them get up to speed. It’s been a wild ride doing it so quickly, but there have been some very pleasing outcomes.

MN: And how do things look in the rest of the world? Are there any obvious winners and losers at this point? Do you have any concise advice for the world?

TB: Computing in schools is a hot topic around the world; the UK have just announced a strong drive to introduce this sort of material to schools, and the US has people working hard to make it available to students. Israel and Korea have had computer science in schools for some time. We’re learning a lot about what is worth teaching, and what the best pedagogy is for the general classroom (most of our experience is for specialist students who have chosen the subject!) The New Zealand path of getting something going quickly with grass-roots support seems to be more effective than waiting for a top-down approach which could take years to develop and prepare teachers for, although it does make for a bumpy ride as problems are ironed out as we go along!

MN: This might be straying out of your area a bit, but do you see CS in a K-12 education setting having an effect on the representation of women in the STEM fields?

TB: Attitudes that affect representation definitely start at school, and to me the biggest goal of teaching CS in high school is not so much to prepare students for further study, but to enable them to find out what the subject is! School students rarely know what CS is, and even worse, it’s common for them to assume that it must be advanced word processing or some other dull area, and hence they avoid it. It’s particularly important for female students to have the opportunity to find out if it’s something that they might be good at, as the stereotypes associated with computing can make them assume that they shouldn’t consider it as a career.

MN: One final question, Tim. The whole world has seen the devastating damage Christchurch has suffered from the earthquakes in the last year. How has the University of Canterbury held up? Have you managed to maintain continuity in your academic calendar?

TB: It’s been quite a year! Thankfully our university has escaped the brunt of the earthquakes (most of the damage is some distance from the university), and we’ve managed to keep a full programme going despite being closed for three weeks for safety checks. Many students joined the “student volunteer army”, who helped with the cleanup in the damaged parts of town, and that was probably one of the most valuable experiences of their career! It hasn’t been without disruption as buildings need to be checked carefully, and some are still under repair, but with a bit of resourcefulness we managed to keep going (for a while I even delivered my classes in a restaurant while lecture theatres were being inspected) The city is now going through a massive program of redevelopment with some pretty creative ideas, and it’s an exciting time to be part of these changes.

Thanks to Dr. Bell for taking the time to share all this with us. My visit to his amazing homeland was a real treat, and the short time I got to spend with Tim in Christchurch was worth the trip all in itself.

Back in the day the Sapir-Whorf hypothesis was all the rage in the study of linguistics. With apologies to those who actually work in the field, I’ll crudely summarize it as the idea that the language you speak both constrains and influences how you think. The idea says that if your language only has one word for snow, for example, you will actually have a hard time seeing any difference between light powder and crunchy ice pack.

Sapir-Whorf was seen as completely discredited back when I learned about it, and while Linguistic Relativity has enjoyed a slight comeback with a weakly restated set of hypotheses, it seems fairly certain that human thought is by no means confined to a cage built out of vocabulary and grammar.

Our field has long had its own Sapir in E.W. Dijkstra, whoe spelled it out with money quotes like these:

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.
The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offence.

Closer to today, we have the famous rant against C++ from Linus Torvalds, who feels that a programmer who uses C++ is going to wreck any project he or she touches:

I’ve come to the conclusion that any programmer that would prefer the project to be in C++ over C is likely a programmer that I really *would* prefer to piss off, so that he doesn’t come and screw up any project I’m involved with.

C++ leads to really really bad design choices.

Dogmatism à la Torvalds

At the large corporate entity that pays my bills, we have a slogan on the back of our badges: No Technology Religion. It might not be easy to live up to this, but yes, I try.

To me, this admonishment means two things:

1. Try to objectively choose the best tool for the job
2. Don’t let your preferences in tools dictate the way the job should be done

Linus is clearly saying in his rant that anyone who programs in C++ is guilty of breaking both of these rules.

I disagree. I think there are times when C++ is clearly the right tool for the job, and that you can arrive at this conclusion fairly objectively. I think Linus is clearly fogged in by his particular Technology Religion.

A Simple Example

As the final assignment for my C/C++ programming class last semester, I asked my students to implement a simple token counting program in C. The goal was to reproduce the behavior given by this C++ fragment:

map<string,int> counts;
string s;
while ( cin >> s )
    counts[s]++;
for ( auto ii = counts.begin() ; ii != counts.end() ; ii++ )
    cout << ii->second << " : " << ii->first << endl;

This particular program highlights a number of features of C++ that are not present in C:

• The versatile replacement for C arrays, vector<T>.
• The string class.
• Safe input using iostreams.
• Associative arrays as part of the standard library.

This program is quite easy to write in C++, and is basically complete. One could flesh it out a bit with of error handling on the input stream, but that’s really not even necessary.

Do it in C

Rewriting this in C is a straightforward task with one big speed bump: the lack of any sort of associative array in the library. There are a number of ways to deal with this – I chose the following strategy:

• Read all the tokens into an array.
• Upon completion, sort the array
• Once the array is sorted, walk through it to get the count for each token.

While this algorithm uses more space than the C++ program, it probably takes up the same amount of time, assuming you don’t bump into one of the pathologically bad cases of qsort().

Since I asserted that this program is easier to write in C++ than C, it behooves me to give a list of reasons why.

• C I/O deficiencies. Reading strings in C is considerably more difficult due to the fact that the C I/O library doesn’t have a standard way to read strings of unbounded length. (Compiler-specific extensions can be used, but that raises other problems.) Your input code has to do a lot of checking for error cases, or you have to build your own string input functions.
• Memory management of C arrays is a very manual task. I have to allocate the original space for my array, take care that I reallocate if I exceed its length, and free the space when I am done.
• Memory management of C strings has exactly the same probelms.
• Sorting the array of strings is just a tiny bit more inconvenient with qsort(), and qsort() doesn’t give me the performance guarantees of the sort() function in the C++ library.

The C version of my function has more lines of code, and more bookkeeping tasks that need to be done manually. There are more opportunities to make mistakes.

A final reason I like the C++ version of the program better is that it lends itself well to working with other types. Any type that has insertion and extraction operators, and a comparison operator, can use that same code with just one declaration change. Turning it into a function template accomplishes the same thing with no code changes needed at all.

Some of My Best Friends are C Programmers

So am I a language bigot for preferring the C++ version of this code?

I hope not. For one thing, I can see that the C version of the program has some nice advantages:

• You can write this program using POSIX system calls for almost everything except memory allocation and sorting, resulting in an extremely small footprint.
• The C version of the program will be faster due to the use of low-level I/O. C++ iostreams get better all the time, but their layered approach will always be at a disadvantage when it comes to efficiency.

So for a program like this, the choice of language really comes down to context. If you believe in the 80/20 rule, you might think this code should be written in C++ if it is outside of the expensive core part of your program. With fewer lines of code you have fewer chances for error, and efficiency is probably not a big consideration.

If this is in a critical section of code that is executed frequently, you might decide C is your best choice. Make sure to put a little extra time into code review to ensure that the code is free of memory leaks and pointer errors, and you are in business.

Sic Temper Linus

So how does Linus’ rant hold up when looking at the C++ code shown at the top of the post? I would venture a guess that given the assignment, any decent C++ programmer would produce code similar to this. Linus says:

You invariably start using the “nice” library features of the language like STL and Boost and other total and utter crap, that may “help” you program, but causes infinite amounts of pain when they don’t work (and anybody who tells me that STL and especially Boost are stable and portable is just so full of BS that it’s not even funny

In this program I make good use of the standard library components that were once part of the STL. Having been part of the standard for over a decade, they work really well and have no portability or correctness issues in any compiler I am aware of. Saying that components like map and vector are problematic is just wrong.

inefficient abstracted programming models where two years down the road you notice that some abstraction wasn’t very efficient, but now all your code depends on all the nice object models around it, and you cannot fix it without rewriting your app.

Well, despite the fact that C++ has made it impossible for me to do so, I managed to write the program without using any abstractions – no new classes, no interfaces. Basically just straight procedural C code that happens to employ a few useful classes.

And at least with the people I work with, I think this is the rule rather than the exception.

In other words, the only way to do good, efficient, and system-level and portable C++ ends up to limit yourself to all the things that are basically available in C.

When C has container classes, a string class, typesafe I/O, and the programmer’s gift from the gods, RAII, then this statement will be true. For now, it is bollocks.

Before modern C++ was available, I probably would have stuck with a simple pipeline to accomplish this task:

tr [:blank:] '\n'  | grep -v "^$" | sort | uniq -c

The fact that I can do the same thing just as easily in a compiled language gives me some flexiblity. I think I can appreciate that fact without being a bigot.

Can you?

Photo by Nick Richards

Ken Olsen, founder of Digital Equipment Corporation, died Sunday, February 6 at the age of 84.

Looking back on the amazing arc of Ken Olsen’s life brings home an important point about the pace of change in the technology business. When I graduated with a CS degree in 1980, DEC was on top of the world, and the minicomputer was the driving force behind a paradigm shift in business and scientific computing. A generation later, DEC was gone, picked up by Compaq in a fire sale in 1998, and sold for parts.

In that short period of time, DEC’s minicomputer business was overtaken by changing technology. The corporate IT world we live in now is centered around networked PCs; if there is a legacy hunk of iron anywhere at the center of it all, it is likely to be a mainframe. The transcendental VAX 11/780, sitting on top of the world in 1980, was scrapped long ago. And Ken Olsen famously sat out the revolution, convinced it was not going to affect his business.

I think business leaders in today’s environment are more likely to realize that sea changes in technology can and will force them to completely rewrite their game plans on a regular basis. Companies like Netflix, Apple, and HP seem to have incorporated reinvention into their DNA. And if the CEOs don’t get it, their boards will replace them with someone who does.

There’s a quip that shows up on tee shirts, bumper stickers, and refrigerator magnets from time to time: Maybe your purpose in life is only to serve as a warning to others. That may not the legacy of choice for a corporate titan, but perhaps it is better than no legacy at all.

You may have never heard of Vinay Deolalikar, but there is a chance that he may become next year’s Turing Award winner, not to mention an overnight millionaire. It seems that Vinay dropped the news at the start of this week that he had proven that P does not equal NP. In short, this proof means that many problems we suspect are hard to solve are in fact provably hard to solve. Whether his proof succeeds or not, the Interwebs are abuzz with the news.

P versus NP is arguably the biggest unanswered problem in Computer Science, and proving it one way or another will cascade into new results and conclusions in many areas of computing and mathematics. The problem is at least 40 years old, and despite people literally devoting their entire careers to the problem, has yet to be decided. Some researchers suspect it can’t be proved, although this has not been proven either.

When a famous problem like this has been stewing for so long, it is not unusual to see sporadic cries of proof, but these need to be taken with a grain of salt. The first test is to see if the person is a flake with a proof that won’t survive a casual scan. Passing that test, the proof then has to endure the normal peer review process. Proofs that plumb difficult new ground can take years to be accepted.

It appears that Deolalikar has passed the first test – people are seriously studying his proof. However, given its complexity and length, you can expect that general acceptance will not happen overnight. More likely, specific problems will arise, challenges will be published, and Deolalikar will either respond, modify his proof, or retreat for more study. If he indeed has the golden ticket, he will be cashing a million dollar prize from the Clay Mathematics Institute sometime soon.

It’s remarkable news indeed, and perhaps the only way it would have been more remarkable would have been with a proof that P equals NP.

Feel like weighing in? Check out the proof yourself and see what you think.

Yes, another middle manager in my organization has decided to part ways with my employer, Cisco Systems, Inc. Dave Corley has been a key part of Cisco since he was acquired along with the rest of Selsius in 1998. He’s now giving us a friendly handshake and showing himself out after just shy of 10 years service. His departure marks the loss of one more innovator from the team that literally created the voice business at Cisco, a business that now generates billions in revenue per year.

A Little Bit of History

Dave didn’t spring fully formed from the brow of Zeus destined to become a father of VoIP. Aafter graduating from the United States Naval Academy in 1975, he found himself at the Naval Postgraduate School writing Heat Transfer Analysis of a Rotating Heat Pipe Containing Internal, Axial Fins, still considered a classic by many. Submarines, not telephones, seemed to hold the key to his future.

A sterling career followed as Dave served as an officer on a trio of nuclear submarines, including the USS Hawkbill, USS Corpus Christi, and USS Parche, serving variously as engineering division officer, weapons officer, and navigation/operations officer, with a rank of Lieutenant Commander on his last two boats.

But all good things must come to an end. The USS Parche went into drydock for a multi-year renovation, and not long after that, Dave’s career in the USN came to its conclusion, retiring from active duty with as Commander. (Later upped to Captain while serving in the reserves.) However, unlike the Parche, which was decommissioned in 1994, Dave still had many years ahead of him, and he settled down in the Dallas suburbs and entered the telephony business.

Corley 2.0

After spending some time with the now defunct Voice Control Systems, Dave hooked up with the crew at Intecom that was to eventually be spun off as Selsius Systems under the leadership of Richard Platt and David Tucker. While at Intecom, Dave worked with Richard and others to develop what is arguably the first commercial VoIP phone system, an achievement that culminated with the purchase of Selsius by Cisco Systems, Inc. in 1998.

When Cisco bought Selsius, they got a company that had a working phone system, but it was not really ready for market. It was still too expensive to manufacturer, the infrastructure was not particularly scalable, and it had a long way to go before it was really managable by enterprise IT departments. There was a lot of work to do.

Dave rose to the task. As a product manager at Cisco, he shepherded Call Manager through several years of major releases, prioritizing features, haggling with development, marketing, and sales, and maintaining a vision that within just a few years, Cisco would own this voice segment.

You really can’t think of a less glamorous job than managing feature lists in a big company like Cisco. Life is an endless series of phone calls, meetings, small victories, and painful setbacks, with very few breaks for catching your breath. Dave oversaw the process of moving Call Manager from a single NT 4.0 server with an Access database to a distributed, redundant server farm using a replicated SQL database. He saw Call Manager progress from limping along with a couple of hundred phones to becoming a system that handles tens of thousands of phones inside Cisco. He worked through complex product integrations, like the deployment of Unity Voice Mail, another product line Cisco acquired around 2000.

When you look back on it today, it all seems inevitable. Cisco now owns the VoIP business, and went from a standing start to sharing leadership in enterpise telephony sales with Avaya. Back in the 90′s it would have seemed impossible for anyone to break into the tightly controlled PBX market, but VoIP opened the door, and Cisco charged in.

Dave’s part in all of this reminds me of the similar role of Lt. General Leslie Groves played in the Manhattan Project. Historians love to write about the enigmatic Robert Oppenheimer, but Groves is the one who oversaw the enormous effort required to get a gigantic bureaucratic enterprise on its toes and sprinting to a successful conclusion. Not glamorous work, but pivotal. In many respects, Dave was the Leslie Groves of Call Manager.

The Linksys Saga

As Cisco’s VoIP business matured, I got the impression that Richard and Dave were becoming restless. My suspicions were confirmed in 2004 when Richard announced that he was creating a new group within Linksys (part of Cisco by then) to build a PBX targeted at the small business market. Apparently Richard and Dave weren’t familiar with Fitzgerald’s maxim that there are no second acts in American life.

The Linksys project was succesfully executed, with Dave in his familiar role, managing features, creating requirements documents, and convening the usual endless rounds of meetings. But unbeknownst to us all, there was trouble afoot.

It turns out that in 2007, Richard Platt reached the point of no return in his quest for job satisfaction. You could characterize Richard as an entrepeneur, or perhaps even a free spirit, but regardless, the constraints of a big company like Cisco apparently wore him down, and he exited on short notice.

Leaving Dave in charge of the group.

Managing Expectations

I’m not going to tell you that entering management at a company like Cisco is guaranteed to drive you stark, barking-dog mad. After all, there are people who take to it with relish – John Chambers seems both sane and happy.

But for some people, it can be a soul-crushing, stultifying existence – divorced from the reality of real work and instead enmeshed in an endless series of political battles, policy altercations, reeducation camps, and regular group recitals of the company song. I couldn’t take it, and at the same time I’m astonished to find that there are people who thrive on it. I’ve mentally divided humanity into two groups: us (people like me), and them (people who are attuned to management). I assume that we are of two different species.

Watching Dave’s experience as a manager at Cisco had me wondering: is he really one of us? He didn’t seem to be enjoying his new position as head of our engineering group. In the past, Dave’s level demeanor was a source of reassurance to me, now that demeanor seemed to have been turned down a notch or two.

My suspicions turned out to have a solid foundation in early May, when Dave announced that he was leaving. Word soon leaked out that he was joining Ripcode, a company that makes a video transcoding engine for the data center. (It’s kind of sad that companies today are restricted to naming themselves using the unregistered portion of the .com name space. Makes for some pretty strange monikers.) Dave knows a bit about building a company, and can no doubt help these guys get their feet on the ground and then go someplace.

Unfortunately, that leaves this ship without its captain.

Jeff Bezos Hawks the Kindle

It’s still not clear whether electronic books are the wave of the future or a consumer products cul-de-sac. Technology continues to improve, and there are certainly lots of good reasons for a device like Amazon’s Kindle to be the leading edge of a major wave of adoption. A few of the more obvious arguments include:

• Reduced cost of distribution. The publishing industry wastes a lot of money printing and shipping books, and because of historical practices, creates huge numbers of books that never even get sold. Not very green, and a waste of money.
• Niche markets that can be very well-served. For example, high school and college students can replace those 30-pound backbacks with a 30-ounce tablet-sized device.
• Removal of barriers to publication. The news and magazine businesses are being revolutionized by self-publication in the form of blogs. Self-publication is possible in the printed book world, but it is still a rather awkward process. Publication to electronic format is presumably a trivial problem.
• Integration of information resources. A device like the Kindle allows you to consult the Internet, written reference materials, and your personal notes all from the same device, making it a true information portal.

But consumer acceptance is a fickle thing, so we don’t know if these rational arguments are going to fly. And of course, every writer who reviews a device like the Kindle or the equally capable Sony Reader feels compelled to write something along the lines of “But I just can’t imagine forgoing the pleasure of curling up on my couch with a good book.” I’m sure that when Gutenberg introduced movable type there were millions of industry reviewers posting notes to their blogs saying “The uniformity of the type is just esthetically unpleasing – I love it when I recognize the script of one of my favorite transcribing brothers.”

And then of course there’s the Steve Job’s money quote on the subject:

“It doesn’t matter how good or bad the product is, the fact is that people don’t read anymore, forty percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don’t read anymore.”

So it’s really pretty hard to be sure just which way this is going to go.

Looking Beyond The Reader

Sony’s Reader

For electronic books to succeed, one thing is certain: the physical reading experience needs to match up well with the one we have right now for our printed media. In some ways this problem is already solved – readers like the products from Amazon and Sony are book-sized and lightweight, with displays that are doing their best to match the various good qualities of paper.

But there are still issues that need work. One of the most important is in the area of layout and markup. A presentation format such as HTML is designed to work with multiple display sizes, repositioning elements as needed. This doesn’t necessarily work so well with textbooks, magazines, etc., where graphic artists invest huge amounts of time and energy on positioning, font selection, and other esthetic issues.

Additionally, there is am staggering amount of material that is simply not in a format compatible with today’s electronic books. Even magazines being published today are not always ready for transfer to an eBook format, and there is of course a massive backlog of valuable material in the world that has never existed in digital format.

Into this void steps Robert Maxwell Case, who has developed a system called ReadAllOver, and is attempting to exploit it via his company, SeeAndBelieve.com. SeeAndBelieve.com has created a digital layout technology called ReadAllOver (the company seemingly has a love affair with awkward CamelCase constructions) that does a superior job of preserving the look of printed materials. The company web site gives a good demonstration of exactly what they are capable of doing – it is definitely worth your time to take a look.

I asked Robert if he could take the time to answer a few questions about ReadAllOver, and he was gracious enough to respond.

Questions With Robert Maxwell Case

Mark Nelson: Hi Robert. I just recently became aware of your company, SeeAndBelieve.Com, and your imaging system, ReadAllOver. Before we get into the details of your technology, can you tell me a little bit about the history of your company? How long have you been at it? Are you working mostly solo or do you have some help? What kind of background do you have that got you into your current work?

Robert Maxwell Case: Sure, Mark. I come from a background of being a full-time musician and a part-time graphic designer. Around 1991-92, I was unhappy with then-current digital halftoning routines and began experimenting on my own. So I’ve been at it 15-plus years.

Jimmy Kung (left) and Robert

I’m not a programmer, so I have had a succession of programming assistants. In recent years, they have come from my affiliation with the Computer Science department at Texas State University-San Marcos where I am a seven-year member of the Industrial Advisory Board. Currently SeeAndBelieve.Com, in addition to myself, has one full-time employee, Jimmy Kung, and several part-timers.

The first of my five U.S. patents (three issued, two pending) was filed in 1993 in response to some interest expressed by Steve Carlsen, developer of the .TIF graphics file format at Aldus (he’s now with Adobe Systems.)

MN: Can you give me a capsule summary of ReadAllOver? How does it differ from page layout systems like we see in web browsers or PDF viewers? Does it differ from the rendering systems used in Sony and Amazon’s current eBook readers?

RC: Well, ReadAllOver in a nutshell is a digital halftone-based graphics system suitable for eBooks. It renders on the screen a digital page with the “look and feel” of a printed page, with all included graphic elements, typography and images, placed precisely as the graphic designer intended. It differs from existing web browsers and .PDF viewers in that it relies less on text files and font metrics and instead places more emphasis on a simplified, highly-compressible bitmap image. In many respects, it is a “picture” of the page, with an ancillary text file.

The Sony and Amazon eBook readers are primarily text-based, offering a limited number of typefaces and few graphics. They both use the E-Ink subtractive screen and we think ReadAllOver’s halftone system can be tailored to enable a good fit with that screen.

MN: It looks like your technology emulates the halftone process used to render photographic images in newspapers and magazines. How does it improve on that process to achieve smaller file sizes? Do you have data showing the level of compression you get for specific images? And do you also render type as halftone images? That would seem a lot less efficient than treating type as marked-up text.

RC: That’s right, ReadAllOver does emulate the halftone process with one major difference, and that is that typography can be processed with it and not fall apart.

Samples of ReadAllOver output (Detail may not be representative, images were resized)

The big idea is that the output image starts as an interim monochromatic checkerboard pattern, beginning from gray. To simplify, we derive the output image by rendering local areas of the input image that are darker than checkerboard gray by turning corresponding output monochrome white pixels to black. Conversely, we render local areas of the input image that are lighter than checkerboard gray by turning corresponding output monochrome black pixels to white. The result is a checkerboard-based ordered dither that can be re-ordered for variable-length run-length compression.

We presently are comparing compression levels for specific images and plan to publish the results. So with typography, abrupt gray level changes such as font outlines fall on the black pixels of the interim checkerboard to render any font without reliance on font metrics or hinting. For example, ReadAllOver pages containing only typography are competitive in file size to marked-up systems, and with the additional attribute that our system is able to efficiently render any font, in any language, and any image, placed correctly. That’s exciting.

MN: I get digital delivery of some magazines already. For example, IEEE Spectrum is delivered using technology from Qmags. Do you think you can do a better job than they are already doing?

RC: Qmags has selected Adobe PDF as its preferred file format, so the comparisons I’ve made previously between ReadAllOver and PDF come into play here. Taking a quick look at Qmags’ file sizes, I would say that ReadAllOver is competitive and looks subjectively more “print-like.”

MN: Have you released any sample code or SDKs for people to work with your compression technology?

RC: Not at the present time … we have been too busy developing our prototype (you can see it at SeeAndBelieve.Com ).

MN: The idea of an electronic book reader has been floating around for a long time, but right now it seems like we’re finally seeing designs that are actually gaining some traction. There are still a lot of naysayers, however. What do you see in the future for the electronic book reader? Will it eventually do for reading what the iPod did for music? Or will it forever be a niche product that is stuck on the verge of popular success? And how hard will it be for you to get the manufacturers of eBook readers to adopt ReadAllOver?

RC: Mark, I do believe that an electronic book reader will achieve iPod-like popular success, and, hopefully, in the near-term. In my opinion, the more book-like the readers become, the closer we will get to that so-called tipping point. Then the public will recognize the added benefit of having access to any publication, including one’s own library, available whenever and wherever they desire. There are some of obstacles still to be overcome, like screen pixels that are pretty large, and wired and wireless transmission pipes that are pretty small.

I personally would like to see a reading device with 8-1/2″ x 11″ facing screens, about the size and weight of a coffee-table book, with graphics-intensive magazines and newspapers rendered on screen nearly indistinguishable from paper and ink. I certainly can imagine a college student not having to carry a backpack full of textbooks around campus.

Here’s my vision for our product: We’re hoping to make adapting to ReadAllOver as seamless and easy as possible. We envision three communities of ReadAllOver users:

1. The reader

2. The publisher/bookseller
3. The hardware manufacturer.

We plan to provide each of these groups with a cost effective, easy-to-use solution. The ReadAllOver Viewer will be free for readers. They’ll read .RAO files on dedicated eBook devices, and most likely on other devices that could emulate a book-like reading experience. The inherent look and feel of printed material should reduce eye strain and, of course, we plan to offer such extras as page flipping, content search, printing, etc.

Publishers and booksellers will use ReadAllOver Publisher, our media content production system. They’ll be able to convert content from scanned material, as well as from existing editing and page layout systems and standards, including Adobe InDesign and FrameMaker, Quark XPress, MS/Office, Open Office, etc. (and, of course, we’ll include a .PDF to .RAO converter.) This community will appreciate a built-in high level of content protection with ReadAllOver’s emphasis on bitmaps. Hopefully they won’t need more, but if they do, our product should easily be able to incorporate additional encryption and DRM mechanisms.

We also plan to collaborate with eBook (and other display device) manufacturers in order to provide built-in support for the ReadAllOver rendering system. We feel we can efficiently adapt our system to display components with limited available grayscale and color levels. We also hope to offer a fixed-bit-rate option where every delivered page is the same file size.

Conclusion

Thanks Robert, this is all interesting stuff. I can see advantages to ReadAllOver that we don’t get from layout systems like PDF or HTML, so perhaps you will be able to hammer out an effective market position. I don’t have an eBook reader yet, but I think when the technology reaches the point where I can have a color Kindle I’ll probably jump on board. I’m almost there now, but since the Kindle is perpetually sold out at Amazon.com, my dollars are still safely in my wallet.

Chances are good that you don’t know who Richard Platt is. Richard was never in the business of self-promotion – had he been, he could have been a household name among the cognoscenti. But Richard didn’t spend his time talking to the press, speaking at conferences, or teeing up with the titans of industry. His passion was devoted to building great products and taking care of the people whom he asked to share that passion.

A Short History of VoIP

In early 1997, David Tucker and Richard Platt were running a division of Intecom called Incite. They were developing and promoting an innovative product designed to optimize the use of video on corporate networks, using a backbone of isoEthernet. I spent some time contracting for Incite, and believe me, in 1997 it was pretty cool to be able to watch four channels of broadcast TV on a PC. But the technology that made this possible, isoEthernet, was also the weakness of the Incite solution. Interface cards were expensive, and isoEthernet wasn’t compatible with emerging Fast Ethernet standards.

David Tucker

This could have been the end of the story, one that is played out daily in technology companies – a bet that didn’t pay off. But instead, a couple of remarkable things happened. First, Richard and David sat down and decided that their existing call processing software and engineering expertise could take a U-turn and instead produce a Voice PBX that ran over the network – all signaling and media would travel over the corporate network instead of using a separate wiring system. The software that had been routing multimedia traffic over isoEthernet would have to now route voice calls over IP, and a complete set of apps and hardware would be needed to complete the effort.

A willingness to scrap a big project and divert a skilled engineering team to a completely new venture is unusual enough. The second amazing thing that happened was that Richard and David were able to convince Intecom management that this was a viable scheme. Company President George Platt (no relation) gave the concept a green light, and suddenly the pressure was all on Richard’s shoulders. Richard’s engineering team had to show that the concept of network-based telephony was viable. There were no commercial VoIP systems at the time, and in fact there was a lot of skepticism that the concept could get off the ground. The pressure was on.

Engineering Crunch Time

Paul Hahn

What ensued was classic Richard Platt. Having successfully made his pitch to management, he now had to get his Engineering team to produce a completely new product from a standing start – and time was of the essence.

Fortunately, Richard has an asset that he shares with other great technical leaders: a high-powered Reality Distortion Field. Richard told his engineering managers what had to be done, and his belief in their ability to produce infected them with the confidence they needed to do the job. (I’ve seen this again and again with Richard. His belief in the people who work for him is so sincere and so infectious that people respond with herculean effort to make his visions come to life.)

Before the year was out, Paul Hahn’s team had hacked a basic Intecom phone into a VoIP network appliance, John Alexander’s team had created the call processing software needed to perform basic PBX functions, and Jeff Sanders’ team had a soft-phone and initial set of applications ready to go.

The system was debuted to Industry Pundit Jeff Pulver in October 1997, perhaps marking the first public demonstration of a commercially ready VoIP phone system. It was followed a few months later by a demonstration at the PBX 2000 show in Washington CD, and the debut of a new company, Selsius Systems, Inc. Despite claims to the contrary by 3Com, Selsius Systems was the first company delivering a scalable PBX that ran over the network¹.

The Next Level

John Alexander

Being first to market is a great honor, but it doesn’t give you a lock on success. Witness Netscape, the Apple Newton, eToys, etc. In their first year of selling systems, Selsius was able to move a few thousand phones, but mostly to tire-kickers. The big enterprise buyers weren’t going to buy a new technology from an unknown company.

At this point David and Richard again worked some magic – convincing Intecom that it was in their best interests to sell Selsius to a big, established company that could give the technology the sales push that it needed. Cisco Systems stepped in with a good offer and took the deal away from runner-up Northern Telecom. (Given the astonishing fall in Nortel’s fortunes over the next few years, this decision seems charmed in hindsight. At the time it was not so obvious that Cisco, with little telephony experience, was a better partner for this new business.)

Without the Selsius acquisition, it is very unlikely that Cisco could have become the dominant player in VoIP in just a few short years. Although our Cisco badges say we have “No Technology Religion,” the truth of the matter is that the Selsius CallManager architecture could never have been built in-house at Cisco.

CallManager was a Visual C++ call control app running on Microsoft servers, using an Access database, and communicating with telephones using the proprietary SCCP protocol. While these decisions were all perfect for the product, they just wouldn’t have happened inside Cisco engineering.

Fortunately, when the Selsius acquisition was complete in late 1998, the rest of Cisco Engineering just had to swallow hard and make the best of it. And make the best of it they did. A huge effort was launched, involving hardware engineering, sales, marketing, and software. As this progressed, Richard took on progressively greater roles in Cisco Voice, climbing the ladder to the lofty twin perches of VP and GM of the Voice Group.

Jeff Sanders

After just a few years of hard work, Cisco’s CallManager became a billion dollar business, with at least that much in additional pull-through for other Cisco hardware. Cisco now ships almost half the IP phones lines (see Infonetics) in the world, and as people move inexorably to VoIP, their total share of the entire voice market will only increase. CallManager has beefed up to a distributed system that runs on Linux or Windows servers, and can talk to SCCP or SIP phones. Systems can support 30,000 users in a single cluster. TV viewers know that even US President Bartlett uses Cisco IP phones, along with his crack anti-terrorism team at CTU, including Jack Bauer.

Five Years at the Helm

After guiding Cisco’s voice products to this level of success, the entrepreneur in Richard started to get itchy for a new venture. With his track record, he could have easily raised money to start a new company, but instead he decided to try to create a startup within Cisco.

Richard and David moved to Linksys in 2005 with the goal of creating a competitive small business VoIP product. Like in the Selsius days, this required some deviation from standard Cisco engineering practices, and Linksys seemed like a safe harbor.

The resulting product, the Linksys One product line, is just now achieving a level of maturity that is allowing it to compete in businesses much smaller than those that CallManager can service. Will it be the home run that CallManager was for Cisco? It’s too early to tell, but with ongoing commitment from Linksys and Cisco management it certainly has a fighting chance.

Coda

At the end of May, Richard started circulating the word that he was retiring from Cisco. There was much wailing and gnashing of teeth from his devoted followers, but at the same time, most of us understood his reasons for leaving. If you love engineering, creating products, and working with people, life as a Vice President at Cisco combines incredible demands and frustration with little joy. It’s not an easy job, and it is often not a fun job. In fact, general consensus among those who knew Richard well was surprise that he lasted as long as he did.

And so in June, after the usual round of farewell celebrations, souvenir gifts, and a few tears, Richard was gone. He was coy about his future plans, saying right now he’s just going to relax and spend some time in that time-honored pursuit of the well-heeled, getting his private pilot license. It may be that a no-compete agreement with Cisco has something to do with that storyline. Or it may be that Richard’s life outside of work is full enough to keep him engaged the rest of his life. Time will tell.

Richard, David, and the rest of 30 or 40 original Selsius team₂ deserve credit for launching an industry, and for creating a huge new business for Cisco. You can argue that VoIP was in the air, was inevitable, and that somebody was going to do it if Selsius didn’t. That’s true, but Selsius was not only first, they managed to turn that first mover advantage into total dominance through hard work and smart business. Richard was a huge part of that, and Cisco is going to miss him.

As will I.

Footnotes

1) The truth about who delivered the first commercial VoIP PBX may never be truly resolved. 3Com and Selsius both unveiled their solutions nearly simultaneously. One of the biggest problems with 3Com’s initial release was that their system was limited to 255 phones, which was a problem when trying to sell to the enterprise. 3Com’s product is still available, and has seen a great deal of enhancement, but Cisco’s giant gravitational field sucked up so much market- and mind-share that 3Com has had to settle for a role as niche player.

If you want to make the case for 3Com’s role as #1, and have some good facts, by all means please add them to the comments section!

2) I’d be interested in creating a roster of everyone who was a Selsius Systems employee or contractor during its short lifespan. Send an email if you’d like to help!

The IEEE has announced its list of medal winners for 2007, and this year’s Richard Hamming medal was awarded to Dr. Abraham Lempel:

For pioneering work in data compression especially the Lempel-Ziv algorithm.

This is a timely award, because it comes on the 30th anniversary of the publication of the first of two seminal papers by Dr. Lempel and Jacob Ziv, his associate at Technion – Israel Institute of Technology.

In 1977 Ziv and Lempel published A Universal Algorithm for Sequential Data Compression, describing what would come to be known as the LZ77 algorithm. In 1978, they followed this with Compression of Individual Sequences via Variable-Rate Coding, which described what came to be known as the LZ78 algorithm.

Both of these algorithms use macro substitution to compress arbitrary data. By replacing a long sequence of bytes with a short macro, compression is achieved. Both algorithms build this library of macros dynamically, adjusting to the input text as it is read. They differ in the way they build the library of macros.

LZ78 was eventually reduced to practice by Terry Welch with the publication of the LZW algorithm, used in UNIX compress, GIF files, and elsewhere. LZ77 provided the core of the deflate algorithm, which is used in the Zip standard, arguably the dominant lossless compression algorithm since it was introduced by PKZIP in 1993.

It would be hard to overstate the impact of these two papers in the world of data compression. While there are other lossless algorithms that can compress as well as LZ77 and LZ78 (e.g. PPM), the two LZ algorithms have won the battle of speed and efficiency almost 20 years. Applications for the algorithms vary from desktop applications to graphics file formats to tape drive controllers. As an example of their influence, Citeseer lists 458 citations for the 1977 paper, and 293 for the 1978 paper.

Dr. Lempel is currently employed by HP, performing and directing research at HP Labs in Israel. He was kind enough to answer a few questions for this article.

Mark Nelson: In 1977 and 1978, you published two now-famous compression papers with Jacob Ziv that provided the background for some of today’s most effective and popular compression algorithms, such as the deflate standard used in Zip-compatible programs. At the time, did you imagine that your work would be so broadly used 30 years later?

Abraham Lempel: We were so excited with the results of our work, that we did not give too much thought to this question at the time. As more and more extensions and versions were published by other researchers, we recognized the seminal nature of this work and viewed our approach to lossless data compression as a long term method.

MN: The papers you published provide a mathematical and theoretical description of universal compressors, but don’t dig into implementation details. Do you have much personal interest in converting theory into practice, or do you get the most satisfaction from the research work that provides the foundation?

AL: Following the publication of 1977 and 1978 papers, we both participated in writing an invention disclosure which included all the details of a preferred embodiment implementation. This was done while I was on sabbatical at Sperry Research, and the 2 granted patents ended up as Sperry, now Unisys, Intellectual Property.

MN: Since your seminal work, I think the most significant advance in lossless compression has been the creation of block-sorting algorithms, as described by Burrows and Wheeler. Do you think there are any revolutionary new techniques on the horizon, or should we just expect minor incremental improvements?

AL: Hard to tell. When we worked on our method, the Huffman algorithm was considered the last word in lossless data compression. Now, the various LZ versions are treated as such. All this is in the context of text compression. I do expect radical progress in lossless image compression, which is a different beast from text.

MN: After a distinguished career at Technion – Israel Institute of Technology and almost 25 years with HP Labs, you’re at a point where many people would be thinking of retiring and slowing down. Is that in your near-term plans?

AL: I do plan to retire in 2008.

It is great to see the IEEE honor Dr. Lempel for his work. It would have been hard to understand how important this was in 1977. Today, with the benefit of 30 years hindsight, it clearly stands out as a masterpiece.

Reference Links

• Citeseer page for A Universal Algorithm for Sequential Data Compression (1977)
• Citeseer page for Compression of Individual Sequences via Variable-Rate Coding (1978)
• IEEE Bio Page that accompanied the Hamming Medal Announcement
• List of Hamming Medal Recipients on the IEEE site.
• Wikipedia entry for Abraham Lempel
• Wikipedia entry for Jacob Ziv
• HP Bio page for Abraham Lempel
• Wikipedia entry for LZW compression.
• My 1989 DDJ article on LZW data compression, including C source
• The zlib home page. zlib is the free and widely used library that implements the deflate algorithm using in Zip programs. deflate combines an LZ77-style algorithm with a Huffman coder.

The eminently quotable Samuel Johnson had a very pragmatic view about writing, and was quoted by Boswell as having said:

No man but a blockhead ever wrote, except for money.

Personally, I think Johnson is pretty close to the mark on this one, but I will add one caveat. Ask any writer about their first book, and the thing they remember best is the thrill of seeing a volume with their name on it up on the shelf – money has nothing to do with it.

My first effort was The Data Compression Book, published all the way back in 1992, when there was a lot less interest in the field than there is now. It was unbroken ground, which meant there was room for an amateur in the field, and I nearly had it to myself.

With the help of DDJ Editor and mentor Jon Erickson, I convinced M&T Books I could do a creditable job on this book. They took me up on it, and believe me, the first time I walked into Taylor’s books in Dallas and saw three or four of these on the shelf, it was a thrill you can hardly imagine.

Fifteen years later, with a handful of other books behind me, I’m much more Johnson-esque and blase these days. Show me the money.

But every once in a while, something manages to pierce this hard-boiled shell and remind of what it was like to first see that book in 1992. This week it was an email out of the blue from somebody named Steve Johnson, who was kind enough to let me reprint his email in its entirety:

Dear Mark,

I have had a copy of The Data Compression Book in my possession for a long time. It was a book of instrumental importance to me when I was starting out a fledgling business long ago – in 1992 – which turned into an image compression company partly because of your book. Your book captivated me with the compression problem, and taught me the basics of information/coding theory (enough to become a dangerous dilettante). Out of that understanding sprang a company called Johnson-Grace Company, which created the first streaming online media ever. AOL used my algorithm to put pictures online in early 1994 when the world dialed in at 2400bps or 9600bps, and ‘digital pictures’ were as fantastic as radio in 1920. My little company went on in 1995 (around the time MSN was launched) and created ‘streaming sound’ and ‘moving pictures’ (slideshows with sound) and then simple telephony (still over dialup, now at 14.4kbps, we added a ‘talk’ button to AOL’s Instant Message box – and then subtracted it after seeing the challenge of creating a consumer grade experience over low bandwidth dial up!).

AOL bought my company in 1996, and the algorithm (AOL’s proprietary ‘ART’ format) still compresses billions of images everyday on the backend of their web delivery system. I owe a great deal to your book – for its clear accessible style, and excellent coverage of the subject. It quite simply taught me (an economist) how compression works, and I managed to put it to a use that solved an important problem.

So here’s why I’m writing (besides finally thanking you after all these years!).

My eldest daughter Emma graduated from high school earlier this year (we live in Boston) and has just commenced her freshman year at Oberlin College this fall. As a graduation gift, I’m presenting her with a bound set of my favorite books called “Dad’s Great Books,” of which I have included my rebound copy of Data Compression. It is no doubt one of the most influential books I’ve ever read, and I hope Emma cherishes the book as much as I did (she happens to be fascinated with information theory at the moment).

If it isn’t too much trouble to ask, I would greatly appreciate it if I could send this bound copy to you for your autograph and have you return the book in the FedEx envelope that I would include.

Please let me know, at your earliest convenience, as well as a good address to use for sending this your way.

Warm Regards,

Steve Johnson

When I read this email to my wife she had tears in her eyes, and yes, I was a bit verklempt myself.

Steve sent me a nicely bound copy of the book and I inscribed it to he and Emma (without even asking for an honorarium!), and with luck she’ll have it on her shelf in a few days, a curious souvenir of bygone times.

All in all a Hallmark moment, although there is a scary side to the whole thing. Steve comes from a background in Economics, knowns as the Dismal Science. Emma is interested in Information Theory, which has no nickname yet, but perhaps should be known as The Yet More Dismal Science.

Should this continue, I fear for Emma’s children. They’ll have to find a branch of mathematical science even more obscure, confusing, and impossible to explain at family functions. Will they be Set Theorists? Transfinite Algebrists?

Regardless of their intellectual or business pursuits, it seems inevitable that anyone from this line of people will be delightful to have around.

Thanks, Steve, for a reminder of what it was like to write that book in 1992, and best of all, to know that at least for you, it was exactly what I had hoped it would be.

I received a sad email from Andy Johnson of Greenleaf Software yesterday, giving me the news that Greenleaf’s founder Don Killen died in his sleep Monday, August 21, 2006.

Don was a pioneer in the world of desktop C and C++ programming, since he sat down in 1982 and hammered out his flagship product, Greenleaf Functions. Back in the days when MS-DOS programmers still had to use the Lattice C compiler, Don saw that there was potential big market for a library that provide glue between the BIOS, the DOS API, and C programmers. Armed with references and a copy of MASM, he created the product, placed a few ads, and found himself with enough orders to go into the business full time. (Mind you, this was back in the early 80s, and Don was managing to get over $200 a pop for this library.) You can follow Don’s version of that history here.

I went to work for Greenleaf in 1989 after burning most of my bridges with telephony startup SRX (later absorbed by Teltronics.) It was a big change for me – Greenleaf was a small family business and I had to try to learn how to operate in that kind of environment. During all my years with Greenleaf, I could never forget that it was Don’s world, and we were lucky to get to play in it.

Greenleaf had quite a few good years, but as the 90s waxed the product mix wasn’t attracting the kind of sales that it needed to keep the place going, and Don’s health was always an issue. Despite a fortunate kidney transplant, as he got older, I don’t think he had the energy and vigor he needed to reinvent the company for the new millenium. Seeing the writing on the wall, the development team branched off to form a contract programming company called Addisoft Consulting. Tammie Williams, Patrick Little and Ruby Hjelte are still there, and worked with Don and Greenleaf from time to time.

Working with Don was a great experience, and I can honestly say that I learned more about the software business from both him and Tammie than I could have managed anywhere else. It was a rare opportunity for an inexperienced programmer, and I really feel luck to have had the opportunity, and to have gotten to know Don.

May he rest in peace.