Jeff Bezos Hawks the Kindle

It’s still not clear whether electronic books are the wave of the future or a consumer products cul-de-sac. Technology continues to improve, and there are certainly lots of good reasons for a device like Amazon’s Kindle to be the leading edge of a major wave of adoption. A few of the more obvious arguments include:

• Reduced cost of distribution. The publishing industry wastes a lot of money printing and shipping books, and because of historical practices, creates huge numbers of books that never even get sold. Not very green, and a waste of money.
• Niche markets that can be very well-served. For example, high school and college students can replace those 30-pound backbacks with a 30-ounce tablet-sized device.
• Removal of barriers to publication. The news and magazine businesses are being revolutionized by self-publication in the form of blogs. Self-publication is possible in the printed book world, but it is still a rather awkward process. Publication to electronic format is presumably a trivial problem.
• Integration of information resources. A device like the Kindle allows you to consult the Internet, written reference materials, and your personal notes all from the same device, making it a true information portal.

But consumer acceptance is a fickle thing, so we don’t know if these rational arguments are going to fly. And of course, every writer who reviews a device like the Kindle or the equally capable Sony Reader feels compelled to write something along the lines of “But I just can’t imagine forgoing the pleasure of curling up on my couch with a good book.” I’m sure that when Gutenberg introduced movable type there were millions of industry reviewers posting notes to their blogs saying “The uniformity of the type is just esthetically unpleasing – I love it when I recognize the script of one of my favorite transcribing brothers.”

And then of course there’s the Steve Job’s money quote on the subject:

“It doesn’t matter how good or bad the product is, the fact is that people don’t read anymore, forty percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don’t read anymore.”

So it’s really pretty hard to be sure just which way this is going to go.

Looking Beyond The Reader

Sony’s Reader

For electronic books to succeed, one thing is certain: the physical reading experience needs to match up well with the one we have right now for our printed media. In some ways this problem is already solved – readers like the products from Amazon and Sony are book-sized and lightweight, with displays that are doing their best to match the various good qualities of paper.

But there are still issues that need work. One of the most important is in the area of layout and markup. A presentation format such as HTML is designed to work with multiple display sizes, repositioning elements as needed. This doesn’t necessarily work so well with textbooks, magazines, etc., where graphic artists invest huge amounts of time and energy on positioning, font selection, and other esthetic issues.

Additionally, there is am staggering amount of material that is simply not in a format compatible with today’s electronic books. Even magazines being published today are not always ready for transfer to an eBook format, and there is of course a massive backlog of valuable material in the world that has never existed in digital format.

Into this void steps Robert Maxwell Case, who has developed a system called ReadAllOver, and is attempting to exploit it via his company, SeeAndBelieve.com. SeeAndBelieve.com has created a digital layout technology called ReadAllOver (the company seemingly has a love affair with awkward CamelCase constructions) that does a superior job of preserving the look of printed materials. The company web site gives a good demonstration of exactly what they are capable of doing – it is definitely worth your time to take a look.

I asked Robert if he could take the time to answer a few questions about ReadAllOver, and he was gracious enough to respond.

Questions With Robert Maxwell Case

Mark Nelson: Hi Robert. I just recently became aware of your company, SeeAndBelieve.Com, and your imaging system, ReadAllOver. Before we get into the details of your technology, can you tell me a little bit about the history of your company? How long have you been at it? Are you working mostly solo or do you have some help? What kind of background do you have that got you into your current work?

Robert Maxwell Case: Sure, Mark. I come from a background of being a full-time musician and a part-time graphic designer. Around 1991-92, I was unhappy with then-current digital halftoning routines and began experimenting on my own. So I’ve been at it 15-plus years.

Jimmy Kung (left) and Robert

I’m not a programmer, so I have had a succession of programming assistants. In recent years, they have come from my affiliation with the Computer Science department at Texas State University-San Marcos where I am a seven-year member of the Industrial Advisory Board. Currently SeeAndBelieve.Com, in addition to myself, has one full-time employee, Jimmy Kung, and several part-timers.

The first of my five U.S. patents (three issued, two pending) was filed in 1993 in response to some interest expressed by Steve Carlsen, developer of the .TIF graphics file format at Aldus (he’s now with Adobe Systems.)

MN: Can you give me a capsule summary of ReadAllOver? How does it differ from page layout systems like we see in web browsers or PDF viewers? Does it differ from the rendering systems used in Sony and Amazon’s current eBook readers?

RC: Well, ReadAllOver in a nutshell is a digital halftone-based graphics system suitable for eBooks. It renders on the screen a digital page with the “look and feel” of a printed page, with all included graphic elements, typography and images, placed precisely as the graphic designer intended. It differs from existing web browsers and .PDF viewers in that it relies less on text files and font metrics and instead places more emphasis on a simplified, highly-compressible bitmap image. In many respects, it is a “picture” of the page, with an ancillary text file.

The Sony and Amazon eBook readers are primarily text-based, offering a limited number of typefaces and few graphics. They both use the E-Ink subtractive screen and we think ReadAllOver’s halftone system can be tailored to enable a good fit with that screen.

MN: It looks like your technology emulates the halftone process used to render photographic images in newspapers and magazines. How does it improve on that process to achieve smaller file sizes? Do you have data showing the level of compression you get for specific images? And do you also render type as halftone images? That would seem a lot less efficient than treating type as marked-up text.

RC: That’s right, ReadAllOver does emulate the halftone process with one major difference, and that is that typography can be processed with it and not fall apart.

Samples of ReadAllOver output (Detail may not be representative, images were resized)

The big idea is that the output image starts as an interim monochromatic checkerboard pattern, beginning from gray. To simplify, we derive the output image by rendering local areas of the input image that are darker than checkerboard gray by turning corresponding output monochrome white pixels to black. Conversely, we render local areas of the input image that are lighter than checkerboard gray by turning corresponding output monochrome black pixels to white. The result is a checkerboard-based ordered dither that can be re-ordered for variable-length run-length compression.

We presently are comparing compression levels for specific images and plan to publish the results. So with typography, abrupt gray level changes such as font outlines fall on the black pixels of the interim checkerboard to render any font without reliance on font metrics or hinting. For example, ReadAllOver pages containing only typography are competitive in file size to marked-up systems, and with the additional attribute that our system is able to efficiently render any font, in any language, and any image, placed correctly. That’s exciting.

MN: I get digital delivery of some magazines already. For example, IEEE Spectrum is delivered using technology from Qmags. Do you think you can do a better job than they are already doing?

RC: Qmags has selected Adobe PDF as its preferred file format, so the comparisons I’ve made previously between ReadAllOver and PDF come into play here. Taking a quick look at Qmags’ file sizes, I would say that ReadAllOver is competitive and looks subjectively more “print-like.”

MN: Have you released any sample code or SDKs for people to work with your compression technology?

RC: Not at the present time … we have been too busy developing our prototype (you can see it at SeeAndBelieve.Com ).

MN: The idea of an electronic book reader has been floating around for a long time, but right now it seems like we’re finally seeing designs that are actually gaining some traction. There are still a lot of naysayers, however. What do you see in the future for the electronic book reader? Will it eventually do for reading what the iPod did for music? Or will it forever be a niche product that is stuck on the verge of popular success? And how hard will it be for you to get the manufacturers of eBook readers to adopt ReadAllOver?

RC: Mark, I do believe that an electronic book reader will achieve iPod-like popular success, and, hopefully, in the near-term. In my opinion, the more book-like the readers become, the closer we will get to that so-called tipping point. Then the public will recognize the added benefit of having access to any publication, including one’s own library, available whenever and wherever they desire. There are some of obstacles still to be overcome, like screen pixels that are pretty large, and wired and wireless transmission pipes that are pretty small.

I personally would like to see a reading device with 8-1/2″ x 11″ facing screens, about the size and weight of a coffee-table book, with graphics-intensive magazines and newspapers rendered on screen nearly indistinguishable from paper and ink. I certainly can imagine a college student not having to carry a backpack full of textbooks around campus.

Here’s my vision for our product: We’re hoping to make adapting to ReadAllOver as seamless and easy as possible. We envision three communities of ReadAllOver users:

1. The reader

2. The publisher/bookseller
3. The hardware manufacturer.

We plan to provide each of these groups with a cost effective, easy-to-use solution. The ReadAllOver Viewer will be free for readers. They’ll read .RAO files on dedicated eBook devices, and most likely on other devices that could emulate a book-like reading experience. The inherent look and feel of printed material should reduce eye strain and, of course, we plan to offer such extras as page flipping, content search, printing, etc.

Publishers and booksellers will use ReadAllOver Publisher, our media content production system. They’ll be able to convert content from scanned material, as well as from existing editing and page layout systems and standards, including Adobe InDesign and FrameMaker, Quark XPress, MS/Office, Open Office, etc. (and, of course, we’ll include a .PDF to .RAO converter.) This community will appreciate a built-in high level of content protection with ReadAllOver’s emphasis on bitmaps. Hopefully they won’t need more, but if they do, our product should easily be able to incorporate additional encryption and DRM mechanisms.

We also plan to collaborate with eBook (and other display device) manufacturers in order to provide built-in support for the ReadAllOver rendering system. We feel we can efficiently adapt our system to display components with limited available grayscale and color levels. We also hope to offer a fixed-bit-rate option where every delivered page is the same file size.

Conclusion

Thanks Robert, this is all interesting stuff. I can see advantages to ReadAllOver that we don’t get from layout systems like PDF or HTML, so perhaps you will be able to hammer out an effective market position. I don’t have an eBook reader yet, but I think when the technology reaches the point where I can have a color Kindle I’ll probably jump on board. I’m almost there now, but since the Kindle is perpetually sold out at Amazon.com, my dollars are still safely in my wallet.

Windows Developer Magazine July, 2002

Back in March 1998, Tom Armstrong and I published an article in Dr. Dobb's Journal showing a technique for implementing transparent regions in bitmapped images. In those days, Windows developers had to roll their own transparency solution, and the path was quite arduous. Today, Windows programmers can render images with transparency using Microsoft's IPicture COM interface, which is considerably easier than the solution Tom and I suggested. It's an easy way to accomplish the same task - if you watch out for its single notable drawback

The IPicture interface is a good way to draw images with transparency today. In the future, you'll probably want to use GDI+, Microsoft's new GDI interface for C++ programmers. I'll show you how to convert the sample program to write cleaner code with GDI+, and discuss the improvements it brings to the table.

Transparent GIFs

When creating a GIF file, you have the option of designating one color in the image as transparent. This transparency feature is normally used to make it appear as if an irregularly shaped image is floating on the background. An example of how this can be useful is shown in Figures 1 and 2.

In Figure 1, two news articles displayed on a Web page have graphical icons that illustrate article topics. The users of this particular web site have the ability to change their personal settings to use a different layout, which changes the background color of the page. In Figure 2, a different theme is used to show the same articles.

In both Figure 1 and Figure 2, the background of the GIF file used to illustrate the article is transparent. Web browsers know that they need to let the background color show through in the transparent regions of the graphic. You can see that the browser deals with the transparency properly in both cases, with nice effect. A beige background in Figure 1, a white background in Figure 2, and in both the topic images float nicely above the background.

Figure 1 - Transparent GIFs on a light brown background

Figure 2 - The same GIFs on a white background

Loading the GIF File

Getting an external file into a format Windows can display can be a chore. Tom and I chose to skirt the issue in our original article by working with files stored in Windows' Device Independent Bitmap format. Although not a popular file format, it was one of the few that Windows was able to read and write without help from third-party libraries.

The situation has improved quite a bit since then. You can now use Microsoft's OleLoadPicture() function to load both JPEG and GIF files. Given the popularity of these two file formats, you might find that OleLoadPicture() covers all your needs in this area.

OleLoadPicture() loads data from an IStream object and stores it in an IPicture object. You'll find that having the data in IPicture format is quite convenient when it comes time to render it. However, the odds are good that you want to be able to load your data from a file, which is not quite the same thing as an IStream interface.

The easiest way I know of to get from a file to an IStream object is to simply read the whole thing into memory and then use CreateStreamOnHGlobal() to create an IStream. A fragment that accomplishes that is shown here:

At this point you have an IStream object ready for use, and all that remains is to make the call to load the IPictureobject:

Although the code isn't particularly elegant looking, it has accomplished the goal of reading a GIF or JPEG file into memory in just a dozen lines or so. All that remains is the task of drawing the code on the appropriate display.

The Drawing Part

In our original article, Tom and I presented a straightforward method for displaying bitmaps with transparency. It required the use of three bitmaps:

1. The background bitmap.
2. The bitmap to be displayed, with a known transparent color.
3. A special mask bitmap. The mask bitmap should have all bits set to zero in areas where the display bitmap is opaque, and all bits set to one in the areas where the display bitmap is transparent.

Displaying the bitmap is simply a matter of performing the following sequence of operations:

Output = Bitmap1 XOR Bitmap2 AND Bitmap3 XOR Bitmap 2

With just a little bit of work, you can see that the output will be a copy of the pixel from Bitmap1 in every position where the mask (Bitmap3) has a value of all ones. The output will be a copy from Bitmap2 in every position where the mask has a value of all zeros.

That's all you really need to understand in order to draw bitmaps with transparency in any sort of graphics environment. What made it into a long article in 1998 was simply all the plumbing you had to do in order to get the background bitmap, create the mask, get the display bitmap, and so on.

The good news is that Microsoft's IPicture interface can take care of all of the messy details here with one call to its Render() function. Once you have loaded the GIF file into an IPicture object, the snippet of code below is all you need to get it onto your display:

Note that the code shown here centers the image in the client window, which complicates things just a bit. It also has to deal with the fact that IPicture uses the HIMETRIC mapping mode, which I have to convert to the more convenient pixel measurements.

A First Pass

Pressing this code into service reveals one striking problem with the IPicture interface. Calling its Render() function with the intention of writing directly to your display window leads to a somewhat unpleasant surprise. It turns out that IPicture uses the standard algorithm I describe earlier to implement transparency. Unfortunately, it performs each of the three steps directly on the display, so you get to see each intermediate bitmap in the process. The four screen shots below show it in all its glory.

Figure 3a - Opening the file

Figure 3b - The background has been XORed with the image

Figure 3c - The resulting bitmap has now been ANDed with the special transparency mask

Figure 3d - After a final XOR, the image is rendered with transparency

This sequence might be interesting to the average DDJ reader, but the end users of your program are hardly going to want to see these three images flashing in quick succession across their screen. The flickering is barely noticeable with small images, but on slow machines, big images go through the painting sequence with painful precision.

The solution to this problem is straightforward. Instead of drawing directly to the screen, you can create an off-screen bitmap, have Render() draw to that bitmap, then copy the completed product to the display. Having done that, you'll find that not only is the flicker eliminated, but the drawing operation actually completes much more quickly.

An MFC routine that accomplishes this in conjunction with the rendering routine above is shown here:

You can see that it only takes a few lines to create the off-screen bitmap. The code above creates a bitmap of the correct size, fills it with the correct background pattern, then calls the rendering routine. Once that is done, all that is left is a BitBlt() call which copies the result to the screen.

The Sample Program

To demonstrate this, I created an MFC program named TransparentGif, forgoing the usual document/view architecture. Instead, I just paint the entire client background of the window with a pattern, the better to view the transparency of the loaded image.

When the program first starts up, by default it draws directly to the screen. If you are blessed with a sub-Gigahertz CPU, you should be able to see the flickering effect quite clearly when loading the sample GIF included with the source. I added code to force a redraw every time you click on the image, which allows you to see the effect in all its beauty.

You can then use the Options|Off Screen Draw menu item to switch to the use of the off-screen bitmap. A little sampling between the two should be enough to convince you of the value of the off-screen drawing technique.

Switcing to GDI+

Looking forward to the future, you might find that this task is going to become a bit easier. Microsoft has developed a new interface to GDI for Windows programmers called GDI+. The class libraries and supporting DLL for GDI+ are shipping as part of Windows XP, and can be installed as an add-on to Windows NT, 2000, 98, and ME.

The header files and import libraries required for GDI+ are part of Microsoft's Platform SDK. To get this SDK, you'll have to have to subscribe to Microsoft's MSDN program at the Operating System Level or better, which has a list price of $699.

You need to perform two installations to develop and test GDI+ programs: the Win32 Platform SDK and the GDI+ runtime files. Both of these can be found on your MSDN CDs or can be downloaded directly from the MSDN site The GDI+ runtime kit can also be distributed with your software, so your end users need no special licensing.

Additions to the project

I made a copy of the TransparentGif program, renaming all the files and classes so that the new project goes by the name TransparentPlus. I then set to work making the changes needed to use GDI+ as the graphics interface.

The first few changes are the ones you typically have to make when using a new library. I added an include of the master header file, gdiplus.h, to stdafx.h. This ensures that I have all the correct prototypes and class definitions. To make sure that the compiler was able to find the header file, I had to add the SDK include directory to the preprocessor settings for the C++ compiler. I used the default installation location for the SDK, so I simply inserted the path:

C:\Program Files\Microsoft SDK\Include

into the list of additional directories for the compiler.

All of the classes and identifiers in GDI+ are placed in the Gdiplus namespace, so I also added a using namespace Gdiplus; declaration to stdafx.h. In a larger project, I probably wouldn't do this; the use of explicit namespaces in the source would add readability to the code.

Likewise, the import library was in the default SDK directory, so all I had to do was drag:

C:\Program Files\Microsoft SDK\lib\GdiPlus.lib

into my workspace pane, thereby adding it to the project.

Startup code

GDI+ programs require a few lines of code for initialization, as well as a cleanup call. I added the following lines to the InitInstance() function in my application class:

This code is lifted cookie-cutter style straight out of the GDI+ documentation. The token initialized by the startup function is passed to the GdiplusShutdown()method when you are done using the library.

File Loading

As you recall, the file loading code in the standard GDI program was a real rat's nest. The hoops I had to jump through in order to load a file into an IPicture object gave a classic demonstration of an ugly Microsoft API.

I'm happy to say that someone working in API design in Redmond has seen the light in this area. Loading images is an order of magnitude easier with GDI+. Instead of loading into an IPicture COM object, GDI+ supports a C++ Image object. This requires that I change the type of CMainFrame::m_pPicture in the appropriate header file. I can then reduce the file loading code from the ugly mess I had before to this picture of simplicity:

This is a huge improvement, and by itself is almost worth the cost of the SDK.

Drawing

The drawing code with GDI+ is a bit simpler as well, mostly because the Image object uses pixels for measurement instead of the HIMETRIC units from IPicture. Other than that a simple replacement of the Render() function with a call to DrawImage() constitutes most of the work. The updated routine is shown here:

GDI+ makes my code look a lot nicer, and informal tests show that GDI+ renders a bit faster as well. Better yet, GDI+ is smart enough to avoid rendering all three steps directly on to the screen, which means you won't see a flash of the XOR'd or masked image.

Despite the speed of GDI+, I think you'll still find that you need to do this image rendering in an off-screen bitmap. Experimenting with the second test program shows that although you don't see intermediate images, you still get an annoying flickering as the image as drawn. Rendering off-screen eliminates that problem.

Conclusion

This article gives you a couple of different ways to effectively use transparent GIF files in your Win32 programs. For today, the IPicture interface does a pretty good job. When GDI+ rolls out as a standard part of Visual Studio, you'll probably want to switch over to it for your imaging needs. Either way, you've got good options that make things look a lot nicer than they did in 1998.

Links

Source for both demo programs

Dr. Dobb's Journal July, 2002

Update: Intel is no longer giving these libraries away for free - be prepared to fork out real money for this code. Does this render the information in this article useless? I don't know, add your comments below!

In this article I take a look at Intel's Win32 JPEG Library, and present a slide-show app I built to test it. I found it quite easy to render JPEG images using this library, and I think the demo code I provide will provide you with a good starting point for your own work. I'll also perform a short comparison of Intel's library against Microsoft's standard IPicture component, both in terms of performance and ease of use.

Intel's Performance Libraries

Not too many people are aware of the fact that Intel dedicates a substantial amount of energy to its software products. As you would expect, most of the software Intel creates is designed to pull through more hardware sales, not to generate revenue of its own. One good example of this is Intel's JPG Library.

Intel added the MMX instruction sets to their CPU chips starting with their 486 Pentium MMX processors. The MMX instructions are designed to speed up certain operations needed in multimedia applications. One of these is the Inverse Discrete Cosine Transform (IDCT), which is used to when decoding JPEG images.

Developers typically can't take advantage of extended instruction sets in products designed for a widespread market, because they can't count on their target machines having the correct extensions. Companies like Intel or AMD can still leverage their proprietary processor features by providing device drivers, browser plugins, or other components, but this doesn't completely satisfy their desire for market penetration.

For example, my sample slide-show program doesn't use operating system components to decode JPEG files, meaning I can't take advantage of the MMX instruction set without writing specific routines to detect the target processor type, then branch to one of two different decoders depending on the result.

Intel correctly figures that they can convince me to offer support for their hardware if they dish it up on a silver platter, and that's exactly what they do with their series of Performance Libraries. At this time, Intel is offering the following roster of libraries:

• Intel Integrated Performance Primitives - an abstraction layer for multimedia processing.
• Intel Math Kernel Library - linear algebra and FFT routines.
• Intel Recognition Primitives Library - code for speech and character recognition software.
• Intel Signal Processing Library - access to Intel CPU functionality similar to that used in Digital Signal Processors.
• Intel Image Processing Library - image manipulation functions, including filtering, thresholding, morphing, FFT, and DCT.
• Intel JPEG Library - encoding and decoding of JPEG images.

All of these libraries have processor specific routines for various families of Intel CPUs, and all have generic code that will also run on any standard Intel Architecture CPU.

What's the Catch

In a clear case of enlightened self-interest, Intel is making these libraries freely available to software developers. You will have to go through a licensing procedure to in order to redistribute the libraries with your products, but as far as I can tell there aren't any circumstances under which Intel will require payments or royalties.

Support is another matter, as you might expect. Intel's web site has FAQ material, known issues, and so on. Talking to a person will require subscription to a premium support service.

Where to go

You can find Intel's JPEG library on their Web site, www.intel.com, by following the links to
Products/Software/Performance Libraries. At the time this article is going to press, the direct link is at:

http://www.intel.com/cd/software/products/asmo-na/eng/perflib/ipp/index.htm.

The library has many of the niceties that you would expect from Intel. Figure 1 shows one of the first screens from the professional installation program, which naturally allows for a full uninstall as well.

Figure 1 - Installing the Library

As you install the library, you'll also see that it includes examples, white papers, documentation, and support for C/C++, Visual Basic, and Delphi.

Using the Library - an Example

I originally stumbled upon Intel's library while searching for a solution to a pressing problem. My friend Darrick Deel and I were intent on supplying some entertainment to a company shindig at Cisco's development offices in Dallas. We were going to act as roving digital photographers, snapping candid shots of our coworkers as they engaged in atypical mirth and merriment.

Our goal was to use a notebook PC and a projector to randomly display shots of partygoers during the event. What we needed was a slideshow
program that would randomly display JPEG photos from a directory tree. We were hoping for a program that would run in unattended mode, but would still be able to periodically scan the photo archive looking for new shots. This would allow Darrick and I to add photos across the network without having to break into the show.

Slideshow software is easy to come by. As Figure 2 shows, a simple search on a popular download site turns up literally hundreds of possibilities. A surprisingly large percentage of this software is either free or quite cheap.

Figure 2- Searching for Slide Show Software

Despite the bountiful selection of software, Darrick and I were suffering from the Goldilocks syndrome - nothing we tried out was just right. Following in an age-old tradition among programmers, I decided to reinvent the wheel. Having recently downloaded Intel's JPEG Library, I knew I had just the tool I needed to make a quick
job of it.

Using Intel's Library

The library ships from Intel with a pretty decent manual in PDF format, and a nicely fleshed out sample program called JPGView. The manual is full of code fragments that do a good job of illustrating the various features of the library. After reading the manual and examining the sample code, the process of displaying a JPEG image on the screen looked pretty straightforward:

1. Initialize the library.
2. Read the JPEG file header to get the basic image information, including size and color space.
3. Set up the various members of the data structure that will hold a Win32 Device Independent Bitmap (DIB.)
4. Read the JPEG image data into the DIB.
5. Display the DIB.

The first four steps here are accomplished in a single routine I wrote called CreateImage. This function takes an input file name as a parameter and returns a pointer to a JPEG_CORE_PROPERTIES structure, which is where the Intel library stores all its information.

The final step is done in the WM_PAINT handler for the program's WndProc. Since the first four steps create a fully functional DIB, all the WM_PAINT handler has to do is set up some parameters and draw the DIB to the output device context.

SlideShow

I put this knowledge together in an imaginatively named program called SlideShow. Although the code is built using Visual C++, it doesn't use MFC, so you should have good luck using the program with Brand X C++ compilers.

SlideShow starts off as a console program that takes a directory name as a command line argument. That directory name should be the root directory of your album of JPEG files. Slideshow then creates a full-screen main window, starts a five second timer, and proceeds to process the message loop.

Other than a little bit of glue code, the message handling routine for this program really only cares about two messages: WM_PAINT and WM_TIMER.

WM_TIMER

The main window of SlideShow gets a timer tick once every five seconds. It has three main jobs to do when this timer tick occurs:

1. If needed, reload the list of pictures to display.
2. Destroy the current picture object.
3. Create a new picture object.

SlideShow keeps a list of all the pictures it finds in the JPEG directory in a C++ vector named WindowData::picture_names. It also keeps an index of the next picture to decode in WindowData::current_picture. As soon as the timer handler is entered, it checks to see if the current_picture value is at or past the end of the picture_names vector. If it is, the current contents of the vector are erased and the picture directories are scanned for a new list of picture names. (Note that this happens naturally the first time the WM_TIMER tick is processed.) The picture names are randomized so the sequences aren't repeated each time the program is executed.

After potential maintenance of the image name list, it's a simple matter to delete the current image and create a new one. CreateImage,
described a bit earlier in this article, does all the processing needed to load the JPEG image into a Device Independent Bitmap. This new image is ready to be painted on the screen, and I force that to happen by calling the Windows API function, InvalidateRect(). This forces a WM_PAINT message to be sent to the main window.

WM_PAINT

The WM_PAINT message means it is time to paint the JPEG image onto the screen. I've characterized this as being an easy task since the image is now in a DIB, and examination of the WM_PAINT handler in Listing 1 bears this out.

The actual painting is done via a call to Windows API function StretchDIBits(). This routine not only copies the image to the screen, but scales it as well. There are a dozen or so lines of code that set up the call to this function. I scale the image so that it fills as much of the screen as possible without clipping any bits, filling any unused space with the desktop background color.

Figure 3 shows what this looks like on the screen of a PC. The program quietly paints a new random image on the screen every five seconds. The machine shown here is a modified I-opener, which runs Windows 98 and has full network access. With no cooling fan to create ambient noise, the I-opener is an ideal desktop photo album.

Figure 3 - The Slideshow in Action

Making Windows into Objects

If you are used to using MFC for all of your C++ projects under Windows, you might find this to-the-API program a bit different. One thing I did to try and make it a little more structured is to encapsulate the Window data in a C struct, which could just as easily be a class.

When the program first starts, I create a WindowData object, which contains all the information regarding the currently selected
picture, the list of file names, and so on. When the window is created, I stuff a pointer to this object into the GWL_USERDATA DWORD owned by the window. From that point on, every time I enter the WndProc I can get a pointer to the structure, eliminating the need for global variables.

Integrating with Visual Studio

The download package for this program contains a project file, but you may need to do a bit of modification after you install Intel's JPG Library.

First of all, the SlideShow.cpp file includes Intel's header file, ijl.h, as shown in this section of code from the source file:

Naturally, this file won't be in your normal include path. I take care of this by going to the Project|Settings dialog in Visual C++, selecting the C/C++ pane, then the Preprocessor category. You should then enter Intel's header directory in the Additional Include Directories edit box. If you take the default installation, your dialog will look like Figure 4.

Figure 4 - Configuring the Include Path

The project file I built for SlideShow includes the static library, ijl5l.lib. If you took the default path for Intel's install, you don't need to make any changes whatsoever. Figure 5 shows the path that is defined in the current project.

If you would like to use Intel's DLL instead of the static library, or if you chose a different install path, you'll need to delete the current library from the project. After deleting, use menu option Project|Add to Project|Files to add the correct library.

Figure 5 - The correct properties for the Intel Library in Visual Studio

Intel vs. Microsoft

There are any number of reasons you might choose to use Intel's library to display JPEG images. But what about Microsoft's solution? Does the standard distribution of Windows come with components that can perform just as well?

As it happens, Microsoft ships with a standard COM object called IPicture that shares much of the functionality with Intel's library. It not only handles JPEG files, but works with GIF, BMP, and other formats as well.

Ease of use

To get a handle on its ease of use, I reworked my SlideShow.cpp to use an all Micrsoft solution. The resulting program, SlideShowMS.cpp (available for download) shares the same structure, and is probably nearly identical in terms of structure and complexity. The major changes are:

• Microsoft's image data is stored in the IPicture component, Intel's is in a JPEG_CORE_PROPERTIES structure.
• Reading the file data requires two steps and a bit of coding for Intel. Microsoft makes reading easier with the simple OleLoadPicture function, but loses points for not reading from standard files. Setting up the IStream object Microsoft needs throws away any advantage they have in this area.
• I used a standard Windows GDI function to draw the Intel object to the screen. IPicture has a Render method that is somewhat easier to use and requires less setup.

After examining these points, I think you'll agree that there's no clear winner here. If all that matters to you is how much work you have to perform to solve your imaging problems, you can stick with your personal preference and not worry about making a big mistake.

Performance

When it comes to decoding JPEG files and rendering them to the screen, Intel's library stole the show. I ran the Slideshow program through a batch of 20 JPEG files that were mostly 1600x1200 pixels in size. My non-scientific test game the following performance results:

Intel's library seems to take Microsoft's time and cut it in half! It appears that Intel's attention to performance paid off. (Interestingly, I would have expected to see a bigger delta between Intel and Microsoft on the Celeron CPU, but was disappointed.)

I'm sure there are ways to tweak my SlideShowMS program to get better performance from Microsoft's implementation. But the point of this article is to look at an off-the-shelf solution for use by programmers who may not be graphics experts. And for a straightforward implementation, Intel appears to really deliver when it comes to CPU cycles.

What about IJG?

The JPEG file format enjoys great acceptance in the software world. This is due in no small part to the availability of the Independent JPEG Group's (IJG) source code. The IJG created a non-commercial source code package that has been used as a reference and/or library by literally hundreds of programs out there. It enjoys a great deal of respect for its flexibility and feature depth.

Intel knows that many developers have already integrated the IJG code into their products, so they have created a 26-page white paper that discusses what you need to perform open-heart library replacement. There are major structural differences between the two sets of code, so this isn't an exercise for the faint of heart. But it can be done.

Summary

I found Intel's library quite easy to work with, which is probably evident given the length of my sample application. While I didn't do too much objective performance testing, I was happy with the speed I achieved when decoding and displaying multi-megapixel JPEG files on various PCs.

My satisfaction with the library is certainly only enhanced by the absence of a price tag. Intel's JPG library is definitely part of my image processing toolbox, you might want to add it to yours as well.

Article Source Code

PC Magazine July, 2002

Remember when PC Magazine used to give away their utilities for free? Ah yes, those where the days. The earliest PC Magazine utilities were distributed as PC-BASIC programs printed up in the Utilities section, which made good typing accuracy an important skill. Eventually this was supplanted by electronic distribution via BBS and then the Internet. A lot of useful programs were thrown our way for free, and PC Magazine presumably bathed in good will and reflected glory.

Back in July, 2002, I wrote one of those free utilities for PC Magazine. CubeShow was a screensaver that featured a tumbling representation of a 3D cube, adorned with photos of your choice. Having never written a line of DirectX code in my life, it was quite an adventure, but I think I managed to produce something that worked and was popular.

Some of the more interesting problems involved with writing this utility were:

• Finding the right rotation parameters to achieve smooth animation
• Figuring out how to change photos on a cube fact only when it was hidden
• Keeping a photo upright as the cube rotated

A lot of fun!

Figure 1 - CubeShow in action

Links

CubeShow on PCMag.com
CubeShow download link

Dr. Dobb's Journal March, 1998 by Mark Nelson and Tom Armstrong

Introduction

Implementing transparent ActiveX controls seems to be one of those programming tasks that is rapidly approaching FAQ status. The ability to display transparent GIF files in most popular Web browsers has apparently put quite a few developers into "me-too" mode. Unfortunately for programmers, Windows 95 and NT don’t provide much native support for the display of graphical images with transparent components. In this article, we provide a fairly simple set of steps that can be used to work around the shortcomings in the O/S. Using Microsoft’s Active Template Library (ATL) we develop an ActiveX control that will display a bitmap with a single transparent color. The control can be used with current and some past versions of Internet Explorer, Netscape Navigator (with the ScriptActive plug-in), Visual Basic, and most other ActiveX containers.

Transparency under Windows

When referring to bitmapped images, the term transparency has a straightforward meaning. A given image has an arbitrary number of transparent areas. When the image is drawn, the transparent areas don’t obscure the area behind the image in the Z-order. In the simplest case, we've all seen this used to good effect on the Web, producing images of complex objects that appear to be floating on top of a background.

Figure 1 shows a pair of transparent GIF files that float nicely above a background on a typical Web page.

Figure 1 - Screen capture of a pair of transparent GIF files used in a banner

Sites such as Mr. Showbiz typically have a standard background pattern that shows up on many or all of their pages. In Figure 1, the GIF on the left side has navigation bars that are used in an image map. A GIF file with the site logo is shown on the right. Both images feature transparent areas that let them blend in nicely with the background.

How do they do that?

After seeing this esthetically pleasing effect on the Web, we set ourselves to the task of duplicating it in an ActiveX control. For Windows programmers, encapsulation of a feature in an ActiveX control is definitely the method of choice.

Research turned up a couple of different ways to implement transparency in a control. The most straightforward of these methods requires the programmer to set up a windows HRGN object, used to define a non-rectangular drawing area. Figure 2 shows a simple example of a drawing region. As long as you can define your region as a path connecting a series of points (or elliptical regions), you can create a drawing region that Windows understands.

Figure 2 - A Windows Drawing Region

Defining a drawing region works well with containers that implement the OCX 96 specification from Microsoft (shipping with the ActiveX SDK). In the best case, you can implement two pass drawing, which lets you draw the foreground part of your control first, then lets the container draw the background. This provides fast and flicker-free drawing of controls.

Ointment ready, enter fly

Defining drawing regions is great for some applications, but it might not be the best general purpose technique. We wanted to be able to draw any type of image with randomly configured transparent areas. This means having transparency controlled on a pixel by pixel basis. Attempting to define a region for any arbitrary bitmap is simply asking for a headache!

For our control, we instead used a masking technique that is described by Ron Gery in an MSDN article. This technique uses a simple masking technique that makes it trivial to use arbitrarily complex transparent regions.

The Gery algorithm assumes that you have a bitmap with a single color that defines the transparent area. Given that, the drawing portion of your code needs to execute the following steps:

• Create a monochrome bitmap the same size as the bitmap you are going to draw.
• Set all the transparent pixels in the monochrome bitmap to 1, and all the opaque pixels to 0.
• XOR the screen region with the bits in your image bitmap.
• AND the screen with your monochrome bitmap. This has the effect of leaving all the transparent areas unchanged, and setting the opaque areas to be black.
• XOR the screen region with the image bits again. This sets all the transparent areas back to their original color, since the two XOR operations cancel one another. The opaque areas now contain the desired image bits, since XORing with solid black is the same as simply setting the bits.

This drawing algorithm is simple to implement using standard Windows raster operations. If you step through it and watch the effects when using the debugger, it will even start to make sense after a few passes.

Background checks

This drawing scheme works pretty well with containers that adhere to the OCX 96 recommendations, but it makes one important assumption. The XOR/Mask sequence preserves the background behind transparent areas, but only if the background has already been drawn.

To programs like Internet Explorer 3.0, the ActiveX control is simply a child window that is responsible for drawing its entire rectangular area. So when IE3 is drawing the background for a Web Page, it will exclude the areas occupied by child windows, allowing them to draw their own background. So if I've set up a pleasing (or not!) background GIF, IE3 won't bother to draw it in any areas covered by my control, making any attempt at transparent drawing doomed from the start.

Microsoft has documented a way around this problem in Knowledge Base article Q165073. This article gives a code snippet that you can drop into a control's WM_ERASEBKGND handler. MFC code similar to that in the Knowledge Base article is shown in Figure 3.

In a nutshell, this code sends a WM_ERASEBKGND message to Internet Explorer, along with the device context for the ActiveX control. This convinces Internet Explorer to draw the background behind the control as if it didn't belong to a child window. This code doesn't cause any trouble for more sophisticated containers, such as IE4, because the WM_ERASEBKGND never gets sent to controls. Newer containers that implement the OCX '96 specification don't create child windows for each embedded control. Instead, each control renders itself directly on the container's device context.

Putting it together using ATL

To demonstrate the concepts discussed here, we created an ActiveX control called TransCtl that displays an eight bit BMP file, and treats all solid white areas ( RGB(255,255,255) ) as transparent. To keep things simple, the control doesn’t do any palette management, so if you are using it on 256 color displays you need to stick to the 20 system colors. (The system colors are generally always going to be present, regardless of the remaining 236 colors in the palette.)

We created this control using Visual C++ 5.0, and Microsoft's framework of the month, the Active Template Library. Programming with the ATL is somewhat more difficult than with MFC, but the resulting ActiveX controls are usually faster, smaller, and can eliminate dependencies on corpulent DLLs.

Working with the ATL is a Wizard-driven process, which means that programming isn’t just typing in code, it also involves quite a few menu selections and button depressions. The next section of this article will walk you through the process from start to finish.

Follow the bouncing ball

In the old days of command line compilers, we could have just included a single C file that held all the source code for this project. Things aren't so easy for authors using IDEs that feature Wizard-based development. The instructions for building this project with Visual C++ look more like a recipe in a cookbook, with an interesting mixture of dialog boxes, button presses, menu options and code snippets.

To build any ATL project using Visual C++, you select File|New|Project|ATL COM AppWizard. The dialog box asks you for a Project Name, which we set to TransparentControl, and a Location, which we filled in with C:\. We selected the default values on the next dialog, creating a DLL with no MFC support, and no merging of proxy/stub code. The Wizard provides the following dialog to confirm what you’ve just done:

Figure 4 - ATL COM AppWizard output

At this point we need to create the actual ActiveX control that will display our BMP files. To do this, we use Insert|New ATL Object, which brings up the ATL Object Wizard, shown in Figure 4. We select Controls|Full Control and click the Next button, which brings up a tabbed properties dialog. In the edit box asking for a short name, we entered TransCtl. In the Miscellaneous sheet of the properties dialog, we turned off the Opaque option.

The Stock Properties tab allows you to specify which stock properties your control will support. Our control needs the ReadyState stock property, which is completely supported by the ATL, but for some reason is absent from the Stock Properties tab. So, to include it in our control’s implementation we have to add the code ourselves, which we discuss later.

Figure 5 - The ATL Object Wizard

Clicking the OK button at this point generates all the files necessary to create the basic ActiveX control. You can build the control at this point, and insert it into Microsoft’s ActiveX Control Test Container (accessible from the Tools menu item.) But before it can do anything interesting, we need to add the code that assigns an image file to the control, loads the image file into memory, and draws it on the screen.

Dealing with the image file

Since the goal of the TransCtl is to display an image file, we clearly need a property that gives the name of the file. Adding properties to an ATL project is another Wizard driven process. In the ClassView tab of the Workspace window, you should have two classes defined: CTransCtl, and ITransCtl. Right click on ITransCtl item in the tree, and select Add Property. Choose a property type of BSTR, which is the canonical string type used in ActiveX controls. Choose a property name of ImageFile, and use the default get and put functions the Wizard suggests.

Figure 6 - Adding a property

While the Visual C++ Wizards help you quite a bit by adding the properties to the control, they don’t actually supply the internal implementation of the property. This needs to be done manually. First, we add a new member variable called m_bstrImageFile, of type CComBSTR to class CTransCtl in file TransCtl.h. Next, we initialize it with an empty string in the CTransCtl constructor. Finally, we implement the get_ImageFile() and put_ImageFile() functions in TransCtl.cpp.

The put_ImageFile() (shown in Listing 2) function has to do more than just copy a string into our member variable. It clears our internal bitmap that actually holds the image, starts up the download process, and informs the user that the control is in an incomplete state. We’ll cover the details of this whole process in the next few sections.

Asynchronous Properties

Until recently, a control's property values, such as the bits of an image, had to be stored directly within a file managed by its container. As the control is instantiated within the container, its property values are provided by the container via COM-based interfaces. This process of binding a control's property values is a synchronous operation. The container opens its storage file, locates the control’s persistent data, queries for the control's persistence interface (usually IPersistStreamInit or IPersistPropertyBag), and then calls IPersist*::Load, whereby the control sets its property values.

This technique works fine for most control properties such as background color, font, and so on because the data is small. However, large property values like the bits of an image, can be quite large, and in today's low-bandwidth environments, where 28.8 modems are considered fast, synchronously binding a control's properties just isn't tolerated. As an example, a Web page containing four ActiveX controls that each displayed a 100KB image would take several minutes before any aspect of the page would actually "display."

Back in early 1996, when Microsoft finally realized that the Web was much more than a passing fad, they developed a COM-based technology to solve the synchronous binding problem. This new specification provided a way for controls and containers to bind their properties through something called an Asynchronous Moniker. Microsoft also provided an implementation of the specification and deemed it a URL Moniker.

Monikers are used to name specific instances of COM classes. A moniker is itself a COM object that implements the IMoniker interface and whose purpose is to encapsulate the details of instantiating a particular object. The moniker hides, from the client, the process of locating, instantiating, and initializing a specific COM class. In other words, clients work through the standard IMoniker interface or the MkParseDisplayNameEx API and can ignore class-specific details.

Typically, monikers are used to bind to a specific instance of a COM object. However, they can also be used to bind to a remote storage or stream. Microsoft's URL moniker implementation allows a client application to asynchronously bind to a Web resource specified by a URL. The client application, by implementing the IBindStatusCallback interface, can treat the resource as a stream of bytes (via an IStream pointer). In other words, a client can download a remote file in an asynchronous manner by specifying only its URL, which is exactly what we need for our ImageFile property implementation.

The ImageFile property, then, becomes a string that holds just the URL for our bitmap file. The persistent data for our ImageFile property is no longer the bits of the image itself, but an embedded reference to them. To give you an idea of how this looks in a Web page, examine the ImageFile property in Figure 7.

Loading the image proceeds like this. As the control is instantiated by the container, it passes the image's URL as part of the synchronous property binding process. Once we have the URL, we initiate the asynchronous download process. The download occurs in a background thread, and when it is complete, we render the transparent control using our image data.

As developers we don't typically want to mess with all this detail, because the implementation is basically the same for any URL that we might access. This, of course, is why frameworks such as the ATL are so popular. ATL provides the CBindStatusCallback class for handling asynchronous downloads, and that's what we'll use to download our image data.

If you look closely at Listing 2, you'll notice that instead of using ATL's CBindStatusCallback class, we instead developed and uses a derived class named COurBindStatusCallback. The CBindStatusCallback implementation provided with version 2.1 of the ATL isn't quite ready for prime time. We had to tweak a few of its methods in order to get everything to work properly. You can examine the minor changes by downloading the code for our example.

The download process begins with a call to COurBindStatusCallback::Download. The Download() method is shown in Figure 8. The implementation is a little hard to understand at first, as is nearly every aspect of ATL, but with a look through ATL's source, we can figure it out.

We are now ready to render our transparent image.

Displaying the CDib object

Once the image has been loaded into memory, all we have to worry about is displaying it. As we discussed earlier in this article, compatibility with IE 3.0 and other older containers means we need to add a handler for WM_ERASEBKGND, and use it to spoof the container into drawing the background for our control. Listing 1 shows the code that is added to the CTransCtl definition to process this message. First, we add an entry to the message map for the control, then we add the message handler itself. Essentially, all this handler does is pass the WM_ERASEBKGND message up to the parent window, along with a copy of the control's device context. (Note that the ATL Object Wizard doesn’t help you add message handlers, this is strictly a manual process.)

The actual drawing of the image is done in the OnDraw() member function of the CTransCtl class. If you've slogged through all this lengthy article wondering when we were actually going to show how to do transparent drawing, you have finally made it!

The OnDraw() function is called by the ATL framework, passing in a structure that contains various bits of information needed to actually paint information on the screen. Our implementation isn’t much more complicated than it would be if we were simply displaying the BMP file. We just have to add the code to create the monochrome bit mask, then perform the XOR/AND/XOR drawing method discussed earlier in this article.

The code shown in Figure 12 creates a monochrome bit mask in a compatible DC, relying on a Windows specific characteristic of monochrome bitmaps. When copying from a color bitmap to a monochrome bitmap, any color pixel that is identical to the color bitmap’s background color will get set to 1 in the monochrome bitmap. All other bits will get set to 0. With this knowledge in hand, you can see how the code shown below will handily create the bitmap needed by the masking algorithm.

Details, Details.

Developing controls with the ATL isn't quite as easy as with MFC, and we've left out some of the details of finalizing the implementation of our control. The details are covered briefly below and can be clarified by downloading and browsing the source.

Controls need to implement two additional interfaces to work effectively in the Web environment: IPersistPropertyBag and IObjectSafety. The IPersistPropertyBag interface allows a control and its container to work together to provide textual persistence. The older control persistence mechanism uses IPersistStreamInit, through which the container treats a controls properties as a stream of bytes. With IPersistPropertyBag, a control’s properties can be saved in an HTML-friendly format. Implementing IPersistPropertyBag is easy with ATL, just include IPersistPropertyBagImpl in your derivation.

A Web-enabled control implements IObjectSafety to inform its container that potential users cannot use it for malicious behavior. In other words, the control does not expose any functionality through which a user could harm the local system. Implementing IObjectSafety is only slightly more difficult, and this is because ATL's version of IObjectSafetyImpl doesn't work correctly. Instead of deriving directly from IObjectSafetyImpl, you derive from IObjectSafety and provide implementations for its two methods: GetInterfaceSafetyOptions() and SetInterfaceSafetyOptions().

Another important aspect of a Web-based implementations is that of control signing. An ActiveX control is really just a Windows DLL and so can easily corrupt (especially if the developer is trying to) a user's system. Microsoft's approach to dealing with this is possibility is to let you, the user, decide which ActiveX controls you choose to trust. Trust is handled by ensuring that each control is developed and distributed by a known entity. To be downloaded and installed on a user’s machine, a control must be digitally signed by such a known entity. To obtain a digital signature, just browse to www.verisign.com and agree to hand over your first born child (and $20.00). The rest is easy.

The final product

Since this is an ActiveX control, we can not only use it in our desktop applications, but in our Internet Applications as well. For a quick demo, we've set up an HTML page on Tom's web site that uses this control to display some transparent BMP files. You can view the control with either Internet Explorer or Netscape Navigator (with NCompass Lab's ScriptActive plug-in) by following this link to www.widgetware.com, (mn: sorry, this site is no longer functional) and looking in the table of contents.

You can still use our control verbatim in your Windows applications, as long as your framework supports control containment. Fortunately, this includes the current releases of nearly all Windows development tools. But fear not, if you want to avoid the rigors of being an ActiveX user, you should still be able to take the old-fashioned approach to re-use by cutting and pasting our drawing code directly into your application. Sometimes the old ways are still best ways!

Project Files