American Express is so excited about having me as a customer that they were willing to pay me $5 to take part in a survey:

American Express Needs YOUR Feedback!

Dear American Express Blue Cardmember:

American Express would like your feedback. We would like you to participate in a survey about your Blue card from American Express. Your participation will provide us with valuable feedback and help us tailor card benefits to better meet your needs.

As a token of our appreciation, you will receive $5 from American Express*. Please note that this survey will be running for a limited period of time. To increase your chances of receiving the honorarium, please complete the survey at your earliest convenience.

The web address for the survey is shown below. To begin the survey, simply double-click on the address to go directly to the questionnaire. However, if you are unable to double-click on the address, please copy and paste the text below into your browser’s address bar.

Well, I need five bucks, and a little bit of checking gave me moderate insurance that this was really from American Express, not a phisher. The survey was being outsourced to Confirmit, a real company, and they don’t seem to be on any malware site lists. Furthermore, as things went on, there was an enormous amount of content in the survey and seemingly no payoff for phishers, so it seems unlikely that this is a scam.

However, right off the bat there was some cause for concern. Double-click on the address? How many browsers do you use that need a double-click before they follow a link? If Confirmit is a real company, they clearly didn’t assign their top copywriter to this project.

The real fun started when I actually started the survey. The classic “make a good first impression” rule that your mother taught you is just as true for web pages as it is for anything else. And the first page of the American Express survey was the equivalent of showing up for your first date with a big gravy blotch on your tie:

This screen cap from the survey has a field called 'Hidden Question' and another called 'To decide Cell'

Yes, that’s right, the first thing I see is some inner workings they’ve inadvertently exposed. No doubt the URL I clicked was supposed to preload a survey question and zip me right past this. This question was undoubtedly in use to stage their testing of the survey, and was supposed to be removed in the published version. There’s a lesson in that.

From Bad to Worse

The rest of the survey only served to further tarnish my impressions of Amex’s IT outsourcing choice. After making it through a few innocuous questions, another page I wasn’t suppose to see popped up:

This creen capture has an obviously incorrect field labeled '--HIDDEN--', and anothe rline that says 'This screen is shown for testing purposes only. It will be hidden when the survey goes live'

Apparently I was going to be be seeing ths particular error quite a few times:

This screen capture shows the same line about testing purposes

Survey Hell

Finally I seem to have made my way through the setup questions. The progress bar showed 50%, and I detected that I was entering the first ring of survey hell. This is where the survey designer starts trying to milk you for a ton of information by repeatedly varying some scenario, then asking you a bunch of detailed questions about it. (This is sheer idiocy on their part. Once they start trying to find the correct adjectives for how a particular ad makes me feel about the product/company, it’s over.)

In this case AmEx had a list of perhaps 15 benefits that their card offers. They started tossing them up in various combinations on the screen, and in each case asking me on a scale of 1-10 how that made me feel about the card. They then tried to quantify the results by asking me how much more I would put on my card each month given that benefit. (Again, totally moronic. No data retrieved this way could possibly have any value.)

This would all be great if their page actually worked. In this case, I’ve told them that I put a tidy $5,000 on the card each month, and they want to see how much I’m going to bump that up if someone comes to my house to give me a back rub. (No joke!) Just to see what would happen, I put down a lower number, and got this nifty error message:

In this screen capture, the survey is asking me how much I would spend on my card if certain benefits were offered. It reports that the number 1,000 is too large, despite the fact that I said I spend $5,000 a month on the card

It turns out the only way I could get past this page was by putting in 1, yes $1. But that’s okay, because at this point there’s obviously no reason to cooperate with the tragically broken survey.

Unending Hell

AmEx then applied the coup de grace by inserting a fiendish logic error in the survey. I started looping through various permutations of three possible card benefits, over and over, for each one picking a number from 1-10 and then assigning it a cash value. I realized that the progress bar at the top of the page was actually moving back to 50% after each answer! In other words, I was stuck in the survey forever. They had a bug that prevented them from exiting the loop.

Nicely done, Amex!

So I bailed. In theory I can come back in a day or two and finish. Perhaps they will have fixed things up by then. More likely they will have fired Confirmit. Will I get my $5? It doesn’t seem likely. AmEx put the fix in here as well, by breaking the payment page. As you can see here, I’ve entered data in all the fields, but the survey software refuses to accept it, somehow thinking I’ve left something blank. This is a big PR win for Amex, putting people through a 20 minute broken survey, then refusing to pay!

The final screen capture promises me a $5 honorarium for filling out the survey.

Moral

The moral of the story is obvious. If you’re going to send an email out to thousands of customers, all of whom are most likely uninformed civilians, it would be a good idea to thoroughly test your product first. The fact that Confirmit clearly didn’t do that is a nail in their coffin, considering this is their livelihood. The fact that AmEx hired this joke of a company doesn’t say much for them, either.

Priceless Web Design:

  • Designing a survey for your customer base? $10,000
  • Deploying the survey and passing out rewards? $25,000
  • Making yourself look like an incompetent pack of rubes for the whole Web to see? Priceless.