American Express is so excited about having me as a customer that they were willing to pay me $5 to take part in a survey:
American Express Needs YOUR Feedback!
Dear American Express Blue Cardmember:
American Express would like your feedback. We would like you to participate in a survey about your Blue card from American Express. Your participation will provide us with valuable feedback and help us tailor card benefits to better meet your needs.
As a token of our appreciation, you will receive $5 from American Express*. Please note that this survey will be running for a limited period of time. To increase your chances of receiving the honorarium, please complete the survey at your earliest convenience.
The web address for the survey is shown below. To begin the survey, simply double-click on the address to go directly to the questionnaire. However, if you are unable to double-click on the address, please copy and paste the text below into your browser’s address bar.
Well, I need five bucks, and a little bit of checking gave me moderate insurance that this was really from American Express, not a phisher. The survey was being outsourced to Confirmit, a real company, and they don’t seem to be on any malware site lists. Furthermore, as things went on, there was an enormous amount of content in the survey and seemingly no payoff for phishers, so it seems unlikely that this is a scam.
However, right off the bat there was some cause for concern. Double-click on the address? How many browsers do you use that need a double-click before they follow a link? If Confirmit is a real company, they clearly didn’t assign their top copywriter to this project.
The real fun started when I actually started the survey. The classic “make a good first impression” rule that your mother taught you is just as true for web pages as it is for anything else. And the first page of the American Express survey was the equivalent of showing up for your first date with a big gravy blotch on your tie:

Yes, that’s right, the first thing I see is some inner workings they’ve inadvertently exposed. No doubt the URL I clicked was supposed to preload a survey question and zip me right past this. This question was undoubtedly in use to stage their testing of the survey, and was supposed to be removed in the published version. There’s a lesson in that.
From Bad to Worse
The rest of the survey only served to further tarnish my impressions of Amex’s IT outsourcing choice. After making it through a few innocuous questions, another page I wasn’t suppose to see popped up:

Apparently I was going to be be seeing ths particular error quite a few times:

Survey Hell
Finally I seem to have made my way through the setup questions. The progress bar showed 50%, and I detected that I was entering the first ring of survey hell. This is where the survey designer starts trying to milk you for a ton of information by repeatedly varying some scenario, then asking you a bunch of detailed questions about it. (This is sheer idiocy on their part. Once they start trying to find the correct adjectives for how a particular ad makes me feel about the product/company, it’s over.)
In this case AmEx had a list of perhaps 15 benefits that their card offers. They started tossing them up in various combinations on the screen, and in each case asking me on a scale of 1-10 how that made me feel about the card. They then tried to quantify the results by asking me how much more I would put on my card each month given that benefit. (Again, totally moronic. No data retrieved this way could possibly have any value.)
This would all be great if their page actually worked. In this case, I’ve told them that I put a tidy $5,000 on the card each month, and they want to see how much I’m going to bump that up if someone comes to my house to give me a back rub. (No joke!) Just to see what would happen, I put down a lower number, and got this nifty error message:

It turns out the only way I could get past this page was by putting in 1, yes $1. But that’s okay, because at this point there’s obviously no reason to cooperate with the tragically broken survey.
Unending Hell
AmEx then applied the coup de grĂ¢ce by inserting a fiendish logic error in the survey. I started looping through various permutations of three possible card benefits, over and over, for each one picking a number from 1-10 and then assigning it a cash value. I realized that the progress bar at the top of the page was actually moving back to 50% after each answer! In other words, I was stuck in the survey forever. They had a bug that prevented them from exiting the loop.
Nicely done, Amex!
So I bailed. In theory I can come back in a day or two and finish. Perhaps they will have fixed things up by then. More likely they will have fired Confirmit. Will I get my $5? It doesn’t seem likely. AmEx put the fix in here as well, by breaking the payment page. As you can see here, I’ve entered data in all the fields, but the survey software refuses to accept it, somehow thinking I’ve left something blank. This is a big PR win for Amex, putting people through a 20 minute broken survey, then refusing to pay!

Moral
The moral of the story is obvious. If you’re going to send an email out to thousands of customers, all of whom are most likely uninformed civilians, it would be a good idea to thoroughly test your product first. The fact that Confirmit clearly didn’t do that is a nail in their coffin, considering this is their livelihood. The fact that AmEx hired this joke of a company doesn’t say much for them, either.
Priceless Web Design:
-
Designing a survey for your customer base? $10,000
Deploying the survey and passing out rewards? $25,000
Making yourself look like an incompetent pack of rubes for the whole Web to see? Priceless.
6 users commented in " I Trust American Express With My Money? "
Follow-up comment rss or Leave a TrackbackLooks like Bangalore (or Moscow, or Beijing) really screwed the pooch on that one.
Oh well, I guess it’s better than speaking with Tarbash, I mean “Ben”, in a Bangalore call center & trying to understand the questions as they’re read to you.
As an update, I complained to AmEx customer service and have not heard back from them. 24 hours after the email was first received, the page was still refusing to accept my name and address for the payout. A couple of hours after that an attempt to access the page announced that the survey was closed and I should try again on Friday.
One way or another, American Express will pay me my $5!
I wouldn’t mind sending $5 to hear the happy ending of this story! Or do I think too highly of Amex?
@Jonas:
I wish there was a happy ending, but really, just more bumbling. Amex customer service sort of distanced themselves from the problem, but did promise to credit my account $5 – never happened though.
The company did fix the survey, and I got to see it working properly, but no $5 from them either.
Personally I think this explains it:
http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/archive/2000/01/18/MN73840.DTL
There is a little irony to that link. I suspect that you view yourself as supremely competent and above something as outrageous as goofing up on testing a system that is customer facing (like those incompetents at AMEX!)
Considering the general smugness of the post (and the whole site really), maybe a link proving that the most self confident tend to be the most incompetent wasnt the best choice.
@quantum:
You see irony, I see nothing more than an object lesson for all of us, and in fact a pretty simple one. Sorry if whatever humility I have doesn’t come across in my writing, but yeah, I make as many mistakes as anyone else. I try to learn from my own, as well as those of others.
For the most part, the articles you find annoying are just fluff that fill in the spaces between the meatier technical articles.
If it helps, at least 50% of my traffic goes to these five technical articles, and I defy you to find any attitude at all in them, much less smugness, self-satisfaction, or arrogance:
Well maybe not defy, how about if I say I just think they are not particularly judgmental, which is what I’m betting you find objectionable.
One final disclaimer: the people at AMEX didn’t have too much to do with how badly botched this survey was. Their only mistake was not performing more oversight on a project they farmed out.
Leave A Reply
You can insert source code in your comment without fear of too much mangling by tagging it to use the iG:Syntax Hiliter plugin. A typical usage of this plugin will look like this:[c]
Note that tags are enclosed in square brackets, not angle brackets. Tags currently supported by this plugin are: as (ActionScript), asp, c, cpp, csharp, css, delphi, html, java, js, mysql, perl, python, ruby, smarty, sql, vb, vbnet, xml, code (Generic). If you post your comment and you aren't happy with the way it looks, I will do everything I can to edit it to your satisfaction.int main()
{
printf( "Hello, world!\n" );
return 1;
}
[/c]