The Ineffectiveness of Double Entry Email Validation

Way back when, pre-2002, I started maintaining an ecommerce website that delivers ordered products via email. This means that customers have to enter their email address during the order process, and that is more challenging than one might expect.

In fact, I recall that the bulk of the customer service revolved around resending orders to people that screwed up their own addresses. Unacceptable! So I whipped up the latest in luser-combatting technology - a double entry email verification system. It was totally vanilla, two entry boxes that got compared (minus whitespace) on the review page.

This seemed to cut down problem users significantly, and there was much rejoicing!

Now, at the time of implementation I added a logging function and periodically reviewed the results to make sure that my syntax validation wasn't too strict. A couple years ago I realized that this data was quite a bit more interesting than I previously thought, and started saving it for later analysis.

Later has arrived.

A short Python script that heavily abuses generators filters out the really stupid attempts and groups the rest into sessions based on difflib comparisons of the email addresses and the log timestamp (one hour cutoff).

I expected to see lots of people struggling with entering an address twice, possibly to the extent that I should not reject them on the basis that it is more important to take customers' money before they get frustrated and give up. Not so at all.

• Only 4.22% had any kind of trouble - everyone else got their syntactically correct address entered twice identically on the first try and kept it that way. Huh... that's a lot of users having their time wasted for no reason.
• 41.92% of trouble sessions gave up and never gave us any money.
  
• Of the trouble sessions, 92.62% had trouble entering the same address twice.
• Of the trouble sessions, 33.01% "round tripped", which is to say they finished with two matching email address that happen to match the very first email address they entered. Not helping those guys!
• Of the trouble sessions, 4.6% started out with a good double email entry but then sabotaged themselves in some way (we have single page data entry forms...).
  
• Average attempts for a trouble session before giving up: 1.44. So if they don't get it right on the first try, they probably aren't coming back.
• Average attempts for a trouble session that eventually results in success: 2.23

A lot of these categories overlap, so summaries are tricky, but one clear finding is that 75% (gave up + round trippers) of the people that had trouble were not helped by the double entry system.

What I don't have is information on which sessions resulted in orders, or even include multiple orders. Demographics wise, I can confidently say that customers skew to the over-30 side, but beyond that I couldn't say.

I am currently leaning towards completely scrapping the double entry system, and possibly making the syntax validation a warning. This would move a certain number of customers into the post transaction support queue, but would hopefully increase the total number of transactions.