Bigger Data Isn’t Always Better Data

By Cathy O’Neil, Bloomberg View

When making a decision such as whether to hire, insure or lend to someone, is more data better? Actually, when it comes to fairness, the opposite is often true.

Consider a recentHarvard Business Review experiment, which involved sending 316 fake applications to the largest U.S. law firms. All the applicants were among the top 1 percent of students at their schools, but other information -- such as their names, college clubs and hobbies -- provided hints about their gender and social class.

The result: Upper-class males were four times as likely to ...