Notre Dame’s Master of Science in Data Science program hosts author and data expert Cathy O’Neil
Organizations are increasingly counting on mathematical models to make hiring, admissions, lending, and other decisions. But according to mathematician Cathy O’Neil, over-reliance on big data and algorithms has become a destructive force in society, further causing divisions between the “haves” and the “have-nots.”
Moral and ethical decisions are embedded into algorithms that benefit organizations and often not the people they are claiming to assist, described O’Neil, the New York Times best-selling author of “Weapons of Math Destruction: How big data increases inequality and threatens democracy,” as she spoke to an audience at The University of Notre Dame in September.
“If you think about a life, starting with going to high school or college, or getting a job, or a mortgage, or insurance….during every moment you are funneled by algorithms as a winner or loser,” she said. “It’s not just destructive for the individual, but it’s also destructive for society.”
O’Neil earned her doctorate in mathematics from Harvard University, taught at Barnard college, worked in the private sector, and spearheaded the Lede Program at Columbia University School of Journalism, which is designed to equip journalists and storytellers with the computational skills needed to turn data into narrative. Sponsored by the Master of Science in Data Science program and Mendoza College of Business, O’Neil’s talk about ethics and data science fit perfectly with the data science program’s goals of training technically proficient and ethically focused graduates, according to Roger Woodard, director of the data science program.
Although a data scientist herself, O’Neil’s skepticism about how mathematical models are used is outlined in her book. By tracing subjects’ lives and describing how algorithms affected them, she melds social justice issues with what many think are the black-and-white answers of mathematical algorithms. However, “these are done in secret, like a scoring system,” she said. “The score gives you a bonus for certain types of information. If your score passes a certain threshold, you are given that opportunity. If you are not high enough, you aren’t.”
During her talk she shared several examples. She described how algorithms more often allowed the release from prison of white inmates over black inmates, how certain financial programs for students were less beneficial for university rankings (making some universities hesitant to use them), and how skewed child abuse data more frequently caused children in lower-income homes be removed from their families than children higher-income homes.
“The algorithms destroy exactly what they were trying to improve,” O’Neil told the audience. “They undermine science in a very important way—and this is a claim I’m making that you can challenge me on—but my problem is it’s called ‘data science,’ but there’s very little science. Data scientists are asking us just to trust them, but science is based on the idea of testing, or evidence. And algorithms, because of their secrecy, are by definition not tested.”
Woodard agreed with the premise of O’Neil’s talk, noting that algorithms can have hidden biases from a variety of sources based on how data was collected and which potential audiences are left out. Notre Dame’s online Master of Science in Data Science program is designed to provide its graduates with the ethical training O’Neil feels is lacking, he noted. “How the data scientist prepares the data for the algorithms can also introduce bias, and we train our students to recognize these issues and consider the impact of their actions,” he said.
In addition to her talk, O’Neil recorded video examples that will be used in the data science program’s Data Ethics and Policy course, Woodard said. She also met with faculty across campus, including The Law School at Notre Dame and the College of Arts and Letters.
“Examples of Weapons of Math Destruction are everywhere,” O’Neil warned the audience. “Who gets into high school, who gets into college; who gets this job, who keeps this job. If you think about a bureaucracy and a difficult decision for a bureaucracy to make, it’s probably being replaced by an algorithm right now.”
Originally published by at science.nd.edu on September 26, 2018.