Financial Regulations Struggle to Keep Up With AI

Machine learning is “remarkably successful at extracting correlations … from data sometimes that we didn’t see before,” said Davis Wright’s Katherine Sheriff. The big three challenges are bias, explainability and privacy, Sheriff said. If the data has bias in it, “you might be embedding some biases inside the data and that’s a problem,” she said. Governments worldwide have told credit card and other financial companies those kinds of biases aren’t allowed, she said. Data analytics was a big issue in the 1960s, which led to the Fair Credit Reporting Act, which requires credit decisions to be explainable, she said. “In country after country, we embed that into our law.” Privacy of data has also been a longtime concern, she said.

There are questions about what's AI, said Aaron Klein, senior fellow-economic studies at the Brookings Institution. “Is this a question of when Netflix gives a suggestion for what movie I’d like to watch next” or is it only when a machine “comes up with a concept or idea” that advances thinking, he said. Machine learning “is a next threshold down in which a machine simply takes a set of information and learns through the repetitive process and then is able to replicate that or even to some degree go beyond its coding to make its own choices,” Klein said: “Big data” is “we have a huge amount of data available on which to run previous analyses or algorithms that produce new correlations and findings.”

Current rules “are completely broken and largely irrelevant,” Klein said. If he's turned down for credit, he can ask why, he said. “The computer spits out a lot of reasons why I could be denied, and the lender just has to pick one.” The laws date to the 1960s and 70s “when there was no automated process,” he said. “Society is deeply conflicted on what should be permitted.”

Bias can easily slip into how data is analyzed, said Brenda Leong, director-AI and ethics at the Future of Privacy Forum. If you use an IP address, Mac versus non-Mac, as a data point, “why and what does that give you?” she asked: Maybe it indicates the Mac user probably has a higher income “but what else might that imply about that particular community?” Is the data collected to train a system “sufficiently diverse or representative of your customer population” or do you need to seek outside data, she asked. “What we have always done is very, very biased in almost all instances, whether intentionally or not, whether for good or bad purposes, or not.”

Kareem Saleh, CEO of FairPlay, said companies using AI in the financial industry need to answer: “Is my algorithm fair? If not, why not? Could it be fairer? What’s the economic impact to our business of being fairer? And, finally, did we give our declines … a second look to see if they might resemble good applicants or applicants that were approved” in ways the algorithm didn’t take into account?"

Use of AI and machine learning “really are being impactful on both financial institutions and their customers,” said Kevin Greenfield, deputy comptroller-operational risk policy, Office of the Comptroller of the Currency. “There’s a lot of good opportunity here” to provide banking services for people who often can’t get credit, he said. “As with everything, there are a lot of risks,” he said. “There is a lot of discussion of privacy concerns, risks of bias, especially unintended bias, and the overall reliability of the systems.” AI “is only as effective as its design,” he said.