Skip to main content

When Bots Misbehave

Feb 22, 2023
AI Artificial Intelligence

Black Knight white paper looks at the pros and cons of using AI in the mortgage industry.

You know the future is now when a chatbot gets offended and compares you to Adolf Hitler.

That happened recently during an “interview” conducted by the Associated Press with Microsoft’s ChatGPT-powered Bing search engine. The journalists challenged the search-engine’s accuracy, and the artificial intelligence-driven program ended up accusing them of spreading lies and comparing them to the German dictator.

That outcome is both amusing and a little frightening, conjuring images of HAL, the AI-powered computer program that went insane in “2001: A Space Odyssey,” or of Skynet, the AI network in the “Terminator” movie franchise that became self-aware and decided humans were a threat.

It’s all fun and games until the robots come to hunt you down.

Such uncomfortable thoughts about AI and its possibilities led Black Knight — the software, data, and analytics company for the mortgage lending, servicing, and real estate industries — to explore strategies for using and managing AI and machine-learning  (AI/ML) tools in the mortgage industry. 

The company has produced a 25-page white paper on the topic, titled “Management of AI and Machine Learning (ML) in the Residential Mortgage Industry.” It explains how AI and ML have recently raised the concerns of regulators and other stakeholders, especially regarding their use in matters of fair lending. The paper also focuses on model explainability and transparency, and outlines potential processes to respond

“Science fiction author Arthur C. Clarke once wrote that ‘Any sufficiently advanced technology is indistinguishable from magic.’ That certainly applies to many applications of AI/ML, particularly so-called ‘black box’ algorithms,” said Rich Gagliano, president of Black Knight Origination Technologies. “In our daily lives in the digital world, AI is making things happen behind the scenes, with decisions being made and events triggered — all as if by magic. But in an industry as tightly regulated as ours, we can’t trust important lending decisions to magic.”

That’s why Black Knight is offering the white paper to everyone in the mortgage industry, he said. 

“Regulators have justifiable concerns, particularly in matters of fair lending, that consumers are not harmed by bias in the application of AI/ML in the lending process,” Gagliano said. “That is what makes model explainability and transparency so critical.”

According to Black Knight, the paper addresses the many challenges and industry-specific concerns that must be considered when bringing AI/ML into the mortgage process. It also details when AI/ML is the right approach, and when it isn’t. 

For example, the paper notes that traditional business rule management software remains the appropriate technology for meeting most mortgage origination requirements, since inputs must be directly traceable to precise expected outputs — otherwise known as a “deterministic solution.”

AI/ML technology, on the other hand, is appropriate for addressing business problems that do not yield to deterministic approaches — i.e., where inputs are varied or a range of possible outcomes exists — and therefore a probabilistic approach is required.

“Mortgage industry executives face a challenge,” Gagliano said. “Existing accounting rules, government regulations, business process standards and auditing methods designed to assess traditional software are not viable when it comes to AI/ML, and yet there are no well-established standards to fix that. This paper attempts to bridge this gap for our industry, with a deep exploration of the things lenders should consider as the technology — and its oversight — progresses.”

The white paper is offered at no charge and can be downloaded here.

About the author
David Krechevsky was an editor at NMP.
Published
Feb 22, 2023
More from
Tech
Was It You Or Was It AI?

As deepfakes get convincing and AI-driven fraud escalates, real estate industry faces critical tests on multiple levels

Jun 16, 2025
Former FHFA AI Chief To Lead Better’s AI-Driven Origination, Homeownership Platform

Leah Price will serve as lead for Better's tech as it heads to market — and company takes aim at ICE’s Encompass platform

Jun 12, 2025
New AI Tool Lets Homebuyers Pinpoint Search By Affordability

Platform augments home search with live mortgage rates, pre-approvals, property-specific special financing

Jun 05, 2025
Not All AI Is Developed Equally

Be wary of new systems that rely on basic consumer-focused AI systems

May 30, 2025
‘Something Big On Mortgage Fraud’: Fannie Mae Launches AI-Driven Crime Detection Unit

GSE to deploy AI fraud-detection tech amid more talk of listing Fannie Mae, Freddie Mac on major stock exchange

May 29, 2025
ServiceLink Enhances Its Digital Closing Solution

EXOS Close allows lenders and borrowers to choose ‘when, where, and how’ they close

May 20, 2025