When Chatbots Go Rogue

Threats are posed to customer service and needs

Kate White and Ioana Gorecki
When Chatbots Go Rogue

CFPB Spotlights the Use of Chatbots

In June, the CFPB released a new issue spotlight on the use of chatbots by banks and other financial institutions. The report notes that banks have increasingly moved from “simple, rule-based chatbots towards more sophisticated technologies such as large language models (“LLMs”) and those marketed as ‘artificial intelligence.’” While these chatbots are intended to simulate human-like responses, they can end up frustrating consumers’ attempts to obtain answers and assistance with financial products or services.

Some of the CFPB’s listed concerns are:

•  Limited ability to solve complex problems, resulting in inadequate levels of customer assistance (for example, difficulty understanding requests, requiring use particular phrases to trigger resolution, difficulty knowing when to connect with a live agent). The CFPB argues this is particularly concerning in the context of financial services, where consumers’ need for assistance could be “dire and urgent.”

•  The potential for inaccurate, unreliable, or insufficient information. In contexts where financial institutions are required to provide people with certain information that is legally required to be accurate, such lapses may also constitute law violations.

•  Security risks associated with bad actors’ use of fake impersonation chatbots to conduct phishing attacks at scale, as well as privacy risks both in securing customers’ inputted data or in illegally collecting and using personal data for chatbot training purposes.

The CFPB notes that it is actively monitoring the market to ensure financial institutions are using chatbots in a manner consistent with customer and legal obligations.

FTC Raises Concerns Regarding Chatbots and “Dark Patterns”

The FTC addressed the intersection of chatbots and “dark patterns’’ in a recent blog post. (As explained in more detail here and here, “dark patterns’’ are sometimes defined as practices or formats that may manipulate or mislead consumers into taking actions they would not otherwise take.) The commission is worried that consumers may place too much trust in machines and expect that they are getting accurate and neutral advice.

The agency cautioned companies that using chatbots to steer people into decisions that are not in their best interests, especially in areas such as finance, health, education, housing, and employment, is likely to be an unfair or deceptive act or practice under the FTC Act.

In addition, the FTC warned companies to ensure that native advertising present in chatbot responses is clearly identified so that users are clearly aware of any commercial relationships present in listed results. The blog was very clear that “FTC staff is focusing intensely on how companies may choose to use AI technology … in ways that can have actual and substantial impact on consumers.”

Given the regulators’ avowed interest in this space, companies should take care that their use of chatbots comports with this most recent guidance.

Kate White and Ioana Gorecki

Katherine White, a former Federal Trade Commission attorney, is a partner at Kelly Drye in Washington, D.C. Ioana Gorecki is of special counsel to the firm.

This article was originally published in the Mortgage Banker Magazine August 2023 issue.
Published on
Aug 14, 2023
Mortgage Banker Magazine
If You Build It, They Will Come

Ratings will go a long way toward growing the market for home equity-backed securities

Ryan Kingsley
Mortgage Banker Magazine
Investors May Be Losing Interest In The Single-Family Rental Market

Rise in interest rates, drop in values have them pulling back in certain regions

Xander Snyder
Mortgage Banker Magazine
A Top-Tier Team

Longtime pros launch new company, see early success

Erica Drzewiecki


DealDesk: Focus on Quontic Bank's Lite Doc Program

At Quontic Wholesale, their bank's CDFI status allows them to validate income with a unique approach to qualif...

Aug 02, 2023
Investor Confidence in Today’s Non-QM And Why Originators Are Paying Attention... A Virtual Town Hall

We host Angel Oak Mortgage Solutions for a special 2021 edition of their virtual town hall series they ran fro...

Apr 08, 2021
How to Help Real Estate Pros in a Post-Refi World

Hear from Melissa Merriman, REALTOR® with The Melissa Merriman Team at Keller Williams, on what real estate pr...

Mar 18, 2021
Rocket Pro TPO Takes the Lead With Early Loan Limit Increase

EVP Mike Fawaz underscores the move's competitive edge, marking the firm's second consecutive year ahead of FHFA's anticipated adjustments.

Congress Passes Legislation To Extend National Flood Insurance Amid Shutdown Concerns

The move ensures continued flood insurance coverage, essential for many homeowners in high-risk flood areas, during peak hurricane season.

Bank of America Whistleblower Lawsuit Dismissed by Judge

Allegations of fraudulent reporting in the "Down Payment Grant Program" meet their end as the US District Court dismisses the case, with the government choosing not to intervene.

Connect with your local mortgage community.

Meet your your colleagues, both national and local, by attending an event in your area.