The Mortgage Market’s Disappearing Middle

An Entrepreneurial Achilles’ Heel

This year’s spring homebuying season was supposed to be an oasis in an origination desert. 

Instead, the wave of mergers and acquisitions that began in the second half of 2022 has continued to thin the ranks of IMBs this year. Analysts predict that industry consolidation will continue as long as low inventory, elevated mortgage rates, and soaring repurchase demands continue to starve lenders. In large measure, it’s entrepreneurial optimism leading lenders astray.

“Most of these lenders in the third and fourth quarter last year,” Ludden recalls, “said, ‘I just need to get through a couple of quarters. I made a lot of money in 2020 and 2021, and therefore I can make it through. I just need to get to that spring season when things are gonna get better.’” 

Watch it on The Interest:The Mortgage Market’s Disappearing Middle

But things did not get better. Instead, the market deteriorated further, and for the majority of lenders, profitability was piecemeal through the first half of the year as total mortgage balances fell for the first time since 2015. Sales volume rose in March when mortgage rates dipped, but rising rates in May and June disappointed lenders who had hung their hats – and their companies – on the spring homebuying season. 

“A lot of them made that decision because they had just been through so much pressure related to staffing,” says Garth Graham, senior partner at Stratmor Group, the mortgage advisory firm. “They said, ‘Gosh, I can’t imagine getting rid of all these people I fought so hard to get in 2020 and 2021.’” 

Which isn’t to say that no lenders read the writing on the wall last summer. 

As the Federal Reserve embarked on its crusade against inflation, raising interest rates 75 basis points for four consecutive months, some lenders prepared for lean times in 2023. Ludden watched several hundred well-managed lenders cut costs at the very beginning and thus be able to manage over the past year at a relatively low loss rate.

Moving forward, Ludden thinks that companies that allow their profit-and-loss (P&L) branches to operate at losses or with high compensation for loan officers (LOs) will struggle substantially relative to companies that don’t. But don’t be too quick to call the lenders who’ve survived this long without taking strategic action “the lucky ones.” 

Waiting too long to sell a company or alter LO compensation can be worse than acting too soon. Graham has spoken with many lenders who have reported four consecutive quarters of losses, which makes counterparties such as regulators, agencies, and warehouse lenders nervous. It’s the counterparties who, contractually, can force lenders into making hard decisions.   

“We have lenders all the time who call us, and I feel bad, but we can’t help them,” Graham says. “Whether their top producers have already left or their locked pipeline is too low, there’s just not anything really of value left other than maybe the tickets and what’s referred to as the shell.” 

And yet, for every three “no value left” conversations that Graham has, there are mortgage bankers who, at this very moment, still stand to gain by making the difficult decision to sell. “It’s really a story about corporate overhead. The buyer can bring synergies in terms of additional revenue and expense savings, and then they become valuable,” he explains. 

Lenders Feeling Burned By Vendors

For many lenders, surviving the winter depends on whether they had a cohesive cost-saving strategy from the beginning. That strategy involves the vendors who sell silver-bullet technological solutions for cutting costs. 

“There’s a lot of over-promising. To some degree, that’s normal in the vendor community. But in our industry, that tends to be over-promising a more holistic or deeper solution,” says Jeremy Potter, a former senior director of capital markets at Rocket Companies who is now president of titleLook, a software company. 

Potter has spent the past three years on the vendor side of the mortgage industry as an investor and founder of various technology companies. While some vendors question whether lenders have cut staff as dramatically as necessary, he says that lenders are tired of getting burned by vendors who are promising more savings than they can deliver. 

“The vendor community is ‘the boy who cried wolf’ to a degree at this point to a CEO of a mid-size IMB,” Potter admits. CEOs and decision-makers at IMBs have been burned by promises of savings, promises of speed, and promises of differentiation. “In many ways, the vendor tech community has not lived up to that,” he observes.

Every vendor pitch, booth, and email blast highlights vendors’ abilities to improve efficiencies and lower costs in a holistic manner so employers can “get more” out of their people. But, Potter says the efficiencies (à la savings) haven’t materialized. As a result, vendors are finding it increasingly difficult to convince CEOs and decision-makers who have been burned in the past that the best option for saving money is spending more money, especially after those stakeholders have had to make difficult staffing decisions. And yet, dragging their organizations through multiple rounds of layoffs, rather than cutting deep from the beginning, has increased the pain for both employers and employees. 

Of course, the silver bullet cost-saving measure is a holistic approach that happens to be the most difficult balance to achieve: keeping “the right people” for CEOs and decision-makers to build additional savings and efficiencies around. What tends to happen, Potter says, is the one person left after layoffs does more work, but they don’t do it more efficiently or as effectively. 

He argues that the piecemeal way companies evolve in the industry will not work in the future. 

“That’s the real trouble here,” says Potter, “for CEOs and decision-makers at midsize IMBs. What tools and solutions are real that you can give that person so that now they can do their job three times as well, instead of doing three jobs?”

Cashing Out The Back Office

Rick Roque, previous founder of Menlo Company, a mergers, acquisitions, and capital fundraising firm, believes the market of the past 18 months has exposed a financial inflection point for small- to medium-sized regional lenders that he calls “the regional ditch.” 

Essentially a cash problem exacerbated by current market conditions, the ditch results in polarization within the mortgage finance industry. 

“You need the cash to be able to make mistakes,” he explains, but for many lenders, the time to make mistakes has come and gone. 

Sustained by the annuities of mortgage servicing rights (MSRs), the nation’s largest lenders are well-positioned to expand their market share whenever the market rebounds. Larger lenders (more than $2 billion per year in originations) also tend to be better capitalized, with more and easier access to cash through secondary markets and higher net-worth ownership. 

Meanwhile, small lenders are circling the wagons in local markets, doing their best to retain top talent as Rocket and PrimeLending move to recruit local LOs. With less than $500 million in annual originations, small lenders can pivot with the market more easily than larger lenders by reducing their liquidity requirements, staffing levels, and core back-office expenses. Small lenders tend to have strong referral partners, which helps sustain purchase volume during periods of declining refinances. On paper, small lenders are also less risky for capital partners like local banks to support if they need extra liquidity.

Being thinned from the herd are medium-sized lenders, a group that includes approximately three-quarters of all IMBs. These lenders lack the agility to pivot quickly or hybridize due to contractual obligations and financial covenants required to remain operational. 

More than a cash problem, though, the regional ditch is a trap for lenders with “a lack of market knowledge and lack of cash to hire correctly,” Roque explains. For a growth-oriented company that wants to hit $5 billion in annual originations, he advises, ignore the industry’s pervasive tribalism — hire leadership who’s been there before. Don’t hire from within. 

Though Stratmor’s Graham prefers the phrase “volume tranche” to “regional ditch,” he agrees that lenders who are not exceeding certain financial thresholds will struggle to compete when the market rebounds. Lenders producing roughly $500 million to $2 billion annually are finding it very difficult to make money right now, Graham says. 

Seventy-eight percent of mortgage banks reported losses in the first quarter, but nearly all fall into the “finding it very difficult to make money” category this year. “It doesn’t mean you have to sell,” says Graham, “because if you’re willing to lose money or break even and keep your capital in there, that’s okay. Wait until next year; that’s a reasonable strategy. But, between those two, it’s very difficult to make money.” 

With few employees left to cut from the front office, the back office is now getting expensive, making mergers and acquisitions deals all the more attractive to buyers. During the boom years of 2020 and 2021, corporate back office functions such as closing and post-closing, executive compensation, quality control, human resources, and technology cost about 50 basis points. Now that cost is more than 100 basis points.  

“That’s the driver of the M&A,” Graham says. “The buyers can say that a 100 basis points are gone – I can buy you without adding one human in the back office. That’s the economic impetus for the deal.” 

What buyers are willing to pay for is production. And pay they will.  

Managing Unprofitable Branches

When business is strong, small- and medium-sized lenders will often branch into new markets using expense management branches (EMBs), otherwise known as profit and loss (P&L) branches. 

However, Roque has witnessed how lenders can struggle to manage P&L branches when the market turns. “When times are good, they’ll grow in markets that aren’t in their backyard and then close those branches because they don’t know how to manage those branches remotely,” he says. Given the cyclical nature of the mortgage industry, this is an unsustainable growth strategy that Roque points to as evidence of the lack of leadership he sees among small- and medium-sized regional lenders. 

In the run-up to rate contractions, this pattern exemplifies how the “regional ditch” is a cash problem that a lack of cash exacerbates. “Once you’re in the ditch, it becomes more expensive to get out,” Roque explains. Two levers lenders pull in a tight market are margins and staff, but this popular branching strategy outsources those decisions to branch managers. 

“It doesn’t mean there’s not some very good independent mortgage bankers who run these models and are successful … but a lot of them struggle,” says Stratmor’s Graham, who’s seen a lot of disruption to this model as the market shrinks.

P&L branches struggle during periods of margin compression and industry consolidation because operational control is split between the branch manager and the corporate office. Usually, branch managers are responsible for day-to-day operations, including staffing, while the corporate back office covers underwriting, closing and post-closing risk, technology, and human resources, among other services. As margins have shrunk, tensions have risen between unprofitable branches and corporate offices that are carrying the losses. 

“Suddenly, you’re negotiating with the branch,” says Graham, explaining that the conversation usually goes something like: 

Closing branches outright is the quickest way to cut losses, but abandoning a foothold in a market is a painful decision. What’s more, most mortgage companies lose $30,000-$50,000 when opening a new branch before that branch turns a profit – if it ever does. Half of newly opened branches never turn a profit, Roque says.

When it comes to small- and medium-sized lenders struggling to grow (let alone stay afloat), Graham says that centralization improves efficiency. Well-organized companies are also more valuable in the offing. Not only does centralization make it easier to leverage staff in various ways, but it allows companies to better focus their relationships and marketing. 

“Generally, the companies that have a very defined market share in a few markets are more valuable than the small independent mortgage bankers that may be in a bunch of markets,” he says, speaking from experience. 

Multi-Year Margin Compression

What remains to be seen is whether margin compression or margin erosion will persist even after mortgage rates drop. Ludden looks to the housing crisis of the 1990s when margin compression was a multi-year phenomenon impacting lenders well after the market had turned. 

As interest rates rose in the early 1990s, refinance rates significantly dropped, forcing lenders to grow through purchase volume – but lenders then didn’t face the same constraints on housing supply that lenders face now as they make a similar pivot toward purchases. Still, the mortgage industry in the 1990s experienced a dramatic reduction in total volume followed by a period of margin erosion that extended for multiple years, not several quarters. 

If history is bound to repeat itself, lenders should be learning that the mortgage industry wines and dines by refinances but lives and dies by purchases. An increase in refinance volume eventually lifted lenders out of the crisis in the 1990s. However, the mini refinance boom that many lenders hoped would materialize in the second half of 2023 seems less likely to materialize now that the second half of 2023 has arrived and mortgage rates are holding altitude.

A former loan officer (LO) himself, Graham says that LO compensation is a major driver of margin erosion that needs to be addressed. “We keep paying basis points. Basis points are on a loan amount, and the loan amount keeps going up, yet the margin or revenue per loan in this type of market is going down.” As average loan amounts rise, LOs pull more money out of every transaction. “That creates a lot of pressure on the P&L for owners,” he says.

Lenders contribute to long-term margin compression when they lower rates to compete for loans. Ludden has talked to lenders who experienced month-over-month, double-digit reduction in margins in the second quarter. These lenders found that originating loans for less profit per loan only increased their losses due to fixed origination expenses. Doing the same volume or increasing volume, but making less money per loan, means lenders have nowhere to go to recoup that loss. 

“Even if you still do the same number of loans,” Ludden explains, “you still have the obligation to keep the people in place that are doing the loans. You’re still paying the loan officer some origination fee, you’re just making less money on that loan you sell. That’s even worse than less volume because at least with less volume, you can reduce your headcount.” 

Secondary Impact

Secondary markets are also squeezing lenders’ margins. An anticipated spike in early payoffs has soured investors’ appetites for newly originated, residential mortgage-backed securities (RMBS) that have coupons averaging between 7-8%. This has investors demanding higher premiums as historical buyers of RMBS, such as banks and the Federal Reserve, are exiting secondary markets altogether. Lenders fear that even when mortgage rates drop, investors will be able to name their price, preventing rates from dropping as low as borrowers would like. 

Warns Ludden, “It could be that you see substantially worse economics for lenders even after we get through what we believe to be the hardest interest rate environment.” 

Nevertheless, all lenders are focused on strategic actions right now, whether investing in servicing infrastructure or looking for an exit strategy. No holdout seller wants to become a walkout seller in a fire sale.

“The question they have to ask is: Do I want to run the risk of having to put more cash into this business after I’ve already gone through a year and a half of losing money, or do I want to be smart and look at where things might be headed and choose to make an exit now while I control my own destiny?” To be in control of his own destiny, for now, Ludden is grateful.