Over the past decade, R.I.P. Medical Debt has grown from a tiny nonprofit group that received less than $3,000 in donations to a multimillion-dollar force in health care philanthropy.

It has done so with a unique and simple strategy to tackling the enormous amounts that Americans owe hospitals: buying up old bills that would otherwise be sold to collection agencies and wiping out the debt.

Since 2014, R.I.P. Medical Debt estimates that it has eliminated more than $11 billion of debt with the help of major donations from philanthropists and even city governments. In January, New York City’s mayor, Eric Adams, announced plans to give the organization $18 million.

But a study published by a group of economists on Monday calls into question the premise of the high-profile charity. After following 213,000 people who were in debt and randomly selecting some to work with the nonprofit group, the researchers found that debt relief did not improve the mental health or the credit scores of debtors, on average. And those whose bills had been paid were just as likely to forgo medical care as those whose bills were left unpaid.

“We were disappointed,” said Ray Kluender, an assistant professor at Harvard Business School and a co-author of the study. “We don’t want to sugarcoat it.”

Allison Sesso, R.I.P. Medical Debt’s executive director, said the study was at odds with what the group had regularly heard from those it had helped. “We’re hearing back from people who are thrilled,” she said.

In a survey the group conducted last year, 60 percent of people with medical bills said the debt had negatively affected their mental health, and 42 percent said they had delayed medical care.

Studies had shown significant mental health and financial improvements for other types of debt relief, such as paying off student loans or mortgages. But those debts have more urgency: Homeowners who do not pay their mortgages could quickly lose their homes, whereas a hospital bill can languish for years with little consequence.

New federal rules implemented last year, which removed medical debts of less than $500 from credit reports, have further lessened the impact of unpaid hospital bills.

The study, published as a National Bureau of Economic Research working paper, is one of the first to look at the impact of medical debt relief on individuals. “It’s a big policy area right now, so its important to show rigorously what the results are,” said Amy Finkelstein, a health economist at the Massachusetts Institute of Technology whose research has shown significant positive effects of gaining health insurance.

Ms. Finkelstein is also a co-director of J-PAL North America, a nonprofit group that runs randomized experiments on social programs and provided some funding for this project.

“The idea that maybe we could get rid of medical debt, and it wouldn’t cost that much money but it would make a big difference, was appealing,” Ms. Finkelstein said. “What we learned, unfortunately, is that it doesn’t look like it has much of an impact.”

Mr. Kluender and one of his co-authors came up with the idea for the study in 2016 when they saw R.I.P. Medical Debt featured in a popular segment from John Oliver’s television show. They and two other economists teamed up with the nonprofit group to run the experiment, which wiped out $169 million in debt from 83,000 debtors between 2018 and 2020.

Those patients, like others R.I.P. Medical Debt typically helps, were not making payments on those bills, which were at least a year old. The economists monitored the patients’ credit scores and sent them surveys asking questions about their mental health and the barriers they had faced in getting medical care.

They compared those results to a control group of 130,000 people who had not had their debts relieved, and they found few differences. The two groups reported similar financial barriers to seeking medical care and similar access to credit. The patients whose medical debts had been paid off were just as likely to have trouble paying other bills a year later.

“Many of these people have lots of other financial issues,” said Neale Mahoney, an economist at Stanford and a co-author of the study. “Removing one red flag just doesn’t make them suddenly turn into a good risk, from a lending perspective.”

For some in the study with no other debt in collections, the erased medical bills did lead to a 3.6-point bump in their credit score, on average.

The researchers were startled to find that for some people, particularly those who already had high levels of financial stress, debt relief worsened their depression. It’s possible, the researchers speculated, that being told about the sudden payoff had inadvertently reminded debtors of their other unpaid bills.

R.I.P. Medical Debt has “evolved” since 2020, when the experiment concluded, Ms. Sesso said. Major donations now allow the group to buy up billions in debt in a single city, which she said could have a larger impact on beneficiaries’ finances.