Debating the Realities of Ranking Charities



Editor’s Note: The following essay, written by Charity Navigator President and CEO Ken Berger, is one part of a two-part point/counterpoint series. The other essay in this series, by William Schambra, director of the Hudson Institute’s Bradley Center for Philanthropy and Civic Renewal, is available here.

In order to discuss the merits of a charity ranking organization like Charity Navigator (hereafter CN), we must define at least a couple of basic concepts. First, we must specify which charities CN evaluates. One of the major criteria that we at CN use is that the charity must be over $1 million in annual revenues. Only about six percent of U.S.-based charities are that large. Furthermore, we do our best to follow the money and focus on the biggest charities first.

Members of the Occupy Wall Street movement have protested the fact that one percent of Americans control approximately 43 percent of the country’s wealth. In the case of charities, one percent of the organizations collect 86 percent of the $1.5 trillion in revenues that comes into the charitable sector each year. That is double what Occupy Wall Street is angry about! Most of the roughly 6,100 charities CN currently evaluates fall into this top one percent. So when I discuss charities, it is the largest charities that I am specifically talking about – not the 94 percent of charities that have less than $1 million in annual revenues, not the small charities that garner around six percent of sector revenues annually.

The other important definition we must take into account refers to what we evaluate. We believe there are three critical dimensions a charity must report on and that a donor must consider in order to be a wise social investor/charitable giver. For the sake of brevity, I refer readers to the methodology section of our website, as well as information on the site regarding the critical new feature that assesses the quality of how a charity publicly reports on the results of its work.

The Donor Problem

Many observers have noted that the vast majority of donors make their donations based on their heartfelt motivations and do not consider a charity’s performance when making charitable decisions. Some observers believe that this is and will always be the case. Others, including CN, do not agree. In our case, we can point to the record breaking six million site visits we had last year to indicate that an increasing number of donors are looking for evidence of performance by charities. Furthermore, the research from Hope Consulting (Money for Good) indicates that 90 percent of donors surveyed consider organizational effectiveness what matters most. On the other hand, their research also indicates that only two to three percent of donors actually go looking for evidence of effectiveness.

How can we reconcile these apparently contradictory findings? I think the answer can be found in the four years of research we did at CN to develop the results reporting dimension of our evaluation system. From this research effort, we discovered that the vast majority of nonprofits do not publicly report on the results of their work. Therefore, most donors are making a rationale choice to not seek out data on effectiveness because there is “no there there!” It is our belief that we can increase the two to three percent of donors who do research to a substantially higher percentage once CN 3.0 (our three-pronged rating system that evaluates a charity on financial health, accountability and transparency, and results reporting) “nudges” enough nonprofits to publicly report the results of their work.

The Nonprofit Problem

Unfortunately, I think the charitable sector has an even bigger problem than not publicly reporting their results. As noted by David Hunter in his article, “The End of Charity,” “…there is virtually no credible evidence that most nonprofit organizations actually produce any social value.” This is the sobering reality in most of the charitable sector today. Therefore, the greatest challenge for the largest nonprofit sector in the history of the world (which resides in the USA) is to know that what it is doing is actually leading to meaningful and lasting change in communities and peoples’ lives (i.e. creating social value). For this to happen, charities need to develop and utilize a robust performance management system to measure the results of their work. We hope that our efforts will help foster this change.

The Beneficiary Problem

The for-profit sector has a more direct feedback loop from its customers because they are both the payer and the recipient of the product or service. However, in the case of the charitable sector, the customers (beneficiaries, clients, consumers, patients, etc.) often do not fund the charities that serve them. Therefore, it is typical that their perspective is not being fully considered in the operations and performance of the charities that are supposed to serve them. The results reporting dimension of our evaluation system includes a measure of beneficiary feedback (we call it constituent voice) to help solve this problem.

The Problem Solved

Here is a critical data point that further encourages us to believe that we can help the movement to solve the aforementioned problems: since launching CN 2.0 in September of 2011, roughly 50 percent of the charities that we evaluate have made changes to the way they operate (i.e., board composition, ethical and other best practices, etc.). Therefore, we are hopeful that CN 3.0 will produce a similar result, encouraging more and more charities to focus on results. We believe that is why Paul Brest, the former president of the William and Flora Hewlett Foundation and one of the preeminent experts on results in the nonprofit sector, has told us that we are doing “the most important work in the nonprofit sector today!”

Responses to William Schambra’s Claims

William Schambra, director of the Hudson Institute’s Bradley Center for Philanthropy and Civic Renewal, has made a number of statements on this subject that I believe warrant a rebuttal.

1) Schambra quotes a finding from the Hope Consulting report: “While donors say they care about nonprofit performance, few donate to the highest performers.”

Although the Hope Consulting report notes that only two to three percent of donors actually use information based on relative performance of charities, the report also indicates that there are 15-16 percent of donors who seem to be very interested in focusing their social investing on charities with evidence of meaningful results. If this 15-16 percent who are motivated to do more were given the information to do so, it is estimated it would unleash a $25-$50 billion in additional giving. This is a potential increase of 17 percent! CN 3.0 was created not only to unleash that potential social investing, but also to further direct the estimated $5 billion to $10 billion donated by our existing users, as well as to increase the number of people who become social investors.

2) Schambra suggests that CN 3.0 is just the latest chapter in an ongoing effort in the history of scientific philanthropy that shows no evidence of changing things for the better but instead buries charities in the burden of reporting meaningless information.

The majority of the history of so-called “scientific philanthropy” focused more on counting activities and outputs, not outcomes. It is only within the past couple of decades that the “outcomes movement” has truly taken form. We agree that many charities are drowning in the requirement to generate data for funders that is largely meaningless. Therefore, all of us must be advocates for funders to engage in a dialogue with the charities they fund to assure that the data reported is related to measurement that indicates the creation of social value. Furthermore, those same funders need to help pay the infrastructure costs and provide expertise to assist charities in creating the performance management systems required to manage, measure and report on results.

The other significant historical change that indicates we are at a critical tipping point is access to powerful information technology (IT). The improvements in IT have made it much easier, less expensive, and quicker to monitor, manage, compile and analyze data.  There are now quite a few early adopters in the charitable sector that are taking advantage of these developments and they are showing the way for many more charities to improve their results.

As to evidence that the focus on performance management and measurement is changing things for the better, here is one of countless examples I could offer: There are 117 cystic fibrosis clinics around the country. With their intervention, 110 clinics extend people’s lives, on average, by about 23 years or so (the average without intervention is seven years of age), taking them into their thirties. In the seven other clinics, people’s lives are extended an additional 20 years into their fifties. Why? Because these seven clinics operate with a relentless and rigorous focus on outcome monitoring and measuring. Where would you want your child to go?

3) Schambra argues that donors give with their hearts and that this is crucial to the survival of the American democratic system, which requires an engaged citizenry. Charities, he says, provide the perfect vehicle for this.

I am all for encouraging democracy, getting people involved, and encouraging citizens to join small local groups to become engaged and empowered. Please remember, though, that roughly 50 percent of charities – those typical local groups – operate with revenues of under $25,000 and control less than one percent of the revenue in the nonprofit sector. The reality is that this is not where the majority of contributions are going.

Furthermore, what are you supposed to do when there is a natural disaster somewhere on the other side of the globe? Is the small local charity that you know from personal exposure to it going to satisfy the job required? Don’t you want to know not just whether citizens were engaged but whether the social investment in that disaster really helped people in a meaningful way?

Why Rating Charities Matters

On a personal note, I have spent 30 years in the sector trying to help others. I have run homeless shelters, provided health care to the poor, and worked with people with physical and developmental disabilities. Therefore, I am not speaking from a theoretical standpoint. I have seen first-hand many people that could have been helped more, but weren’t, because we weren’t managing our resources, programs and services very well. If we can get more charities to focus on better managing their efforts to provide measurable social value, it means we can truly save many more lives. In other words, this is a life and death discussion we are having today. I urge you to think deeply about this and hope you will join with us as we strive to win this “battle for the soul of the nonprofit sector.”


  • Peter Miragliotta

    Ken makes a compelling argument for the benefits of measuring charities’ results.

  • Garth Nowland-Foreman

    Yes, I hope everyone does in fact follow the link to the fine print of Charity Navigator’s methodology. First, Charity Navigator has a faulty compass when it comes to measuring so-called “financial efficiency”. This mainly relates to 3 different ways of measuring percentage of funds allocated to so-called programme costs and other (so-called admin and fund-raising costs). This thoroughly contracts the above articles emphasis on outcomes; it is in fact an input measure. Regardless of the results achieved, a charity is considered most efficient when it spends nothing on so-called admin and fund-raising costs. In fact, research in a number of countries has shown that the greatest problem for non-profits is an UNDER-investment in ‘back room’ or so-called admin costs – and can include items such as R&D, training, supervision, monitoring & evaluation, IT systems, etc etc. (Some Australian research, for example, suggests a 40% gap between non-profit and equivalent commercial organisations. Letts, Ryan & Grossman (1999) outline the systemic or structural reason non-profit under-investment in crucial infrastructure, which I don’t have space to repeat here.) At its extreme, an organisation that only employed one paid manager (and her associated on-costs), and happened to use all volunteers to deliver food to elderly risks being rated at the lowest extreme of “financial efficiency”, while another organisation that engaged a volunteer manager in donated premises, and used paid staff to deliver donated food to elderly might be able to pull off your highest extreme score of “financial efficiency” (even if in the latter case it cost twice as much to deliver the same number of food parcels). Yes, this is a bizarre example, but it also illustrates how bizarre it is to pay too much attention to such a measure. What are admin or overhead costs anyway? Either every dollar spent helps achieve its mission, vision and values or it doesn’t. An efficient organisation does NOT require all its resources to be at the front-line. This is an illogical assumption. Measure’s such as Charity Navigator’s so-called “financial efficiency” only serve to steer organisations off-course – either by encouraging them to hide and mislead in their reporting, or otherwise to under-invest in important infrastructure or back-room costs. If you were really interested in measuring efficiency, you would ignore what inputs are used, and compare instead the costs per output achieved (or even costs per outcome if you felt really ambitious!) If i get a chance, I’ll come back to comment on the other factors…

  • Ken Berger


    Your characterization of our view of overhead is not accurate. If a charity claims zero overhead or even simply zero fundraising, we put up a donor advisory to caution our users because we believe that overhead is important but should be managed well. Fundamentally, we believe there are three critical dimensions to be considered regarding a high performing charity, not just results (aka mission accountability). Financial and organizational (esp governance) accountability are critical to ongoing success as well.

    We do not pay “too much attention to the measure” in that it is a part of the financial dimension (which also includes other measures such as sustainability). Currently questions of overhead impact about 30% of a charities score in our rating system. We anticipate this will further decline once the results dimension is fully operational. We agree that results should be the most important consideration but since there is no data currently available on most charities for results, we need to help change the culture. To expect us to measure results when there is nothing available is a ridiculous expectation. Furthermore, to entirely discard questions of overhead is to throw out the baby with the bathwater. I ran charities for 30 years and it was critical to know that we were balancing infrastructure needs with ongoing operational needs. Inputs as well as outputs and outcomes are all important consideration to make an operation function well and for a wise social investor to assess an organization. Long term, meaningful and lasting social value, requires keeping your eye on all these issues in our view.