Using Outcomes to Measure Nonprofit Success

Print Share on LinkedIn More

 Cups

Those who donate to nonprofit organizations naturally want to feel their gifts will be used successfully in a way that will improve society or some part of it, like children, the sick, students, etc. But how can donors evaluate whether or not a charity will ultimately deliver on their promise or mission?

Success in the business world is generally measured by the amount of profit—the bottom line—that is reported in the business’s financial statements. There are other aspects that can mark a successful business: Does it treat its workers fairly? Does it protect the environment? Is its advertising truthful, and are its products or services of good quality? But failure in these aspects eventually leads to diminished profits, as workers, customers, and investors desert that business for more socially responsible companies whose products or services are of better quality.

In the nonprofit world, however, there is no common, easily understood measure of success. In fact, having a large positive bottom line may be an indicator that the organization is not doing as much as it could to fulfill its mission. The true measures of success for most nonprofits are statistics related to its programs, but such data are difficult even for management to obtain and understand, much less outsiders. For example, an obvious measure of success for an educational institution would be how much students learn from attending classes. But, much to the chagrin of educators and public policy makers, actually measuring this learning is very difficult for a variety of reasons. (Think about how a church might measure its own success. Souls saved per pew-hour preached? And where would those data points come from?)

The Success Metric Conundrum

There are three types of data that might be used to measure a nonprofit’s success, but only one of them is a true measure:

–        Inputs describe how much in the way of resources (both financial and non-financial, such as volunteer time, materials, equipment, etc.) was used to conduct an activity.

–        Outputs measure the activities conducted by the organization, such as the number of classes held, the number of students enrolled or graduated, the number of concerts performed and number of concertgoers attending, the number of members enrolled, and the like. The problem with this type of data is that, while it shows the quantity of program services provided, it does not indicate whether any real benefits resulted. Did the students learn anything? What was the quality of the concerts? How well were the members served?

–        Outcomes measure how much better off the organization’s clients, or society as a whole, are as a result of the organization’s activities. For example, by how much has the teenage pregnancy rate in a community been reduced through the efforts of a charity whose mission includes educating children about the undesirable results of getting pregnant so young?

Of these three types of data, only the first is traditionally found in financial statements, although some organizations present certain output data in footnotes, as supplementary schedules or in management reports. However, true success is measured only by outcomes, and these data are never found in financial statements, if they can be obtained at all.

Evaluating Nonprofit Overhead

So, lacking access to most output data and almost all outcome data, donors are often left with input data as a surrogate for measuring success for nonprofits. But these data are very flawed when used for this purpose, as they do not necessarily have any direct relationship to true organizational success. For example, in a hospital, input data show how much money is spent treating patients, but not whether any patients are actually cured of the conditions that brought them to the hospital in the first place. Further, these data are necessarily past-oriented, and do not offer any assurance that the gift I make today will be used the same way in the future.

The particular input data that have been widely used to evaluate charities demonstrate how an organization spends its resources. Accounting standards require nonprofits to report their expenses in three categories: program, management, and fundraising. The knee-jerk reaction by users of financial statements is to consider program expenses as good and management and fundraising expenses (so-called overhead) as bad. Usually data are expressed as ratios of each category of expenses to total expenses, summing to 100 percent.

Various attempts have been made over the years to define acceptable ratios, with desirable program expenses usually somewhere in the range of 60 to 80 percent, and/or acceptable fundraising ratios generally no more than 15 to 30 percent. Of course, some expenditure on overhead is necessary for any organization to operate, but too often more attention is paid to this than is warranted. Recently, a major Internet news site published a list of what it called “The 50 Worst Charities in America.” Its sole criterion for being on the list was related to the overhead ratio. While expense ratios provide valuable information for nonprofit executives charged with making decisions about the organizations’ activities, these ratios don’t serve any value as an indicator of organizational success. Although most people understand that it takes money to raise money, and it takes money to manage an organization, sometimes one sees expressions in a fundraising appeal of an expectation that 100 percent of all contributions raised will go directly toward program expenses. That is clearly unreasonable and indicative of an attempt to mislead the public about how a charity uses contributions.

Low Overhead vs. High Performance

Some states—most recently, Oregon—have passed laws attempting to regulate how much charities that solicit contributions in that state are allowed to spend on overhead. The Internal Revenue Service and federal law have no such requirements. The IRS, however, evaluates tax-exempt organizations on how well they appear to be fulfilling the mission for which they obtained exempt status; an extremely low program-expense ratio would naturally raise a question as to whether that mission is likely being fulfilled. Definitions and penalties vary, but the logic behind such a law is clear: to protect the innocent citizens of the state from being bilked by unscrupulous charities that try to give the impression that donating to them will result in wonderful benefits to society, when in fact most of the money raised will go for overhead. Unfortunately, there are some so-called charities out there that spend little, if anything, on programs while paying exorbitant amounts to professional fundraisers and to their own managers. The problem for donors and regulators is to distinguish the bad actors from the well-intentioned charities. There can be various legitimate reasons why a charity’s overhead ratio may be on the high side. For example, the organization espouses a cause that is relatively unpopular with the public, so it naturally has to expend more effort to raise the money it needs; the organization’s constituency is largely in an economically depressed area, so is less able to contribute or pay for services; the organization is new and still building its infrastructure; or the organization has experienced some problems recently, but is now getting back on its feet.

Several years ago, the Wall Street Journal published a single-panel cartoon showing a homeless individual with his hat held out. Around his neck was a sign saying, “No portion of your contribution will be used for administration.” Would you contribute to this nonprofit “organization”? The sign is presumably telling the truth, and the “charity” likely would not end up on the fifty-worst list, but what is the program here? What beneficial outcomes are to be expected? If the program is aimed at providing him with food, clothing, job training, and a job, then maybe this is a cause worth donating to. But if the “program” consists of enjoying the offerings of the nearest tavern, donors would probably give this one a pass.

Earlier this week, the presidents of three well known nonprofit organizations—the BBB Wise Giving Alliance, GuideStar, and Charity Navigator, whose missions include evaluating charities and making their evaluations available as guidance to donors—issued a joint letter titled, “The Overhead Myth.” In this call to action, the three organizations urged donors and the public to place less reliance on expense ratios when making giving decisions. They correctly point out that how money is spent is often not a very reliable indicator of the outcomes achieved by the nonprofit. In fact, they suggest that many nonprofits should probably be spending more on overhead to improve the quality of management, strengthen internal control, gain operating efficiency, provide for long-term stability, and the like.

The last sentence of that letter reads, “The people and communities served by charities don’t need low overhead, they need high performance.” This author could not agree more.

  • Stephen Viederman

    Pleased to see “outcome” rather than the over used “impact.”

  • Renee McGivern

    Finally, the sector is having this conversation and I was pleased to see that the leaders of the big three organizations took a stand and wrote and signed a letter for distribution saying that performance trumps overhead. The letter was not written for donors, though. It’s not simple and direct enough. I would love to see what on-the-ground nonprofit leaders did with the letter and how they reworked the text to be understandable to board members and donors.

  • Ira Kaminow

    Outcome measurement is a great idea and I strongly favor it.in principle. I fear however, that it is extremely difficult to measure outcomes in our complex world where charities often play only a small role in achieving success or suffering failure. Moreover, I am concerned that the new focus on outcomes will force charities to develop meaningless measures of outcome in order to attract donors’ dollars, just as the old focus on overhead forced charities to underspend on overhead and under-report overhead spending. Let’s face it, charities are hard to evaluate. They should be judged along many dimensions one of which should be outcome measures which are necessarily imperfect.

  • Keenan Wellar

    Measuring outcomes is challenging. But in reality, many non-profits are not even trying – they aren’t pursuing outcomes so they don’t have to worry about trying to measure them!

    A great many non-profits measure and report solely on outputs, mainly for the purpose of generating or maintaining inputs ($$$) be they funders, donors, or sponsors. This is why there is so much resistance to a focus on a social change model in the social services sector, for example – the systems inputs and outputs have been established and they often have little to do with community-based outcomes. They have to do with an Excel sheet created in 1992.

    On a familiar grand scale, we can think of this as building more and more shelters and bigger and bigger food banks, but not reducing homelessness or hunger. In my own world, it would be the housing of people with intellectual disabilities in group homes, “occupying” them in day programs and sheltered workshops, and channeling them away from other citizens into specialized recreation, education, and basically every aspect of life through expensive segregated programs.

    We’ve known the outcomes of this are horrible, and we knew it 30 years ago. And yet the model persists and dominates, and at times seems immovable. I could probably write a funding proposal tomorrow and get funded to do all the wrong things. Funders bear a lot of responsibility for this but that’s no excuse – if the sector had a stronger focus on outcomes then funders could be helped to break their addiction to program and project outputs and get into the exciting and much more effective world of CHANGING communities.

  • Keenan Wellar

    I’m working on that Renee, I’ll be happy to share with you when it happens. I think the significance of the news was lost – it certainly doesn’t compare to all the attention given when the pendulum swung the other way and suddenly everyone had to eliminate overhead. It should have been major and ongoing headline news but if not for places like NPQ, I’m not sure who would have noticed.

  • Lora-Marie Bernard

    I also thank you for understanding the difference between outcome and impact. One additional comment on this very timely article is to simply expand on high-performance nonprofits. So many organizations are in capacity growth right now and/or are struggling to emerge from the economic downturn. As a society we expect an emerging business to build its overheard in order to be successful. However, we have an expectation that a nonprofit will grow without an attention to overheard. This is a difficult position to be in. The idea of looking at the whole organization instead of its financials has intrigued me for several years as an executive director of an emerging environmental organization. I’m glad this discussion is coming to the front burner, getting the buzz it deserves and a head nod from our watchdogs. It will be interesting to see how effective Oregon will be.

  • Paul

    Rather than criticizing the use of overhead measures as a measure of success the nonprofit community should embrace it and other measures but in their proper place. Simply noting that overhead is not a perfect measure and contrasting the imperfect measure with a, nearly impossible to obtain, ideal measure comes across as objecting to being measured. That’s not the world we live in. Measures are everywhere. High overhead is often an indicator that something is amiss. Nonprofits would do better to propose a series of measures that would be useful indicators of effectiveness. The series of measures would need to be broad enough to allow for the wide variety of measures for the wide variety of nonprofits. Ideally the specific indicators would be linked to the organizations mission (different specific measures for a mission of feeding the hungry vs. curing hunger). But constantly criticizing the measures for their imperfections is not the long term need of the NFP community.

  • Keenan Wellar

    I think the main reason for the counter-reaction to the “overhead” measures came about because a determination of “high overhead” can only be reached after extensive examination of an organization’s work – inputs, outputs, and outcomes. To me it’s like pointing out that a trailer truck gets poor gas mileage…true! But if your job is to haul three tons of groceries cross-country, then saying a Prius is more efficient is actually not accurate.

    Sorry the analogy is far from perfect, but the same goes for charitable organizations. For me looking at expense ratios and the like without any attention to PURPOSE (and outcomes) offers little value.

    Unless you can find two organizations that are identical in every way but one of them is somehow expending more on overhead for no discernible reason, it’s hard to find a lot of utility in overhead measurement. In my 20 years or so in the non-profit community with a mix of experience in board and staff roles, I can’t think of one occasion where issues of waste would have been identified through an examination of overhead.

    To me where there are gross inefficiencies in the sector (and individual organizations) they are more likely to do with investment in outputs that don’t actually help improve the community or are not very effective in creating change. There are many programs with totally wrongheaded outcomes that have wonderful balance sheets with overhead that won’t ever raise an eyebrow.

  • Ken Steensma

    Perhaps we might consider extending our nonprofit measurement paradigm beyond outcomes to a focus on lifetime collective impact. This focus enables us to view performance from a systemic perspective that drives collaboration between organizations that are absorbing the costs and organizations that are tasked with reducing the costs. For example, what might be the lifetime collective impact of one high school dropout – personal costs, family costs, community costs, lost tax revenue from underemployment, multi-generational concerns, etc? Who will be absorbing these long term costs once the individual leavers the educational system? How long will they be willing to absorb this cost? What would it cost to enable this individual to graduate prepared to direct his/her future? Wouldn’t it make ‘cents’ to to collaborate? Who would like to drive the strategy?

  • Veronica Kulon

    WIth today’s philanthropic environment, funders are focusing more on impacts rather than outcomes and financial statements. As a nonprofit consultant addressing on the issues of strategic planning and fund development with a Masters in Nonprofit Management, I advise clients to operation their organization with best business practices using a logic model to document inputs, outputs, outcomes and both long and short term impacts/

  • Bert T. Edwards

    BBB Wise Giving Alliance, GuideStar and Charity Navigator and now Dick Llarkin are totally correct. OPutcomes are what a donor needs to know, and this is not easily derived from audited – or unaudited – financial statrements. During my 34+ year CPA career, I was engagement partner on several Washington-based nonprofits – Girl Scout council of the Nation’s capital (GSCNC), Martha’s Table, and the Cathedral Choral Society. GSCNC was and continues as the largest Giorl Scout Coun cil in america in terms of girls of 3-18 enrolled in programs and adult volunteers; the GSCNC staff was principally involved with recruiting and training volunteers, fuind-raising administration (GSCNC sells nmorecookies than any Girl Scoutcounsil in the US), and similar “overhead” activities. Martha’s Table operates a feeding program for homeless and indigents for which all food and food-preparation and distribution is peerformed by volunteers, and an after school “latch key” child program, also clonducted by volunteers. The cathedral Choral slociety has a pat-time administratior and all activity is performed by volunteer menbers of this magnificent chorus.These three organkizations was widelhy kinown in the Washington area for their outstanding programs, but the accounting rules generally do not permit quantifying the value of the volunteers. Each is supported by donated funds from leading Washington charitable foundations and citizens. none of these funders would discontinue their support based on nhigh ovehead and fund-raising expenses because their oputcomes are reatly appreciated by their supporters.

  • RJ Jankowski

    I appreciate articles like this. The Center for Association Resources always seems to offer added value.

  • TREVOR REISZ

    This article does an excellent job of clarifying the challenge of determining some myths of evaluating the success of a non-profit. Traditional metrics of looking at percentages of of expenses, or overhead, are very misleading and are dependent on many, not so obvious factors.
    Unfortunately, although very well written, this article really does not provide tangilbe answers as to how to determine the performance of the non-profit. The answer(s) are far more complicated and require an effort to determine how the non-profit assess its own performance and sometimes as importantly, how it communicates that performace to potential contributors.
    Each association is very unique and there are no generic methodologies.
    Most require proper education and assistance in first understanding the importance of this subject, and obtaning some professional assistance of those that are expert in disecting the performance metrics, and also communicating the data to the potential contributors in an efficcient and clear manner.