The Tyranny of Success: Nonprofits and Metrics

Print Share on LinkedIn More



The tyranny of success often can’t be bothered with complexity.”

—Father Gregory Boyle, founder of Homeboy Industries, Los Angeles

Over the past year or so, the Nonprofit Quarterly has hosted a lively conversation about the merits of metrics, strategic philanthropy, and other cutting-edge practices associated with sophisticated contemporary grantmaking. (It is worth noting that NPQ is almost the sole venue for this sort of spirited, balanced back-and-forth about issues that are too seldom discussed in the nonprofit world.)

As useful as the discussion so far has been, we’ve only begun to deal with the larger problem beneath questions of method and technique. That problem is—as Jesuit priest and founder of Homeboy Industries, Father Gregory Boyle, puts it—the “tyranny of success.”

Modern philanthropy famously demands success in the form of tangible, objective, measurably positive outcomes. Each new generation of grantmakers comes onto the scene and utters this sentiment as if it had never occurred to anyone before. But, of course, it’s been at the heart of philanthropy since John D. Rockefeller enunciated it one hundred years ago.

This demand, in turn, requires nonprofits or those who audit or evaluate them to make sharp distinctions of their work into success and failure. And yet, especially among hard-pressed populations that have always been the peculiar charge of the nonprofit sector, that distinction often is not so clear. What today looks like a success in a few months may turn into a failure, and what today looks like a failure becomes a success. More important, failure and the way it’s accommodated contribute decisively to success. But the measurers have no way to countenance such complexity.

In order to construct the elaborate edifice of Big Data, there simply must be, deep down at its very foundation, a clear, distinct separation of the sheep from the goats, an assignment of “1” (success) or “0” (failure). Just as the vast digital world is built on “0s” and “1s,” so is the empire of metrics. The most remote and abstract regression analysis depends on that initial, radical division—no hedging, no hemming or hawing, just putting it down on the scoring sheet as a “1” or a “0,” leaving the rest to the evaluators at foundation headquarters or at the university.

But this grand divide isn’t as simple as it seems. Alicia Manning, program officer with the Lynde and Harry Bradley Foundation in Milwaukee—where I once worked—recently took me to meet George Bogdanovich, cofounder of that city’s Community Warehouse. Among many other things, the warehouse accepts donated surplus construction material, appliances, plumbing fixtures, and so forth, and sells them at reasonable prices to those who are rehabilitating houses and stores in the inner city. They’ve now begun using some of the materials to prefabricate doorframes and sink cabinets, which provides jobs while teaching entry-level carpentry skills. All Warehouse employees have come out of prison or off the streets.

As happens so often when I visit frontline nonprofits—okay, sometimes it happens because I raise the question—we started discussing the problem of measurement: the demand that nonprofits classify their efforts into either success or failure. This clearly is a problem that George wrestles with regularly and without any prompting from impatient funders..

He brought up the example of Joe, a pimp and drug dealer who had been one of his first employees. Things had gone well for a while, but then there had been a couple of thefts. George chose to follow the advice dispensed by the director of the recovery community where Joe lived: “Fire his ass.” Even as we stood in the drafty warehouse talking the question over, though, George was awaiting the call he gets from Joe every year around Christmas. He always checks in, telling George how things are going, and thanking him once again for giving him the opportunity for his first job out of prison—even though George had, indeed, fired his ass.

“So,” George asked—probably more of himself than of us—“was my experience with Joe just a failure?”

No way you don’t count that a success,” Alicia immediately insisted.

In moments of pridefulness, I refer to Alicia as my student, since twenty years ago I took her around on her first site visits among Milwaukee’s grassroots leaders. But the fact is, she has far surpassed whatever I had to offer her and now navigates with ease among the Bradley Foundation’s smallest inner-city grantees. Her grant recommendations are based less on numbers than on the “local wisdom” that she gathers with her own eyes and ears and finely tuned BS-detector. She has no problem dealing with the inadequacy of “1s” and “0s” and the ambiguity that Joe poses to classification. But how many other foundation program officers, academics, or evaluators can claim that hard-won skill?

I first struggled with this problem in the 1990s as Alicia’s predecessor at the Bradley Foundation, helping to support the National Commission on Philanthropy and Civic Renewal, which became the namesake for my present center at the Hudson Institute. Chaired by now-Senator Lamar Alexander, half of the commission’s membership had been drawn from grassroots leaders whom Alexander had met on his travels or whom Bradley had come to support in its efforts to stimulate interest in faith-based civic institutions. The commission visited each of the grassroots members’ projects in turn.

Commission member and Episcopal priest Father Jerry Hill, for instance, had initiated a program in Dallas, the Austin Street Shelter, which ministered to a very marginal group: the vulnerable, mentally ill homeless (that is, homeless individuals who were all too likely to be victimized in the mainstream shelters) in that tough part of town. It was nothing at all fancy—a warehouse-like structure where the homeless would come each evening to pick up a pallet and sleep on the floor. Sponges and a bucket of bleach water stood beside the stack of already clean pallets, for those who wanted to wipe them down again. But around the periphery of the main room were smaller, semiprivate sleeping quarters for those who wished to begin their journeys off the streets..

Father Hill introduced us to one such journeyer. She had been a Braniff stewardess until the airline went belly-up, leaving her jobless in a bad economy. That began a downward spiral to the streets. There she became known as the “Bug Lady,” because of the vermin in her hair. By the time she reached the shelter she had become so unstable that female staff had to help her shower. But now she was on her way back up, with her own bed in a semiprivate room, and a job—saving up so she could move to her own apartment.

The whole process—the climb from the streets to the semiprivate room and now possibly onward to full independence (a climb by no means yet complete, and by no means guaranteed to have a happy ending)—had already taken over a year and countless hours of intensive staff attention, racking up who knows how much per capita cost.

The commission’s final report concluded that we need to be more generous toward the work of smaller, often faith-based groups like the Austin Street Shelter that do so much of the unheralded heavy lifting among America’s most vulnerable and marginalized. Still, its number one recommendation for philanthropy (John D. Rockefeller, yet again) was to focus on overcoming the gap between American generosity and the “actual impact of giving on those in need,” a gap that “must be closed by reorienting our giving toward organizations that get, and can demonstrate, real results.”

But how, I wondered, did the commission’s own example of the Austin Street Shelter fit into its demand for “real results”—for success? After months upon months of concentrated, intensive staff time and effort with the Bug Lady, do we finally just tally her up as a “1”? Is that even remotely like the “1” also used to depict, say, the depressed college dropout who quickly recovers from his fling with drugs and gets back on his feet with minimal help from the staff?

And if, in spite of monumental efforts, the still-fragile Bug Lady slips back onto the streets, do we now count her as a failure? Do we erase the “1” and replace it with a “0,” discounting entirely the shelter’s time, investment, and promising initial progress that might bear fruit later on?

Our assignment of “1” to the Bug Lady is at once far too inadequate a way to describe the shelter’s work and yet far too firm and final, given the realistic prospects for someone in her situation. But the shelter’s outcomes-oriented funders would insist nevertheless that it make these sorts of stark and sterile judgments, even among those least susceptible to being sorted so conclusively into successes or failures.

Another example of the insufficiency of “0s” and “1s”: This past fall, I attended a memorial service in Denver for the late Bob Coté, founder of Step 13, a program aimed at bringing alcoholics and addicts off the streets. Coté could be cranky, outspoken, and cantankerous—utterly defiant of the pieties and orthodoxies of the large social service establishment. He broke onto the national scene briefly in the 1990s, testifying before Congress against the federal practice of mailing SSI checks to local bars, where they would become, in essence, running tabs for alcoholic recipients. The Social Security Administration was subsidizing “suicide on the installment plan,” he vociferously charged.

I had always wondered where he had acquired this burning animus against SSI, and in the testimonials to Bob I heard the reason: he had himself come off the streets in the 1980s, and brought a dozen other guys with him into sobriety. One of them was Billy, his proudest success, an old drinking pal who was the “smartest guy he had ever met”—able to spout pages of Shakespeare from memory. But once sober, Billy had signed up for SSI. And, for the first time, he could afford the “good stuff”—Crown Royal—if he so chose. He did. He was dead within months of leaving Step 13.

A success become a failure, a “1” replaced with a “0.” And yet, far from discouraging Coté it gave him the fire to testify before Congress and, more importantly, the determination to redouble his efforts to bring more of his old acquaintances off the streets. To the end of his days, Bob talked wistfully about losing Billy; a small plaque devoted to Billy’s memory hangs on the wall of Step 13’s main meeting room. Without that “failure” and the resolve it enkindled, how many successes would never have happened?

Failures that we should count as successes, toehold successes that tremble perpetually on the edge of failure, and failures that stoke the energy to keep striving for success. These nuances and complexities leave nonprofit leaders troubled and uneasy about the demand to freeze-frame their work into “1s” and “0s.” Everything about their daily experience tells them that success and failure are far more complicated and nuanced than that.

No one captures this problem more directly or bluntly than Father Boyle. Homeboy Industries’ work focuses on young Hispanic people caught up in L.A.’s powerful gang scene. It offers a dizzying array of social services and enterprises, including a day-care center, graffiti removal, silk-screening, a farmers’ market, catering, a bakery and café, a grocery, and tattoo removal.

Like all the groups we’ve been discussing, Fr. Boyle has any number of “success stories” to tell, many of them recounted in his recent Tattoos on the Heart: The Power of Boundless Compassion. When readers arrive at chapter 8, promisingly titled “Success,” we of course expect a final crescendo of successes, especially after the teaser opening lines: “People want me to tell them success stories. I understand this. They are the stories you want to tell, after all.”

Then he does tell us stories, six of them, about Scrappy, Raul, Shady, Manny, Ronnie, and Angel—all young people whom Homeboy Industries had been easing out of the gang scene. Each of the stories begins with hope and expectation, as Fr. Boyle and his staff take heart from promising first steps toward more productive lives. Each looks to be a “win” for Homeboy—a “1.” By the time chapter 8 concludes, however, each of their lives has been snuffed out by gunfire. “Success” doesn’t give us a single happy ending.

Fr. Boyle isn’t being morbid. He’s challenging at its root what he calls the “tyranny of success”—our unforgiving system of “1s” and “0s.” He draws some comfort, he maintains, from Mother Teresa’s famous reminder that “we are called upon not to be successful, but to be faithful.” He adds, “This distinction is helpful for me as I barricade myself against the daily bread of setback[. . . .] For once you choose to hang out with folks who carry more burden than they can bear, all bets seem to be off. Salivating for success keeps you from being faithful, keeps you from truly seeing whoever’s sitting in front of you.”

Scrappy’s gangland-style execution while working on a Homeboy graffiti crew, for instance, prompts these ruminations from Fr. Boyle: “Was he a success story? Does he now appear in some column of failure as we tally up outcomes? The tyranny of success often can’t be bothered with complexity. The tote board matters little when held up alongside Scrappy’s intricate, tragic struggle to figure out who he was in the world.”

Fr. Boyle’s faith tradition directs him toward the biblical figure of Jesus, of course, who challenged the world’s framework of success and failure. Jesus snubbed the rich and powerful, seeking out instead the company of prostitutes, publicans, the blind, halt, and lame, and other marginalized individuals of his time. As Jesus stood with them, Fr. Boyle insists, so we are called to stand with the unemployed, the adjudicated, gang members, and other marginalized figures of our time.

Yes, of course, we work for success in bringing about healthy changes, he maintains. But we cannot come to regard those changes as the sole purpose of the work. “You stand with the least likely to succeed,” he continues, “until success is succeeded by something more valuable: kinship.”

I discussed these matters recently with my mentor, Bob Woodson, founder of the Center for Neighborhood Enterprise (CNE), which works to strengthen grassroots groups around the country. As a strong believer in measurable outcomes, he comes at this from a slightly different angle. He makes the case that this isn’t an “either/or” question, and that faithfulness and success are, in fact, intertwined.

I was in his office several months ago when he had to take a call—“had to,” because it was from a prison pay phone. Sammy had once been part of a major success story for CNE—a neighborhood truce it had negotiated that quelled gang violence in one of Anacostia’s toughest neighborhoods. But unlike so many others from that truce who had gone on to productive lives, Sammy had lost his way and was now finishing up a ten-year stint in prison. Bob had to work hard to overcome his initial hurt and disappointment, but finally he had and now talked to him regularly. As I listened, he offered to put some money into Sammy’s commissary account and to fix him up with some suits when he was released.

I pointed out later to Bob that, even after hearing him speak for thirty years, I had nonetheless never heard about this aspect of his life and work. I knew he would do anything to pull young men and women out of gang life, and I had often heard him celebrate their successes as he carried their hopeful stories to audiences across the nation. But until the coincidence of a call he had to take with my presence in his office on other business, I had never appreciated his willingness to stand as well with those who hadn’t made it—his unheralded determination to be faithful to them until they were ready to try again.

But as he explained, were he just to stand with the successful, there would be far fewer of them. Those who “carry more burden than they can bear” don’t yield their fates readily to outsiders—those whom society trains and pays to “fix” others through programmatic interventions. They might, however, share their burdens with those for whom they’ve come to feel kinship—those who were once on the same destructive paths but left them and are now ready to stand faithfully and unflinchingly beside others who are ready to leave, through all their advances and setbacks, their small wins and occasional substantial losses.

This doesn’t mean the sort of tolerance for any and all infractions that verges on enabling. But it does mean knowing when to make stern demands for more responsible behavior and when to display mercy, to give second chances. Even Fr. Boyle, who confesses that he is a softie, must nonetheless occasionally “fire [someone’s] ass.” But this is how he approaches it. “I call the person in and say, ‘The day won’t ever come when I will withdraw love and support from you. I am simply in your corner till the wheels fall off. Oh, by the way, I have to let you go.’ They always agree with me. Nearly always.”

The tension between faithfulness and success, however, can never be entirely resolved. As Fr. Boyle notes, “I’m not opposed to success; I just think we should accept it only if it is a by-product of our fidelity. If our primary concern is results, we will choose to work only with those who give us good ones.” Groups like the Austin Street Shelter, which set out initially to minister to the hardest of hard cases among the homeless, might be tempted to mix in more of those who are only momentarily down on their luck—who require far less effort and attention than the Bug Lady and who show results far more readily—in order to boost the “1s” that can be reported to funders in thrall to the tyranny of success.

But the temptation to “cream” isn’t the worst aspect of the tyranny of success. The much larger burden it imposes on those who work at the outermost margins of society, I think, is that it taps directly into and aggravates an inner turmoil experienced by the most effective grassroots leaders. As suggested by our examples, they already torment themselves every day—and, more than likely, deep into the night—with the question of success and failure. “Was I too tough today? Too soft? Did I ask too much? Too little? What could I have done differently?”

These questions don’t disturb their rest because funders have raised them; they are questions that, as faithful servants, they already incessantly put to themselves, and probably more sharply than any funder ever could. The suggestion that it takes the imposition of an external metrics framework to awaken their yearning to be effective is at best a well-intentioned misunderstanding and at worst a gross insult.

When faithful nonprofits lose someone—to drugs, homelessness, gang violence—they’re upset not because it messes with their tally of “1s” and “0s”; they’re disappointed, perhaps even heartbroken, because they were hoping, betting, counting on that person to make it this time, perhaps against a long history of failure, relapse, and disappearance. In turn, that degree of faithfulness and trust by the nonprofit staff and other program participants may well be, in the darkest moments for the person on the edge, the sole factor that keeps one foot stepping in front of the other in the long, seemingly impossible climb back to a healthy life.

Ironically, one way the tyranny of success does not manifest itself is in the ruthless defunding of “losers” in the name of supporting only “winners” among grantees. For all the tough talk from foundations and evaluators about paying only for outcomes rather than efforts, the fact is that this resolve seldom gets beyond verbal swagger, as surveys for the Center for Effective Philanthropy have repeatedly found. Indeed, one of the reasons frequently offered by foundations for not releasing publicly the results of their evaluations is that it might embarrass poorly performing grantees.

If the day ever comes when metrics bluster at last acquires some teeth—a day fervently wished for by so many philanthropy leaders—we may well begin to cull out the smallest, least “efficient” nonprofits working with the most vulnerable. Then the tyranny of success will have at last consolidated its domain.

Meanwhile, success’s tyranny manifests itself more subtly, I think, in the remorseless demand for good news from the front line, the insatiable appetite for stories about winners rather than losers. Program officers—and here I report from personal experience—make site visits and listen with smug self-satisfaction to stories about the wonderful changes that have been wrought with the foundation’s help. Photographers and public relations specialists come down and capture in word and picture heartwarming scenes for the annual report. The measurers render those stories into “0s” and 1s” and ship them off to the foundation or university for further processing.


But once the program officers, report writers, photographers, and evaluators have left the site loaded down with success stories—probably without touching the plastic platters of food and drink that were purchased at some sacrifice for the occasion—the nonprofit’s staff is left to deal alone with the daily diet of bad news. They must bear by themselves the quiet awareness of the kinfolk they’ve lost or are losing, the “failure stories” no one wants to hear, the setbacks for which a “0” is hardly adequate to describe the hurt and disappointment they cause.


Not only that, we even tend to dismiss as “unprofessional” the inner turmoil that failure may induce in frontline leaders. They shouldn’t get so involved, we advise from afar; they shouldn’t take failure so personally. It’s the peculiar luxury of program officers, evaluators, and writers to retire quickly to their downtown suites bearing neat packages of good-news numbers without ever becoming intellectually or emotionally involved in the complex and ambiguous process that produced them in the first place. After all, they can say to themselves, they’re simply being rigorous, objective, neutral, and detached—all of which are prerequisites of professionalism.

But, as Lisbeth Schorr pointed out years ago in Within Our Reach: Breaking the Cycle of Disadvantage, the most successful social service workers are precisely those willing to step outside self-imposed professional limitations and bureaucratic boundaries in order to form immediate, personal bonds with clients—bonds that are essential to but by no means contingent upon “good outcomes.” As she put it, “these professionals have found a way to escape the constraints of a professional value system that confers highest status on those who deal with issues from which all human complexity has been removed [. . . .] This suggests a fundamental contradiction between the needs of vulnerable children and families and the traditional requirements of professionalism and bureaucracy.”1

As a nation, we’ve learned to compensate for the constraints of professionalism, at least in theory, through “third-party government,” which channels relatively inflexible and unresponsive federal programs through local nonprofits. They in turn can be counted on to introduce the flexibility and responsiveness we know to be essential to success. Insofar as federal social programs are effective, chances are they’ve found ways to preserve their flexibility and individually tailored adaptiveness in the face of bureaucratic professionalism’s relentless drive to regulate, rationalize and regiment.

The tyranny of success is reflected not in a ruthless exposure of failure—as many have noted, there’s very little of that. It is reflected, rather, in our denial of failure—in our determined averting of eyes from its omnipresence, our refusal to acknowledge its direct but complex connection to success. The outcomes-oriented among us find ways to strip-mine the “good stories” from frontline nonprofits while leaving behind slag heaps of “bad stories” for them to dwell upon alone, in their difficult hours of self-criticism. We find ways to reap the benefits of deep, often painful personal commitment by frontline leaders without ourselves having to share their suffering. Indeed, we praise ourselves for keeping that suffering at arm’s length through the norms of professionalism. But in rare moments of honesty we confess that those norms must be violated on a regular basis if we are, in fact, to do our work.




  1. Lisbeth B. Schorr with Daniel Schorr, Within Our Reach: Breaking the Cycle of Disadvantage (New York: Anchor, 1989).


William Schambra is director of the Hudson Institute’s Bradley Center for Philanthropy and Civic Renewal.



  • Kelly White

    This piece beautifully conveys a main core value that I would hope all organizations try to incorporate into their work – always strive to do the right thing versus just doing things right.

    I currently run a heavily regulated agency that is working toward the prevention and intervention of child abuse and neglect. Doing things “right” is essential, as the potential outcome is the loss of governmental licensing that allows the programs to operate. But sometimes doing it “right” gets in the way of doing what is right. The author primarily talks about the tyranny of metrics associated with philanthropic funding but I would like to add the overlay of governmental funding and regulation to the discussion.

    At times it seems as though it has become almost impossible to do the right thing for our most vulnerable populations because of the regulation and measurement required by philanthropic and governmental funders. I watch agencies receive funding increases year after year because of their so-called success rates. These are the same programs I have quit even trying to engage with my clients because I know they only accept referrals of those individuals that are already virtually assured success – with or without intervention; what the author (and I) refer to as “creaming”.

    There simply must be a better way of evaluating and reinforcing our successes than the current metric driven outcomes so in vogue.

  • Alan Arthur

    The determinations “success and failure” must be applied in equal measure to each of us. Thomas Edison and Albert Einstein, Kurt Warner and Junior Seau, and Bill Clinton and Richard Nixon are a few well-known examples. To paraphrase a formerly homeless friend of mine, “Love is what we each need and must each give – and though its impact is impossible to measure, from our own experience we know it.”
    I am not worried that foundation officers will devolve into unfeeling automatons with counting clickers in their hands. They are, after all, also human beings.

  • Brooke Barrett

    Working at the intersection of education & community development, I’ve been a part of creating metrics & data systems and am all too aware that they are imperfect. However, good measurement doesn’t just create binaries of “success” & “failure” nor should it just be calculating numbers or be used as the only means to tell the story of an organization’s impact. It only creates a “tyranny” when used incorrectly. We can’t abandon the use of metrics or the pursuit of creating better metrics on the soft excuse of working with humans instead of widgets. We need metrics to understand how we can better serve our communities and to keep the sector honest about it’s impact. It’s not enough to say, “we tried” or “we exist”. There is always a way to tell impact stories using hard (quantitative) data and soft (qualitative) data on the human stories behind the numbers. Rather than creating an either/or proposition, we need to better equip the sector to use both.

  • Arlene Spencer

    Well put. Nonprofits attempt to adhere to best practices, yet, (as is going to happen) they run into the tight spaces or common practices that exceed or do not fit the benchmarks acquisition, tallying, goals acquisition, user demographics gathering, etc. criterion set forth by the organizations themselves, the donors (i.e. grant donors), best practices, different professions’ ethics, or (as shown) by the clients. To whom does what mean, and what quantitative, qualitative, or valuation do we (as a society, class, industry, etc.) adhere to those words (or jargon)? Tricky waters. We arrive, tomorrow, at the next best practices by treading the water, today, observing realities, and then innovating to face or surmount dilemmas or contradictions (as best we can). It’s brave to face and admit real truths, hard as they might be to admit.

  • Sarah Stachowiak

    This compelling piece does a beautiful job of illustrating what can go wrong if metrics are considered too simplistically. I do think, also, that an equally compelling piece could be written about the potential risks or lost opportunities that can occur when measurement isn’t undertaken. Measurement has the potential to help people understand their programs differently, segment groups to different kinds of services, or supplement their gut with data. Measurement should be just one piece that can support learning and innovation not used solely for the tyranny of success described here. I know a group, for example, that’s using an innovation process to identify families that aren’t benefiting from Early Head Start and trying new approaches so that they can see better results than they were otherwise. On the surface, they were achieving a 90+% rate of success. It could have been easy to ignore the small but meaningful group of people who weren’t part of that “1”. But without some kind of data, it’s not clear that this pattern could have been identified and addressed in this way. Rather than bemoan the potential misuse of outcomes and measurement, I hope more groups can find ways to make measurement useful and empowering for them and the people they work with.

  • David Stravers

    Bill Schambra’s essay is itself a “1” (success) because he discusses in depth the issues that too many in the nonprofit sector gloss over or just ignore. This discussion itself is exactly what we need to engage when evaluating our efforts, along with the humility and willingness to redefine what success means. Are we willing to drill down deep enough into every success or failure to understand the real impact or lack of impact of our efforts, and to determine whether the long term outcomes justify our short term celebrations of victories? The challenge for many of us is that our funders / donors often do not have the expertise or the time to invest in the discussion. Perhaps this is why, as Bill points out, there is such a wide gap between what we say is important to us, and what we actually decide when making our funding decisions? The problem is not whether to use metrics, but which metrics. Anything can be measured, even the experiences of the Bug Lady and Billy of Step 13 and those who learned from them. We need a willingness to measure deep and measure long, and funders need a willingness to invest deep and invest long.

  • Carlo Cuesta

    This great article raises a deeper issue beyond metrics. How do we deal with our own failure and how we think others perceive our lack of success? I think every nonprofit leader would like the world (or at least their funders) to have a nuanced understanding of the impact their organization is having. Yet accountability inherently drives fear, pushing us to express a simplified narrative about our work. We need to change the notion of what an acceptable narrative is, building the trust needed to freely express the complexity of our work.

  • Susan Philliber

    Perhaps Mr. Schambra just isn’t familiar with all the choices in evaluation. His depiction of evaluation as leading to a simple success or failure diagnosis creates a straw man that he spends lots of time knocking down. Who has he been working with? We have so many techniques to parse results–effect sizes, levels of significance, confidence intervals, disaggregating data to find for whom programs are most and least successful, cost/benefit analysis, scales that allow ratings of achievement along a continuum, goal attainment analysis, and many other strategies that are more sophisticated–and more cognizant of nuance–than he seems to give us credit for. The funders we work with who are the most knowledgeable about evaluation also know about all this and I have never, ever been asked for a simple 1-0 judgment in my 30+ years of doing evaluation. I would love to help him find a better evaluator who can give him information for program management and not what he seems to think we do–cast 1/0 judgments with naivete.

  • Phil Cubeta

    Bravo, Dr. Schambra! If only your article would spark a real discussion, not a debate or position paper posturing, among those who care about the state of civil society. I wonder, for example, if a conscientious thought leader on metrics, like Mario Marino, might be induced to respond. It would seem that a real discussion of these issues would have to begin with granting the validity of so many points made in this article.

  • Keenan Wellar

    To those who take this article to be a criticism of measurement itself, I really don’t think that is the point that is being made. The point is that in response to criticisms related to accountability (and consequently measurement) we are seeing the pendulum swing perhaps too far, in a classic case of the tail wagging the dog.

    I co-founded a charitable organization that is in the broad category of “human services” and like some of the stories in the article point out, human lives are very messy, and there is no matrix that will ever be able to capture all of the elements or assign accurate valuations to our work. Am I confident that we are doing great work? Yes. Can I explain it? Yes. Can I provide a mathematical formula that ties a donor dollar to quality of life outcomes related to living situation, employment situation, and social inclusion in the community? Not easily and certainly (if it was easy) not very honestly.

    And this is why, I believe, I observe with increasing frequency a trend towards dishonesty in non-profit reporting, or certainly a tendency towards gross over-simplification of social change. More reporting is not always better reporting, and in fact an increase in dishonest reporting would obviously have the potential to be very damaging.

    Please understand I am not opposed to measurement! Far from it. One of the biggest challenges in my own sector is a scarcity of resources concurrent with the ongoing funding of many “worst practices” because what is being reported and counted has little to do with what the sector is supposed to be achieving. As said in the concluding paragraph:

    “The tyranny of success is reflected not in a ruthless exposure of failure—as many have noted, there’s very little of that. It’s reflected rather in our denial of failure, in our determined averting of eyes from its omnipresence, our refusal to acknowledge its direct but complex connection to success.”

    It is true that these sorts of debates are hard to find. Kudos to NPQ for shining a light on this topic. What we really need is to have this type of debate within non-profit sectors within the communities where they operate , because rather than the need for a microscopic analysis of outcomes and impacts that are assigned through an Excel-compatible matrix, I think we really need to first pay more attention to the broad strokes.

    In my own sector it would mean challenging the systems norms of group homes and day programs, where a massive proportion of funding is invested, all in the name of helping people with developmental and intellectual disabilities to “be included” in the community. If this sounds counter-productive to you 1) I agree with you 2) you probably don’t work in my sector because most organizations in my field have made themselves quite comfortable with that contraction.

    These programmatic solutions are measured and lots of data is collected – but few of the right questions are being asked. Pretty pie charts may be impressive to some, but whether or not they demonstrate efficient and effective use of resources towards the betterment of individuals and the community…that often requires a bit more insight, and an understanding that if lives are messy, then so too is the process of understanding how those lives are impacted by the work of non-profit organizations.

    From what I have seen from organizations responding to the demand for “impact measurement” is that it inevitably lends itself to doing that which comes easy – so it can be “done” and therefore “counted” and therefore “celebrated” as a success to funders, donors, etc. If we persist in this direction, the sector will destroy itself. Our existence and necessity comes from the need to solve problems that neither government or the private sector is solving. If we respond to demands about accountability by focusing only on what is easily done, we’ll not only have eliminated the ability to innovate, we’ll have rendered the sector irrelevant. Government and the private sector are better suited to take on the easy bits. Creative problem-solving for social problems has always been driven and given voice by the non-profit sector and we need to resist any tendencies away from ownership of that important space.

  • Bill Kennedy

    While this essay makes good points about how arbitrary statistics can be, the problem can be addressed.

    All you need to recognize is that the charity’s story can be told in different ways through different channels. While some of those channels are determined by the donor / funder, the charity still has control of the others. There is nothing to prevent charities from including stories of individual success along with the metrics. In fact, most do.

    Even within the statistics, not all of them require the success/failure judgement call. You can also measure the inputs, and the steps along the way. For example, a program to help homeless people become financially self-sufficient does not have to limit itself to the metrics of its successes/failures. It can talk about the number of volunteer hours, pounds of food and clothing donated (inputs), as well as the number of health and financial sessions attended and certificates earned by participants (e.g. driver’s licenses, forklift operator certificates, food handling certificates, etc. whatever qualifications were needed for the people to find jobs). These measures can be used by both managers and funders to gauge the success of the program without the tyranny of 1’s and 0’s.

  • Leah Goldstein Moses

    What a wonderful, thoughtful piece that highlights the passions and faith that underly the sector. However, I think that while the tyranny of success overestimates the power of measurement, this piece underestimates people’s ability to see, hear and understand nuance. As an evaluator, I am constantly striving to help people in the sector LEARN. What are their successes that should be held up to inform others? What are the failures? What are those in-between, muddled instances that could help us understand our context and environment so much more clearly if we spend time looking at them? I find program staff, leaders, volunteers and funders alike are happy to have short snippets of information, but even more engaged when you help them dive into more complexity.

    On a scale as vast as the federal government, there needs to be some loss of nuance when you are aggregating from Florida to Alaska. But in between, in all those places, are people that are open to a lot more complexity.

  • Robin

    Agreed. Measurement is far more complex than “success” vs “failure”. Maybe the nuance has not reached all corners (or even most corners) of grantmaking practitioners, but researchers have developed methods for measuring progress in a far more diverse fashion. You can look at change over time, you can look at comparisons to comparable groups, you can break down “big” goals (like self-sufficiency, legitimate work) into smaller goals (like obtained legal employment, stayed out of trouble for 6 months, etc.). You can even measure how long before recidivism takes place again – if your program is successful in keeping ex-prisoners out of trouble for longer, even if they eventually go back to crime you can point to some degree of success.

    There are two really big reasons not to abandon measurement. 1) Measurement holds a lot of promise and is just as complex as the problems it seeks to measure. 2) You’ve missed the boat already – even if you don’t agree with measurement on principle, every single funder, from government to private to non-profit, wants to see numbers. If you want to stay in business and continue to serve those in need, you have to figure out how to do measurement that works for you.

  • Steven Mayer

    While there’s no doubt that that measurement is often a tool of abuse in the philanthropic sector, the suggestion by Mr. Schambra that measurement is all about assigning a binary “success” vs. “failure” is a straw man. I’ve not seen foundations attempt it, evaluators promote it, or nonprofits sit still for it. Perhaps he suggests this to promote discussion, but better he expose himself to the rich variety of measurement options out there. There’s more to this than “numbers equals tyranny.”

  • Clark McCain

    Modern philanthropy SHOULD demand success in the form of tangible, objective, measurably positive outcomes not primarily to encourage better non-profit performance. It should do so because it will result in better performance by philanthropists!

    Having spent time in high-performing academic and corporate environments as well as working several years as a program officer, I am still stunned by the level of mediocrity which exists in philanthropy. As Tom Tierney states, “philanthropy’s natural state is underperformance. There are no “market forces” that influence philanthropy. Whether a donor achieves great results or mediocre results, there’s no consequence.”

    Funders MUST identify clear goals and strive to achieve measurable results through their grantmaking as a method to introduce some form of internal accountability and foster a culture of continuous improvement.

    Schambra’s thoughtful piece laments the “tyranny of success” in the non-profit space. What of the “tyranny of mediocrity” in philanthropy?

  • Heidi Atkinson

    I think an important flaw in the current use of metrics is the lack of program refinement, letting go of poor programs and increasing investment in great ones. There is little incentive to measure incredible results when it doesn’t bring new funding. Within a single agency I have seen securely funded programs whose impact is questionable at best, while their sister program is allocated the same amount of funding despite being much better “bang for one’s buck”. The ineffective program was part of a philanthropy fad and the program officer at the funder was invested in making it look good. The agency could not afford to present their interpretation of the data because it contradicted what the funder needed to hear.

    And while evaluating what works and what does not is important, having a hodgepodge of evaluation methods, forms, metrics, and time periods imposed by funders does not translate by default to reflective, insightful management. Despite what we hear from foundations that they are partners, and that they want to help nonprofits better fulfill their missions, it is hard to believe them when that partnership only takes the form of quarterly requests for information.

    One point unaddressed in the comments is the burden of systematically tracking and reporting on nuanced successes. It is so much simpler to set up a system that tracks one metric, say graduation rates from a job training program, and then use that as a clear indicator of success than to develop a tiered system that shows X% achieved steps 1 and 2 before dropping out. And many funders will ask about inputs in a proposal, but then simplify to a yes/no metric at reporting time. “Did your clients graduate or not?” It is crystal clear which answer is success and which is failure.

  • Rev. Chris Morton

    this is a great article that highlights a very real struggle and one with which I have moved back and forth on.

    Where I am today in my journey around measurements is that I see failure and success as two terms that really don’t describe human beings except in the minds of those who want stark contrasts upon which to evaluate good and bad. And in my life I am increasingly moving away from good-bad dichotomies, and living into accepting paradoxes and polarities.

    At Associated Ministries of Tacoma-Pierce County, we have brought a strengths-based perspective to who we are as an agency. Through the strengths-based approach, we look at everyone with whom we come in contact — the 2,000 households we meet each month of which about half are new to us each month, and our myriad of community partners, and our staff/colleagues and volunteers including board members — we look at each person as people with stories to tell about the journey that they are on. And as we listen to the stories we are listening for their strengths, while also paying attention to our own strengths. So that, together, we can partner in our journey down a common path, even if it is for a brief period of time.

    In much the same way that we keep data on our donors and volunteers, when it comes to client contact, we have a program database that tracks every contact, and we gather information about each person with whom we come in contact and enter it into at least one database. Data, then, informs our work. But I hope that the day does not come that the data dictates what we do, how we do it, and when we do it.

    Yesterday, May 31st, was the last day of a one-year project called Peer Navigation Services. We hired a full-time case manager, and two people who were formerly homeless, to work with 80 of the 1,200 households we have on the community’s Placement Roster (fancy words for “waiting list”) for housing services. Of those 80 households, nearly 50% of them are now stably housed. That data point tells a great story. But, the anecdotes that we often tell are far less linear paths of success. In fact, the story of one of the women with whom one of our Peer Navigators worked is on our web site — the woman wrote a wonderful, lengthy poem about her journey (; her progress and changes, and her continued struggles. This woman volunteered to share her story with us, and wanted it to be told. So, we are telling it as best as we can, while upholding her dignity.

    Associated Ministries is building itself into a storied-cultured organization. We want to tell stories. Real stories. Varied stories. Stories of struggle and challenges. Stories of real human beings whose lives move up and down, forward and backward. So, we often tell stories that society might feel do not have a happy ending. But, human stories do not have endings — they have points along the path that lead to one ending, death. And when the path is littered with social dynamics like “we don’t provide housing to felons,” or “we won’t employ someone with that background,” or “we demand a certain level of educational achievement,” and will not at least come to know the person for who s/he is, then our job is not to ignore the litter on the path, but to name it as a part of the story. And when the path is littered with choices and decisions that create barriers, like “I started using again,” we don’t ignore that and blame society for such choices. Our effort is simply to tell the story, without the blame and shame that is so prevalent in our society. Instead, we let the story speak for itself.

    Associated Ministries has developed an array of housing programs and services in recent years, so we launched a fundraising breakfast to support those programs. Our inaugural breakfast was in 2012. So, I chose a story as a part of my remarks. I don’t remember the details as I am writing this comment to the article, but I do remember that this young woman was not on a linear path — her path was crooked and bent, sometimes because she made choices (to sell drugs as a source of income) that took her away from her focus, and sometimes because there were social dynamics (business closing so she lost her job) that pulled her away from her focus. And I remember how I told that story; with fear and trepidation, because I was afraid that our donors would not appreciate the success of the story — the success being that human beings engaged with a human being for a brief period on her journey’s path, provided what we could for the time we were together, and we let her go. Whether she had a job, or was housed, was not the measure. The measure is that there were connections made.

    It is difficult to tell such stories and not have a happy ending to them. People like stories with endings. So, I often lace story-telling with an often vague reference to my own stories — when I try to change some aspect of my life, whether it is exercising, or how I am handling situations emotionally, or when I want to change a spiritual discipline, I often do well at first, and then I falter, or stumble. And sometimes I simply quit. I don’t have the energy, or the desire, or the will. Other times, I realize that my human-ness showed up and I simply need to keep trying the new behavior. But remember, this is me; someone who is seen as living a stable lifestyle, with plenty of income, living in the same home four years, a car to drive, and plenty of clothes to wear — in other words, not many difficult choices about my basic needs. Now, ask someone to change their lifestyle while they are challenged to meet their own basic needs, and maybe you can understand why the paths bend — just like mine, and maybe yours.

    No, data could never capture my story. Success? There are plenty of places on my path that I can point to and say, “see what happened while I was there.” Failures? There are places on my path that I can point to and say, “see what happened when I was there.” When I pray for a meal, I often thank the hands of those who were a part of bringing the food to the table — not only those whose hands I can see, but also the hands of the laborers I will never meet. Success and failure are much like that — there are a lot of people involved in one story, some of whose hands we can see, and some whose hands we will never see. I think that Fr. Boyle, as he so often does, had it right — ours is not a clinging to society’s measure of success, but a fidelity to our purpose.

  • Joan

    Such an interesting piece! I recently read an article on nonprofits, and how they are finding ways to inflate efficiency ratios in order to gain donors. Worth a read: