Writing in 1987, management scholars Rosabeth Moss Kanter and David V. Summers framed the essentially problematic nature of evaluation in nonprofit settings. Because nonprofit organizations define themselves around their missions or the services they offer rather than around their financial returns, they are particularly difficult to evaluate. Services are “notoriously intangible and difficult to measure,” they write. “The clients receiving them and the professionals delivering them may make very different judgments about their quality, and donors may hold still another standard. And ‘doing good’ is a matter of social values about which there may be little or no consensus (1987, 154).” Invariably, as Peter Drucker wrote in 1968, any measurement of organizational performance is a value judgment: should we measure the effectiveness of mental hospitals by how well their beds—a scarce and expensive commodity—are utilized? This measure leads to “patients being kept in the hospital—which, therapeutically, is about the worst thing that can be done to them.” But “lack of utilization, that is, empty beds, would also not be the right yardstick. How then does one measure whether a mental hospital is doing a good job within the wretched limits of our knowledge of mental disease?” (Drucker 1968, 196-197)

Organizations Without Owners

These difficulties are compounded by the fact that, as ownerless organizations, it is unclear to whom the accountability inherent in evaluation is due: To donors—to demonstrate the efficient and effective use of their contributions? To clients—to vouch for quality of service? To trustees and directors—to show the competence of managers? To government agencies and other institutional funders—to give evidence that the effectiveness of the organization’s programs? To the general public—to prove the value of the subsidy offered through tax exemption and deductibility of donations? Each of these constituencies, as Kanter and Summers suggest, is likely to prefer a different performance standard. And none, except an organization’s governing board (which is the organization in a legal sense) and the Attorney General (who represents the interests of the public) has legal standing to demand accountability. In contrast, stockholders, customers, voters, and legislators can demand—and in some instances have the authority and legal standing to require—accountability from business firms and government agencies.

If nonprofits are generically difficult to evaluate, grantmaking foundations pose the knottiest problems of all. Often framed to serve broad and barely specified purposes (like the Rockefeller Foundation’s mandate to “serve mankind”), their trustees have enormous discretion to define and change their goals and purposes. Unlike universities and hospitals, they are not engaged in selling goods and services of any kind—so there are no clients or customers to whom they must be accountable. And, since they are not in competition with other firms, they have no obligation to offer themselves or their activities for comparison with other grantmakers. Because foundations’ main business is managing their endowments and distributing its income, they neither seek support from individual donors, other grantmakers, or government agencies, nor, for that reason, have any obligation to account to them except with regard to compliance with tax, regulatory, and fiduciary obligations.

Although in many instances established by individuals and families, donors cannot hold foundations to account for the use of their gifts. Once a charitable gift is completed, donors have no legal standing to demand that their wishes be followed. While donors can seek to increase the likelihood that the foundations they establish remain faithful to their wishes, by serving on their boards and appointing as trustees family members and representatives, the law discourages donor control by subjecting such entities to more searching regulatory scrutiny and by reducing the tax benefits that accrue from charitable gifts made under such conditions. In any event, no matter how carefully donors may proceed in their efforts to ensure fidelity to their intentions, foundations are generally perpetuities and, with the passage of time and the procession of generations, there is no assurance that a donor’s descendants will remain faithful to his original intentions.

Organizations Without Constraints

Businesses exist to fulfill their charter purposes and to maximize returns on capital. Government agencies exist to fulfill the legislative mandates on which they were founded. Nonprofit producers of goods and services exist to fulfill their missions. All of these entities, depending on the industry in which they are located, are likely to be subject to professional or industry standards. For example, all hospitals, whether public, proprietary, or nonprofit, must be licensed and must operate in accordance with pertinent government regulations.

But most grantmaking foundations operate under no comparable public or private constraints. Beyond compliance to a generic set of fiduciary and regulatory obligations, they are free to define their goals and purposes as they see fit with no substantive accountability to anyone.

Why, then, would foundations want or need to subject themselves and their grantees to evaluation—a procedure that even its proponents concede is expensive and time-consuming, and onerous and that its critics argue yields results of questionable value? Recent history suggests that the evaluative impulse has less to do with any genuine desire to measure the efficiency and effectiveness of foundation initiatives than from efforts by grantmakers to protect themselves from government regulation, from the desire of foundation executives and boards to deal with the dilemmas of decision making under ambiguity, and from the eagerness of entrepreneurial academics and consultants to market evaluation as a product.

“Do We Know What We Are Doing?”

Three decades ago, as foundations faced what seemed to be a real possibility of extinction in the face of public and congressional hostility, this question was asked by the head of one of the nation’s leading grantmakers. He offered evaluation as the answer. Today, when public sentiment is probably more favorable to foundations and other nonprofit organizations than it has ever been, we are still asking the same question—and still, apparently, offering the same answer.

The difference is that evaluation, while framed with the same rhetoric of rationality and purposiveness, in practice has taken on a very different function. Results-oriented boards demand proof of foundation efficacy, but are indifferent to evaluation findings. Foundation management pressures staff to do evaluation, but does not use the information it generates. Foundation staff do evaluation, but generally lack the resources or the competence to do it with any rigor. Grantees are compelled to participate in evaluation, but—in instances where they have access to its products—seldom find them useful.

It is easy to understand why evaluation as practiced by most foundations today is little more than conventional wisdom cloaked in a rhetoric of evaluation and planning. The same conditions prevail that Orville G. Brim, Jr., former president of Sage Foundation, identified in his 1973 essay on evaluation: foundations continued to be institutionally isolated by the lack of accountability to external stakeholders—or as Brim put it, “they don’t know whether they are doing what they think they are doing—or whether what they are doing makes any difference to anyone or not. Institutional isolation breeds narcissism and illusory feelings of power, and separates administrators from the frontiers of thought.” There is still “no readily accessible hard-headed impersonal way to evaluate the performance of foundation administrators. There are no performance statistics. . . no bottom line showing profit or loss.” The best contemporary evaluation can do is raise subjective standards, based on the aligned expectations of grantmakers, grantees, and relevant stakeholders about purpose and process. Though foundation executives and trustees are not as “socially encapsulated” as they were thirty years ago—boards are no longer overwhelmingly white, male, and Protestant—to say this is a little like saying that Harvard is no longer an elite institution because it recruits most of its students from public rather than private schools, as it did half a century ago. Foundation executives are no more likely to “deal in truth in exchanges with their primary constituency—the grant applicants” than they were in Brim’s time. Because few efforts have been made to address—or even study—the problematic “social relations of philanthropy,” as Susan Ostrander calls them, foundation officers are unlikely to hear criticism from grantees (including, of course, evaluators, a substantial number of whom do their work under foundation grants and contracts). Finally, then as now, “foundations lack natural enemies in our society.” They have no competitors. They are not regularly monitored by journalists, public agencies, or trade associations.

The situation is perhaps best captured by the title given by a task force of nonprofit executives to a 1991 report produced by Independent Sector, Obedience to the Unenforceable: Ethics and the Nation’s Voluntary and Philanthropic Community. Foundations are compelled to obey basic fiduciary duties of loyalty and care, to operate within the law, and to make certain required periodic reports to federal and state authorities. Beyond those minimal obligations, the law has little to say about what foundations do and how they do it. And the public, donors, beneficiaries, and other affected have little or no recourse under the law if they are unhappy with what foundations do.

Conclusion

It may well be that the basic premise of evaluation as it applies to foundations is mistaken. Standards of efficiency and efficacy are appropriate to organizations that have constituencies that can not only demand but legally compel accountability. No one has the power or the standing to set such criteria for grantmaking foundations. The law is largely silent as to what constitutes the substance of charity or the best ways of administering it. And, the episodic fears of foundation leaders about “outbursts of regulatory enthusiasm” notwithstanding, it is inconceivable that government would ever seriously attempt to eliminate foundations from the American scene.

Evaluation research was the product of modern welfare state liberalism, premised on the belief that people’s lives could be changed through changing the conditions under which they lived. The early evaluation literature makes clear the extent to which liberal policy makers and academics were as eager to find ways of persuasively demonstrating the efficacy of government interventions as conservatives were to find ways of showing that they were a waste of taxpayers’ dollars. The conservative revolution that began in 1980 not only succeeded in killing off the liberal welfare state, it also shattered the faith, shared by mid-century liberals and conservatives alike, that policy propositions could be tested and were amenable to disinterested, objective analysis. The conservative revolution substituted moral yardsticks for social scientific ones. The utilitarian commitment to the greatest good of the greatest number was replaced by an evangelical commitment to saving the world one soul at a time. Careful methodology and quantitative proof was supplanted by policy based on scripture and anecdote (i.e., the alleged efficacy of faith-based substance abuse programs).

In the current political climate, it hardly matters whether foundations are efficient or whether they can demonstrate the effectiveness of their interventions, since public policy no longer even makes a pretense of being based on scientific knowledge. One thing is clear: that foundations are free to seek or to avoid efficacy and efficiency; to use or not use evaluation. The existence of real needs—hunger, suffering, ignorance—are relevant to what they do only to the extent that their officers, directors, and staff choose to respond to them.

But it is important to keep in mind that the instrumental aspect of foundations—their demonstrated capacity to initiate and sustain certain kinds of social and political change—is only one aspect of what they are. The other aspect—the expressive—cannot and should not be ignored. The conception of charity as a results-oriented enterprise intended to produce social change is barely a century old. And even in its twentieth century heyday, “scientific” philanthropy comprised only a small portion of the funds given by Americans for charitable purposes (the vast majority was devoted to—and continues to be devoted to—expressive activities, primarily religious ones).

The charitable universe in the United States undoubtedly has more than its share of charitable endowments created for expressive rather than instrumental reasons. But the fact that they may not address current needs as defined by some scheme of evaluation does not mean that they serve no social purpose. Who could have guessed that an enterprise as seemingly irrelevant to the needs of a nation in the grips of the Depression—the Institute for Advanced Study, an enterprise devoted to supporting theoretical work by physicists and mathematicians—would not only help the U.S. achieve decisive victory in World War II, but profoundly shape the character of America’s actions as leader of the “free world?”

Expressive philanthropy is a roll of the dice. It allows individuals to dedicate money in perpetuity to any legal purpose without reference to any measurable standard to usefulness or charitableness. While donors and their trustees are free to seek such standards, they are not required to do so—nor should proponents of performance evaluation in the foundation world seek to require them to do so. Even if they attempted to hold forth evaluation as a requisite of philanthropic best practice, it seems unlikely, given the nature of foundations as organizations, that they could succeed.

For a lengthier consideration of the issues raised in this article see Marc T. Braverman, Norman A. Constantine, & Jana Kay Slater (eds.); 2004. Foundations & Evaluation: Contexts and Practices for Effective Philanthropy. San Francisco, CA: Jossey-Bass.

 

References

Brim, O.G., Jr. 1973. “Do We Know What We Are Doing?” In F.F. Heimann (ed.), The Future of Foundations, pp. 216-258. Englewood Cliffs, NJ: Prentice-Hall, Inc.

Drucker, Peter. 1969. The Age of Discontinuity. New York: Harper & Row.

Independent Sector. 1999. Obedience to the Unenforceable: Ethics and the Nation’s Voluntary and Philanthropic Community. Washington, DC: Independent Sector.

Kanter, R.M., & Summers, D.V. 1987. “Doing Well While Doing Good: Dilemmas of Performance Measurement in Nonprofit Organizations and the Need for a Multiple-Constituency Approach.” In W.W. Powell (ed.), The Nonprofit Sector: A Research Handbook, 154-166. New Haven, CT: Yale University Press.