Writing in 1987, management scholars Rosabeth Moss Kanter and David V. Summers framed the essentially problematic nature of evaluation in nonprofit settings. Because nonprofit organizations define themselves around their missions or the services they offer rather than around their financial returns, they are particularly difficult to evaluate. Services are “notoriously intangible and difficult to measure,” they write. “The clients receiving them and the professionals delivering them may make very different judgments about their quality, and donors may hold still another standard. And ‘doing good’ is a matter of social values about which there may be little or no consensus (1987, 154).” Invariably, as Peter Drucker wrote in 1968, any measurement of organizational performance is a value judgment: should we measure the effectiveness of mental hospitals by how well their beds—a scarce and expensive commodity—are utilized? This measure leads to “patients being kept in the hospital—which, therapeutically, is about the worst thing that can be done to them.” But “lack of utilization, that is, empty beds, would also not be the right yardstick. How then does one measure whether a mental hospital is doing a good job within the wretched limits of our knowledge of mental disease?” (Drucker 1968, 196-197)
Organizations Without Owners
These difficulties are compounded by the fact that, as ownerless organizations, it is unclear to whom the accountability inherent in evaluation is due: To donors—to demonstrate the efficient and effective use of their contributions? To clients—to vouch for quality of service? To trustees and directors—to show the competence of managers? To government agencies and other institutional funders—to give evidence that the effectiveness of the organization’s programs? To the general public—to prove the value of the subsidy offered through tax exemption and deductibility of donations? Each of these constituencies, as Kanter and Summers suggest, is likely to prefer a different performance standard. And none, except an organization’s governing board (which is the organization in a legal sense) and the Attorney General (who represents the interests of the public) has legal standing to demand accountability. In contrast, stockholders, customers, voters, and legislators can demand—and in some instances have the authority and legal standing to require—accountability from business firms and government agencies.
If nonprofits are generically difficult to evaluate, grantmaking foundations pose the knottiest problems of all. Often framed to serve broad and barely specified purposes (like the Rockefeller Foundation’s mandate to “serve mankind”), their trustees have enormous discretion to define and change their goals and purposes. Unlike universities and hospitals, they are not engaged in selling goods and services of any kind—so there are no clients or customers to whom they must be accountable. And, since they are not in competition with other firms, they have no obligation to offer themselves or their activities for comparison with other grantmakers. Because foundations’ main business is managing their endowments and distributing its income, they neither seek support from individual donors, other grantmakers, or government agencies, nor, for that reason, have any obligation to account to them except with regard to compliance with tax, regulatory, and fiduciary obligations.
Although in many instances established by individuals and families, donors cannot hold foundations to account for the use of their gifts. Once a charitable gift is completed, donors have no legal standing to demand that their wishes be followed. While donors can seek to increase the likelihood that the foundations they establish remain faithful to their wishes, by serving on their boards and appointing as trustees family members and representatives, the law discourages donor control by subjecting such entities to more searching regulatory scrutiny and by reducing the tax benefits that accrue from charitable gifts made under such conditions. In any event, no matter how carefully donors may proceed in their efforts to ensure fidelity to their intentions, foundations are generally perpetuities and, with the passage of time and the procession of generations, there is no assurance that a donor’s descendants will remain faithful to his original intentions.
Organizations Without Constraints
Businesses exist to fulfill their charter purposes and to maximize returns on capital. Government agencies exist to fulfill the legislative mandates on which they were founded. Nonprofit producers of goods and services exist to fulfill their missions. All of these entities, depending on the industry in which they are located, are likely to be subject to professional or industry standards. For example, all hospitals, whether public, proprietary, or nonprofit, must be licensed and must operate in accordance with pertinent government regulations.
But most grantmaking foundations operate under no comparable public or private constraints. Beyond compliance to a generic set of fiduciary and regulatory obligations, they are free to define their goals and purposes as they see fit with no substantive accountability to anyone.
Why, then, would foundations want or need to subject themselves and their grantees to evaluation—a procedure that even its proponents concede is expensive and time-consuming, and onerous and that its critics argue yields results of questionable value? Recent history suggests that the evaluative impulse has less to do with any genuine desire to measure the efficiency and effectiveness of foundation initiatives than from efforts by grantmakers to protect themselves from government regulation, from the desire of foundation executives and boards to deal with the dilemmas of decision making under ambiguity, and from the eagerness of entrepreneurial academics and consultants to market evaluation as a product.
“Do We Know What We Are Doing?”
Three decades ago, as foundations faced what seemed to be a real possibility of extinction in the face of public and congressional hostility, this question was asked by the head of one of the nation’s leading grantmakers. He offered evaluation as the answer. Today, when public sentiment is probably more favorable to foundations and other nonprofit organizations than it has ever been, we are still asking the same question—and still, apparently, offering the same answer.
The difference is that evaluation, while framed with the same rhetoric of rationality and purposiveness, in practice has taken on a very different function. Results-oriented boards demand proof of foundation efficacy, but are indifferent to evaluation findings. Foundation management pressures staff to do evaluation, but does not use the information it generates. Foundation staff do evaluation, but generally lack the resources or the competence to do it with any rigor. Grantees are compelled to participate in evaluation, but—in instances where they have access to its products—seldom find them useful.
It is easy to understand why evaluation as practiced by most foundations today is little more than conventional wisdom cloaked in a rhetoric of evaluation and planning. The same conditions prevail that Orville G. Brim, Jr., former president of Sage Foundation, identified in his 1973 essay on evaluation: foundations continued to be institutionally isolated by the lack of accountability to external stakeholders—or as Brim put it, “they don’t know whether they are doing what they think they are doing—or whether what they are doing makes any difference to anyone or not. Institutional isolation breeds narcissism and illusory feelings of power, and separates administrators from the frontiers of thought.” There is still “no readily accessible hard-headed impersonal way to evaluate the performance of foundation administrators. There are no performance statistics. . . no bottom line showing profit or loss.” The best contemporary evaluation can do is raise subjective standards, based on the aligned expectations of grantmakers, grantees, and relevant stakeholders about purpose and process. Though foundation executives and trustees are not as “socially encapsulated” as they were thirty years ago—boards are no longer overwhelmingly white, male, and Protestant—to say this is a little like saying that Harvard is no longer an elite institution because it recruits most of its students from public rather than private schools, as it did half a century ago. Foundation executives are no more likely to “deal in truth in exchanges with their primary constituency—the grant applicants” than they were in Brim’s time. Because few efforts have been made to address—or even study—the problematic “social relations of philanthropy,” as Susan Ostrander calls them, foundation officers are unlikely to hear criticism from grantees (including, of course, evaluators, a substantial number of whom do their work under foundation grants and contracts). Finally, then as now, “foundations lack natural enemies in our society.” They have no competitors. They are not regularly monitored by journalists, public agencies, or trade associations.
The situation is perhaps best captured by the title given by a